A BYOAC Lightgun, are we up to it?

[crashwg]

if a ps2 and a $30 camera can figure out where your hands are, surely a 3 Ghz pc could with minimal cpu load.

The EyeToy doesn't detect where your hands are.  It detects movement by seeing changes in brighness...

[sWampy]

if a ps2 and a $30 camera can figure out where your hands are, surely a 3 Ghz pc could with minimal cpu load.

The EyeToy doesn't detect where your hands are.  It detects movement by seeing changes in brighness...

Which tells it where your hands are.

[ErikRuud]

Here is another interesting project.
http://www.perceptual-vision.com/Nouse/download.html

It's looking like a BYO RGT is possible.

Those two projects look mighty similar 

Oops, sorry, that was the wrong link!  I have corrected it up above.

[sWampy]

Here is another interesting project.

http://www.cs.ucsb.edu/~matz/HGI/HandVu.html

It's looking like a BYO RGT is possible.

Oops, I duplicated the link from above.  I have now corrected it.

Nice find,  with the source to that, and http://sourceforge.net/projects/opencvlibrary/  this might be a fairly easy project.

[tmasman]

The biggest issue I see right now is that the current solutions work by moving the mouse cursor... Howerver, you can't have 2 mouse cursors... So you wouldn't be able to track 2 guns this way...

Perhaps you could figure out a way to make an analog device drive the inputs (like the T2 guns) then you could build a driver that sends similar inputs to track the 2 guns.  Maybe that's not  the best way to word it, but I know what I'm talking about... (Drive a gun with something other than a mouse)

Anyhow, a gun with an IR Diod in the barrel & a camera should work for 1 gun, we'd just need to find a way to track 2 diods... (different flashing patterns maybe...) but then you're dealing with FPS of the camera & being able to adjust for different cameras...

Interesting...

[Silver]

The IR sensor approach is easily the best. It's also how Namco do it in the recent "light gun" games (HOTD3 etc...)

Its also fully customisable - make your own ideal guns and stick a detector on the front. The whole buttons/recoil thing is entirely seperate - nothing to do with light guns - and easily implimented.

The "hard" part is indeed working out the pulsing/tracking through the sensors. Then a driver etc....

Andy of Ultimarc has posted before saying he is looking into this exact setup (Namco IR sensors). I hope he is successful as he already has the experience - I imagine the Cypress microcontollers he uses on the Optipac/Ipac are capable of pulsing the emitters and probably coding into basic X/Y data on chip. This would mean he could simply use the USB HID interface that the analogue inputs of the Optipac already use to send data to the PC.

Please note all the above is speculation on my half - I was planning to look into this myself but stopped when a) I saw how much Sega charged for there "gun sense" boards which are clearly just 2 IR emitters each (and a tiny IC I think).  b) Saw that Andy was looking into it.

[RitchieTheBrit]

I second HOAGIE!  Crap at stuff like that, but as long as there was international shipping, I think I ouwld be interested in buying one.  As for the shell, wouldn't you be able to strip a Desert Eagle BB gun?  They're damn big pistols, and two of them would look rather splendisimo!

For those with a DC, BTW, Cybergun actually make one with recoil action, and it's not bad, either!  More accurate than the MadCatz version, and it looks the dogs danglies!  If anyones intersted I cou post a pics!

[ErikRuud]

The biggest issue I see right now is that the current solutions work by moving the mouse cursor... Howerver, you can't have 2 mouse cursors... So you wouldn't be able to track 2 guns this way...

Perhaps you could figure out a way to make an analog device drive the inputs (like the T2 guns) then you could build a driver that sends similar inputs to track the 2 guns.

[tmasman]

...Don't use IR leds with a USB camera. Use colored LED's. The driver could be setup to recognize different color leds.  One gun could use blue led's and another could use red.  You coud have even use more colors for more guns, but I don't think there is need for more than two.

That's not a bad idea, I was just thinking that actual light would get annoying... You'd see it's reflection off of your monitor or the glass in front of it. IR could be seen by the camera without being seen by the users.

[1UP]

The problem with this whole thing is Mame's cockeyed support of lightguns.

[ErikRuud]

Getting Guncon2 support into MAME itself would be great, but only for those using TV's.

I think a more universal "Guncon2" approach would involve a hardware solution that could get its synch from VGA, S-Video or Composite.

I really like the camera gun idea the more I look into it.  It is independent of the monitor type and size. The gun could be wireless, and almost any gun design could be used.

[RitchieTheBrit]

Erm, 'K, I know nothing, but I like the way 1UP is thinking.

There's some really neat looking GCon guns out there, it would open up a world of possibility!  The GCon system seems to offer the most accurate and reliable bas on which to work.

And I know that Cybergun have a GCon version of the Desert Eagle 50 cal, as well as a neat little Scorpion...

Oh, and I saw an UZI, AN UZI...

Oh, and...   

[Silver]

I'd love just to plug in a gun-con and have real-time no white flash tracking. Nice and affordable too!

Theres a mame issue - where there does not appear to be native lightgun support. Feeding raw x/y data into the emulated game board would depend on how they got the data. The IR/LED arcade gun systems calculate the postion from the multiple IR data on the game board itself, so emulating that would not be a good idea.

Lightgun issue is simply that there no lightguns for pc's that are close to console ones for style/accuracy. However, if we BYOLightGun you could just put the technology inside a guncon case...

Personally, I think a BYO gun using a USB camera and LED would be much harder than using an array of IR LEDs around the screen. To use a camera you are going to have to write/acquire software that will analayse a video image and track both movement AND angle (you have to work out where its pointing and where it i). This would take a lot of processing power I believe. This is exactly what eRealgames are doing for there gun - so I would either keep emailing them to release the PC version this year, or if they fail - it looks very hackable as apparantly it already comes with a USB connector.

[Thenasty]

I'll spend $200 for one when you guys have it working specially for DOS/Arcade Monitor (15khz).

[sWampy]

Personally, I think a BYO gun using a USB camera and LED would be much harder than using an array of IR LEDs around the screen. To use a camera you are going to have to write/acquire software that will analayse a video image and track both movement AND angle (you have to work out where its pointing and where it i). This would take a lot of processing power I believe. This is exactly what eRealgames are doing for there gun - so I would either keep emailing them to release the PC version this year, or if they fail - it looks very hackable as apparantly it already comes with a USB connector.

Most of the software is already written, there is pretty complete library on sourceforge to do a camera tracking an object such as a nose/gun/eye. 

How does a bunch of led's around your monitor tell you where the gun is?

[RayB]

Starburst in Toronto is currently selling Happ guns for $70 canadian. Is there no way to get these to work with MAME?

[tmasman]

Starburst in Toronto is currently selling Happ guns for $70 canadian. Is there no way to get these to work with MAME?

Currently... No.

[shawnzilla]

Since I first posted this thread, it certainly seems that as the BYOAC community, we have the ability to make this happen. What I'd like to do is form a task group to pull this togther. The task group will have four divisions:

1) Hardware/software - This group will determine the best way to design/implement a lightgun

2) Design - What should it look like, and where will we source the parts

3) Public Relations - This group will serve as liason to the Mame dev team, as well as setting up the financial structure of this project (i.e., ensure that this will be a community project with all profits (after paying back investors) being re-invested into either A) The lightgun project or B) Future BYOAC projects or C) Un-dumped boards

4) Investors - Need I say more? I count myself as one of the primary investors. But, I certainly can't foot the bill alone.

Is this acceptable to everyone? If so, please PM me indicating your interest in any of the four groups. I truly believe we can make this happen, so that we don't ever have to lean on a company like Act-labs again.

[Trimoor]

Here's my take:

1) Use the camera tracker approach, with three IR LEDs per gun.  Why three?  That way it can track it in true 3D space, unlike the dual LED eRealGames which forces you to recalibrate every time you move toward/away from the monitor.  They can be placed anywhere, so long as they are all visible to the camera and are not in a straght line.

To differentiate between LEDs, there are two ways.  You can use color LEDs, but this can be annoying, and colored objects can cause interference.

The alternative is flashing at different rates.  My Logitech Quickcam allows me to set the refresh rate at 60Hz for lower resolutions, but I have no idea if it actually works this fast.  To control the flashing, use the serial port.  Set it to pulse once per camera refresh, where a decade counter cycles the LEDs so only one is on per camera frame.  At 60Hz, this gives both guns a full 10 samples per second.  You might run into problems if you move the gun too fast, but hopefully this can be corrected in software.  A filter should be used on the camera to remove anything but IR light.

I can help with hardware, especially circuitry.

Anyone who decides to help with software: Please, please use portable code.  I don't want to be stuck with windows.  Solid C is probably best for something like this.  No VB!!

2) This might not go over well, but what about using the case from real guns?  We could try to order the two halves directly from the factory, without any deadly firing mechanisms installed.  You can't beat the realism.  Otherwise, we should contact the manufacturer of the Happ guns and order some plastic shells.

[Gamecab]

Has anyone seen the VR Gun that you can make from a Gyroscopic mouse and other parts? http://www.geocities.com/mellott124/

It looks like a good start if you want the gun to work on all monitors.

Thanks
Charlie

[Lilwolf]

btw, as another way...

there are some very cheap crappy guns that work in mame.  Whats the chance if we hacked them into a true arcade gun, and the arcade guns original optics... that it would be better then the original?

It might make sense to use either the bio gun or the other crappy but kinda working still selling guns and fit them in a better case... with better optics.

might be an option.

[quarterback]

http://www.geocities.com/mellott124/

The link is bad.

Works for me

Hi, my name is Kevin Mellott and I like to mess around with VR. I graduated with Honors from High Tech Institute, a trade school in Phoenix, AZ and currently have my Associate degree in Computer/Electronics, FCC GROL Certification, C.E.T. certification, Fiber Optic Certification, and my A+ Certification. I currently work for Agilent Technologies in Santa Rosa, CA.

VR is one of my passions. My first Virtual Encounter was enough to hook me on this amazing concept. I have surfed the web countless hours looking for everything from Power glove information to old Vr Systems that never where, like Atari's IVR system. I am going to update my page as much as possible with all the information I have found on any subject regarding Virtual Reality and so forth. If you ever have any questions or comments please feel free to email me. I have much more info then listed here and I am always happy to help. So here goes.

[1UP]

I think the camera gun is not going to pan out.  A cheap USB camera just doesn't have the resolution to get the accuracy you can get with a proper light gun.  I'd be willing to bet that motion tracking on two guns simultaneously will bring any system to its knees.

What you're doing is trading screen flash for precision.  There is a better compromise: rather than reinventing the wheel, improve on it.  The screen flash on most PC guns is usually pure white, which not only gives you a headache pretty quick, but also distorts the monitor, further degrading accuracy.

A better way would be similar to the Guncon 2/PS2: flash the screen gray.  Easier on the eyes, less distortion, and still enough contrast for the gun to time the flash.

I think by far the best thing would be to use existing hardware as much as possible.  I think by the time you buy a gun to gut it, and a camera to hack inside, you'd actually be better off buying a Happs gun (the basic model is $70 in standard colors, and it even fits their holsters.)  If someone could come up with a cheaper interface for these, and a Mame driver, you'd be on to something good.

Speaking of which, I am working with Jack over at R0R3 to get one of his boards for testing, and a couple recoil guns on the way from Happ.  If someone wants to work with me on creating a Mame driver, speak up now...

BTW, in the absence of the Act Labs VGA guns, there are always these:

http://www.lik-sang.com/info.php?category=270&products_id=1429&PHPSESSID=6ab3f4545866b56802e980b3c167ccda

I've found them to be decent when properly calibrated (video must be 640x480, and need to tweak mouse sensitivity settings a bit), and two will work with Analog+.  Out of stock currently, but hopefully they'll be back.

[Silver]

i agree that motion tracking of cameras is not the most effecient way to go about this.

I also have my doubts about just improving the video-gun idea. If we can get a good solid gun a la guncon2 then great, but to work with composite/s-video/and PC's would be rather akward, increasing the work.

Plus plahying around with my really old monitor I find that globally increasing the brightness - even just to light gray - has a rather large distortion effect. I'm convinced this would affect the accuracy.

If we are starting from scratch, I say go for the prefect solution - remove the display from teh equation altogether, 1-off calibration at installation, real-time tracking. Aim high!

[Xiaou2]

 Well, a 1mega pixil camera with propper lenses would probably be more accurate than an arcade lightgun.   However... many cameras lack the speed needed for such a fast realtime situation.  The bandwidth would be insane, and again, something would have to interpret the images in realtime. 

  Also, a typical lightgun may not work for many different types of displays. 

  A typical arcade machine is spec'd out to precise means:

 1)  Known arcade monitor, refresh rates, ect.
 2)  Perfect sync arcade system.

 A pc running windows isnt the most stable thing as far as 'timing' goes.  Windows itself may use methods which throw timings off.  Special video card drviers may update displays in odd ways or timings as well.   

 Some tvs use strange updating methods too - which is why my panasonic tv wouldnt work with the act-labs tv gun.

  Im no expert on this kind of hardware... but from the things Ive seen, Im pretty sure that a typical arcade lightgun will not function well with mame and a non arcade monitor. 

  An alternative method is needed... or should be sought.   However... it may well be beyond the expertise on this board.   Or, at least - maybe beyond the financial
capability.   


   

 

[menace]

From the gun design stand point several things have to be considered.  it should be no heavier than the act labs gun--anyone who has actually played a game of area 51 knows that after 10mins, your arms are killing you and that thing weighs mere ounces.  Also, would an accurate sighting mechanism be possible.  This would make eventual calibration even more precise.

Also the length of the cables should be longer than an act-labs standard length so that the control box (if required) doesn't have to be at the front center of the cabinet.  wireless would be the cats --I'm attempting to get by the auto-censor and should be beaten after I re-read the rules-- but that seems a tad more difficult.

[ErikRuud]

Have any of you looked at the videos on the HandVu page?







Tracking a few LED's would be a lot easier than what they have done, and they don't appear to be using anything really high end.  The minimum cpu they reccomend is 1.5ghz.

[ErikRuud]

Has anyone seen the VR Gun that you can make from a Gyroscopic mouse and other parts? http://www.geocities.com/mellott124/

It looks like a good start if you want the gun to work on all monitors.

Thanks
Charlie

The big problem with the Gyro mouse/Gun idea is that there is no relationship between the gun and the screen at all.  You can stand anywhere with in range of the receiver and the mouse will still work. You could stand behind your cab and it will still work!  You would not be aim with the gun.  Others have said that you have to aim by twisting your wrist instead of moving your arm to mae these work.  This is not the most natural motion for using a gun.

[Silver]

Tracking a few LED's would be a lot easier than what they have done, and they don't appear to be using anything really high end.  The minimum cpu they reccomend is 1.5ghz.

I wouldn't fancy running mame on my pc alongside an app that has a minimum cpu requirement of 1.5GHz, and then streaming 60fps video down the USB port.

[sWampy]

Tracking a few LED's would be a lot easier than what they have done, and they don't appear to be using anything really high end.  The minimum cpu they reccomend is 1.5ghz.

I wouldn't fancy running mame on my pc alongside an app that has a minimum cpu requirement of 1.5GHz, and then streaming 60fps video down the USB port.

Well 90% of the cpu load they are doing is tracking the hand and trying to figure out how the fingers are held in realtime.   This would be tracking a single point of light, or even better a target pattern on the end of the barrel, that is static and never changes shape.   

I don't think you would need 60fps of full resolution video either.   You should be able to track the gun accurately enough  in a 320x200 mode, or maybe even 160x100 black and white at fairly low fps, just enough to show you a cursor updated a couple of times a second, then when you pull the trigger it could then take a high resolution shot to get exact location of gun.

I was playing with the nose mouse project with a $40 usb 2.0  camera I picked up at officemax yesterday, and it worked suprisingly well for playing arkanoid and tetris, a little tiring though.   

[ErikRuud]

Exactly!

The HandVu system is tracking dozens of points on a hand, and trying to recognise gestures.

The NOUSE sytem is track the nose and eyes.

Both of these systems are tracking based on skin color and shape.  They have to do a lot of calculations to determine what is a hand/head and what is not.

For a gun system, you would be track a few bright points of light which is a lot easier.  A single gun system wouldn't even need to scan in color, so the overhead is not as high as you might think.

[sWampy]

Exactly!

The HandVu system is tracking dozens of points on a hand, and trying to recognise gestures.

The NOUSE sytem is track the nose and eyes.

Both of these systems are tracking based on skin color and shape.  They have to do a lot of calculations to determine what is a hand/head and what is not.

For a gun system, you would be track a few bright points of light which is a lot easier.  A single gun system wouldn't even need to scan in color, so the overhead is not as high as you might think.

Need to play with the camera a little and see how well they see IR Led's it might be possible to just have a pattern on the end of the guns (checker board on one, stripes on the other) shine a few ir led's at the players, and have it light up the targets enough for the where they can be tracked.

[Silver]

Hmm, well I could be wrong about the cameras-not-a-hot-idea then.

If you have the system to play with, can you try one of the constant-tracking gun games? I think Opwolf or T2? Would be interested to see - if it works fine, then hey - build your own gun this weekend!

Also - as I'm still curious - could you take a look at cpu usage while constantly moving the sensor around in windows?

I've always doubted the accuracy of these 1 camera systems too - especially if you move slightly since calibration.

EDIT: And there is not much shortage of these camera trackers:

http://www.naturalpoint.com/trackir/index.html
http://www.naturalpoint.com/smartnav/

This TrackIR already just works on 1 reflective dot so maybe simpler than the nose-trackers. And you can just stick a reflective dot on your arcade guns....

[ErikRuud]

Take a look at the TrackIR 3.

It uses a three point reflector to give a full 6 degrees of freedom.

That is the closesedt I have seen to how I think the gun system could work.

All it needs is a way to register a trigger pull, and calibration mode so that it knows where the edge of the screen is in relation to the gun.

[Silver]

Looks very nice (if expensive).

I notice that they have withdrawn generic mouse support in recent drivers, and only support particular games in the drivers - there is no full directX input.

The trigger pull is just a button - so stick the detector in the front of the gun, and wire the button microswitch to your ipac/keyboard hack/keywhiz.

The biggest issue for me is no 2 players - that would require either a driver rewrite to track 2 points (and not get confused) or 2 systems on a different IR frequency or pulse timing (I notice they use LED IR tracking in this system - they just use a camera to detect the points rather than IR LDRS)

Hmm This suggests writing a driver to track 2 points by 1 camera is quite do-able - different shaped detector point or something.

I still like the idea of building-your-own from scratch though. I'm going to keep looking for the IR LED/LDR/PIC approach...

[AndyWarne]

I am already looking at the issue of light guns and have got to the point of writing code and having a working IR mechanism. My intention is to make it system-independent but if this does not work out it would be necessary to have close integration with MAME. If this becomes necessary then some kind of open-source arrangement might be a good idea. The problem with open-source is that it makes recouping any financial investment difficult and this kind of project would require investment to do it properly.
Andy

[Silver]

That's fantastic news Andy.

Do you mean code for a PIC/microcontroller rather than an OS?

I see your problem though. Are you looking into making it supply raw x/y co-ordinate data? and then a driver in the os?

Maybe its also time to look at the lighgun code in mame, which I believe just emulates a mouse.... time to brush up my C skills.. (or try anyway).

[tmasman]

Take a look at the TrackIR 3.
It uses a three point reflector to give a full 6 degrees of freedom.
That is the closesedt I have seen to how I think the gun system could work.

This still isn't exactly like what we want, but it's definately a proof of concept!

This device is designed to move the camera view around as you move your head... (Real-time tracking)...  There's absolutely no reason why you couldn't design something similar with reflectors or IR LEDS attached to a hand gun.

I've been thinking through the math behind calculating the gun's aim... It could be a bit tricky... If you move left or right fron the point you calibrated the gun, it would be tricky to re-figure the actual gun angle... (Although still doable).  But what if you changed your distance from the screen?  That would drastically change the angles... ug... I'd be willing to mess with it, but I've never messed with image recognition or manipulation programatically...
gr...

I might mess with some of it in my free time, but meanwhile, I'm gonna be watching Mr Warne for any further progress.

[Trimoor]

3 LEDs is enough for full 3D tracking.  When moving away, the dots appear closer together.  The tracking algorithm is relatively simple; it's a video card in reverse.  It measures the distance between each dot and uses cosine to calculate a 3D object.

AndyWarne, how does an open source make investment difficult?  The investors are willing to pay for the finished product, or a finished idea.  We don't care about selling it.

[Hoagie_one]

do we have an ETA on light guns Andy?