Main Restorations Software Audio/Jukebox/MP3 Everything Else Buy/Sell/Trade
Project Announcements Monitor/Video GroovyMAME Merit/JVL Touchscreen Meet Up Retail Vendors
Driving & Racing Woodworking Software Support Forums Consoles Project Arcade Reviews
Automated Projects Artwork Frontend Support Forums Pinball Forum Discussion Old Boards
Raspberry Pi & Dev Board controls.dat Linux Miscellaneous Arcade Wiki Discussion Old Archives
Lightguns --- Bug Reports --- Site News

Unread posts | New Replies | Recent posts | Rules | Chatroom | Wiki | File Repository | RSS | Submit news

  

Author Topic: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen  (Read 1133 times)

0 Members and 1 Guest are viewing this topic.

voradori

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 5
  • Last login:January 04, 2019, 11:08:59 am
  • I want to build my own arcade controls!
Hi guys

I've found an very interesting video
Author uses PSeye looking at display, some software making transform projection from display edges
on camera image to the screen coordinates and a gun with IR laser pointer and mouse parts inside.

This system allows a perfect line-of-sight calibration and no need to recalibrate everything every time when changing distance or angle.
No need to worry about screen sizes, distances and other things we are used to with Aimtraks, Wiimotes or other pointers.

There's no details from uploader, so any ideas how can we reproduce this?

It's basically clear that PSEye here may need modifying with IR filter,
to make it see only the point from laser,
and that we must solder custom buttons to mouse mainboard instead of its native.

And I don't recognize the software he's using.
There's big chance that it's his own implementation, but may be someone can recognize it oh have an idea what can be used here?
Personaly, I think the same result can be achieved by using something like FreeTrack, but it needs modification for screen projections.

Another thing is that such setup allows only one gun usung that way.
Otherwise, the software should somehow differ the points, but it doesn't look that way.

So, please, share you ideas here, it may be a very interesting discussion

----------------
My channel
with Wiimote Lightgun setups and playthroughs

BadMouth

  • Trade Count: (+6)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 8502
  • Last login:Yesterday at 09:22:54 pm
  • ...
Looks like it works well.  Not convinced tracking is perfect, but it looks better than other demos and the required distance to screen is definitely better.  I've always thought tracking laser dots would remove a lot of the problems, but never thought about an IR one.  I've seen laser based systems at (real) gun shows marketed for law enforcement training, but the screens always look washed out so the camera can see the dot.

Unlimited ammo and spraying bullets on house of the dead is just wrong.  Those smaller first level enemies should have been one shot one kill.  The zombies are only two shots.   Playing like a positional gun game hurts the demo.

Titchgamer

  • Trade Count: (0)
  • Full Member
  • ***
  • Online Online
  • Posts: 2989
  • Last login:Today at 05:59:13 pm
  • I have a gaming addiction.....
Wont let me load the video atm but I will look on laptop later.

My only real concern with this without looking at the video would be the laser itself.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
That's interesting.  A wiimote could probably pull that off with an appropriate lens affixed to the front.  How would you track player 2 though?  UV laser and another camera?
Ps eyes are dirt cheap if you know where to look.  I wonder about the math involved to straighten out that trapezoid.... that's the kind of stuff I always stumble on. 



*edit*  I feel the need to point out that on an arcade cabs we would have more obstacles to get this working.  A bezel would probably diffuse the laser which could cause issues.  The camera might also have problems seeing over the controls on the control panel... unless it was mounted from a pole attached to the top of the cab or something. 
« Last Edit: December 03, 2018, 01:52:25 pm by Howard_Casto »

BadMouth

  • Trade Count: (+6)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 8502
  • Last login:Yesterday at 09:22:54 pm
  • ...
I can see it working with the camera mounted under the marquee and either no monitor glass or DK style will the camera behind the monitor glass.

In my theoretical setup with regular lasers (which I do not have the skills to execute), P1 would use a red laser and P2 would use a green laser.
That puts us back to needing dark or washed out picture for the dots to stand out though.

Titchgamer

  • Trade Count: (0)
  • Full Member
  • ***
  • Online Online
  • Posts: 2989
  • Last login:Today at 05:59:13 pm
  • I have a gaming addiction.....
Also lasers reflect off glass screens so likely to catch yaself in the eye (speaking from experience!)

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
I can see it working with the camera mounted under the marquee and either no monitor glass or DK style will the camera behind the monitor glass.

In my theoretical setup with regular lasers (which I do not have the skills to execute), P1 would use a red laser and P2 would use a green laser.
That puts us back to needing dark or washed out picture for the dots to stand out though.

Well the more that monitor image is at an angle, the more horizontal resolution you lose on the pinched end of the trapezoid.  I'm not exactly sure how much is too much though.  The easiest and cheapest way to filter off garbage is via filters on the lens of the camera.  So like I mentioned, one camera with and ir filter and one with a uv and then that particular camera can only see the opposite of it's filter.  It could be done with visible colors as well.  If you put a red filter on a camera, only the most brilliant of reds show up and I'm guessing a laser would be bright enough. 

You know, with an appropriate webcam and a good library to control it, something like this might be doable on an Arduino, eliminating the need for processing on the pc.  Wiimotes already have dot processing, but obviously pc-side software is need to translate the data into controls.  I should have a laser pointer around here somewhere.  I'll get it out and see what I can do. 

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Ok, I played around with what I had on hand and the results were mixed.  Just using a wiimote as a camera, visible lasers seem to barely emit any ir light.  I could get a dot to show up with both a red and green laser, but only when I shined it directly into the wiimote camera..... it couldn't detect it when reflected off a monitor.  So I'm pretty sure an IR laser is required.  Just to play around, I took a sensor bar and pointed it towards my monitor and the wiimote was able to track the reflection of the ir leds, so with enough ir light, I think it could track the screen just fine. 


Btw... thinking about the 2 player issue a bit more, a wiimote can track up to four ir dots, so if p1 used a single dot and p2 used a cluster of dots that might work.  It'd only have issue when both players aimed at the same spot, which would probably be rare.  Obviously with a regular webcam, we can process it however we want, so a similar method would also work. 

BadMouth

  • Trade Count: (+6)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 8502
  • Last login:Yesterday at 09:22:54 pm
  • ...
I spent a little bit of time looking for a cheap IR laser.  Apparently IR thermometers do not use IR lasers.  :'(
Also looked at security beams.
This is the best I came up with: https://www.amazon.com/Focusable-Infrared-Adjusted-0-15KHz-12x30mm/dp/B015Z1EBC6/ref=asc_df_B015Z1EBC6/?tag=hyprod-20&linkCode=df0&hvadid=312186726383&hvpos=1o1&hvnetw=g&hvrand=17634641023638740559&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9006544&hvtargid=pla-584380015398&psc=1&tag=&ref=&adgrpid=65813410767&hvpone=&hvptwo=&hvadid=312186726383&hvpos=1o1&hvnetw=g&hvrand=17634641023638740559&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9006544&hvtargid=pla-584380015398

Someone mentions using it with a Wii camera in the reviews.

EDIT: Did some quick research and 30mW sounds dangerous.   :(  Especially since you won't see it and therefore won't blink.
DOUBLE EDIT: Title says 30mW, but description says 1mW which would be perfectly safe.  :dunno
TRIPLE EDIT:

Quote
Question: What power is this? The title says 30mw but description says 1mw
Answer: It's 30.
By AixiZ OEM Electronics Seller on June 19, 2018
« Last Edit: December 04, 2018, 02:34:10 pm by BadMouth »

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
My most recent meltdown has kind of knocked the wind out of my hacking motivation, but I'm still super interested in getting this off the ground.  Civil laser products seem to have ir lasers in any configuration one would want but I'm pretty ignorant about what we would need and safety concerns.  If you guys can find a suitable diode I'll give it a try.  In the meantime I'm going to temporarily mount a wiimote to my cabinet and play with some ir leds to see just how far back a wiimote needs to be to see the whole screen and if a wide angle lens is required.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #10 on: December 12, 2018, 01:12:18 am »
Went digging through the guys youtube page and eventually found a comment where he linked to this:

https://item.taobao.com/item.htm?spm=a230r.1.14.40.ebb2eb2BpVsM2&id=547838903438&ns=1&abbucket=19#detail

The text in the description is all images, so if somebody with better google fu than me wants to figure out what it says then have at it.  It looks to me though like they are selling a hacked gun and a ps eye camera for 200 bucks.  If the driver is somewhere in that mess of links we have a ready made solution. 

argonlefou

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 561
  • Last login:March 05, 2019, 12:18:48 pm
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #11 on: December 12, 2018, 03:47:30 am »
I got this cool website which is able to translate chinese from pictures: https://translate.yandex.com/ocr
Worked well by saving images and selecting the files with yandex
It's basically advertising, with some interresting text in the end :
 
Quote
Suitable for people: there is a basic operation of the computer (download and install the computer software )
The Light Gun Game love the good guys .

Composition of electroencephalogram light gun :
Game light gun + wireless receiver + PS3 camera + positioning software computer light gun principle :
The game light gun emits laser light onto the display, and the computer uses the PSS camera to shoot the video on the display
Light spot, calculate its position on the display, and control the mouse to its position.
The game light gun uses wireless transmission of its key information to the computer .

Quote
The advantage of a computer light gun :
High positioning accuracy, after calibration mouse cursor and laser spot deviation can be controlled to 20 pixels or less
2. No accumulation of deviation, how long the game will not produce mouse cursor and laser spot accumulation deviation
3. Fast response, PS3 camera 75HzFPS, has exceeded the normal LCD screen 60Hz refresh rate
Simple debugging, application of fast Matrix conversion of image processing algorithms, 4: 00 calibration light gun positioning
5. The use of N system standard high-speed PS3 camera, light gun game and video chat Entertainment two phase should
6. Light gun model fine level high, appearance, clear texture, good sense of hand, full grip

And then there is the big list of links, with sample videos and files to download. There are a lot of emulator binaries and game patch (I still didn't find out how to really download the files from this crappy chinese cloud though)
And PSEyes drivers but you can find that anywhere else
They are mentionning some other software(in RED !) :
Quote
The copyright of the computer light gun positioning software is owned by XuHai personally. It is used by computer light guns and is continuously updated. If you have purchased a computer light gun, please contact customer service to obtain the "Computer Light Gun Setup Tutorial".



So at the end, and because of what is written in the last part, I think they are providing in this list  all modified exe/emulators to play with absolute coordinate instead of Windows' relative mouse positionning :
Quote
Computer Light Gun Game patch support :
PS1 emulator ePSXe original program is to use the relative displacement of the shop has been a unique release absolutely
A light gun version of the main program, to match the shop computer light gun use. Perfect support " solution crisis 》
And " chemical solutions crisis machine taitan plan ".
PS2 emulator pcsx2 the original program is to use the relative displacement of the shop has been a unique release of absolute
The main program to match our computer light gun use. Perfect support " solution crisis Machine 2》
, "Chemical solution crisis Machine 3", " chemical solution crisis machine hazard zone 》、
"VR", " Ninja sudden
"Attack", "vampire night"."Battle Fire Fly: AFU Khan" and " Battle Fire Fly: broken
Spear " the original program is to use the relative displacement, the shop has been alone to publish the absolute positioning light gun version
The main program to support the use of our computer light gun.
"Rambo game" is the use of the original program of the relative displacement, the shop has a unique release of absolute
A light gun version of the main program, to match the shop computer light gun use.


But I found no evidence of any driver or interesting thing like that :-\
« Last Edit: December 12, 2018, 03:49:44 am by argonlefou »

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #12 on: December 12, 2018, 01:00:37 pm »
Very, very useful thanks.  So it sounds like this is a homebrew solution as I thought since something like this would have been made available in the states if an actual company had released it in traditional markets.  You know when adapted to US dollars it's only around 25 bucks.  I figure it would be a nightmare to get imported though. 


Technically I can write a piece of software to scan webcam stills to detect motion, but I have no clue how to optimize that, so it'd probably be terribly slow.  Since wiimotes do all the heavy lifting with dot that might still be the way to go.  That being said there are webcam motion tracking software suites out there that might also get the job done. 

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #13 on: December 13, 2018, 12:17:10 am »
I did some testing and man, maybe the wiimote isn't the way to go.  Put a sensor bar in front of my cab and put the wiimote at the very edge of the control panel (about two feet away).  Couldn't see the four corners of the 27 inch screen.... wasn't even close.  I'm not sure about what kind of lens to get either. 


The only way I could see the wiimote working as-is would be if it's mounted from the ceiling a few feet back.  The added height would increase the distance and that might barely be enough for medium sized monitors. 

As for the ir laser, what about the lasers used in some laser tag sets?  I'm not sure if they use ir lasers or just an ir light. 

We really need someone to chime in that has experience in optics.  Lens filters could really make things easier if we knew what we were doing.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #14 on: December 13, 2018, 12:59:26 am »
Sorry for the multiple replies... I can't sleep again tonight. 

Checked out some of those old wii whiteboard posts from back in the day.  It looks like it's difficult to find an ir laser that will register with a wiimote. 

https://www.wiimoteproject.com/wiimote-whiteboard/ir-laser/15/

One of the solutions mentioned on page two is to simply remove the ir filter so visible light can also be used.  This could be combined with gel filters and multiple wiimotes to have, for example, one gun use a red laser and one use a green.  I'm wondering if the ir lasers will then work without the filter though.... Personally I would like to have an invisible laser. 


I also looked at Johnny Lee's old whiteboard video.  He seems to be able to see the whole screen with the wiimotes fairly close, so perhaps my aim is just off.  He does mention that the wiimote can only see about 45 degrees in either direction. 


I've got an old generic wiimote I got for 3 bucks that I don't care about.  I might try removing the filter and/or adjusting the lens tomorrow night. 

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #15 on: December 13, 2018, 02:34:11 am »
Still can't sleep.... tried some stuff and it isn't looking good.  With the ir filter removed, the third party wiimote can't detect a laser shined at the screen.  I could track it against a white wall though, proving that the camera can see the laser.  I'll try with an official wiimote, but I think the wiimote is out as a camera option. 

Titchgamer

  • Trade Count: (0)
  • Full Member
  • ***
  • Online Online
  • Posts: 2989
  • Last login:Today at 05:59:13 pm
  • I have a gaming addiction.....
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #16 on: December 13, 2018, 08:21:39 am »
Trouble with removing the IR filter you will probably get a white out effect in daylight or with and lamp that gives out IR?

I made a homemade NV unit in the past by removing the IR filter from a camera and using a IR torch to light up the area I was looking at through the camera.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #17 on: December 13, 2018, 12:56:38 pm »
Yeah there are certainly drawbacks, but right now I'm just trying to replicate what we saw in that video and worry about other issues once I get that far. 

The wiimote has some built in smarts for it's dot detection... it only registers "roundish" dots, so that helps a lot with the exception of lamps and other traditional light sources. 

I don't think it matters anyway.... I just can't get it to track a laser.  I'm wondering if the anti-glare coating on my monitors is throwing it off.  Here's the thing though... if that is the issue, I don't really want to work on it as it wouldn't work for my situation.

I can pick up a ps eye, but I'd have to write my own motion tracking software.  If I thought I'd have a snowball's chance in hell of getting that kit I linked to I'd go ahead and order it.  I just don't see how a package with a "gun" in it is going to make it through customs and that's assuming I could get past the language barrier to actually order the dang thing. 

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6242
  • Last login:Today at 04:55:48 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #18 on: December 14, 2018, 01:37:37 pm »

The PSEYE setup looks exactly like the other supposed upcoming system mentioned in one of the other lightgun threads.  I suspect it will show itself soon on the general market, but probably not at such a low price point.  I'm sure the effectiveness is also going to be highly reliant on the screen surface type/coatings.  Projector owners shouldn't have an issue though.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #19 on: December 14, 2018, 01:47:08 pm »
I'll probably order a ps eye after the holidays.  I'm like you, in that I think that we'll get a similar product, but a costly one.  The main thing that keeps me from buying a couple of aim tracks is the fact that I would be forking over 200+ bucks. 



Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #21 on: December 15, 2018, 08:47:46 pm »
Well that first sensor is 45 bucks and apparently only has one IR led.  The other two are super expensive.  Just by looking it looks like some bootleg arcade games running off of wii tech.  I guess any info is good info though. 

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #22 on: December 16, 2018, 11:12:21 pm »
I was trying to think what I had lying around and I do have some laptops with webcams that actually work on win 10.  I might play with them a little this week assuming I can get my shopping finished for the holiday. 

I really just need something with visual feedback so I can see how well various lasers show up on a running monitor.  It'll give me an idea of how well this could work in various conditions.  I worry about using the ps eye cams because they rely on third party drivers.  It'd be nice to find a cam with the same wide angle lens but using a generic HID interface.  Found some example code that does rudimentary motion detection... it should be enough for me to figure out how to do efficient dot tracking.

lodoss118

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 163
  • Last login:March 14, 2019, 06:48:16 am
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #23 on: December 17, 2018, 02:16:59 pm »
Hi I have found the guys page where he sells the kit:

https://item.taobao.com/item.htm?spm=a230r.1.14.40.ebb2eb2BpVsM2&id=547838903438&ns=1&abbucket=19#detail

You can see more info and videos for setup, I might buy his kit but gun looks a bit cheap.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #24 on: December 17, 2018, 05:14:52 pm »
Heh... thanks for linking to the page I already linked to a few posts ago.  ;)

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 17085
  • Last login:Today at 03:29:51 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: [Theory] Custom PC Lightgun With PS3Eye\Webcam IR tracking and IR laser pen
« Reply #25 on: December 20, 2018, 02:53:08 pm »
So I was playing around a few minutes ago and you know what the wiimote can track surprisingly well on some screens?  A simple single-led keychain light.  It doesn't work on all of my monitors but the ones with a glass-like finish can track it really well when the ir filter is removed due to the fact that it leaves a distinct reflection.  So an ir led shined at the screen might work for some monitors.  After all the holiday madness I'll try to source a decent webcam and experiment a little.