Updated IR light gun Arduino sketch

[JayBee]

Hey guys,

I thought I would share my IR light gun Arduino code I've been working on lately, for whoever is interested.
This project started as a small update of SAMCO's great IR Cam code ( https://github.com/samuelballantyne ), but since I rewrote most of the code I decided to make my own Github.
It uses the same hardware (beside pin numbers since I'm using a ATmega32U4 chipset), and I also added more hardware for a solenoid (will post about that later).

The Arduino sketch can be found here: https://github.com/JayBee-git/IR_Light_Gun

Here is the list of notable differences compared to SAMCO's original code:

• A bit more precise mouse cursor; I kept SAMCO's efficient position calculation code, but made it float instead of integer. The difference is not huge (few pixels), but it removes a bit of cursor jittering.
• Faster cursor and less latency; since I rewrote everything without delay function in the main loop, the exection is faster and the mouse has less lag.
• Out of range detection; Detects if there are less than 2 points visible (out of the screen aiming, obsctacle...), leaves the cursor at the last known point (to avoid accidental cursor jump), and releases the mouse control (to use a normal mouse).
• Calibration parameters; You can now edit where you want the calibration points to be on the screen.
• Calibration error check; To avoid calibration problems, it will not let you save calibration points if it's out of range (for instance if your IR Cam doesn't see one of the IR LED). It will let you repeat the aiming as long as it's not correctly detected.
• Joystick mode support; Now you can also use your Gun as a joystick for games that don't have mouse support. You can enable/disable it by holding the calibration button more than 2 seconds. There seems to be slightly more latency in this mode, so I would advice always using the mouse mode whenever possible.
• Save calibration points and joystick/mouse mode in the arduino EEPROM; Now your last calibration points, as well as the last mode used (joystick/mouse) are saved in the arduino EEPROM memory. You can disable the option easily if you don't want to use it.
• Button holding detection, with timer for each button; A hold function will be triggered on whatever button you hold, with the timing you want. Useful for things like fullauto recoil mode or simply having a secondary button function when holding.
• Sleep timer; This timer is used to pause any calculation when not aiming at the screen for a while. I wanted to also shutdown the camera, but it doesn't seem to want to be switched OFF and ON again. (limitation of the Wire library?)
• Position filter; You can activate this option to make motion become a lot smoother, and removes lot of jittering, but it also adds more latency. I recommend to leave it off unless you really have precision trouble.
• Rumble motors and solenoid support; If you connect a solenoid board or a rumble motor to pin 7 (or other pin if you change it), you will get rumble/recoil support. It will be triggered once when the left button (gun trigger) is pushed, and switch to fullauto mode if the button is hold. Be very careful if you use solenoid as you need the correct circuit for it, and using a bad timing settings might also damage it. As the other options, you can change the settings at the beginning of the sketch file.

Thing to add next:

• Add button combo detection (for instance calibration + trigger).
• A way to enable/disable cursor filter with a button combo.
• Make a Windows tool to manage those options, and help with the configuration.
• Investigating in new ways of improving even further the accuracy and viewing angle.
• Trying to support PS2/XBox or other old hardware. If somebody can help me with the PS2/XBox protocol, that would be great
• Add a RGB LED support for status/mode control and errors.

Tell me what you think about it, and if you have anything to add/change, any suggestion is welcome

External libraries used from DFRobot, Jonathan Edgecome and Matthew Heironimus (big thanks to them). You can find them here: https://github.com/DFRobot/DFRobotIRPosition https://github.com/jonathanedgecombe/absmouse https://github.com/MHeironimus/ArduinoJoystickLibrary

[lightgungamer]

Hey, this looks awesome. It’s great to see people working on this.

So would your code convert to work on the itsybitsy m0 used in the samco project just by changing the ports used in the code?

Do you think there would be any benefit to using more led points?

[JayBee]

Thanks, I really like working on it, I will keep updating it as much as needed

Yes of course it should work on the istybitsy m0.
Although I forgot to add a code to disable the solenoid/rumble function if you don't have one, might add it today.

I thought about adding more led too, but here is the big problem with it: the camera is not able to recognize which led is which anymore if you stop pointing them.
For instance if you have 4 LEDs, and aim outside of the screen, and come back to see only 2 of them, the camera has no idea of which 2 led it is.
When you have only 2 led it's pretty simple, there is always one on the left, one on the right, easy to know where they should be.

But I am currently investigating for a better aiming solution that will be fully compatible with this hardware, stay tuned for more

[lightgungamer]

That sounds interesting. I’ll definitely keep an eye on it.
The viewing angle and issue with losing sight of the points does seem to be a problem. There is some chat on the samco github page which looked interesting about dealing with more points and keeping track of them. It’s a bit beyond me though!
https://github.com/samuelballantyne/IR-Light-Gun/issues/1

[JayBee]

That sounds interesting. I’ll definitely keep an eye on it.
The viewing angle and issue with losing sight of the points does seem to be a problem. There is some chat on the samco github page which looked interesting about dealing with more points and keeping track of them. It’s a bit beyond me though!
https://github.com/samuelballantyne/IR-Light-Gun/issues/1

Yeah Toni Spets code looks so nice and promising, I really want to study it and see if I can use it for our project, it would be awesome to have such precision, range and auto calibration for it

[samco]

Nice work JayBee, just had a proper look through your code theres some really cool stuff in there. I don't think I'll be implementing any of it in my code  (apart from taking out the delays which in hindsight should have been a no brainer ) just because I want to keep my code as simple as possible for the your average DIYer so if they having any hic-cups I can easily help them out. But it's great to have a place for all these extra features for more advanced coders. Cheers Man.

[JayBee]

Thanks Samco,
Yes it's nice to leave the option open for everyone
By the way, I figured out how to get a 4 points setup, with the IR camera we have it looks like it's even more simple and reliable than with the wiimote.
Now I just need to learn some more math and 2D to 3D space coordinates calculation (which is my weakness) to code that 
If you have any good docs on the subject, that would be awesome 

[JayBee]

After a lot of investigation, it seems that without tilt/gravity sensor, the 4 IR point system seems very hard to do, it would especially limit the tilt we can actually do with the gun.
I will see into it a bit more, see how bad it would be.

[lightgungamer]

Oh no. What’s the challenge with it?  Is it losing sight of points?

I would say that having it be accurate when you moved a bit (where you stand) would be more important than being able to tilt the gun if that makes a difference.

[lightgungamer]

Did you say you had another solution in mind as well?

[Howard_Casto]

From what I did with the wiimotes, the challenge is both.  You have to be really really far back to always have all four corners of a monitor in view at all times.  Additionally, with all four points in the corners, there's no easy way to tell which way is up. 

[JayBee]

The very good feature of the camera we are using in this project is that it keeps track of every single point it sees (up to 4) as long as they stay on screen.
For the setup I prefer to put the leds right in the middle of each side. Makes it perfect to know which side you are aiming, and reduce the view angle needed by almost half

Actually I just managed to make the Arduino able to recognize which point is which, with 3 or 4 points on screen.
Once those 3/4 points get detected, it keeps track of them, and you can aim anywhere as long as you keep at least one point in the cam viewing angle.
Of course the more points the camera gets at the same time, the more precise the aim and tilt will become.
You can even move around or get closer

Oh and if you put the LEDs right next to the LCD panel, or just do one calibration (to detect the screen borders) , you won't need any more calibration for a perfect sight aiming

[JayBee]

Oh no. What’s the challenge with it?  Is it losing sight of points?

I would say that having it be accurate when you moved a bit (where you stand) would be more important than being able to tilt the gun if that makes a difference.

Yeah I totally agree with you, that's the main reason that pushed me to want to do the 4 LEDs detection, along with the reducing of distance needed to make it work.
It was very hard but I think I found a way to make it work decently.
Now I will need more math and geometry to calculate all of that properly.

[JayBee]

Just a quick update, my sketch won't work on the itsybitsy m0, because this board doesn't have native EEPROM and doesn't support the joystick lib I am using.
I might make 2 versions of it, one for the Arduino atmega32u4 I am using (which I recommend because it is cheaper, easier to find and has more functionality), and one for the istybitsy.
It's sad the pinout isn't the same for both, that would make it way easier to use SAMCO's PCB 
But I guess if nobody saw it didn't work, it means nobody actually uses my version, so maybe I should focus only on the 4 IR leds setup and forget this one 

[ryoken]

Just for ask .. what if.... the ir camera is in the monitor and the lightgun had the 4 leds.

[JayBee]

It won't work, the LEDs need to be far enough apart to use them in any kind of calculation.
The ps move was doing that only because it had tons of other sensors in it.
Even if it worked it would need to have at least twice the number of arduinos, a software to combine their both input into one (if that's even possible) and would be a pain to use in multiplayer.
But of course you are welcome to try to build a system like that, people were saying 4 LEDs system were unluckily to work, and I still made it, so who knows 🤣

[Zebra]

How do they manage it with arcade IR guns which have 10 - 12 led sensor boards positioned around the screen bezel?

The House of the dead 4 arcade has been dumped for use with Teknoparrot. perhaps there is some code ideas to borrow given that the PCB is basically a regular PC?

The arcade IR guns I have used seem to be a lot less sensitive to changes in the players position and they seem slightly more accurate (than an Aimtrak). I assumed this was due to them using 10 led clusters instead of 2.

[JayBee]

How do they manage it with arcade IR guns which have 10 - 12 led sensor boards positioned around the screen bezel?

The House of the dead 4 arcade has been dumped for use with Teknoparrot. perhaps there is some code ideas to borrow given that the PCB is basically a regular PC?

The arcade IR guns I have used seem to be a lot less sensitive to changes in the players position and they seem slightly more accurate (than an Aimtrak). I assumed this was due to them using 10 led clusters instead of 2.

From my understanding, those arcade gun systems use a wide angle/high speed IR camera, and I believe they either make the surrounding leds flash one by one really fast, to make the camera know which is which, or they have a very good algorithm to keep track of them all without having to make them blink.
There is an arcade center close to my place that has it, next time I go there I will try filming the leds at 240fps to see if they blink.
The limitation of our current setup is that the camera has reduced framerate (not fast enough for blinking), can only see 4 IR points at a time (not enough to have this 10+ led setup), and has a low viewing angle (23~33 degrees).

But the good news is that I still manage to make a very similar system with 4 leds!
Like in the arcade it has auto calibration that allow you to have perfect accuracy no matter your position, and reduces the minimum distance requirement greatly.
You can see it here:
http://forum.arcadecontrols.com/index.php/topic,161189.0.html
I just finished the first version of it, I will post it soon for whoever wants to try it.

[Zebra]

I keep meaning to try one of those Extreme IR sensor bars made for Aimtraks to see if it delivers the wide angles it promises and the improved accuracy. They apparently can be seen (by the gun) from 140 degrees  and work much closer to the screen.

The arcade guns definitely have nicer looking cameras with noticeably larger lenses. The difference between Raw Thrills gun camera's like this and an Aimtrak camera are night and day:

https://na.suzohapp.com/products/arcade_game_parts/820-00010-01

With that said, the guncon 3 works about as well as the Time Crisis 4 arcade gun in terms of accuracy. It's just irritating in how it loses sight when you get within 8 feet of the screen. If that can be solved with better led boards, I'd take it over an Aimtrak. I'd never give up my real light guns on a crt though.

[JayBee]

I keep meaning to try one of those Extreme IR sensor bars made for Aimtraks to see if it delivers the wide angles it promises and the improved accuracy. They apparently can be seen (by the gun) from 140 degrees  and work much closer to the screen.

The arcade guns definitely have nicer looking cameras with noticeably larger lenses. The difference between Raw Thrills gun camera's like this and an Aimtrak camera are night and day:

https://na.suzohapp.com/products/arcade_game_parts/820-00010-01

With that said, the guncon 3 works about as well as the Time Crisis 4 arcade gun in terms of accuracy. It's just irritating in how it loses sight when you get within 8 feet of the screen. If that can be solved with better led boards, I'd take it over an Aimtrak. I'd never give up my real light guns on a crt though.

yeah it would be nice to have better angles and all, but I honestly can't stand the fact you have to recalibrate every time you move. It's especially annoying when you want to play with a friend that love arcade gun games but don't understand anything to this calibration thing.
I'm still searching better solutions for those damn IR led. Even very powerful ones don't work perfect.
I'm investigating with SMD leds, since they seem to have way better power and angle.