The NEW Build Your Own Arcade Controls
Main => Lightguns => Topic started by: Toasty833 on August 22, 2021, 12:59:15 pm
-
Lighthouse tracking, used with the HTC Vive and Valve Index VR systems, has an accuracy of within a few mm, and they come with 2 controllers out of the box. It wouldn't be too hard to 3D print or strap some gun sights to the top of a controller, define your screen area and positional offsets, and create a lightgun with no distance requirements by just ignoring the headset.
There was an attempt made here: https://www.reddit.com/r/Vive/comments/751j8d/vive_controller_as_a_mouse_pointer_v03/ to do something similar, but unfortunately SteamVR has since updated and I can't get the software to work to see if the accuracy is good enough for use as a light gun. Current LCD systems all require a minimum distance of about 1.5x screen size to work, which isn't so bad on smaller screens, but if you want to play on newer 4k screens you might have to stand very far away. With a tracked gun, it doesn't matter where the gun is in relation to the screen, you can be an inch away and get the same performance.
-
Went looking around for this kind of thing again today, and found https://imgur.com/gallery/vBaXSkW, using openVR (https://github.com/asingh57/openvr_lightgun). Very user unfriendly, but I managed to get it working with my Vive wand. In terms of tracking accuracy, it's fantastic, dead on the same mark every time, very low latency. The problem is with calibration, the tool is very basic and doesn't really let you dial in the screen position properly, a proper GUI could offer crosshairs and multi-angle shooting to define the screen much more accurately. And this was clearly early software. I also couldn't get it to compile so I had to use the debugging version with clicks disabled, but I wasn't really expecting it to be feature complete, I just wanted to know what kind of accuracy was possible.
https://imgur.com/a/ZVay1zS is a quick example of the tracking quality from about a foot away, which isn't possible on any existing LCD based setups to my knowledge. The alignment issues are all down to the shoddy calibration, what's important is how well it follows the movement.
-
Well, it's been another 2 years, and I went looking around again. Nothing has changed in that timeframe, I've seen a couple of WIP projects for this kind of thing pop up and then fizzle out without any release.
But today I read a comment somewhere about how you could just use Virtual Desktop, align it to your real screen, and then use a VR controller as a light gun on your monitor. After about 20 minutes of fiddling with distance, taking the headset on and off, I honestly got it pretty close. Not perfect, but it was about 5mm variance for most of the screen. The main problem is my 40" monitor is shiny and reflective, which throws off the tracking, I imagine other screens would work better. More than responsive enough to play with a cursor though, so I played through a bit of Blue Estate, which worked great. Only issue is that one of the buttons in VD seems to be mapped to moving the screen around if you hold it by accident, and obviously you don't get anything fancy like off-screen reloading.
I might try and mess with this concept a bit more. I've got a tracker puck, I might try and attach it to the bottom of a guncon, use a bit of card in front to block reflections, and hook up the buttons via the pogo pins. It'd be really nice if I could somehow calibrate the virtual mouse that VD creates, but that might be beyond the scope of the software, and find a program that takes any left click inputs near the edge of the screen and converts them into an alternate button press for quasi off-screen reloads. Maybe it's something AHK can do with a macro?
EDIT: Got the system half working with a vive wand, which seems to be a lot more robust for tracking than the index controller I was using yesterday, tracking is much more stable even with a reflective monitor. Unfortunately, I couldn't work out a way to remap any of the vive buttons, so you're stuck with just the left click trigger. In order to get the vive wand at a usable angle (by default VD has it coming out straight like a lightsaber), I used an updated version of OpenVR input emulator found at https://github.com/Louka3000/OpenVR-InputEmulator-Fixed to adjust the angle to be more gun like. Sadly, I couldn't get the input remapping element of this tool to work, and attempts to use the SteamVR remapper to add control options proved fruitless, so I used a keyboard button to reload instead. My hope is that with a vive tracker I get similar tracking performance and usable iron sights after attaching it to a gun shell, and maybe I work out a way to get the pogo pin inputs as usable buttons.
-
Hi Toasty,
I tried the vive tracker as a light gun route myself and used the precompiled vive controller as a mouse pointer as well. I ended up getting this gun (Hyperkin Hyper Blaster) to attach the vive tracker to:
(https://unboundxr.eu/media/catalog/product/cache/14/thumbnail/600x600/e4d92e6aceaad517e7b5c12e0dc06587/6/1/61dvjhfnlil._ac_.jpg)
https://unboundxr.eu/hyper-blaster-vr-gun (https://unboundxr.eu/hyper-blaster-vr-gun)
https://www.amazon.com/Hyperkin-Hyper-Blaster-Vive-Tracker-pc/dp/B077Y9WBDZ (https://www.amazon.com/Hyperkin-Hyper-Blaster-Vive-Tracker-pc/dp/B077Y9WBDZ)
It did all work in the end but it was a lot of mucking around. I ended up getting an off the shelf solution that uses 4 IR emitters around the TV / Monitor.
-
I tried the vive tracker as a light gun route myself and used the precompiled vive controller as a mouse pointer as well. I ended up getting this gun (Hyperkin Hyper Blaster) to attach the vive tracker to:
How did you get the calibration to work? For me it's always all over the place, and there's no good way to adjust it.
I saw the hyperkin guns, but importing one would've been overly expensive, and they mounted the tracker on the top so you can't use the iron sights without heavy modification anyway. I found it easier to just mount to a guncon, you can put a tracker in the hole for the cable with minor modification to fit a tripod mount, and then you can use the iron sights and wrap a couple of wires around the pogo pins for a trigger and A/B. I've tried some alternatives, but the required minimum distance is just too much for my setup, even with fish eye lenses. I could use a smaller screen or move further away, but I want to play on a big screen, and that requires both a large screen, and a low minimum distance. Right now I'm playing with my gun barrel about 9 inches from the display, I think even camera based laser detection would have problems here due to occlusion.
As a basic example, at this distance on a 4:3 game I have a horizontal view angle of 50 degrees. If we assume you can get about 1x diagonal resolution away from a screen with a fish eye, and you're playing with an outstretched arm instead of shooting from the hip, you would need to be around 60 inches away (40 inches due to screen, 20 due to distance from barrel to eye), giving you a view angle of 26 degrees, or almost half as small a picture. With a much larger screen, barrel to eye distance gets smaller proportionally, but even with a 200" display (which would give you a 163" 4:3 screen), you'd still only manage a 33 degree horizontal view angle.
Obviously, there's an argument to be made about what view angle is good enough. Time Crisis 3 deluxe had 50" 4:3 screens with the player about 60" away from the screens, which corresponds to a horizontal view angle of 37 degrees. Standard cabs had smaller screens with lower view angles and are still totally playable, but I'm looking for arcade level without having to rent a warehouse to fit a 300" screen in so I can stand 27 feet away.
-
It's been a while since I played with it but I used version 04 of the software you linked in your first post in the reddit thread. Which I found by changing the URL to:
http://olli.wtf/vive/ (http://olli.wtf/vive/)
so it is this one:
http://olli.wtf/vive/vive_mouse_v04.zip (http://olli.wtf/vive/vive_mouse_v04.zip)
So I run the config app I think and then I think I chose the option where you put the tracker in the corners of the screen. Being careful to do it in the order that it wants them being done, top, left, right etc, whatever it says.
Pretty sure that gives the best calibration rather than the aiming one.
I also had to adjust the X axis or Y axis value or something by 90 or -90 degrees so that it wasn't aiming straight down.
I had a python program that could pick up the pogo pin keys, there is a library for it. I had to manually map a key for a game but I was going to make it scriptable.
I had it working on a 86 inch tv and was like 4 feet away or something and it was accurate.
If you want I can dig out the gun and setup the light houses and do it again. Record a video or whatever.
You are right about the hyperkin gun though, the iron sights thing isn't good.
-
So I run the config app I think and then I think I chose the option where you put the tracker in the corners of the screen. Being careful to do it in the order that it wants them being done, top, left, right etc, whatever it says.
Pretty sure that gives the best calibration rather than the aiming one.
I gave it another try with 04 and I've got a better calibration now, but it's still not ideal, out in a few places. I could probably try it a few times to get a good set where the tracking is more accurate (bringing my tracker that close to the screen creates tracking uncertainty due to the reflective nature of the display, so I just have to get lucky with the position being accurate 4 times), but I don't know if it's worth it.
I had a python program that could pick up the pogo pin keys, there is a library for it. I had to manually map a key for a game but I was going to make it scriptable.
This is one of the big issues with vive mouse that I can see, the trigger feels a bit weird like there's a delay for precision or the dragging function, there's no off-screen functionality, and it takes total control of the mouse cursor so you need either a kill script with AHK or something to end the program, or to sacrifice a button on your gun for the menu button that toggles the cursor tracking on and off, and you need a secondary program to do basic stuff like map right click. I think ultimately virtual desktop just feels a little better in the cursor even though you have to set it up each time, because I still need to use input emulator to set my tracker coordinates in line with the gun barrel anyway (I can't seem to find a way to automate this through OVIE, you can save profiles but I don't think you can auto-apply them on boot). Vive mouse offers static calibration, but everything else just feels a little cumbersome. I think I'd have to use something like OpenVR2Key to remap the pogo pins, and that doesn't offer mouse clicks so I'd have to map to keyboard keys which brings a new host of incompatibilities.
-
I had a look at what I did and I remember now that I looked into the 04 exe file and I found that it sends the mouse clicks by hooking into USER32.DLL and accessing some windows functions.
So to stop it from doing that I used a C++ program to intercept the vive_mouse.exe process and redirect it to a dummy DLL file for the USER32.DLL calls. (I got AI to give me most of the source code to do it)
It has to be called after starting the vive_mouse exe.
If the calibration can be static (saved) could you put a sheet or something over the TV to stop the reflections just for the calibration phase?
I had a look at the default.config file that the vive_mouse_config.exe creates and it is a text file. The x,y,z co-ords could be adjusted manually or copy pasted from better calibrations.
This is the python library ( triad_openvr ) I used to access the data of the tracker (x,y,z positions, triggers and other pogo pins i.e. buttons of hyperkin):
https://github.com/TriadSemi/triad_openvr (https://github.com/TriadSemi/triad_openvr)
Once I detected the trigger, I made my own calls to user32 dll using python. I don't think I got to doing the other buttons but I could see them being pressed. Right click and keyboard should be available through user32 as well.
I remember I did this for that same reason you were saying, pulling the trigger felt bad and would sometimes be dragging causing issues.
I was trying to write my own calibration and screen co-ordinate system in python but ending up finding that way to make vive_mouse 04 work.
If you want any source code I can send it but it may be a little rough.
By the way, what VR headset are you using Virtual Desktop with?
-
If the calibration can be static (saved) could you put a sheet or something over the TV to stop the reflections just for the calibration phase?
This is a smart idea, I hadn't considered it.
I had a look at the default.config file that the vive_mouse_config.exe creates and it is a text file. The x,y,z co-ords could be adjusted manually or copy pasted from better calibrations.
Yeah, but it's a lot of fiddling. Ultimately, even if I could get a perfect calibration through Vive Mouse it feels like a little more trouble than it's worth. OpenVR2Key works well for remapping additional buttons to keypresses, and I imagine I could just use JoyToKey if I need right click for a game. I've also changed my tracker to be recognised as a controller, which is more useful, so some things that apply to trackers might not work with a spoofed controller.
If you want any source code I can send it but it may be a little rough.
I could take a look at it maybe, but I'm pretty incompetent when it comes to coding. It might be useful to have up anyway in case anyone else is interested in this kind of thing in the future and can make more use of it than me.
By the way, what VR headset are you using Virtual Desktop with?
At this point, an Index. I'm still using 1.0 base stations because they seemed to be better built and I can still use my Vive stuff.
-
Yeah, but it's a lot of fiddling. Ultimately, even if I could get a perfect calibration through Vive Mouse it feels like a little more trouble than it's worth. OpenVR2Key works well for remapping additional buttons to keypresses, and I imagine I could just use JoyToKey if I need right click for a game. I've also changed my tracker to be recognised as a controller, which is more useful, so some things that apply to trackers might not work with a spoofed controller.
Actually I think I may have changed my tracker to a controller as well, probably to get it to work with Vive Mouse or something else.
So what do you mean to use OpenVR2Key to work with, is that to use with Virtual Desktop?
I'm trying to think what would give the best setup for you. I'd be happy to help where I can with programming or testing. If you want to let me know what problems you are having at the moment with it?
I can understand the hesitancy with using Vive Mouse, it gave the best results for me with how far I got with the vive but it's a closed sourced program, if it could be modified it would be a lot better.
I was doing something similar with Virtual Desktop using a Meta Quest VR headset and there was a project posted here by NailBuster that did it with a custom program. You posted in the thread:
https://forum.arcadecontrols.com/index.php/topic,166078.msg1760535.html (https://forum.arcadecontrols.com/index.php/topic,166078.msg1760535.html)
I tried to do the same with a program on the quest where you had a 16:9 box and I would line it up with the screen and it could see where the controller intersect the screen and give mouse co-ords via wifi to a pc.
But it was a lot of peeking back and forth in the headset. If I went further with it then the controllers would allow the box to be moved around. But I didn't go that far before switching to the vive stuff.
I haven't tried Virtual Desktop with a vive headset though (I got the original one).
-
So what do you mean to use OpenVR2Key to work with, is that to use with Virtual Desktop?
Yeah, VD gives me a left click trigger but for some reason I can't get use it to bind any of the other buttons with a vive controller, with an index one I can bind buttons to right click but I guess the guy developing it broke vive support at some point (or it never had any) and nobody ever noticed. It's not a big issue because OpenVR2Key lets me bind VR buttons to keypresses.
I'm trying to think what would give the best setup for you. I'd be happy to help where I can with programming or testing. If you want to let me know what problems you are having at the moment with it?
I can understand the hesitancy with using Vive Mouse, it gave the best results for me with how far I got with the vive but it's a closed sourced program, if it could be modified it would be a lot better.
Yeah, maybe you're right. Static calibration is fairly appealing, the problem is just everything else about the program is a bit half-baked and harder to get working than VD. I guess I could use OpenVR2Key and then something else to bind to mouse clicks. I'd also need to work out how to block Vive Mouse from receiving/sending keypresses like you did, seems like I need VS for the right dlls. Then I need to try and get AHK to intercept left clicks and turn them into right clicks if I'm clicking on the very edge of the screen for off-screen reloads. I also realised that vive mouse releases control of the mouse if you point it far enough off-screen, so I don't have to worry about wasting a button on it.
-
Yeah, maybe you're right. Static calibration is fairly appealing, the problem is just everything else about the program is a bit half-baked and harder to get working than VD. I guess I could use OpenVR2Key and then something else to bind to mouse clicks. I'd also need to work out how to block Vive Mouse from receiving/sending keypresses like you did, seems like I need VS for the right dlls. Then I need to try and get AHK to intercept left clicks and turn them into right clicks if I'm clicking on the very edge of the screen for off-screen reloads. I also realised that vive mouse releases control of the mouse if you point it far enough off-screen, so I don't have to worry about wasting a button on it.
Can I ask what game or emulator you are trying to get working that uses right click for off screen reload? Most games I've seen use the same left click for firing and reload so it seems a bit odd. Maybe I could try it out.
I think using the Inject DLL code I have it would also be able to not just turn off the clicks from Vive Mouse but also intercept them and resend them. It is probably also possible to get the x,y mouse co-ords and then change whether it is left or right click. That should also remove the need for the python code.
If you have any problems compiling the Inject DLL code let me know
-
Can I ask what game or emulator you are trying to get working that uses right click for off screen reload?
Basically every game works with it. The number of games that don't have an off-screen function is less common, and right click is the standard, basically every gun I've seen sends a right click instead of left click if you click outside of screen boundaries. Only titles like Time Crisis that use a pedal instead don't really have off-screen support. The few titles I've tried so far with just a left click gun would just fire at the edge of the screen if I tried to off-screen reload, but maybe there are a few that see "is gun at border? Then reload" rather than relying on the gun to say "I can't see the screen, send right click instead".
I think using the Inject DLL code I have it would also be able to not just turn off the clicks from Vive Mouse but also intercept them and resend them. It is probably also possible to get the x,y mouse co-ords and then change whether it is left or right click. That should also remove the need for the python code.
If something like that is possible it should be pretty useful.
If you have any problems compiling the Inject DLL code let me know
I used the pre-compiled version in the debug folders and it worked fine for disabling the vive buttons, with the exception of the menu button that still disables the cursor but that isn't really a big issue and I'm guessing it's hardcoded in vive mouse rather than something that's passed through and can be intercepted. With trigger, grip and trackpad that's still 3 usable buttons I can remap through OpenVR2Key because SteamVR still sees them fine.