Main Restorations Software Audio/Jukebox/MP3 Everything Else Buy/Sell/Trade
Project Announcements Monitor/Video GroovyMAME Merit/JVL Touchscreen Meet Up Retail Vendors
Driving & Racing Woodworking Software Support Forums Consoles Project Arcade Reviews
Automated Projects Artwork Frontend Support Forums Pinball Forum Discussion Old Boards
Raspberry Pi & Dev Board controls.dat Linux Miscellaneous Arcade Wiki Discussion Old Archives
Lightguns Arcade1Up Try the site in https mode Site News

Unread posts | New Replies | Recent posts | Rules | Chatroom | Wiki | File Repository | RSS | Submit news

  

Author Topic: Quick prototype demo for possible Sinden light gun improvement  (Read 2490 times)

0 Members and 1 Guest are viewing this topic.

greymatr

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 11
  • Last login:Today at 02:45:47 am
  • I want to build my own arcade controls!
Hi,

I just wanted to give a quick demo of an idea I had using a Playstation 4 Camera and a pair of 3d glasses for a TV.

The PS4 Camera has two camera built into it and the glasses are placed on one of the cameras. Because of the polarization filter in the LCD panel of the 3d glasses it can be aligned so that the light coming from a monitor is blocked.

This makes the monitor appear like it is off and is just black. The other camera can see the light coming from the monitor and software should be able to make a quite quick difference between the two.

My idea is that this would allow for a Sinden style gun but without the need for the white border.

Thanks for listening to my TED talk ;)

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #1 on: September 26, 2022, 11:19:24 am »
What good is a video feed from a dead monitor?  Unless I'm missing something, it's no different than not having that camera active at all.

The reason it blocks with the polarizer is because it is oriented 90-degrees to the internal one in the LCD panel.  For this to do anything, you would need a special LCD panel with different polarization on the border from the rest of the screen.  And then you would still only need one camera to use it.

And since your eyes aren't polarized, you would still see the border.
« Last Edit: September 26, 2022, 11:22:30 am by RandyT »

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #2 on: September 26, 2022, 07:15:45 pm »
It's hard to tell what he means, but I can guess.

You have 2 images that are exactly the same except for one specific rectangular area in each image. You can compare the differences to find the display's outline. For example, take all the pixels of the first image that are black. Next, take the pixels from the second image that are white. The display's area would be all pixels that are both black on the first image, and white on the second image.

Another way is to compare pixels and remove all colors that are within a given tolerance range. Theoretically a rectangular object will remain since they're polar opposites.

I think a big problem is that this will be very slow.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #3 on: September 26, 2022, 09:44:24 pm »
I don't think that approach solves anything, even if it worked, which it wouldn't because it relies on border areas being non-black in color.

Honestly, there is no way to make the border visible to the camera, and not to the user, unless the border is polarized in a direction favorable to the one on the camera, and the player is wearing glasses with polarizers which block it.

This is the reason so many lightgun systems are based on IR light.  IR is in the part of the spectrum to which the camera is very sensitive, but also happens to be the part to which our eyes are very insensitive (almost completely.)  There is very little difference in the Sinden method, as opposed to lining the perimeter of the screen with IR LEDs, or even just beacons at the corners.  The LEDs have the advantage of not being visible to the player but have the disadvantage of requiring more math and/or fairly good precision in mounting.  Outlining the screen area directly on the display provides a better edge reference and no extra hardware, but at the cost of having it be constantly visible to the player.

There are probably some additional pros and cons in there somewhere, but that's the basic gist of it.
 

« Last Edit: September 27, 2022, 01:07:21 pm by RandyT »

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #4 on: September 27, 2022, 09:38:02 pm »
The player doesn't wear any glasses... He's saying that you have 2 cameras mounted in your gun. One will have the polarization filter, and the other won't. The software can identify the area of the display using those 2 images in his example. Once we have the area of the display, a border can be calculated to perform something like Sinden, or instead, the midpoints of edges can be found to do something like GUN4IR. The border doesn't actually get rendered. It doesn't need to because the software calculates it using those 2 images.

The problem is that processing the 2 images would be slow. Or at least too slow for a gun game.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #5 on: September 28, 2022, 07:07:42 am »
The player doesn't wear any glasses... He's saying that you have 2 cameras mounted in your gun. One will have the polarization filter, and the other won't. The software can identify the area of the display using those 2 images in his example. Once we have the area of the display, a border can be calculated to perform something like Sinden, or instead, the midpoints of edges can be found to do something like GUN4IR. The border doesn't actually get rendered. It doesn't need to because the software calculates it using those 2 images.

The problem is that processing the 2 images would be slow. Or at least too slow for a gun game.

Either you're missing something or I am.  If the system can identify the border of the visible screen well enough to subtract it from the other, what possible benefit is the dark screen?  The tracking system for the purposes of aiming doesn't give a whit about the physical frame of the TV, only the image generated on it, and then only the edges.  And if the edges of the displayed images just happened to be black, it would look like the blocked image.  Then what?

IOW, if it can see the edges of the lit display well enough to compare it with the blocked one, you already have the data you need.  The image he presented is not a real case scenario, being fully illuminated and blooming all over the place.  The gain adjustment in the camera will properly define the screen without these gymnastics.  The "dead" monitor image is extraneous.

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #6 on: September 28, 2022, 08:03:06 am »
The system would need both images to identify the border of the screen. Let me word it another way; If you layered those 2 images on top of each other, the screen can be identified by finding all pixels where the black and white overlap. All other overlapping pixels in the image will be the same color, or only slightly off. So you'd know those are not a part of the screen. Only the extreme differences, white overlapping black, would be the screen.

From there it's like you said in the second paragraph. You'd have all the data you'd need to perform location calculations.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #7 on: September 28, 2022, 08:42:07 am »

I know what you are saying, I disagree with the value proposition.  If you want to know the value of A, and to get that value you want to subtract the value of A from B, or B from A, don't you already have or need to have the value of A to begin with? And if you do, you don't need to be concerned with B.

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #8 on: September 28, 2022, 02:45:15 pm »
You don't have the value of A or B already though. You have to go through each coordinate/pixel in images to find them. I.e., you need to compare position 0,0 in the first and second image. You'll only know if it's a part of the display's screen when one pixel has black and the other has white. If that's not the case, then it isn't the screen. So you need to search for it... After you found them all, then you have everything you need to work towards Sinden or GUN4IR calculations.

You can't search 1 image alone because like you said, you'd just have a black screen along with a dozen other black areas in the image. It doesn't help. You need to compare it to another image that has the polar opposite color for the screen area. That overlap is how you find the display's screen and also rule out any outliers.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #9 on: September 28, 2022, 02:51:01 pm »
You can't search 1 image alone because like you said, you'd just have a black screen along with a dozen other black areas in the image. It doesn't help. You need to compare it to another image that has the polar opposite color for the screen area. That overlap is how you find the display's screen and also rule out any outliers.

But the polarization of the blank screen not only never changes, it is physically impossible to do so, thanks to the polarizer in the monitor.  The blank screen is literally and permanently just that.  There is no way to make those comparisons.

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #10 on: September 28, 2022, 05:11:09 pm »
Yeah that's the point. The only difference between the 2 images is that 1 has a black screen and the other has white screen. It's pretty easy to compare the images to find the screen. You just have to scan for those 2 differences.

Ok imagine it this way. Place the images on top of each other. Now, what parts are the same on both images? The parts that aren't is the screen. For example, you know his hand is not a part of the screen because his hand is the same color in both images. Only the screen will have polar opposite colors when comparing the images.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #11 on: September 28, 2022, 07:54:26 pm »
The parts that aren't is the screen. For example, you know his hand is not a part of the screen because his hand is the same color in both images. Only the screen will have polar opposite colors when comparing the images.

It's not "polar opposite colors".  I'm honestly not exactly sure what that even means.  Colors are just frequencies of light on the visible spectrum.

It is simply devoid of at least as much light as can be blocked by the polarizer, as it would be in the case of black, so nothing is solved.  To make matters worse, now think about what happens in a room with subdued lighting.

It just won't do what you think it will.  But feel free to try it and prove me wrong ;)

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #12 on: September 28, 2022, 08:52:15 pm »
Nothing is measuring wavelengths or light. These are regular cameras taking pictures. It is comparing RGB hex values between the pixels of 2 images. If the first image has 0x000000 for a given pixel, and the second image has 0xFFFFFF for that same pixel, then you can assume that the difference is caused by the polarizing lens.  If the lens wasn't blocking the screen's light, they'd both be 0xFFFFFF. Since we know the lens is only blocking light from the screen, we'd know that this pixel must be a part of the screen. You do this for the entire image and you've have the screen mapped out.

There is no other way that a pixel would be nearly black on 1 image, but white on the other, unless it were blocked by the polarizing lens. Well, only the screen is being blocked, so you'd know it's a part of the screen.

I was also exaggerating about being polar opposites. It's not going to be 100% black and 100% white between the 2 images. The point is that they aren't even close in value. You can implement thresholds as well since the brightness between colors will be slightly different across the entire image. There is no point in talking about implementation when trying to provide a simplified view on how it would identify the screen.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #13 on: September 28, 2022, 11:05:25 pm »
Nothing is measuring wavelengths or light. These are regular cameras taking pictures. It is comparing RGB hex values between the pixels of 2 images. If the first image has 0x000000 for a given pixel, and the second image has 0xFFFFFF for that same pixel, then you can assume that the difference is caused by the polarizing lens.  If the lens wasn't blocking the screen's light, they'd both be 0xFFFFFF. Since we know the lens is only blocking light from the screen, we'd know that this pixel must be a part of the screen. You do this for the entire image and you've have the screen mapped out.

There is no other way that a pixel would be nearly black on 1 image, but white on the other, unless it were blocked by the polarizing lens. Well, only the screen is being blocked, so you'd know it's a part of the screen.

I was also exaggerating about being polar opposites. It's not going to be 100% black and 100% white between the 2 images. The point is that they aren't even close in value. You can implement thresholds as well since the brightness between colors will be slightly different across the entire image. There is no point in talking about implementation when trying to provide a simplified view on how it would identify the screen.

It's all theory, assumes perfection in the function of all components and ignores how things actually work in practice.  If a camera can pick up subtle shade and color changes with pixel precision, then the border of the standard system wouldn't need to be a bright color.  But it does.   Cameras also get very noisy in low light conditions.  Noise means inaccuracy. In fact, the troubleshooting wiki calls out a dim screen as a possible cause for jitter.  In a perfect world, the camera should be able to pick up the subtle illumination of the leaked backlight in the black areas to define the screen.  But this is not a perfect world.

And another thing standing in the way is that only LCD's use this type of polarization.  What about OLED, CRT and Plasma screens?  This suggestion simply adds more hardware and complexity, and hobbles display compatibility while IMHO, delivering zero gain.

greymatr

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 11
  • Last login:Today at 02:45:47 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #14 on: September 29, 2022, 04:56:27 pm »
Yes this is basically the idea that I had, sorry that I didn't explain it a bit more fully.

As I understand it the Sinden system uses a white border as it uses the distortion of the border to work out how the camera on the end of the gun is viewing the screen so that it can work out where it is pointing at on the screen. Basically if it were looking dead on to the screen then the border would be a rectangle but any other angle and it becomes a parallelogram.

So what I was showing is that if we can work out where the screen is by taking the difference between the two images then we can work out the same distortion and then we don't need the white border to be showing on the screen. That is what would be gained by doing this. I just don't like the white border idea. I think Gun4IR may be the better system but then you have to mount and power the infrared LEDs.

But you are right, I have an LCD screen and that is what I was working with and it wouldn't work for other display types.

RandyT

  • Trade Count: (+14)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 6882
  • Last login:March 26, 2024, 03:33:28 pm
  • Friends don't let friends hack keyboards.
    • GroovyGameGear.com
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #15 on: September 29, 2022, 06:07:22 pm »
Like I stated, this won't do it.  Border not bright enough?  Camera can't see it.  Border not thick enough?  Camera can't see it.  So, if it has challenges just with that, it's more than a stretch to think that the camera would be able to accurately discern a black screen from a dark colored plastic frame in order to get any sort of decent reference for extrapolation.

With data processing "garbage in" always means "garbage out".  That white frame is the best reference you can get.  The only way to get rid of it is to find a way for the camera to see it, or an equally good alternative to it, and not the player.  Hiding things on-screen is exactly the challenge which faced 3D displays.  That's why viewers need to wear active shutter glasses and anyone not wearing them sees a mess on the screen. 

There's only one way I can think that something like this might work, and that would be against a bare wall with IR strips back-lighting it.  Then, with the aid of a second camera with an IR-pass filter and during the calibration, the software figures out the difference to provide the offset to the screen.

But honestly, if you are going to do that, you might as well just use IR beacons and draw virtual lines between them for the reference.  As cheap as LED strips are nowadays, and the fact that they can be had in IR as well as pretty colors, it puzzles me as to why anyone serious about these setups isn't just using those for reference points.  If done right, there are even ways for these to provide more information than just screen delineation.

TapeWormInYourGut

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 110
  • Last login:March 24, 2024, 12:25:44 am
  • I want to build my own arcade controls!
Re: Quick prototype demo for possible Sinden light gun improvement
« Reply #16 on: October 01, 2022, 12:26:39 am »
It's all theory, assumes perfection in the function of all components and ignores how things actually work in practice.  If a camera can pick up subtle shade and color changes with pixel precision, then the border of the standard system wouldn't need to be a bright color.  But it does.   Cameras also get very noisy in low light conditions.  Noise means inaccuracy. In fact, the troubleshooting wiki calls out a dim screen as a possible cause for jitter.  In a perfect world, the camera should be able to pick up the subtle illumination of the leaked backlight in the black areas to define the screen.  But this is not a perfect world.

And another thing standing in the way is that only LCD's use this type of polarization.  What about OLED, CRT and Plasma screens?  This suggestion simply adds more hardware and complexity, and hobbles display compatibility while IMHO, delivering zero gain.
Well I agree there. I didn't say it was possible, I was just trying to explain his idea. The amount of time needed to process the images would make this unfeasible for its cost. Frankly if we a back to adding IR borders, well we may as well stick with Gun4ir. Personally I've found it to be incredibly accurate no matter where I stand. The sight on my gun always lines up. Jaybee has created quite an impressive system.