Universal Console Dashboard Project.

[Howard_Casto]

My hatred of the SuperBowl is your gain!  With absolutely nothing on tv tonight I decided to look into a project I've been meaning to do for a while, after I worked on mamehooker a little of course. 

Anyway, you know where on modern game consoles you can press a button on your controller and bring up a menu (aka a dashboard) that allows you to do various things including load and save states and exit the emulator?  Yeah we'll I'm making one.  There is a great need for this as many of the console emulators expect you to whip out a keyboard to do basic tasks like load a save state or even exit the emulator (often the exit key can't be remapped or binded to a joystick button).

Recently all of the popular console emulators got lua scripting support.  That allows me to draw on-screen and make a menu, which helps tremendously. 


I've made a test lua menu for fceux 2.2.2 (included below) which, while ugly allows you to access a rudimentary menu via your controller of choice.  If while holding start to pause the game you also hold select, the menu appears.  I've already managed to get a reset button and power button (exit the emulator) hooked up.  What I'm having trouble with is save states.  Even though I've followed fceux's documentation to the letter and even tried various other ideas I can get things to save and load properly.  This seems to be a common issue amongst the emulators (several of the 'X' emulators are using a common lua core). 

So I'm thinking I can just launch a keypress simulator app via os.execute... but os.execute steals focus away and displays an ugly command prompt.  Are there any ways to avoid this?  Perhaps there is a way via lua scripting to call the keybd_event api directly?  Any help would be appreciated as if I get this menu working properly it'll almost automatically work in snes9x, gens, pscsx and a few others. 

And yes, I will make it pretty, but we've got to get it functional first.

===================================

Update: 

GensRR now has a fully working menu!



Update 2: 

FCEUX is also done.  Includes Game Genie support!



Update 3:

Snes9x-rr is done!



Update 4:

You can now download the UCDP files from my site:

http://dragonking.arcadecontrols.com/static.php?page=AboutUCDP

[Howard_Casto]

Wow, I figured there would be more interest in this.  Upon further inspection it appears that fceux's lua implementation is borked... I wasn't doing anything wrong coding-wise.  I might see if there is another build out there that works better. 

In the mean-time I wanted to see how portable the system is.  The answer is pretty portable!  I changed only a couple of lines of code and got it working in Gensrr 11b.  Save and Load actually work in this emulator.  Right now I've got it hard-coded to slot 1, but I'll eventually add a menu allowing you to select the slot.  Again, right now this is just a proof of concept deal, I'll make it pretty and add the nice features later. 

If you want to play with it I'll include it below.  With this one you pause, hold and press the "mode" button.  I'm not as familiar with Genesis operations so we may have to change that.  Also note the menu isn't centered due to increased resolution.  I'll fix that later. 

I know there is also lua support for snes9x and I believe a few of the n64/psx emulators support it as well.  Quite frankly that's all I care about in terms of console emulators. 

[Howard_Casto]

Hmm, maybe there isn't any interest due to the title and lack of pics. 

The goal here is to make an in-game menu for all the popular console emulators so that you can do all the major tasks, including loading and saving states, resetting the console and saving the game using the original controller. 

Here's a pic of the Gens Menu, which I've gotten the furthest along with.

[pfcwintergreen]

It's a cool idea, and I share you hatred of the superbowl,  but as far as lack of interest goes it might be because of existing projects...namely Hyperpause; If I understand your concept correctly it's pretty similar(albeit Hyperspin-only). The problem with Hyperpause is that it's a configuration nightmare(at least in my experience) due to the fact that different emulators respond in unpredictable ways to the program's attempts to suspend the emu's process. I suppose going the lua route will fix that, right? But then wouldn't there be a lot of emulators that don't support lua? I am interested in this but I don't know if I would implement it on my cabinet if it could only work with a few(lua-supporting) emulators.

Well, in any case thanks for looking into it and all your other tools(troubleshooter 2 etc).

[Howard_Casto]

Hyperpause, with all due respect is the most convoluted way you could possibly do it.  I'm not a fan at all.  Much respect to whoever decided to tackle that one, but I've tried making various external methods over the years.  They never work and when they do it's so convoluted that nobody will bother anyway.

This is for console emulators only.  If you are on a mame cab you'll have access to a keyboard ect and savestates aren't that important, but on the other hand if you are using a console emulation box, you have a gamepad, and hopefully that's it. 

There is zero configuration involved with this method, unless you count having to add "-lua menu.lua" to the command line string when you launch a rom.  Because it's lua the menu is actually rendered in game, so it's a part of the emulator essentially.  You don't even have to configure the buttons... these lua scripts use the game button memory addresses, not the physical joystick buttons, so it automatically looks for whatever you have mapped to select + start or what have you. 

It looks like there are lua enabled emulators for nes,snes,genesis,n64 and psx and I believe Atari 2600.  That's all you really need.  Earlier consoles don't have anything to save and newer ones should have the ability to save at any point. 

[Le Chuck]

Support for all my fav consoles would be pretty great really, NES/SNES/N64/SG

Keep it up. looking forward to you again making my gaming life easier 

[nullPointer]

Holy Crap!  I just stumbled across this.  What a great idea Howard.  I can't wait to see where you go with this.

Just as a general question, are the visual elements divorced from the "functional bits", or is everything pretty well tightly integrated?  I'm not overly familiar with Lua, but if one were interested in skinning the GUI without screwing up the functionality, is that something that would be possible?  (And yeah, I know you said you would prettify it at some point, I was just curious).

Keep up the great work!

[Draco1962]

Fantastic idea!  In addition to user skinning, will the application be frontend friendly?  I am really looking forward to seeing how this develops!

[Howard_Casto]

In terms of skinning, it's possible, BUT we'll probably want to make two separate versions, a graphical one and a "lite" one that just uses internal rendering. 

The loading of images requires the gd image library for lua to be installed.  While you only have to do it once, it isn't exactly a walk in the park to install.  Also I'm unfortunately limited to the game resolution.  So for snes/genesis, ect something fairly nice can be added, but for the nes / mastersystem ect I don't know how much can be done graphically.

The idea was to try to theme it around the console.  I've since refined it a bit from the screenshot, but the genesis one uses that classic grid pattern, ect.   

Right now my biggest challenge is finding the best versions of each emulator.  Gens works flawlessly, but the others have some issues.  I can't get save states to work in fceux... it might be fixable because the source code is available.  Snes9x is having some sort of issue, but it has to be something I'm misunderstanding because it errors out when trying to read the gamepad, which is pretty basic functionality.  I haven't looked at the others yet. 

These are all offshoots of the old "rr" project that helped people make emulator assisted videos.  So in theory the lua scripting is universal amongst the emulators but in practice there are various minor inconsistencies.  Since Gens is fully working I'll start with that, and modify the script for each of the other emulator's quirks. 

This is one of those spare time deals..... much like fix it felix, it isn't a huge project.  I'll finish the gens script later tonight and work on the others when I get around to it. 

[Howard_Casto]

Ok here's my first completed prototype, a Menu for use with Gens-rr 11b. 

Installation:

1.  Place this lua script in your "Gens11b\lua_samples" folder.
2. Configure your FE of choice for Gens as normal only add the command line flag "-lua c:\gens11b\lua_samples\GensMenu.lua" to the end for launching options.  (Note the full path might not be necessary, I'm not sure.)
3.  Enjoy!

Usage:

(Note:  When I mention buttons I am referring to the virtual buttons on the Genesis gamepad for player 1, what you have them mapped to is dependent upon how you've got Gens setup.)

To bring up the menu, press Start, which should pause your game and while holding start press the "Mode" button.
While in the Menus, Up and Down navigate, the B button selects and the A button backs out.  Start and Mode don't do anything while in the menu.

Menu Features:

A "fake" reset I made via capturing a save state when the emulator starts.
A power button, which exits the emulator. 
Load/Save State selection menus, allowing you to choose from any of the 10 standard slots to save/load a game.

Bugs:

I need to implement better button de-bounce code.  Right now you need to be careful when you go into the load/save menus or you'll accidentally jump two menus and select a slot by accident! Just tap the buttons for now and I'll fix it tomorrow hopefully.

[Howard_Casto]

A set of screenshots (note, I had to do these in windowed mode and Gens doesn't scale well... it looks better at full-screen):

[Howard_Casto]

Ok so I looked into the snes9x issues.....

What happened was that for whatever reason they stopped making a "rr" version at around 1.52 and 1.52 is borked.  If you use an earlier version the scripts work just fine.  I even plugged in my gens script and aside from some save state issues (saves are coded differently in snes) the menu looked and behaved identically.  So we'd have to run 1.51 or earlier.  I'm not sure what has been changed in snes9x over the years, but even version 1.53 is around 4 years old.  The source to 1.52/1.53 is available on google, so if somebody wants to get it working....  I can help but I'm not up for a huge project atm. 

[Howard_Casto]

So I opened an account on the TAS forums... immediately got a response as to my fceux issue, which is always nice. 

So now I've got a way to implement save/load on fceux. 

I also made a script for snes9x that is more or less the gens script save some graphical changes.  There is an issue with snes9x though, namely you can't launch a lua script from the command line.  I might have to do a quick compile of the latest build an add command line support at a later date. 

I also looked into the "gd" image format.  It's all text based, so rather than require the gd library I could just write a quickie program that converts a standard image to a bit of code.  It might take a while for a program to parse an image, but it would be as simple as using the "getpixel" api on each pixel in the image and converting it to a format like so:

"\alpha\r\g\b" and adding it to the string along with a header specifiying the dimensions. 

[Howard_Casto]

Ok GensMenu 2.0.

Second verse, same as the first!  For use with GensRR 11b or higher.

Fixes:

Improved Visuals.
Fixed de-bounce, so now you won't accidentally blast through both menus.
Menu no longer disappears after a selection... you'll have to back out via the "A" button, which seems to make more sense.
Loading and Saving fully working with full error handling. 

Long story short, this version works perfectly.  All the versions for the other emulators will be based on this script.  Enjoy!

[Howard_Casto]

I'm on a roll!  Fceux is finished!  Any version that supports lua scripts should work. 

With this one install/syntax is a little different. 

fceux.exe -lua nesMenu.lua "c:\path\to\rom\rom.nes"

Also put the lua script in the same folder as the exe, NOT in the lua scripts folder. 


It looks a little different (nes color scheme) but it operates exactly like the Gens version.  Just press start to pause and hold select to bring up the menu.  If you have problems fceux not reading inputs, check the "background inputs" box in the emulator's options.

[Le Chuck]

Awesome, keep after it.  You're going to make my HTPC so much more user friendly!

[Howard_Casto]

Thanks, I'm trying. 

I looked at Mupen RR... is that a good emulator?  I always used project64 back in the day.  Anyway, the syntax seems straight forward, but the render loop now works in a different manner, so I can't just drop it in and tweak away like the others.  It would be straight forward to fix, but Mupen's documentation is in Japanese, so there's that. 

I'm not going to do anything newer than the psx/n64.  After that they figured out that it might be nice to be able to save games at will.  It only took 20 years. 

Something I haven't mentioned is that the TAS Video community has moved away from the RR builds and on to bizhawk, which is some sort of multi-console emulator.  If I write a script for it, it'll work on multiple systems.  That being said much like mess it's a jack of all trades, master of none.  Some features and emulation "thingies" that are standard on other emulators aren't in it yet.  So it made sense to me to work on more familiar emulators first. 

It should be a viable emulator for older systems like the 2600 and Master System though.

2600 should be interesting.  I would like to write a menu for it that brings up the dip menu and gives you access to reset and power.  The question is how would you bind a hotkey?  Hold the fire button for a really long time?

It would also be nice if I could somehow read the rom name.  If I could then I could take snapshots when the user saves and use those in the load/save menu instead of numbers.

*EDIT*

Oh btw I've got a working snes9x script as well.  The problem is the emulator doesn't support the loading of lua scripts via the command line?  The source is out there, I can fix it, but it'd be nice if somebody else helped me on this.  I don't have a compile environment setup and call me lazy, but that sort of thing takes a long time to configure. 

[Howard_Casto]

Ok so I was checking my email and ran across an article to read the state of the 360 guide button.  I thought that was pretty cool so adding guide button support to these scripts might be nice.  Honestly I should know this stuff... I had to learn how to use xinput for mamehooker, but people keep finding undocumented functions in the xinput dlls.

I can't (easily) do it directly.... the lua scripting reads virtual buttons on the gamepad, not unmapped joystick buttons.  Since this would be useful for other stuff, an app that allows you to bind the guide buttons to keyboard keys, combined with the "power off" app I linked to in another thread, would probably be the way to go. 

I CAN get the input state of all keyboard keys, so I could map the guide button, to, let's say the tab key and then write the scripts to also respond to tab being pressed. 

[wp34]

I just stumbled across this and am very interested.  I'm starting to look at a console emulator/HTPC and this looks like the cat's meow.

Thanks Howard.   

[Howard_Casto]

So I was perplexed by the lack of support on snes9x... looked into it.  Apparently there is a bsnes (superior) port out there with lua support.  The reason I didn't find it?  It's named lsnes???  Seriously every other "rr" build is either the name of the emulator +"rr" or the name of the emulator +"X".  Why lsnes?

Anyway, I'm going to try to make a script for that tonight.  I honestly would still rather use snes9x though.  While lsnes touts better emulation, man whoever designed the interface must have been an engineer.  That's all I'm gonna say. 

[Howard_Casto]

I mentioned earlier that the lua implementation doesn't natively support the importing of normal image formats and that I might write a "gd" converter.  Well I did that last night, tested it and it didn't work.  I took another look today... it turns out I forgot to set the read mode in lua to binary file.  So yeah with the help of my converter we should be able to skin the menus without the need for the gd library. 

I set a practical limit on the image dimensions to 255x255, namely because I can't figure out how to add large image dimensions to the gd header.  That sounds small, but keep in mind that we are limited to the screen resolution on these games, which is around 240p. 

I did a quick test below.  The cut-off point on the test image is 255 pixels to give you an idea.  That seems more than large enough and we can always tile images.  Before you ask, the magenta all around the image is just due to my sloppy image editing... it's a color key system (full transparent or full opaque) and lacks a true alpha value.

One oddity is that you are stuck with not only the game's native resolution, but it's bit depth as well.  Genesis and Snes are 16 bit systems so graphics look pretty much unmolested, but the NES... ugh... 8 bit graphics people.  We might need a pixel artist to do that one. 

[Howard_Casto]

I wanted to see what I could do graphically with the gdoverlay function so I whipped up a test menu for the genesis.  As you can see either the grid or that font has to go, but the point was to see if it looks stupid.  Looks pretty good regardless. 

Just for the record this massive menu is only 186x180, so yeah, I don't think the 255x255 resolution limitation I imposed is going to be an issue. 

[liquid8]

Great job, Howard. This will be another handy tool from you 

I haven't messed with lua much lately, but if you needed help with any testing or menu creation for other emus, I'd be willing to help out. I'm okay with graphics and have a lot of programming experience.

How limited is the lua for emu control - does it allow you to perform any functions or only ones that are exposed to you? For example, could you change emu-specific settings? Can you load roms on the fly from the menu?

How limited is the lua scripting in general and with image support without gd? Could you load boxart, history info, etc? Can you use other fonts? Is the 256x256 limit for the whole menu or per image?

The genesis menu looks pretty good with the system logo - anything with gridlines is probably not going to be great. I'd suggest going with a simple system theme like:

NES


I hadn't seen these before, but some interesting ideas and menu concepts:

http://devster.monkeeh.com/sega/radica/#whatisit


https://code.google.com/p/genplus-gx/

[Howard_Casto]

It varies from emulator to emulator.  Typically what we have control of is very limited.  Aside from what I've already exposed, you can load roms and sometimes apply game genie codes and that's about it. 

You can run external program via os.execute, but like with most lua implementations, the function works poorly, showing the command line, not retaining focus ect. 

In terms of the examples you posted fceux is probably going to get something basic like that due to the 8 bit limitation.  The gens plus menu is probably beyond what I can do... the resolution I have available is half that.  Ignoring the gd limitation, the genesis screen resolution is 320 x 223 and gens gx uses a 480p menu system.  That's the reason I cut the "sega" off of the genesis logo btw.... at that low a res it looks like a blurry mess. 

Image support without the gd library is non-existant.  The engines will only render/load images in gd format, so they have to be converted.  A manual conversion takes about 1 minute for a 255x255 image.  Now a couple of the emulators have an auxiliary image library built in, but it isn't universally supported. 

Fonts aren't supported at all.  Obviously you can make an gd-based font, but other than that you are stuck with the emulator's "system" font, which is what you see in the earlier screenshots.  It's very small and very ugly.   

The limit is 256 x256 per image.  Each image takes up around 256k of memory, which by modern standards is nothing, but it's something to keep in mind. 

[liquid8]

here's a couple quick mockups:





let me know if you like that.

Also, I took a look at the lua stuff, seems fairly simple. I was trying to play around with the emu.advanceframe and emu.pause - is it not possible to prevent input in the game while in the menu? I guess the lua is part of the emulation, so if you don't advanceframe or if you pause, the menu won't work...

Do you have a new lua you can provide where the graphic can be overlayed?

[Howard_Casto]

Yeah I was rather disappointed with that as well.  Emu.pause doesn't press the pause key, rather it halts emulation.  So your code loop halts as well and thus it's useless.  You can simulate a button press via joypad.set, but seeing as how you wouldn't know if the user already paused the game... again, a dead end.  Now if the state game pause is held in a universal position in memory, there are memory read/write functions available, so maybe it can be done that way... I dunno. 

That's why I binded the hotkey to start.... this way the user hopefully pauses the game as they bring up the menu. 

Both look good.  I think we can do better with the font, but other than that.   

The lua code in using for the graphics test is just that, it isn't hooked up yet and just displays an overlay.  I'll post my gd converter and test script this evening. 

Before we get all graphical I want to look into retrieving the rom name.  I think I have limited access to the winapi, I might be able to read the window caption that way and parse the rom name.  If I could then I could use the screenshot function to save an image each time a save state is created, which would be better really. 

[Howard_Casto]

Ok here is something to play with.  I've included my gdi converter.  It's pretty straight forward. Load a jpg bmp(preferred) or gif then click the save button.  The resulting gdi file can then be read into lua via a generic binary read.  Anything magenta will be made transparent.

I've also included the graphics test script I whipped up.  Keep in mind it just shows the graphic and nothing else.  An odd quirk about Gens is that the root directory as far as lua is concerned is the exe path, not the lua script path, so the gdi file needs to be with the exe.

[Howard_Casto]

Ok I looked into getting the rom name somehow.  Gens doesn't include winapi support... fceux does.  More inconsistencies (*sigh*).  So for gens we have to do it the hard way.  Luckily sega is so anal about labeling their roms.  @ 0x120 there are 48 bytes in each cart that is the rom name.  So problem solved on that one after a bit of clever memory reading and byte/string manipulation. 

I'll see if I can manage to make and reload a screenshot as a gd file. 

[Howard_Casto]

Ok time for another sigh. 

Gens is just being difficult.  Now that I have the rom name I can indeed save a snapshot to file.  Unfortunately the gdoverlay function in Gens doesn't allow you to scale images. 

So yeah, I guess I could do something fullscreen?

[liquid8]

OK, tried the img2gdi, but I'm getting an Runtime error '6' Overflow and just a blank image.gdi file. I tried in bmp, jpg and gif formats.

It looked like fceux has an option to take a screenshot without gd.. does gens not have that?

I don't know if a screenshot for the save state is really necessary - especially with limited real estate.. what about just storing the date and time of the save?

[Howard_Casto]

What are your image dimensions?  Remember 255x255 (one byte by one byte in the header) is the maximum size allowed. 

Nope, gens doesn't have any image libraries attached unfortunately. 

I know on modern consoles (Wii, Wii U, ect) a snapshot is always used to identify the save state and honestly it is a lot easier that way.  I can always draw on top of the snap, so perhaps a "Select a state" message along with up and down arrows indicating you can scroll through them. 

[Howard_Casto]

Ok so I had one of those "duh" moments after I took a break.  The game saved me a full res gdi, so why not look at it's header to determine the format?  It turns out what I thought were null-terminators weren't, so that was the issue. 

This new version will support any resolution, although obviously we are still limited by the game resolution.

I might look into doing a full screen menu with this new found freedom.  I wasn't going to attempt it if we had to deal with tiling. 

[liquid8]

Yep, my mistake - my psd still had the dimension of the system resolution

Playing around a bit.. here's the result



I'll post more tomorrow...

[Howard_Casto]

Looks good man. 

I figured out something as well via a net search.  You can do a "fake" pause which will work for our purposes.  What you have to do is put the frameadvance in a a if statement and call a manual redraw and wait command whenever you want to fake a pause. 

So emu.frameadvance()

becomes:

if MenuDis==0 or MenuDis==nil then
   emu.frameadvance();
else
   emu.redraw();
   emu.wait();
end;

Basically we check to see if the menu is on, if it is, we pause. 

Now this brings up three additional issues.  First off the game renders unthrottled, so the dbounce variable I have towards the top has to be changed to a crazy high number like 40.  Secondly you can't read the gamepads with joypad.get... use joypad.peek instead.  Lastly the sound buffer screws up when you halt the screen, so call a sound.silence() when the menu first pops up. 

So yeah, we are getting somewhere now. 

[liquid8]

bah.. here's a fully working version   

https://www.dropbox.com/s/qop66ag80adyyrc/gens-menu.rar?dl=1

I did modify some of your variables in that though 

[Howard_Casto]

I took a look at some old genesis boxes.  I'm wanting to base the menus off of the consoles artwork.  The aesthetic was very similar to the virtual console on the Wii, so I mocked this up.

I figure I can have the buttons float in the center like wii menus do, only make them rounded black buttons like those on the Genesis rev 2. 

I want to keep working on the script and make the whole thing fully skinnable via swapping out a few variables at the top. 

I know it's ridiculously big btw, but I started thinking that in a htpc/emu box situation you'll be sitting far away from the tv, so maybe a small menu isn't the best option.

[Le Chuck]

Keep it up, that looks great

[liquid8]

I took a look at some old genesis boxes.  I'm wanting to them the menus off of the consoles artwork.  The aesthetic was very similar to the virtual console on the Wii, so I mocked this up.

I figure I can have the buttons float in the center like wii menus do, only make them rounded black buttons like those on the Genesis rev 2. 

I want to keep working on the script and make the whole thing fully skinnable via swapping out a few variables at the top. 

I know it's ridiculously big btw, but I started thinking that in a htpc/emu box situation you'll be sitting far away from the tv, so maybe a small menu isn't the best option.

Looks good.. yeah fullscreen menu would be nice - unless you aren't able to make the "pause" happen, then i'd keep it small so you can still see what is going on in the background.

Is it possible to use external fonts or at least modify the font size? I couldn't figure that one out.

Would it be too taxing to load up multiple images and overlay them on top of one another?

[Howard_Casto]

Pause works well.  Now the only thing that makes me worry is the part about the game running unthrottled.  People might have to adjust the dBounce variable based on their own computer's speed, so I'll put it at the very top of the script. 

Well there's no reason to use text really.  We have 4 buttons and that's it.  So just make an image. 

Now if we start having some reason to draw text on the fly, then we can look into a gdi-based font or something. 

[Howard_Casto]

Alright, I've got the first menu scripted with graphics anyway.

Yeah those buttons are too big.  That's my problem. I'm working in 640x480 resolution and then shrinking stuff down when I'm finished, but I forgot that the Genesis resolution is 320x224, not 320x240, so they turned out way too tall.  I'll go back and fix it later. 

I'm not super fond of the red as a highlight, I might rework that to make the selected button have full on white text and a white border with unselected having a black border. 

Anyway, all the stuff I've added thus far is fully skinnable.  At the top of the script is a X position, Y position, Alpha Value and URL for both an "On" image and a "Off" image.  To make a new skin you would simply run your individual images through the gdi converter and change the variables to your liking. 

Here's how I'm going to do save states, for Gens at least... Snaps with be optional and have a Boolean value at the top of the script to turn them off/on.  The select a slot menu will be similar to how it is now, and if snaps are enabled they will be displayed behind the menu where the game normally shows through as you scroll down the list.  You should be able to see enough of the image to tell which save it is and this way it doesn't look out of place.  When the image is missing a default image will be shown.  If you use the lua script exclusively for saving, this will allow you to tell which slots are already used when saving, which was kind of the point. 

Anyway I'll work on all of that later tonight.