Open-source front end

[Dave Dribin]

One word: XML. Anyone wanna take a stab at coming up with a schema for game definitions?

If your doing this just for MAME, take a look at the output from the "-listinfo" option of mame.exe before writing your own XML schema.  It gives lots of information and its fairly easy to write a parser for it.  It may be all you need.

If you're doing this for games other than MAME (i.e console games), I could see some possibilities for XML.

-Dave

[)p(]

I am all for it but make it as general as possible at least arcade, pinball and console games should be supported...

lets see...this is what I would really like to see...a database like app that can do
-records with a large number of columns where the labels can be set to anything.
-flexible selecting of records to be send to the frontend
-searching directorys for available games and check the crc's
-searching directorys for available games and add them to the database (mostly for console games)
-lots more...


peter

[Howard_Casto]

As usual guys I'm a few steps ahead of you.    After talking with lilwolf a bit I've decided it might be a good idea to "de-intergrate" cetain elements of rd.  I've been thinking about it and the biggest hassle of any fe is searching for the roms and giving the zip files "real" names.  So keeping that in mind I thought a good project for myself would to make a "scanner" app that simply searches paths you've chosen, and compares files in said paths to a rather generic, but very versitile database.  
The database fields would be something like this:

filename|extension|Real Name|Clone|Emulator(s)|path|misc|

I haven't even started yet, so if anyone thinks they can do it in xml or whatever, that'd be great.  We need to make it real simple though, like you send it a lists of paths, and it spits out an array containing the above fields all alpahbetized and ready to go.  Since I seem to be the author that likes to support the "odd ball" emus, I'll be glad to fill in the gaps for emus like modeler and imapct, that don't have the built in information systems that mame does.  

A note to Dave, you've seemed to take my comments personally so I won't comment anymore, I was just trying to explain what you would have to do to get people to contribute to your project.  I'm sorry I won't say anymore.  

[Howard_Casto]

I really DISLIKE the idea of having just a databse and thats' it.  It's simple enough gathering all of this info, but the real code comes into play by actually sorting this info and storing it into variables.  I agree with the previous posts in that there needs to be some type of ocx plug-in developed as well.  

on more category as well.... |category|  

he he, how could i forget that one.

[SirPoonga]

The database fields would be something like this:

filename|extension|Real Name|Clone|Emulator(s)|path|misc|

Ewww, ewww, ewww.  Bad database design.  Ewww, ewww, ewww.  Not take database modelling in college yet?
hehe, DB class in college, using DB2 at IBM, and using SQL Server at my last job taught me alot:)

hehe.  That actually is a bad db design.  Everything in one table as complex as having a generic DB that covers emulators in general.  I could come up with a pretty generic design (with the help of y'all).

Actually, if the db was a flat file than yeah, that'd probably be ok.  But extremely inefficient (especially if a mass change takes place).


now, for the rest of you.
NOOOO, not an ocx.  That's not portable:)
It's have to be an API.  Generically it would be compiled as commandline executable.  Otherwise it'd be a set of C++ classes with db ties and one could make a linux .so lib, a windows .dll, or whatever a mac has and include it in their FE project to make their own gui.  Anyway, It's have to be a library with a simple API.


My current FE creates the DB from scratch if it doesn't exist.  Write now it uses Access with the JET driver.  But all i need to do is change the connection string to use it for SQLServer, MySQL, DB2, etc....

[Lilwolf]

Like I said a few messages back, I'm all for it.

[Lilwolf]

I actually love the idea.  If we had a standard schema for the games, then we could work on writing a few programs for propigating the data and have it work

like this (quick, dirty, not thought out).

database: the game
name, description, system, categories, funstuff.  IE whats to be displayed

system: the system
name, directory, program name, whats needed to start, whats needed to exit.
ie

mame, c:\emul\mame, mame.exe, -resolution 640x480, 'esc key character'

s11, c:\emul\s11, s11emulationstart.exe, -startgame %n (where %n would be replaced with the rom name), 'f10'

The advantage is that we could have someone write a program to parse the roms for S11 and fill in the systems database, and your emulator would all of a sudden support it!  

propigating the database (in my mind) shouldn't be done without a keyboard anyway, and should probably be in a different application (or like I do, have special key configurations to do it)

I really DISLIKE the idea of having just a databse and thats' it.

[Howard_Casto]

s11, c:\emul\s11, s11emulationstart.exe, -startgame %n (where %n would be replaced with the rom name), 'f10'

The advantage is that we could have someone write a program to parse the roms for S11 and fill in the systems database, and your emulator would all of a sudden support it!  

See I realize that you other programmers haven't been as concerned with supporting other emu's, but I have, and there's a fatal flaw in your plan......

Your example for instance......  it wont' work!  S11 uses a numerical scheme to launch games, so you have to have a database of all the rom names within the emulator and then have a case statment which compares the game with your table and laucnhes s11 with the appropriate number as an extension.  

Ok how about impact/modeler?  wont work!  You have to  simulate keypresses to launch a game since theres no command-line options...... you could tell it which keys it has to pass, but then we would have to translate those keys to their appropriate programming language once we get them... (vb uses keycodes as well as ascii, and keycodes have slightly better reponse time.)  U64/u642..... same deal, you could only pass the appropriate keypress sicne there's no command line argument.....

Now daphne and raine would work, but they use standard mame syntax anyway, so they would work on any fe with practially no extra code as it is.

I'm not dismissing your idea, but merely showing that in order to do this right an ocx would be necessary to handle such things...  if it does it for you then all of this converiting wouldn't be necessary, and it would be the only true way to have a virtually codeless solution that would be universal as well.

[ArcadeFX]

With the new version of ArcadeFX that went live today I totally converted the backend to work off a database.  This was a huge step forward for AFX and I am glad I took the time to do it.  I wish I would of done it from the start.  It would of saved me a ton to code changes.

About using XML, I think this is a great idea. Authorware now has an XML Parsing Xtra that I have been wanting to use.  If there is any plan to more forward with this I am in. I have done a little with XML but not enought to jump in and add this to AFX yet.

If someone can give the advantages of using XML I would like to see them.

[Mike]

Ed if you keep the database backend you should switch to MSDE on next go around. It's alot more powerful. It's basically ms sql scaled down. It can hold 2 gb of data. It's also free to distribute if you build an application on it. It can do most things sql server can do, stored procedures and triggers. Also it can hold images and files. It think there is a lot you could do with it as opposed to access ie: stick all your marquees, snips and cabinet pics in one place. Even though it's microsoft it might not be bad to use for the opensource project either. I'm all for xml the only problem I see with it is there really is no benefit to xml in itself. The only advantage you have is you can borrow someone elses interpretor for xml and modify it for our needs. Then it would probably use less resources than a true database engine. But on the downside you'd have to modify or create the interpretor where as with a database engine you just have to install it.

[flemming]

I'm pretty new to XML as well, so perhaps I'm making poor assumptions from the start but I spent a little time thinking about this today and trying to put together a basic tree for what a game might look like.  Most of this info is from listinfo plus catver.ini, but it would of course be a start.  Those of us interested in taking this further should probably do so offline, since this board isn't particularly efficient.  But here's what I was thinking:

<game name="pacman">
<description>Pac-Man (Midway)</description>
<year>1980</year>
<manufacturer>[Namco] (Midway license)</manufacturer>
<category>Maze</category>
<cloneof>puckman</cloneof>
<romof>puckman</romof>
<emulator name="mame">
 <version>0.02</version>
 <driver>
  <status>good</status>
  <color>good</color>
  <sound>good</sound>
 </driver>
</emulator>
<history>
 "Designed by Toru Iwantani \nProgrammed by Hideyuki ..."
</history>
</game>

Of course the message board eats the tabs, but you get the idea.  You could easily get carried away and store the rom file info, crcs, etc but mind you a dump of listinfo with all that information including the history is 9MB so this file would probably end up being larger than that.  I also know ArcadeFX stores a counter and such for however many times the rom is ran, but I think changing data like that is probably better stored elsewhere since in the future you'd probably want to be able to drop in a new version of this file and losing that data would suck.  I've only thought of this from my standpoint of what I'd like to see in the file.  I'm sure others have different ideas and If we want to find some common ground this might be a good place to start.  Of course in theory XML might be a good idea, but in practice it might be PAINFULLY slow, so that's of course something which will have to be weighed.  If someone wants to setup a mailing list or something for us to talk about this further that would be great, if not I can do it as well.

Robert

[Dave Dribin]

Am I alone in thinking that an SQL RDBMS is a tad overkill for a front-end?  I don't want to require my users to install an RDBMS just to run it.  There are in-process databases such as Berkeley DB and GNU dbm that are *much* lighter weight.

http://www.sleepycat.com/

-Dave

[Mike]

i agree i was just suggesting it over access. Because at least you can count on your data being there and most people are famaliar with microsoft products. Maybe I'm delusional since i've been at work for 15 hours staring at poorly written VB code and I can't go home for at least another 6 hours. I should have kept my low paying job doing java development. But dang it I wanted a mame machine so I needed the big bucks microsoft programming pays.  

[flemming]

Am I alone in thinking that an SQL RDBMS is a tad overkill for a front-end?

[Howard_Casto]

Guys perhaps were going about this the wrong way.... I'm a bit confused as to what you guys are suggesting now.  A database should never be manipulated while a fe is running, for speed and compatability issues.  I assumed taht you all realized that a text or binary output of a pre written text file was the way to go.  Trust me on this one... rd alpha had a database (access in the release, but i tried a few)  and it slowed everything down and actually required more code to navigate through it than if i would have hardcoded the searches and read from a static text file...

just something to think about...

(btw im in no way endorizing access, it's just it was there and quick to do.)

[Dave Dribin]

I should have kept my low paying job doing java development. But dang it I wanted a mame machine so I needed the big bucks microsoft programming pays.  

Hrm... you must have got a bum Java job!  ;-) I've been doing Java on Unix for the last 2 years and the pay is nothing to sneeze at.

-Dave, just trying to point out that you can get paid well w/o Microsoft products

[Howard_Casto]

my bad... i mis-read the last post or two.. if it ws managed form within an ocx then you would get acceptable speed

[Dave Dribin]

my bad... i mis-read the last post or two.. if it ws managed form within an ocx then you would get acceptable speed

Ok, this is my Windows ignorance showing through, but how does OCX relieve the overhead of an RDBMS?  Does it run in a separate thread or process?  Keep in mind that I barely know what OCX means.

-Dave

[Mike]

Got a question since I never used Berkley db. Connection wise what do you use to connect from to it from an application? And if anything what would have to be installed on a users computer? Also if you can connect to it say through odbc, how effecient is the driver? The reason I ask is because currently one of the databases I use is progress. Which is a wonderful database when using the progress language. But it moves like a slug on a salt lick when you try to connect to it from anything else. I disagree with howard though on even access being to slow. I don't know how you had your database set up or how you were connected to it but I can't imagine how a text file could be faster. Unless your loading the whole file into memory and performing everything from there. The reason I like the database Idea is for example on your screen when your displaying roms all you have to do is select the rom name from the database and show it. Then get the images and info for the rom that is selected not all roms so you keep the memory usage down. you could do a prefetch for rom info and images on ones above and below and a page down or up if you were having performance with them displaying fast enough. Thats just my thought.

[Dave Dribin]

Plus perhaps it's just the Linux bigot coming out of me but last time I checked MAME ran on a TON of platforms and OSes so I don't really see the point in talking about working together and sharing certain backend functionality and then bringing up MS SQL and OCX files  Berkeley DB does work on a variety of platforms so I think it would also be a valid option should the XML pipedream get tossed aside.

I agree... a common backend that only runs under Windows is useless to me.  I'm targetting my front-end for DOS, Windows, and Linux, so a backend would have to be portable to these platforms.

Maybe it could be broken out into two components, an API and a backend.  The backend could be pluggable with SQL, Berekeley DB, and GNU dbm plugins, for example.

-Dave

[Mike]

ocx doesn't relieve rdbms overhead. It is a seperate component or control that could take care of the database functions. So the program itself wouldn't be burdened with them. I think thats what howard was getting at. He was also referring to using them with access.

[ArcadeFX]

I have everything you suggested for the XML already in my Access database. Fully populated.   It works like a charm and is lightening fast.  I do queries to the database the entire time the use is in ArcadeFX now.   Almost every action you do in AFX now is pulling data from the Access database.  It is head and shoulders faster than accessing any large file like listinfo or catver.ini or any other mame generated file.  I do not store any images in the database at all.  I see no reason to do this.  All image are taken from there respective MAME folder based on data returned form the database.

In AW database access on the fly from a db is faster that I could have imagined.  Mush faster than accessign any of the MAME generated files for sure.

Here is what my database contains and is fully prepopulated when the user downloads it with the AFX files.

gamename, gamedescription, gameyear, gamemanufacturer, gamecloneof, gameromof, gamecategory, gameversion, videoscreen, videoorientation, videox, videoy, videocolors, videofreq, soundchannels, inputplayers, inputcontrol, inputbuttons, inputcoins, driverstatus, drivercolor, driversound, drivercolordeep, available, favorite, favorite2, favorite3, favorite4, favorite5, counter

The last seven are data that is written from AFX for each user.

Any database I have ever worked with is faster than any txt file could every be. This includes PHP/MSLQ, Oracle, Sql Server, Access. I guess it all depends on how you code the front end.

This is only limitation in AW that I have had to work around and that is a 30k limit on variables.  This is a pretty easy thing to work around.

[Dave Dribin]

Got a question since I never used Berkley db. Connection wise what do you use to connect from to it from an application? And if anything what would have to be installed on a users computer?

There is no connection.  It's just a library or DLL you link in with your program.  The library accesses DBM files on disk directly.  These files are in a B-tree format or something.  If you statically link it in, then nothing is needed on the users' computer.  If you go the DLL or shared library route, the user would need the DLL installed somwhere.  

Granted, Berkeley DB cannot do much of what an SQL database does.  There's no concept of tables, joins, transactions or anything like that.  But it does provide you with very fast O(1) lookups on key/value data sets.  I also think it is very memory effecient since it does not need to read in the whole DBM file just to access a few records.

-Dave

[Mike]

One more thing I see no point what so ever in creating anything for dos. I can understand linux, windows and even mac. But dos is a beast in it's self. There is no reason to write anything your going to have to dumb down to be compatible with an out of date OS. We should start it moving ahead with mame where it's going. Not what it's moving away from.

[Dave Dribin]

One more thing I see no point what so ever in creating anything for dos. I can understand linux, windows and even mac. But dos is a beast in it's self. There is no reason to write anything your going to have to dumb down to be compatible with an out of date OS. We should start it moving ahead with mame where it's going. Not what it's moving away from.

While I agree that DOS is a minority (and dying) platform, it is not dead for MAME yet.  There are many people who still use DOS for arcade cabinets.  I'm gonna cling on for another 6 months to a year.

Besides, I see no reason why DOS *can't* be a target.  With the DJGPP environment for DOS, programming for DOS isn't as bad as you might think.  It's almost like programming for Unix as many of the Unix APIs and libraries are available.  Hell, if they can port GCC 3.0.4 and Perl 5.6.1 to DOS, I *think* a  front-end could run under DOS! ;-)

-Dave

[ArcadeFX]

DOS is dead as far as I am concerned.  If you want to make something that looks good then DOS is out.

[Mike]

Hrm... you must have got a bum Java job!

[ArcadeFX]

You have to remember that people have to download this and set it up on their PC.  They are not going to buy anything Oracle to run a front end.  Oracle would give you all the power you need but useless for what we need to do with a front end.

Access is the best option for Window by far.  Relativly small download, free, and more powerful and txt files.

I personally think trying to develop a FE for every platform (DOS, Win, Mac, Linux) will limit you to much.  If it is open then let people make their own port to their desired OS.  And work with them to do it.   Otherwise anything that is done is going to look and function like it did before P-diddy, Howard, and myself started our FEs. BORING!  Yea, MAME32 works great but it looks like every other VB/C++ app. on the net.  That is the reason that I went with Authorware.  AW is not the solution for what an open project like what is being talked about here but is my creation of the perfect front end.  Perfect to me.    

How many other apps can you think of are cross platform like what some of you are suggesting here? I can't think of any off the top of my head.

[Dave Dribin]

But I guess i am also the sql and oracle dba so that may have something to do with my pay. Also may be why I like rdbms so much, hey we could use oracle 9i personal edition. It should run on everything.  

Oh, don't get me wrong.  I have nothing against RDBMSs.  I'm partial to PostgreSQL or MySQL, myself.  Oracle is a fine DB if you need all of its features and you can afford the price.  But why spend thousands of dollars for something you can  get for free?

-Dave

[Mike]

i was kidding on using oracle but personal edition is free if you ever want to download it.

[flemming]

How many other apps can you think of are cross platform like what some of you are suggesting here? I can't think of any off the top of my head.

Well there is MAME itself now isn't there?  BUT, I don't think that is what we were talking about (at least I wasn't).   He topic shifted from opensource FEs to playing nice with each other.  Along with the idea of playing nice with each other came the ideas of a universal database that various FEs could use and even a common library that could be written for extracting and sorting info from that database.  Along the way that got hijacked into RDBMSes and OCX objects, neither of which really go along with the idea of a universal database or common libraries.  Obviously those of use that choose not to use Windows are in the minority, but I'd still like to participate in the reindeer games and if one way to do that is to work with other FE authors on a standard way of doing a few things, I'm sure that will save both myself and other developers time.  Of course the additional benefit being that end users could switch FEs and have some data remain constant between them.

Robert

[Dave Dribin]

How many other apps can you think of are cross platform like what some of you are suggesting here? I can't think of any off the top of my head.

Ugh... MAME!    And even if DOS is dropped out of the equation, many games like Unreal Tournament, Quake 3 and Myth 2 all run under Linux, Windows, and Mac.  UT and Q3 run under all sorts of consoles, too.  And they're not boring.  Well, ok, maybe Q3 was boring, but that's beside the point....

That said, I don't think one uber front-end will ever make everyone happy, anyways.  There's way too many opinions on what makes a good FE.  But that won't stop me from making an open source, cross platform FE that is functional and looks better than a standard Windows program.  

-Dave

[Mike]

You should make emuix. Strip the linux kernel so it just runs emulators and fires off a clean customizable GUI to launch the emulators from nothing but the keyboard controls. That would be ideal for all. Lets see that penguin power.

[flemming]

You should make emuix. Strip the linux kernel so it just runs emulators and fires off a clean customizable GUI to launch the emulators from nothing but the keyboard controls. That would be ideal for all. Lets see that penguin power.

Uh, what's what I'm doing.  Except it's not as black magicy as it seems you'd like to make it sound.  MAME can be build using SDL which can use the frame buffer so X is not needed.  Pygame the library I'm using to write my front end also uses SDL, thus the same applies.  So add the Linux Progress Patch to the kernel with some cheezy graphic and you have a machine that boots up immediatley into a graphical screen while the OS loads then fires off the frontend.  It's pretty brain dead to strip down what gets loaded on startup and build a kernel with only the bare minimum so that your overhead is reduced as much as possible.

Robert

[ArcadeFX]

Okay you guys got me with MAME being cross platform but that doesn't have a univeral frontend that works on all platforms. That is what I was getting at.

I also use Red Hat at home on my cable box so I would be interested in anything you do on Linux.  The perfect MAME OS would really be Linux for the simple reasons you mentioned. You can totally customize the OS for the cabinet.  

Only problem is now how do I play NFL Blitz and Castle Worf?  I can't live without those.

[Lilwolf]

One more thing about databases and XML.

If we make generic frontend tools, we sould write an API to get the data somehow.  Who cares how!

Why?  Database are better!  They really are.  XML parsers usually require you to load everything in memory.  Thats great! But that was my number 1 complaint in my first version that I took to much resources on low end systems.  

So why not use both?  

Have a database plugin and a xml plugin going throught the same API?  That would be easy and we could easily get them writting and working great!  I also working on (hopefully soon) getting mame so we can ask it for the info ingame.  So the only thing I would be interested in would be the name of the game, all the rest would come from mame itself (and I would probably do that from mame directly also).  But I could incorporate that into the same api.  

Last,  Howard, about the trouble with my database stuff.  I have most of that covered.  

first, each game can add to thier own command line.  So for S11 I add a

-listinfo 1

for the first game in the argument location.  But I also have a -argument in the system.  That can take (currently) a %n and have that replaced with the rom name.

so I can have
-game %n.zip

and it can combine together also.

-game %n.zip -gameargumentA

This will run almost everything.  But I can't simulate keystrokes *** until you open that part of your code as a middlewhere piece of course ***

But that can be added

[SirPoonga]

Am I alone in thinking that an SQL RDBMS is a tad overkill for a front-end?

[Lilwolf]

2) Recently played games.

[Lilwolf]

Ok... we are talking about implimentations...  We should be talking about requirements/functions

1)  Crossplatform?  
 yes) This will force it to be java or c basically.  We could go pascal, but that would kill a bunch of stuff.  They both have their advantages.
 no) This allows us to use DLL and activeX components for the parts.  

2)  Moduler?   IE, do you have to compile to get the different parts together?
 yes) Then java is still a great platform.  But most others there is a level of hacking to get it to work.  
 
3)  API's for what?  
 Databases/XML/Whatever
 Emulator starting?  (ie, module for mame, impact, ect)
 Screen setup?  or standard skinner with modules to add?
 Keyboard/Encoder hooks?
 In-frontend configuration?

4)  How about database configuration?  Does this happen from a seperate application?  

IE, Before we argue about how to do the parts, we should decide on how to create the parts, how to connect them, then let different people work on the different parts (and see if they work in the end

[Lilwolf]

btw, if we only care about the .net framework (not sure the extent to that....)

the c# could be the language (very very similar to java) and there is hooks to convert VB and ActiveX component dll's to the .net framework.

I ahven't tried it, but there are a bunch of examples around.

Also, all the tools are free I think