Software Support > controls.dat
Would anyone like to take over?
drventure:
Ok. Just had a thought. My brother just pointed out that you can make URL calls into a publicly shared Google Docs spreadsheet, to retrieve specific cells, or entire sheets
You can play with the concept here
http://ouseful.open.ac.uk/datastore/gspreadsheetdb2.php
so, take the XML, reformat it a little into a spreadsheet, share it (maybe even editable to the right people). then putting a website together to display or reformat it it would be pretty trivial.
Maybe it's too big for a google spreadsheet, but probably not.
Anyway, thought it was an interesting possibility.
Howard_Casto:
Depends... what's the maximum file size of a google spreadsheet?
Since gambling games and pinball have ben added to MAME, we are in 5 digits in terms of the total games in MAME. Obviously we'll never hit 100% but any format chosen needs to be able to handle the full mame set with room to grow on. It's one of the reasons I've always resisted xml btw.... there's so much darn white space in a xml file that the one mame currently generates can be sluggish to parse depending upon the library you use to parse it (if any).
So for me at least, the database needs to be just that... a database and nothing more. I'm talking comma-delimited text file simple. We just need to make the interface capable of outputting it into more user-friendly data. (xls, xml, ini whatever..)
Somewhat releated in terms of approach... are you guys aware of who they do things at the wii hacking scene for the fes there?
http://www.gametdb.com/
It's a very elegant but simple solution for organizing all the data pertaining to a game.
Here's an example:
http://www.gametdb.com/Wii/SU3PMR
The SU3PMR part of the url refers to the game's internal id code (sort of like a mame rom name). The images shown on the webpage, which fes use for cover art, are always found in the "cover" "coverfull" "cover3d" and "disc" folders of the same url. If we could find somebody willing to actually host a repository like this for mame with the same type of url schema a different type of approach could be taken and each game could get it's own individual file ect... But I dunno, mame is so popular a site like that would get hammered.
It's just there are so many errant support dats/files/images for mame out there it sure would be nice if they all adhered to the same formatting for easy parsing.... just a thought.
EightBySix:
Amazons AWS free tier now includes database support. Even when he free year runs out, it's often cheaper than web hosting. Just saying.
edekoning:
I just found this thread through some random search, and a I realize its months old, but I still want to react to a few statements made.
--- Quote from: SirPoonga on October 11, 2012, 02:16:56 pm ---So here's what I am thinking. A new format is probably needed. I think we go strictly XML. INI is old school, but was used because back in the day people were still using Win98 on their cabinets.
--- End quote ---
I never really understand these arguments. Just because something is old (but most likely tried and tested) does not mean it should not be used. And just because everybody is doing XML does not mean you should.
--- Quote from: Howard_Casto on September 02, 2012, 05:39:36 pm ---I personally never liked the xml version just because xml is less readable to human eyes. That being said I eventually dumped support for the ini because it turns out that the windows api calls to read/write inis have a 64kb limit. Now for any sensible db this wouldn't be much of an issue, but mame has an insaine amount of games and thus I hit that limit pretty fast. Seeing as how a person then had to parse the file manually, sans any helper functions, it made much less sense to use the format.
--- End quote ---
I also find XML to be not that human readable/editable. Never understood why everybody likes it so much. I don't really see a problem in the XML file (either mame or controls) being big. Just use a SAX parser, and store it compressed.
Regarding the INI files, why use windows api calls at all? I mean an INI file is just a very simple text file. As long as you keep it simple (no hierarchies, no escape characters, ...) then there is no real reason to use any library at all. Each section represents a game and is followed by a number of key=value pairs. Just process the file line by line.
I'm not sure what you are looking for in a file format:
-must it be human readable?
-must it be a small as possible?
-must it be easy to read/write/edit
-can it be in binary?
Human readable usually means big files, because of all the context that's needed. Note that I don't consider a file, just because its not binary, human readable. Binary files are usually tiny, but require dedicated tools to edit.
Have you ever really considered a binary format? You could define one that's much easier to parse than any text format could ever be. I hope you know a little C/C++ otherwise this next bit wont make any sense. If you define a POD-struct that can only contain fixed width strings, and that explicitly contains all the needed padding bytes. Than you could just read x-bytes from a files and do a direct cast to the game struct type. So parsing would the data for a single game would require one line of code. An example from the top of my head:
struct Game
{
char [8] name;
uint8_t numPlayers;
uint8_t alternating;
uint8_t mirrored;
uint8_t usesService;
uint8_t tilt;
uint8_t cocktail;
char [2] pad2; // explicit padding bytes
char [64] description;
...
};
// reading
char* data; // assume game data read from a file
Game game = *reinterpret_cast <game*> (data); // done parsing
// writing
data = reinterpret_cast <char*> (&game); // done parsing
write (file, data, sizeof (Game));
If you decide on some format that is not standard (CSV, INI, XML, ...) then you should provide a detailed description as well as some sample code to read/write such a file. For my example you would provide a header file containing the structures, and a source file that contains functions to read/write a file. It would also be nice if the same was supplied for multiple languages, e.g.: python.
Howard_Casto:
I can answer a few of your questions.
Controls.dat already does xml, half of the viewers use the xml file, the other half use the ini file.
A big xml file means it takes longer to parse... we aren't talking harddrive space, but rather we are talking processing time and memory. Also Bandwidth... it will be downloaded by users you know. ;)
The windows api calls are amazingly fast for getting data. Unless you are really good at making serach routines, anything done manually would be considerably slower. Searching line by line sounds like a good idea... until you realize were are in the 5 digit area for the number of lines. That being said, I think the 64kb limit only applied to win 98, so it may be a non issue at this point. I'd have to check my code.
Binary wouldn't really save any space or time. You'd still need the bare minimum in terms of data logging, like a section name, the key and the value. So two characters for the brackets normally gets reduced to 1 for the start of a new section and a "=" plus a line return, which is three characters gets reduced to two.... not really much of a savings.
The dat file doesn't have to be humanly parsable, but it has to be easy for novice programmers to parse. Most of the people in this hobby aren't professional programmers and even as simple as the dat file is now, there are only three viewers. That's one of the reasons I always resist xml btw.... yeah it's easy enough to use a plug-in parser, but they are generally slow and unoptimized for the massive xml that is related to mame, so the apps built with them chug along. As PCs have gotten faster it's much less of an issue, but still.... it's bothersome.
I'm not saying definatively one way or another is the correct way to go, I'm just sharing my thoughts.
I would really like to get this thing going again, but controls.dat is a MASSIVE undertaking in terms of managment. Nobody seems to want to do the web side of things, so we are kind of stuck.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version