Main Restorations Software Audio/Jukebox/MP3 Everything Else Buy/Sell/Trade
Project Announcements Monitor/Video GroovyMAME Merit/JVL Touchscreen Meet Up Retail Vendors
Driving & Racing Woodworking Software Support Forums Consoles Project Arcade Reviews
Automated Projects Artwork Frontend Support Forums Pinball Forum Discussion Old Boards
Raspberry Pi & Dev Board controls.dat Linux Miscellaneous Arcade Wiki Discussion Old Archives
Lightguns Arcade1Up Try the site in https mode Site News

Unread posts | New Replies | Recent posts | Rules | Chatroom | Wiki | File Repository | RSS | Submit news

  

Author Topic: Would anyone like to take over?  (Read 34799 times)

0 Members and 1 Guest are viewing this topic.

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Would anyone like to take over?
« on: August 27, 2012, 12:35:55 pm »
Other priorities in life are above this project.  I would love to continue to work on it however I think I need to pass the torch to keep it going.  I will still help out when I can.

So here's the deal.  I think the project's site needs a complete overhaul and brought up to date on technology.  It is currently written in old php which is probably not secure.  The database design is horrible.  I started this project 10 years ago when I just got out of college.  Now when I look at it I see my inexperience.

So I would suggest if someone wants to take it over to start designing a website for it.  Currently all the data is in a MySQL database.  I can help convert the data to a different schema or format.  I could see a complete overhaul of the output file to xml and json with an iphone and android app that works with your cabinet.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #1 on: August 31, 2012, 08:36:09 am »
I've been wondering where you wandered off to.

I was thinking about the project today and I thought that maybe a different approach is in order.  We tried to do it all on the web and in a perfect world, that is the best method.  The problem was it never worked out that way.  Anytime mame did a major output change (which was, and still is a farily common occurance) you ended up having to muck about in the database and manually change stuff.  Users had to wait anytime you modified anything and after you did get the db in line you usually had to modify the xml output and the webpage's wizard interface.  It was doable, but a time-consuming process, and everything had to be put on hold anytime major changes were made. 

On the other hand headkaze and a few others made offline apps.  I don't necessarily think this is the way to go either.  Offline apps lack a centralized database and version incompatabilites as well as lack of user proof, could cause a problem. 

I do think that a hybrid of the two methods might work.

The centralized database, the official version of the dat, and all the output reports would still remain on the site.  Instead of a wizard system there would be an offline app you download.  The app would require a web-connection anyway, and make submissions very similar to how we used to do it.  The only difference is, when something changes all of the wizard stuff as well as control types, ect would be on the offline app, so it would be much quicker to fix.  The only thing the site would recieve would be a pre-formatted DB entry, which could be read via the admin's offline app and approved, just like the old system. 

If the server-side was setup properly, the site should be much more secure.  Normal users would only have read acess to all sections except the submissions ftp, which could be completely cut-off from the rest of the site.

Generation of the xml would also be easier.  That could be done via the admin's app and uploaded via ftp, so a server side solution to generating those files would no longer be needed.  The site could just access the unmolested db directly for reports, which hopefully would make things easier.

But I'm just throwing ideas at the wall here.  I'll still be an admin, I can offer to redesign the xml format and handle mame input changes, and if an offline app is needed I can write it and maintain it.  Online databases are NOT my thing though, and I wouldn't be comfortable doing the server end. 

drventure

  • Trade Count: (+2)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 4152
  • Last login:June 25, 2023, 02:23:57 pm
  • Laser Death Ray Bargain Bin! Make me an offer!
Re: Would anyone like to take over?
« Reply #2 on: August 31, 2012, 11:22:03 am »
I'd love to throw in, but I'm a bit swamped with my own things right now too.

One possible solution I'd throw out.

Would it be feasible/dump/insane to do something like...
1) base the INI on the XML(or vice versa) and generate one from the other
2) then, put the INI or XML file into a public version control system like CodePlex or Github.

Then, different versions could be easily tracked via the normal versioning systems inherit to VCS's and any updates/user mods could be vectored through any of several moderators via pull requests (ie uploads of changes made by a user that aren't integrated into that version until a mod decides it's ok).

Granted, contributing might be a little more difficult (maybe too difficult?) but it might help take some of the burden off the moderators.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #3 on: September 02, 2012, 05:39:36 pm »
I'd love to throw in, but I'm a bit swamped with my own things right now too.

One possible solution I'd throw out.

Would it be feasible/dump/insane to do something like...
1) base the INI on the XML(or vice versa) and generate one from the other
2) then, put the INI or XML file into a public version control system like CodePlex or Github.

Then, different versions could be easily tracked via the normal versioning systems inherit to VCS's and any updates/user mods could be vectored through any of several moderators via pull requests (ie uploads of changes made by a user that aren't integrated into that version until a mod decides it's ok).

Granted, contributing might be a little more difficult (maybe too difficult?) but it might help take some of the burden off the moderators.

I personally never liked the xml version just because xml is less readable to human eyes.  That being said I eventually dumped support for the ini because it turns out that the windows api calls to read/write inis have a 64kb limit.  Now for any sensible db this wouldn't be much of an issue, but mame has an insaine amount of games and thus I hit that limit pretty fast.  Seeing as how a person then had to parse the file manually, sans any helper functions, it made much less sense to use the format.

The plan as of the next upgrade to the format was to cut the cord and remove the ini completely. 

Now xml parsers obviously don't have this problem, but xml files, due to their massive amount of waste (too many flags and white space) up the file size dramatically.  I don't know if you've noticed, but the current listxml produced by mame is just shy of 100 megs! Now obviously controls dat is much smaller, there is less data to contend with, but we've got to assume that as more entries are added, the total file size will become a concern. This isn't really a big deal on modern computers, but it is a killer for any site that hosts the file. 

So I think a regular database is best for the online component simply to reduce the file size. 


I think VCS  is a little daunting for a guy who just bought a dataeast control panel and wants to make a single dat entry.  I could be wrong on that though.

A code repository in general isn't a bad idea though.  The only problem is how to make it user friendly.  Pretty much nobody aside from SirP and myself can write a complex entry manually because nobody but SirP and myself fully understands the craziness of the format.  We would still need some sort of wizard, either offline or online. 

Another major hurdle of the project was mame's constant renaming of roms.  Entries had to periodically be compared/parsed through a diff file to make sure none of the rom names changed.  Now 99% of your controls.dat entries are parents, so it rarely effected much, but when it did, it was a headache (for sirp mostly).

I think now would be the best opportunity to completely ovehaul the db and the format.  So ideas... they need to be posted.

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #4 on: September 03, 2012, 12:43:39 am »
beenbusy


Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #5 on: September 03, 2012, 06:21:29 am »
Bah!

Why ride when you can sit in front of a lcd for hours a day and grow ever paler and fatter like the rest of us?  ;)

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #6 on: October 11, 2012, 02:16:56 pm »
So here's what I am thinking.  A new format is probably needed.  I think we go strictly XML.  INI is old school, but was used because back in the day people were still using Win98 on their cabinets.

If someone wants to redesign the dat file go ahead.  We can start having a discussion here on what the requirements are, how to handle odd games, how to handle NeoGeo and other major parents, etc...  Come up with a design and a website to handle it.  I can help convert our existing data over to the new format.  Then the project will be yours to maintain.  I will still help out but I just don't have the time I use to. 

It's been 4 years that I have wanted to redesign my control panel and make everything flush mount now that the tech is there :)

drventure

  • Trade Count: (+2)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 4152
  • Last login:June 25, 2023, 02:23:57 pm
  • Laser Death Ray Bargain Bin! Make me an offer!
Re: Would anyone like to take over?
« Reply #7 on: October 11, 2012, 02:53:02 pm »
JSON might be a workable and reasonable alternative.

It's a little lighter than XML, there are parsers everywhere, and it's reasonably readable.

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #8 on: October 15, 2012, 03:18:16 pm »
I thought of that.  It would be nice to know which version of the file people mostly use.  what do the front ends use.  If we switch to JSON how man front ends will no longer be able to use an updated file because the developer stopped working on the front end.

LGC

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 9
  • Last login:March 21, 2015, 04:58:05 am
Re: Would anyone like to take over?
« Reply #9 on: October 16, 2012, 11:44:11 am »
With a good database structure (and data to go with it) it should be possible to generate any kind of output.
I've been toying with the idea of making a single, more front end oriented, game info related kind of data store for some time ... just never could settle on what information should be in there
and perhaps even more important, how to maintain/update/verify all the information..

anyhow, just to say I'd like to help out in any way I can
and the best place to get things going would be to discuss and decide what relevant information to store.

drventure

  • Trade Count: (+2)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 4152
  • Last login:June 25, 2023, 02:23:57 pm
  • Laser Death Ray Bargain Bin! Make me an offer!
Re: Would anyone like to take over?
« Reply #10 on: October 22, 2012, 05:26:05 pm »
Ok. Just had a thought. My brother just pointed out that you can make URL calls into a publicly shared Google Docs spreadsheet, to retrieve specific cells, or entire sheets

You can play with the concept here

http://ouseful.open.ac.uk/datastore/gspreadsheetdb2.php

so, take the XML, reformat it a little into a spreadsheet, share it (maybe even editable to the right people). then putting a website together to display or reformat it it would be pretty trivial.

Maybe it's too big for a google spreadsheet, but probably not.

Anyway, thought it was an interesting possibility.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #11 on: November 02, 2012, 07:17:29 pm »
Depends... what's the maximum file size of a google spreadsheet? 

Since gambling games and pinball have ben added to MAME, we are in 5 digits in terms of the total games in MAME.  Obviously we'll never hit 100% but any format chosen needs to be able to handle the full mame set with room to grow on.  It's one of the reasons I've always resisted xml btw.... there's so much darn white space in a xml file that the one mame currently generates can be sluggish to parse depending upon the library you use to parse it (if any). 

So for me at least, the database needs to be just that... a database and nothing more.  I'm talking comma-delimited text file simple.  We just need to make the interface capable of outputting it into more user-friendly data.  (xls, xml, ini whatever..)


Somewhat releated in terms of approach... are you guys aware of who they do things at the wii hacking scene for the fes there?

http://www.gametdb.com/

It's a very elegant but simple solution for organizing all the data pertaining to a game. 

Here's an example:

http://www.gametdb.com/Wii/SU3PMR

The SU3PMR part of the url refers to the game's internal id code (sort of like a mame rom name).  The images shown on the webpage, which fes use for cover art, are always found in the "cover" "coverfull" "cover3d" and "disc"  folders of the same url.  If we could find somebody willing to actually host a repository like this for mame with the same type of url schema a different type of approach could be taken and each game could get it's own individual file ect...  But I dunno, mame is so popular a site like that would get hammered. 

It's just there are so many errant support dats/files/images for mame out there it sure would be nice if they all adhered to the same formatting for easy parsing.... just a thought.

EightBySix

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 523
  • Last login:April 25, 2021, 01:50:16 pm
Re: Would anyone like to take over?
« Reply #12 on: November 03, 2012, 04:13:29 am »
Amazons AWS free tier now includes database support. Even when he free year runs out, it's often cheaper than web hosting. Just saying.

edekoning

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 591
  • Last login:November 10, 2022, 02:21:38 pm
Re: Would anyone like to take over?
« Reply #13 on: February 14, 2013, 03:28:02 am »
I just found this thread through some random search, and a I realize its months old, but I still want to react to a few statements made.

So here's what I am thinking.  A new format is probably needed.  I think we go strictly XML.  INI is old school, but was used because back in the day people were still using Win98 on their cabinets.
I never really understand these arguments. Just because something is old (but most likely tried and tested) does not mean it should not be used. And just because everybody is doing XML does not mean you should.

I personally never liked the xml version just because xml is less readable to human eyes.  That being said I eventually dumped support for the ini because it turns out that the windows api calls to read/write inis have a 64kb limit.  Now for any sensible db this wouldn't be much of an issue, but mame has an insaine amount of games and thus I hit that limit pretty fast.  Seeing as how a person then had to parse the file manually, sans any helper functions, it made much less sense to use the format.
I also find XML to be not that human readable/editable. Never understood why everybody likes it so much. I don't really see a problem in the XML file (either mame or controls) being big. Just use a SAX parser, and store it compressed.
Regarding the INI files, why use windows api calls at all? I mean an INI file is just a very simple text file. As long as you keep it simple (no hierarchies, no escape characters, ...) then there is no real reason to use any library at all. Each section represents a game and is followed by a number of key=value pairs. Just process the file line by line.


I'm not sure what you are looking for in a file format:
-must it be human readable?
-must it be a small as possible?
-must it be easy to read/write/edit
-can it be in binary?
Human readable usually means big files, because of all the context that's needed. Note that I don't consider a file, just because its not binary, human readable. Binary files are usually tiny, but require dedicated tools to edit.

Have you ever really considered a binary format? You could define one that's much easier to parse than any text format could ever be. I hope you know a little C/C++ otherwise this next bit wont make any sense. If you define a POD-struct that can only contain fixed width strings, and that explicitly contains all the needed padding bytes. Than you could just read x-bytes from a files and do a direct cast to the game struct type. So parsing would the data for a single game would require one line of code. An example from the top of my head:

struct Game
{
char [8] name;
uint8_t numPlayers;
uint8_t alternating;
uint8_t mirrored;
uint8_t usesService;
uint8_t tilt;
uint8_t cocktail;
char [2] pad2; // explicit padding bytes
char [64] description;
...
};

// reading
char* data; // assume game data read from a file
Game game = *reinterpret_cast <game*> (data); // done parsing
// writing
data = reinterpret_cast <char*> (&game); // done parsing
write (file, data, sizeof (Game));


If you decide on some format that is not standard (CSV, INI, XML, ...) then you should provide a detailed description as well as some sample code to read/write such a file. For my example you would provide a header file containing the structures, and a source file that contains functions to read/write a file. It would also be nice if the same was supplied for multiple languages, e.g.: python.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #14 on: February 14, 2013, 11:25:46 pm »
I can answer a few of your questions.

Controls.dat already does xml, half of the viewers use the xml file, the other half use the ini file. 

A big xml file means it takes longer to parse... we aren't talking harddrive space, but rather we are talking processing time and memory.  Also Bandwidth...  it will be downloaded by users you know.  ;)

The windows api calls are amazingly fast for getting data.  Unless you are really good at making serach routines, anything done manually would be considerably slower.  Searching line by line sounds like a good idea... until you realize were are in the 5 digit area for the number of lines.  That being said, I think the 64kb limit only applied to win 98, so it may be a non issue at this point.  I'd have to check my code. 

Binary wouldn't really save any space or time.  You'd still need the bare minimum in terms of data logging, like a section name, the key and the value.  So two characters for the brackets normally gets reduced to 1 for the start of a new section and a "=" plus a line return, which is three characters gets reduced to two.... not really much of a savings. 

The dat file doesn't have to be humanly parsable, but it has to be easy for novice programmers to parse.  Most of the people in this hobby aren't professional programmers and even as simple as the dat file is now, there are only three viewers.  That's one of the reasons I always resist xml btw....  yeah it's easy enough to use a plug-in parser, but they are generally slow and unoptimized for the massive xml that is related to mame, so the apps built with them chug along.  As PCs have gotten faster it's much less of an issue, but still.... it's bothersome. 

I'm not saying definatively one way or another is the correct way to go, I'm just sharing my thoughts. 


I would really like to get this thing going again, but controls.dat is a MASSIVE undertaking in terms of managment.  Nobody seems to want to do the web side of things, so we are kind of stuck. 

edekoning

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 591
  • Last login:November 10, 2022, 02:21:38 pm
Re: Would anyone like to take over?
« Reply #15 on: February 15, 2013, 05:58:15 am »
I think you and I are approaching this differently. You talk about searching in the file, implying that whenever some controls data is needed the file is accessed. I would just load the file once and keep it in memory in some kind of associative/hash container. Disk IO is magnitudes slower than Memory IO, so it should be avoided.

Binary will save lots of space, as you would only store values. All the data logging, keys, are all in the code, and not in the file. If you look at my example you see that the game struct has a variable with name numPlayers, whereas the file will only contain its value. So the XML attribute numPlayers="2" will be reduced to a single byte. Now binary has drawbacks: any format changes will most likely brake every bodies code, versioning is a ---smurfette---, so you should always provide a reader/writer sample code and clear documentation.

I've managed to stay away from all web related stuff in my work so far, not planning on changing that any time soon ;)

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #16 on: February 15, 2013, 10:35:33 am »
But controls.dat viewers aren't a resident app.  They are generally launched prior to the game and called via a hotkey.  So you can't hold the file in memory... well you can, but you have to load it each time anyway, which takes just as long.  Loading a 30+ mb file into memory, even on a modern machine, typically takes longer than searching the file by loading a bit at a time.  Again, the massive file size of the dat throws typical approaches out the window. 

Controls.dat is a flexible data file, so there can't be any hard-coded values.  As you said, changes will break the compatability.  Think about the nightmare it would be if I were to add a (just for generic example) "orientation" variable to the game structure.  If it's plain text, any new additions to the format would simply be ignored by older viewers.  As a binary file it would break them. 

Rodent.Vienna

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 7
  • Last login:November 20, 2020, 03:47:26 am
  • I want to build my own arcade controls!
Re: Would anyone like to take over?
« Reply #17 on: March 14, 2013, 12:13:30 pm »
i am interested to be part of whatever continuation this project may lead to.

I understand conceptually and structurally what contents are of different files supporting mame data, also such as controls.ini / controls.xml.

i would understand the direction to take this to as well, but i do not understand what the technical basis / starting pint is, in:

- what is consistent of the project currently?
- active people with which background participating
- technical requirements / existing solutions to promote crowdsourcing in a submitted / approved manner (2 lists to projectize)
- limits in resources (people / information), limits in technology (missing programs to aid the project)
- what is the complete "what we have" to start from as of now?

I am truly interested to participate from a strategic / "ideal scenario" point of view, and to support by finding the right people to fill gaps.

You need to pick me up with the "where are we at" :)
the where do we want to go is something not clear yet.

thanks
alex

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #18 on: March 16, 2013, 07:43:32 pm »
I can summarize. 

We've got the website, we've got the old database and we've got all the tools needed to maintain the status quo. 

The problem is the format needs changing a little to accommodate different control schemes and the database might not be the most optimized thing in the world.  All the stuff on the backend needs to be reworked (the webpage and database) which somebody else needs to primarily handle.  The stuff on the front end also needs a little work (xml/ini output and the viewers ect) which I will gladly handle.   

Rodent.Vienna

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 7
  • Last login:November 20, 2020, 03:47:26 am
  • I want to build my own arcade controls!
Re: Would anyone like to take over?
« Reply #19 on: March 20, 2013, 04:02:41 pm »
thank you for the insight.
i am currently focused on a different project, so all my input can be either handled as theoretical, or as "in preparation".

when you say "all tools to maintain status quo", what do you mean by that?

what is the potential to get an intermediate release out there, that covers the romfilename changes from 0.141.1 to e.g. 0.148?

when you say "we", and i asked about "active people with which background involved", can you put more light on this, or is it secret? :)

what i cannot see here, except actually the technical stuff around "backend needs reworking", is an outline if this is actually the single point of stallment currently,
or if there is indeed progress on the data entry / quality / data update side meantime, despite this technical problem.

when you say "somebody else needs to handle webpage & database", do you mean by that somebody needs to be found who can tackle this, or is this somebody identifies already?

thank you and kind regards
Alex

sandheaver

  • Guest
  • Trade Count: (0)
Re: Would anyone like to take over?
« Reply #20 on: March 26, 2013, 07:22:46 pm »
edit: nevermind.
« Last Edit: March 26, 2013, 07:55:14 pm by sandheaver »

RATMNL

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 4
  • Last login:April 23, 2013, 06:14:30 am
  • I did build my own arcade controls!
Re: Would anyone like to take over?
« Reply #21 on: April 09, 2013, 05:28:16 am »
Stumbled upon this while browsing for extensions for my Arcade Cabinet, and while I'm busy working on a new breed of Catlist (it WILL be different!) I do think someone should pick this up!
While I'm not a complete Code-monkey, I am a rather proper Typing-Monkey (did I tell I started that Catlist from SCRATCH?) so If this project needs some help being kept either alive or Work in Progress, Hit Me Up!

I'll be keeping my eye and ears open! Would be a shame that such a good idea would be discontinued or fall in disrepair.

M.

edekoning

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 591
  • Last login:November 10, 2022, 02:21:38 pm
Re: Would anyone like to take over?
« Reply #22 on: April 09, 2013, 08:48:36 am »
a new breed of Catlist (it WILL be different!) ... (did I tell I started that Catlist from SCRATCH?)

Please tell me you are not going for a single category/subcategory per game. It makes much more sense to be able to specify multiple 'labels' for each game, e.g.: Street Fighter = 'Fighter' + 'Versus'. That way you can also define a single label 'mature', and not having to duplicate every category that contains a mature game. It also means that games that have driving and platform parts can be labelled as 'Driving' + 'Platform', instead of forcing them into a single category. You could also merge in the nplayers file as that would just be a represented by a bunch of additional labels, e.g.: xmen = 'Fighter' + 'Coop' + 'P4 Sim'.

The main challenge will be to decide upon the labels and how to keep their amount as small as possible. Oh, and all FE's will need to be changed as well ;D

RATMNL

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 4
  • Last login:April 23, 2013, 06:14:30 am
  • I did build my own arcade controls!
Re: Would anyone like to take over?
« Reply #23 on: April 09, 2013, 09:36:42 am »
a new breed of Catlist (it WILL be different!) ... (did I tell I started that Catlist from SCRATCH?)

Please tell me you are not going for a single category/subcategory per game. It makes much more sense to be able to specify multiple 'labels' for each game, e.g.: Street Fighter = 'Fighter' + 'Versus'. That way you can also define a single label 'mature', and not having to duplicate every category that contains a mature game. It also means that games that have driving and platform parts can be labelled as 'Driving' + 'Platform', instead of forcing them into a single category. You could also merge in the nplayers file as that would just be a represented by a bunch of additional labels, e.g.: xmen = 'Fighter' + 'Coop' + 'P4 Sim'.

The main challenge will be to decide upon the labels and how to keep their amount as small as possible. Oh, and all FE's will need to be changed as well ;D

You're taking the wrong approach at it, or at least a different one. I'm not looking to re-invent Catlist. Comparing it to Catlist might be a bit off, because while it serves the same purpose, I wrote with my personal priorities in mind and will serve as such (Those being to "work for me" for my Cabinet, while not losing the incredible amount of information and artwork in the process,a dn to display properly in QMC because I'm working on a mac and so is my cabinet). Ok damnit, I had planned not to talk about this until it was finished, but since I sprang my mouth already (and honestly thought no-one would dive on THAT part of the post)

- All Games are in there (up to 0.148).

- Parents & Noteworthy Clones will be sorted as in catlist, only with significantly (!!) less categories (we're talking *only* 78. for instance I really don't care about my point-of-view in a race-game. (Which is not a good example because all games that you race car-like vehicles are sorted in either Racing / Circuits or Racing / Touring, which means going in circles or from A to B).

Note that these will mostly suit my english-speaking needs (except for where history.dat says it's worth it to include the mostly Japanese one as well (yes, I read most of history.dat to figure that all out). Games that only appeared in Japan or such are in there as treated normal.

- Clones that didn't make that cut, but have different flyers as their parent counterparts are stored in a seperate tag -> ZZ - Flyerclones

- In recent efforts to clean up history.dat there's a lot of cases where the parent's entry would just say: export Version, look at the japanese one! separate tag > ZZ - Infoclones

- Only working games in the Main list. Preliminary drivers are stored in ZZZ - Preliminary.

- The ones that didn't make all that are stored in ZZZ - Clonebin, hoping to make it out someday based on new history.dat information. Let's Pray for them.

- Gambling games Is the only thing I did quite selective, If I liked what I saw i put it into Gambling, If I didn't I put it in a folder called ZZZ - Unspeakables. There's really no need to be there.

- The Mature Issue has been broken down into 6 categories (with Mature as first prefix, so it would stay together).

- The [Veradded] "column" is not used for tracking which MAME it was released in, but it displays the Hardware System it was Built on (like CPS, Arcadia, NeoGeo, PC-10 etc.)


- Some other stuff/ideas for dividing the Clonebin to get it in compliance with history.dat, but that's not for now, might not even be for release.

As said I'm not reinventing anything, I just put a HUGE amount of manual labour to build something into something the way I wanted it to be.  ;D
(reading history.dat, checking all flyers and playing next to every game to be able what mattered for me in the sorting of categories and keeping those numbers a bit lower.)

Anyway... On-topic? ;)


sandheaver

  • Guest
  • Trade Count: (0)
Re: Would anyone like to take over?
« Reply #24 on: April 09, 2013, 01:07:47 pm »
a new breed of Catlist (it WILL be different!) ... (did I tell I started that Catlist from SCRATCH?)

Please tell me you are not going for a single category/subcategory per game. It makes much more sense to be able to specify multiple 'labels' for each game, e.g.: Street Fighter = 'Fighter' + 'Versus'. That way you can also define a single label 'mature', and not having to duplicate every category that contains a mature game. It also means that games that have driving and platform parts can be labelled as 'Driving' + 'Platform', instead of forcing them into a single category. You could also merge in the nplayers file as that would just be a represented by a bunch of additional labels, e.g.: xmen = 'Fighter' + 'Coop' + 'P4 Sim'.

The main challenge will be to decide upon the labels and how to keep their amount as small as possible. Oh, and all FE's will need to be changed as well ;D

You are describing game type tags, not necessarily categories.  A category list would be a subset of a tag list for any particular game, I think.

My point is that we needn't redo the categories, but create tags that incorporate the categories as well as other things about the game.  So a category for Tetris might be "Puzzle," and that is certainly valid, but to that, one could add "Versus," "Competitive," and maybe things like monitor orientation or whatever else.

RATMNL

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 4
  • Last login:April 23, 2013, 06:14:30 am
  • I did build my own arcade controls!
Re: Would anyone like to take over?
« Reply #25 on: April 09, 2013, 01:18:21 pm »
... if there was any FE support anywhere for such an endeavour.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #26 on: April 09, 2013, 01:58:41 pm »
That's great and I wish you all the luck in the world, but I fail to see what a catlist has to do with controls.dat. 

Much like mame, controls.dat is a documentation project... we literally document the physical controls of a game and if at all possible the labels that were printed for said controls on the cpo.  Catlist is sort of a gray area thing, seeing as how categories are subjective. 

I'm probably misunderstanding something, but controls.dat is not a part of catlist nor vice-versa. 

RATMNL

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 4
  • Last login:April 23, 2013, 06:14:30 am
  • I did build my own arcade controls!
Re: Would anyone like to take over?
« Reply #27 on: April 09, 2013, 02:02:30 pm »
That we all know, I suppose, Disussion got offtopic (my bad, I suppose.)

Anyway.. controls.dat, similar discussion...

keilmillerjr

  • Trade Count: (+5)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 1847
  • Last login:October 06, 2023, 10:20:39 pm
  • Web Developer.
Re: Would anyone like to take over?
« Reply #28 on: April 23, 2013, 05:02:23 am »
I do not have experience with the controls.dat project... but I am an aspiring web developer and I think you, some one else, or I could create a web app written in ruby on rails. Using the default sqlite database is fine. It's a single file. Easy to work with and you could implement it into a desktop app. Update it online through a web form. You could easily output a json and xml view template with ruby on rails. I don't see why it wouldn't be possible to generate an ini file too. People could view and download a current version instantly. Just an idea.

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #29 on: January 31, 2014, 12:37:55 pm »
Long time no visit. I appreciate saint hosting for me. I am seriously thinking about getting a windows server and redoing this project in ASP.NET MVC. I need to get come experience with modern web app development.

Looking back at this cod it is ugly. I can tell I was fresh out of college when I started the project.

The problem with the project as it stands is it is not that flexible to mame changes. And it doesn't containa change history in the db. The only history is the generated files.  But then when I created this I was trying to get something usable as quickly as possible.


Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #30 on: February 03, 2014, 07:12:52 pm »
Sounds like a plan.  You know mame's input system has changed some as well. 

You should stop by every once in a while... you don't need to code to say hi.  ;)

liquid8

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 156
  • Last login:June 11, 2017, 04:02:02 am
  • I working on it.. it'll be a while.
Re: Would anyone like to take over?
« Reply #31 on: February 28, 2014, 07:36:09 am »
What's the saying.. I don't want too many irons in the fire - but I was talking with Howard about a console info dat, and he mentioned this project.

I hate to see this die out, so I wanted to chime in and maybe offer some assistance.

I am seriously thinking about getting a windows server and redoing this project in ASP.NET MVC. I need to get come experience with modern web app development.

PHP and MySQL isn't dead.. why re-invent the wheel? What is it that the current app isn't doing well that it needs to be re-done? If it is just that MAME input system is different than this was originally designed to handle, maybe you just need a revamped UI and a better structured database?

The problem with the project as it stands is it is not that flexible to mame changes. And it doesn't containa change history in the db. The only history is the generated files.  But then when I created this I was trying to get something usable as quickly as possible.

Why is change history necessary any more than comparing the old and new version? Are you talking about for verifying/approving changes, or who made a certain change? Or do you mean maintaining MAME version history?

Although I know controls.dat and use it, I'm not familiar totally with what's involved in updating it and the MAME changes that have occurred. Just taking a quick look at the site - the biggest problem seems to me that controls.dat isn't being updated :P

I'm no database engineer, and not a professional web developer, but I  have plenty of experience in both to contribute. I don't know I have the time to redo a site/database, but with more input on the issues at hand, I'd be willing to check it out and at least give some input, if not assist in some manner.

keilmillerjr

  • Trade Count: (+5)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 1847
  • Last login:October 06, 2023, 10:20:39 pm
  • Web Developer.
Re: Would anyone like to take over?
« Reply #32 on: April 10, 2014, 11:19:54 pm »
I am seriously thinking about getting a windows server and redoing this project in ASP.NET MVC. I need to get come experience with modern web app development.

Have you used asp before? It's the most god awful language, it costs money to develop in it, and it's tied to one platform. If you need help, I tossed myself out there once. Seriously.

Jakobud

  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 1940
  • Last login:March 11, 2024, 04:45:38 am
Re: Would anyone like to take over?
« Reply #33 on: August 04, 2014, 01:38:31 pm »
What is the status of this project lately?  Where did the database of games go?  What % of games are complete with regards to their control labels?

I've been out of the BYOAC scene for a while.  Are there local software solutions for this these days?  Are the Mame devs still not labelling controls accurately?

What would be the ideal situation for you guys with regards to the way people update and add controls for games?  Seems like it would be pretty easy to have a database of games and allow people to update controls for games.  I am a web developer so I could help out if its not too big of a deal.

drventure

  • Trade Count: (+2)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 4152
  • Last login:June 25, 2023, 02:23:57 pm
  • Laser Death Ray Bargain Bin! Make me an offer!
Re: Would anyone like to take over?
« Reply #34 on: August 04, 2014, 02:28:11 pm »
I've been throwing around some ideas on this as well.

I've been working with Ruby and Rails over the past 2 years, and I suspect I could cobble together a pretty decent administrative style site for this using ActiveAdmin pretty quickly.

Still on the job hunt right now, which is basically a full time job in and off itself, so it might have to wait for any real time till I snag something and get settled, though.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #35 on: August 15, 2014, 09:41:45 pm »
What is the status of this project lately?  Where did the database of games go?  What % of games are complete with regards to their control labels?

I've been out of the BYOAC scene for a while.  Are there local software solutions for this these days?  Are the Mame devs still not labelling controls accurately?

What would be the ideal situation for you guys with regards to the way people update and add controls for games?  Seems like it would be pretty easy to have a database of games and allow people to update controls for games.  I am a web developer so I could help out if its not too big of a deal.

SirP still has the reigns.  He recently removed the scripting from the site as it could be exploited and he said he wants to try and do a new site later on.  Then again he says that roughly once a year (I'm looking at you SirP!  ;)

I'm sitting in a holding pattern because before starting up again we would have to seriously re-design the file format due to recent mame changes. 

Headkaze wrote a stand-alone app... and people have added info to it, but I doubt it's accurate not because people don't have good intentions, but because when we used to do it we were "show your proof" Nazi's to make sure label names and what had you cam from some sort of official documentation. 


So long story short, contact him... keep me in the loop as well because I often do a lot of the grunt work in terms of designing the format and doing entries. 

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #36 on: August 19, 2014, 01:13:32 pm »
The whole project needs a redesign because of the changes that have happened to mame. The big thing is how to handle the newer format of listing and map controls to it.

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #37 on: August 25, 2014, 08:54:50 pm »
Well, it looks different but they haven't changed a whole lot other than the formatting.  You've still got generic control names like "dial" and then you have to go and look up what individual inputs "dial" has.  If anything it is a bit easier to read now, so long as you ignore the flags inside the control node (which I think we could get away with). 

Now on the physical mapping end of things, things have changed significantly.  All analog controls have a digital + and - as well as the analog axis.  That analog axis can also be mapped to a full axis or a split axis (more of a viewer problem in that case). 

Since that needs re-designed we might as well add in diagonal labels as well.  That's something I regretted.  Yeah it's a pain in the butt to display in viewers, but I don't see how that is our problem. 

Also you are going to hate this suggestion, but I think ditching xml might be a good idea as well.  I'm not necessarily suggesting the ini format either, but now that mame has such a massive amount of roms, the xml size of just mame is out of control.  It actually takes a minute or two just for mame to print it out, much less parse it or convert it via another app.  So that is bad enough, but then to parse the controls.dat on top of that... well let's just say viewers could get sluggish.

Either that or start putting each game's xml in a separate file.   

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #38 on: August 25, 2014, 09:01:59 pm »
Oh I probably need to point out that the mame output is STILL inaccurate, unfortunately. 

For example run -lx on outrun.  It says it has 3 buttons... I'm not sure how it got that number.  You've got a gear shift toggle a start button (which isn't supposed to be counted) and two coin slots.  So the number should be 1 or arguably 4, not 3.  It shows a paddle... ok that's alright and a pedal, yup it's got that... but no second pedal?

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #39 on: August 28, 2014, 11:09:48 am »
I think what needs to happen is to not have controls.dat as mame driven. Meaning this it what I envision.  Currently when adding a game to the dat the wizard would get all of the mame controls for that game and display them asking for the real label.

I am thinking about changing this. Instead you will have to manually add controls to a game. The mame entry will be listed as a hint. But you would add something like an 8 way joy or spinner or 49 way joy.  There will be labels associated with that control to fill out. Then each label will be mapped to it's mame entry counter part. This way the game's co trols still keep proper labels even if the mame format changes. Then we would just need to change the mapping when mame changes.

Xml is fine if you know how to use it properly. You can either load it all into memory or use an xml stream. One you do at the beginning of the program during a splash screen, hopefully in its own thread. The other you can do on the fly and it will still be decent speed.

I would think about ditching the ini format and maybe using JSON instead.
« Last Edit: August 28, 2014, 11:15:58 am by SirPoonga »

Howard_Casto

  • Idiot Police
  • Trade Count: (+1)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 19399
  • Last login:March 16, 2024, 05:59:16 pm
  • Your Post's Soul is MINE!!! .......Again??
    • The Dragon King
Re: Would anyone like to take over?
« Reply #40 on: August 29, 2014, 12:31:07 am »
Not really.  I'm sorry but the xml load even on "professional" applications on the rom managers is ungodly slow now.  Have you messed with the recent builds of mame since they added all the e.m. and pinball stuff?

SirPoonga

  • Puck'em Up
  • Global Moderator
  • Trade Count: (+1)
  • Full Member
  • *****
  • Offline Offline
  • Posts: 8183
  • Last login:April 12, 2023, 09:22:35 pm
  • The Bears Still Suck!
Re: Would anyone like to take over?
« Reply #41 on: August 29, 2014, 03:33:41 pm »
But there is a difference between controls.dat and listxml output, controls.dat is not going to be 160megs.  It isn't going to be anywhere close to that.

rassilon

  • Trade Count: (0)
  • Jr. Member
  • **
  • Offline Offline
  • Posts: 1
  • Last login:September 07, 2023, 07:51:59 pm
  • I want to build my own arcade controls!
Re: Would anyone like to take over?
« Reply #42 on: September 06, 2023, 08:16:12 pm »
@SirPoonga,

I'm happy to help out as I'm finally going to put together a control panel and I really want to have a second display for button labels ala CPWizard and company. We could start by archiving the existing mysql data and the php in github maybe?

I figure that is a simpler place to start from instead of producing schema from modern mame list xml and the
control.xml file.

Thanks,
Bill