Aaron removes xml2info as of u12 (in other words, we are screwed)

[Howard_Casto]

I guess the xml2info code from the old builds are included in mame.  Anyone care to look at the changes aaron has made to the xml output and get it working again??

If not this pretty much means that old frontends/utilites are officially dead. 

Btw, this also means that until further notice, dk and j5 no longer work. 

[SirPeale]

With the latest version of Mame, XML2INFO is now gone.

Who wants a crack at implementing XML into ArcadeOS?

[Buddabing]

If you must have xml2info, just take the code from v0.106, add in the new fields, compile it, and post the result on your utilities page. It's not difficult.

[SirPeale]

There's some talk here.

[Lilwolf]

I have to say... you can also just parse the xml... not hard and its a lot more useful.

It has been the standard for 2 years.  Took me about 15 minutes to parse it... another hour to replace the old parser with the xml parser.  You shouldn't be afraid of XML... just wonder why people use it when speeds on the line.

[krick]

I have the xml2info.c file that I submitted with my patch to 106u11 to work with the new XML changes.

I've built an exe too that should work as long as the XML doesn't drastically change in the future.

You can get them both here...

http://mame.3feetunder.com/xml2info/


EDIT#1:  Oops!, I might have spoke too soon.  I didn't notice that uRebelScum added additional control info to the XML.  I'll have to test it with 106u12 and see if it crashes or not.  I'm assuming it will need a little tweaking.  Stay tuned.

EDIT#2:  Yep.  It's broke real good.  Not sure how to fix this one.  The xml2info code was very fragile and dependent on the order of the items.  The new control tag is nested inside of the input tag and therefore won't get processed until the entire input tag is processed first.  Not good.  I'll keep looking at it but *if* I can get it working, it's gonna be *really* ugly code.  It would probably be faster to re-write it from scratch using a DOM package.  I'm quite familiar with JDOM.  Maybe I'll give it a try in JAVA when I get more time.

EDIT#3: I have something working.  It's not 100% optimal but it works.  See my post further down in this thread.

[Howard_Casto]

I have to say... you can also just parse the xml... not hard and its a lot more useful.

It has been the standard for 2 years.  Took me about 15 minutes to parse it... another hour to replace the old parser with the xml parser.  You shouldn't be afraid of XML... just wonder why people use it when speeds on the line.

I must be dumb or something.  I haven't looked into it seriously but this evening I downloaded several xml parsers and example code.  While I can parse any old piece of xml easily I can't get mame's listxml to parse.  Is mame's output fully xml compliant?


For the record, I don't like xml because it's in-general larger and slower than virtually any format out there.  Mind you the listinfo format is a complete mess, but considering the work was already done I never saw any point in converting.  I still don't unedrstand why mame couldn't have outputted plain old text (think list info without all the stupid tabs and formatting characters). 

[krick]

I must be dumb or something.  I haven't looked into it seriously but this evening I downloaded several xml parsers and example code.  While I can parse any old piece of xml easily I can't get mame's listxml to parse.  Is mame's output fully xml compliant?
 

The DTD at the top of the listxml output is broken currently so the XML doesn't conform.  It's an easy fix.

See this post...

http://www.mameworld.info/ubbthreads/showthreaded.php?Number=80169

[Howard_Casto]

yeah figured that out and promptly deleted the whole mess as it isn't really needed if you know the xml version. 

Even so every other parser I try seems to have some issue with listxml.  I found one that works, but it's a manual parser, so it should.

[krick]

Even so every other parser I try seems to have some issue with listxml.  I found one that works, but it's a manual parser, so it should.

If you find parsers that complain, let me know what they complain about.  It's probably something legitimate that needs to be fixed.

[krick]

Ok, I've got something working.  It's not optimal but it should get people at least playing games.  For the moment, I had to hard code input control to "unknown".  Eventually, I or someone else will figure out how to get correct data in there but for now, this is probably the best to hope for.  Enjoy...

http://mame.3feetunder.com/xml2info/

[Howard_Casto]

Even so every other parser I try seems to have some issue with listxml.  I found one that works, but it's a manual parser, so it should.

If you find parsers that complain, let me know what they complain about.  It's probably something legitimate that needs to be fixed.

I wouldn't know what to tell you.  They don't error out, it's just the parsers choke or enver finish ect. 

 
I think the abandonment of the listinfo format (regardless of how horribly flawed it is) is a mistake. 

The xml format is very simple and easy to understand, the xml parsers available aren't though. 

The list output is supposed to be for fe develpers right?  Then why in the world is the xml format used?  Aaron has to remember, most fe programmers are just hobbists. We shouldn't have to learn a whole new language just to interface with mame.  Xml is exteremelly difficult to parse with some of the older languages, even with tools (msxml, sax, whatever).  After looking at my choices that work with vb6 I am rather dis-heartened.  The official m$ stuff is so convoluted with all of it's classes and sub-classes that it takes knowledge of sql-like queries necessary even for looking up something as simple as a sub tag.  Most of the parsers I tried simply can't load the entire listxml output which is essential as front-ends need to load all the data and convert it to whatever type of list file they use.  The ones that did work took an hour to load it.  And I'm not exaggerating, I mean a whole hour. 


Your efforts trying to fix xml2info are appreciated, but the real solution (as it'll be hard to keep up a converter not officially in mame) is to get everyone off this xml kick and have mame output a real format.  What's wrong with ini files?  The ini format is easy to parse both manually and with help.  It's ascii too, the only difference is the lack of sub-nodes which aren't really needed in mame anyway.  Also it is rather small compared to xml.   

Xml is meant for tiny little storage files, not huge databases like mame generates.

[krick]

I think the abandonment of the listinfo format (regardless of how horribly flawed it is) is a mistake. 

The xml format is very simple and easy to understand, the xml parsers available aren't though. 

The list output is supposed to be for fe develpers right?  Then why in the world is the xml format used?  Aaron has to remember, most fe programmers are just hobbists. We shouldn't have to learn a whole new language just to interface with mame.  Xml is exteremelly difficult to parse with some of the older languages, even with tools (msxml, sax, whatever).  After looking at my choices that work with vb6 I am rather dis-heartened.  The official m$ stuff is so convoluted with all of it's classes and sub-classes that it takes knowledge of sql-like queries necessary even for looking up something as simple as a sub tag.  Most of the parsers I tried simply can't load the entire listxml output which is essential as front-ends need to load all the data and convert it to whatever type of list file they use.  The ones that did work took an hour to load it.  And I'm not exaggerating, I mean a whole hour. 


Your efforts trying to fix xml2info are appreciated, but the real solution (as it'll be hard to keep up a converter not officially in mame) is to get everyone off this xml kick and have mame output a real format.  What's wrong with ini files?  The ini format is easy to parse both manually and with help.  It's ascii too, the only difference is the lack of sub-nodes which aren't really needed in mame anyway.  Also it is rather small compared to xml.   

There's two kinds of XML parsers.  DOM parsers that parse the entire file into memory at once, and SAX parsers that parse it sequentially and fire events as tags are encountered.

DOM parsers are nice when you need the whole file in memory, like when you're editing it.  However, they have the entire file in memory and parsed into a tree structure that is even larger than the original file.

SAX parsers are nice when XML is used for a configuration file and you just need to read it in sequentially and do stuff based on what you find.  SAX parsers are the most memory efficient way to go if you don't need the whole file at once.

You need to find the right tool for the job.

Much more info...

http://www.w3.org/DOM/faq.html#SAXandDOM

Xml is meant for tiny little storage files, not huge databases like mame generates.

XML is meant to be a standardized method of describing non-binary data, especially when there are hierarchical aspects to it.  The benefit is that there are tons of off the shelf XML tools and libraries so you don't need to write custom code for every project.  The obvious drawbacks are what you pointed out, the file sizes are usually much larger than other solutions.  However, with the advent of 3+GHz systems with 2+GB of memory and 500+GB hard drives, the benefits of XML outweigh the size issues.

One could argue that each game should be described in its own XML file and have all the files dumped into the same directory.  That would make it somewhat more manageable chunks for memory limited computers.


Oh, and you might want to check out XPath.  It's awesome for dealing with "huge [XML]  databases like mame generates". 

[Minwah]

I could be re-inventing the wheel (& I haven't tried it yet), but I'm tempted to write my own xml parser...

[headkaze]

I wouldn't know what to tell you.  They don't error out, it's just the parsers choke or enver finish ect. 

Let me guess VB6 custom written parsers? heh dosn't suprise me. If you coded in .NET System.Xml.XmlTextReader would make things alot more easier for you.

I think the abandonment of the listinfo format (regardless of how horribly flawed it is) is a mistake. 

The xml format is very simple and easy to understand, the xml parsers available aren't though.

A very bad mistake for breaking legacy support, I agree, but sometimes you have to go with the new and let go of the old. I know it's a ---smurf---, but it's the way the IT industry will always go. Part of being in this industry is keeping up with standards and keeping your skills up to date. This is the case for homebrew developers as well as professionals.

The registry was never supposed to be used as much as it has been for applications on Windows and .NET has dropped native support of reading & writing ini files because they are moving towards XML. It's a format that is being adopted everywhere.

The list output is supposed to be for fe develpers right?  Then why in the world is the xml format used?  Aaron has to remember, most fe programmers are just hobbists. We shouldn't have to learn a whole new language just to interface with mame.  Xml is exteremelly difficult to parse with some of the older languages, even with tools (msxml, sax, whatever).  After looking at my choices that work with vb6 I am rather dis-heartened.  The official m$ stuff is so convoluted with all of it's classes and sub-classes that it takes knowledge of sql-like queries necessary even for looking up something as simple as a sub tag.  Most of the parsers I tried simply can't load the entire listxml output which is essential as front-ends need to load all the data and convert it to whatever type of list file they use.  The ones that did work took an hour to load it.  And I'm not exaggerating, I mean a whole hour. 

I would recommend looking at some XML parsing code written in C++ and compiled to a DLL that can be accessed through the LoadLibrary API or however else your access a library in VB6. The thing is, as one would expect string parsing functions in VB6 are slow as hell, your really need something written in C++ to get some decent parsing speed.

You comments on hobbiest coders mostly coding in old languages is a little off. Alot of people are moving to .NET, but I agree it is a mistake to take it out for the sake of it.

Your efforts trying to fix xml2info are appreciated, but the real solution (as it'll be hard to keep up a converter not officially in mame) is to get everyone off this xml kick and have mame output a real format.  What's wrong with ini files?  The ini format is easy to parse both manually and with help.  It's ascii too, the only difference is the lack of sub-nodes which aren't really needed in mame anyway.  Also it is rather small compared to xml.

Yeah, the ini file format is being phased out by M$ so I guess it's important for coders to keep up with the times. Vista's XAML is an important part of that OS and will be used to separate the UI to the program code. It's a powerful language for something like that.

Xml is meant for tiny little storage files, not huge databases like mame generates.

Good point, it should be an SQL database file, not data generated by Mame.exe on the fly. And even VB6 can read Access databases using ODBC. So that would be the ideal solution. If the authors sincerely wanted to keep up with the times, then they would put this information into a database.

I suggest another solution; a program written in .NET using it's native XML parsing support to read the XML data generated from Mame.exe and output it to an SQL database. Then re-write your applications to read the database.

EDIT: Hmm it seems the Microsoft Access application can import XML directly.

[Lilwolf]

I know how you feel about XML.  I work for a company who uses it as a network protocol...  and then worries about speed.  We have to optimize the hell out of everything... just to make up for XML's bloat.... plus forcing us to do all those string manipulations to read everything in.

But it is nice for adding things like seperating the controls.  I bet uRebel couldn't have done it in the original -listinfo.... heck, one day I would love to see it seperate the players when possible. 

That... and you should almost always use a SAX parser.  It basically just reads the XML and calls methods based on whats next...  DOM parsers kinda suck because (1) they use a SAX parser and then (2) store everything no matter if your interested in it or not and (3) it stores it in a standard / bloated mannor.... even though its already bloated... 

I think they reason they removed the converter was just that it because broken because people didn't update it... and nobody wanted to maintain it.  Sounds like a perfect side project for someone else though...

I have to say... you can also just parse the xml... not hard and its a lot more useful.

It has been the standard for 2 years.  Took me about 15 minutes to parse it... another hour to replace the old parser with the xml parser.  You shouldn't be afraid of XML... just wonder why people use it when speeds on the line.

I must be dumb or something.  I haven't looked into it seriously but this evening I downloaded several xml parsers and example code.  While I can parse any old piece of xml easily I can't get mame's listxml to parse.  Is mame's output fully xml compliant?


For the record, I don't like xml because it's in-general larger and slower than virtually any format out there.  Mind you the listinfo format is a complete mess, but considering the work was already done I never saw any point in converting.  I still don't unedrstand why mame couldn't have outputted plain old text (think list info without all the stupid tabs and formatting characters). 

[Buddabing]

I rewrote xml2info back in '04 using my own flex/bison based parser. It was way faster than the expat library, and I predict it would be faster than any general-purpose parser.

Once v0.107 comes out, I'll update it for the new fields and it should just drop in.

[youki]

As lot of of you , i hate XML.   Didn't see any interrest in this format except it is something standarized.
Just a fashion effect , in few month a new "standard" will arrive and we will forget progressivly XML,  and same thing few months later..etc..Etc..   Some guys are paid to create standard , so they have to produce a new one each time. And generally the new one is always more heavy than the previous... it is the life... in addition most of these "Standard" creator never even write a line of code.

I'm working for a big professionnal software company , I can say you , they produce new Technology  just to be able to sell new producs and update.  Not because there is a need for.  Just a question of marketing after.

Almost all our lastest sofware uses intanscivly XML and others "new technology" , and believe me , all was far better before. (performance, memory usage, flexibility , required hardware..Etc..etc...).  the only good point for us, is it more standarized than before.

Anyway , AtomicFe is not affected by this kind of problems.  It is a choose i did from start , nothing specific to an emulator.   I assume that lot of others good tools exists to create advanced game list, manage roms , etc..etc..  So it is not the job of Atomic.  Atomic is a Front End, his job is to display a game list  in the  nicest way possible  and run game  on the most possible hardware configuration.

[krick]

As lot of of you , i hate XML.   Didn't see any interrest in this format except it is something standarized.
Just a fashion effect , in few month a new "standard" will arrive and we will forget progressivly XML,  and same thing few months later..etc..Etc..   Some guys are paid to create standard , so they have to produce a new one each time. And generally the new one is always more heavy than the previous... it is the life... in addition most of these "Standard" creator never even write a line of code.

You sound like a bitter old programmer.

XML is derived from SGML (as is HTML).  SGML was an ISO standard in 1986 so it's not exactly a "new standard".

The benefit of a standardized way of presenting data is the re-use of tools and libraries that process that data.

With DOM SAX and XPath, you can do amazing things with data in XML format without having to write a lot of tedious (and error prone) parsing code.

You can also use XSLT to transform the XML data into other formats, even non XML formats like MAMEinfo.

[youki]

You sound like a bitter old programmer.

lol , i'm just consterned of crazy solution and standard they output nowadays.

I'm more nostalgic than bitter.    I'm nostalgic from the time where coders knew coding.

We have machine faster than light , and we have program and technology slower and heavier than a turtle.

So when you combine both , you have softwares that runs at the same speed than a sofwares 15 years ago!

Do an experience,  run windows windows 3.1 and ms word 2.0   , on a modern machine. 

[Howard_Casto]

Gotta agree with the peanut gallery on this one. 

I don't think that learning a "new" standard is what bothers me, what bothers me is it isn't any better than the old one. If anything it's worse.  The dom objects and such are supposed to make xml eaiser to parse, but without the help of sax (which is useless if you are trying to look up a specific record) you can't even load a frikkin file as large as listxml.  All I know is I can load the entire listinfo file in any langauge in less than a second and parse it in less than a minute.  Even with the aid of sax I'd say the speed is going to be considerably slower parsing the xml. 


Like it or not xml is very much a fad.  I just hate to see it  used so heavily in mame because it isn't "future proof" and therefore in a few more years we might see all of this happening again. 



Just for the record even puny old vb6 supports all the major xml parsers, it's just none of them are very good, that's what I'm getting at.  Not vb's fault, it's the parsers fault.  I think my only hope is to look into xsl and convert the listxml into a real ascii format.

[Buddabing]

I think control viewer authors like Howard and myself should be excited about this change in u12:

Added multiple input controls in -listxml output for games with more
than one type. Added pedal control type. Added more info on analog
controls as defined in the driver: minimum, maximum, sensitivity,
keydelta, and reverse. This required the 'control' attribute in the
XML to be moved into an element. There can now be more than one
'control' element in the input secction. [uRebelScum]

[Howard_Casto]

Yeah I'm aware of the new data... I'm always excited about new data.  If it just wasn't presented in such a worthless format we'd be all set. 

[Effayy]

Yeah I'm aware of the new data... I'm always excited about new data.  If it just wasn't presented in such a worthless format we'd be all set.

Howard, I may be off base with this, but I don't consider XML to be all that worthless when taken from the standpoint that the output can then be transformed a myriad of different ways to suit external apps.

Yeah, it takes time to transform that data into an individualized format, but it's at least in an industry-standard format now that is versitile enough to be transformable into whatever you see fit for your external app.

- FA

[Howard_Casto]

So can listinfo.  The difference is listinfo can be transformed into various formats without the use of external plugins and with next to no difficulty. 

As I've pointed out several times in this thread xml is difficult to deal with when the file is as large as this one.  I'm all about industry standards, but only when they make sense to use with the data at hand. 

As much as I loathe msAccess, even it would have been a better base format. 

Or delimeted text files.  Any idiot (myself included) can parse text files. 


There's also the issue of other emulators.  My fe and several of the newer ones treat mame like any other emulator.  We can do this and still have advanced filtering due to clrmamepro files, which are pretty much the standard for getting rom names and descriptions (as well as 5 or 6 other fields). Guess what the clrmame dats are based off of?  You guessed it, listinfo.  So in terms of emulators, listinfo pretty much is the "industry standard".   


Anyway I'm done complaining. 

[cuavas]

I can't believe I'm reading this.  My Mac frontend that I have under development parses the entire listxml output, the mameinfo.dat and history.dat files in 15 seconds on a 1.5GHz PowerBook.  I am using libxml2 to parse the listxml output, which is a slow DOM parser.  I could make it a lot faster if I used expat or something else.  I just use libxml2 because it's easy and included with OSX (and I only parse the data once before storing it in a different format).

My code for processing the XML is tiny compared to the code I had for reading the old format.  With the XML I can lean on libxml2, while the old format required me to do all the parsing myself.  I know which I'd rather use.

If your parser chokes, use a better parser.  If your parser is too slow, use a faster parser.  If your code is too complex, you aren't using the libraries properly.

[Howard_Casto]

I can't believe I'm reading this.  My Mac frontend that I have under development parses the entire listxml output, the mameinfo.dat and history.dat files in 15 seconds on a 1.5GHz PowerBook.  I am using libxml2 to parse the listxml output, which is a slow DOM parser.  I could make it a lot faster if I used expat or something else.  I just use libxml2 because it's easy and included with OSX (and I only parse the data once before storing it in a different format).

My code for processing the XML is tiny compared to the code I had for reading the old format.  With the XML I can lean on libxml2, while the old format required me to do all the parsing myself.  I know which I'd rather use.

If your parser chokes, use a better parser.  If your parser is too slow, use a faster parser.  If your code is too complex, you aren't using the libraries properly.

Your using a mac so your points are invalid.  Mac parsers probably handle it better, I dunno, I never touch the things. 

[cuavas]

Excuse me, but think before you post.  libxml2 is not a Mac parser, it is a cross-platform XML parser.  It was originally part of the Gnome and runs on just about anything.  Try going to http://www.xmlsoft.org/ and reading some of the documentation.  expat is also cross-platform.  I expect both these parsers would perform better on Intel hardware than PowerPC due to the higher clock speeds of the chips.

C is a cross-platform language so my comparison of code complexity is also valid.  I would have to write the same amount of crap in C to parse the old info format on Windows as I do on a Mac.  The API to libxml2 is the same on Windows as anything else, so the code to use it would be just as simple.

The only difference is that I would have to include a copy of libxml2 with the distribution under Windows because the library isn't included with the OS.

[Howard_Casto]

I don't mean to offend but think before you post.  You never mentioned what you were using to parse xml.  How was I supposed to know you were using libxml2? 

Also the interface to said libraries is undoubtedly different on various platforms and languages. 


Regardless, if you'll read above before you started I said "I'm done complaining." I've got a parser setup for myself at this time.  It's cludgy and even with sax's help it's noticably slower than listinfo parsing, but it'll work (for now) so I'm happy. 

Also, due to the new control stuff added by rebel, I can add a new "generic output" feature to j5, so at least something good came out of this bad decision.

[cuavas]

Maybe if you actually read my original post, you'd realise that I said "I am using libxml2 to parse the listxml output, which is a slow DOM parser."  Your assertion that my comments are invalid or irrelevant because I am am using some kind of "Mac parser" is what I was objecting to in the second post.

The thrust of my original post was that the comments in this thread about XML taking minutes to parse and being more difficult to write code to interpret are silly.  There are parsers that will eat it quickly, and because you can use an off-the-shelf parser, you require less complex code - the hard work has already been done in writing the libraries.

[Minwah]

Added multiple input controls in -listxml output for games with more
than one type. Added pedal control type. Added more info on analog
controls as defined in the driver: minimum, maximum, sensitivity,
keydelta, and reverse. This required the 'control' attribute in the
XML to be moved into an element. There can now be more than one
'control' element in the input secction. [uRebelScum]

This does sound good

Going OT but does anyone know if sensitivity etc. can be set (& work) in ctrlr files?  Last time I tried (~v0.101) this didn't work...

As for XML, I don't like it but as we have no choice no point moaning about it...

[Buddabing]

XML parsers don't have to be slow. My XML parser takes 2-3 seconds on my crappy 1 GHz laptop.

Keep in mind that a special-purpose XML parser written just for the -listxml output will always outperform a general-purpose XML parser like expat.

It is important to me that other people's code runs well. Therefore, once v0.107 comes out, I will write some code to make my listxml parser available as a DLL, so that it will be usable to any Windows progam. I will also make it available as a plain library for linux users.

[youki]

Keep in mind that a special-purpose XML parser written just for the -listxml output will always outperform a general-purpose XML parser like expat.

Totally correct ,  that the solution i often choose in my job when i have performance and memory issue with parsers.
But you can really do that, if you are sure that the XML won't change to often.   Otherwize each time one XML object or a property will be added, you will have to update your Parser.  (mainly if you go very far in optimization).

I will write some code to make my listxml parser available as a DLL

Great!

[Minwah]

I will write some code to make my listxml parser available as a DLL

Great!

Ditto!

[krick]

I'm almost finished writing an XLS transform stylesheet that transforms the mame XML output to the info format.

I've never written a single XLS file before and this has only taken me about a day to learn.


You can get windows binaries (DLLs) here for the XSLT transformation processor and other libraries that it's dependent on...

http://www.zlatkovic.com/libxml.en.html


I'll put the XLS on my website when I'm done...

http://mame.3feetunder.com/xml2info/

[Howard_Casto]

XML parsers don't have to be slow. My XML parser takes 2-3 seconds on my crappy 1 GHz laptop.

Keep in mind that a special-purpose XML parser written just for the -listxml output will always outperform a general-purpose XML parser like expat.

It is important to me that other people's code runs well. Therefore, once v0.107 comes out, I will write some code to make my listxml parser available as a DLL, so that it will be usable to any Windows progam. I will also make it available as a plain library for linux users.

It'd be appreciated.  Actually that's what I've discovered myself and it was kind of the point to this whole thread.  The "off the shelf" parsers as it were are crap for parsing listxml. 

What I'm using now is sax with a custom event handling class that ignores trees and nodes and all that crap and just scans the document on a element by element basis.  Since the original listxml grouped things similarly, I can just look for key names and go from there. It gets quicker as I tweak it further, but the fact that you can get good performance just by running listxml through joe-blow's pre-built parser is just plain false.  There are too many branches to our little tree.   



XLS is also compelling.  I looked into it and honestly the syntax to make a template was beyond me.  I got confused and left it.  So good work on that one krick. 


Reagardless, since I've been forced to translate the data myself, I probably won't be converting it back to listinfo just to have it converted for my needs again. I'll just put a special case function in for mame, which isn't what I like to do, but it'll keep things speedy.  I think the other methods are important though, manily because nobody will want to use my crappy code and aside from dat-util, many of the dat managers are going to be broken in the create your own dat dept. 


Minwah, about the ctrlr files, they have had "issues" like this for a while now, and it needs to be addressed.  Something I discovered the other day is that you can no longer re-map the special playchoice 10 buttons anymore via ctrlr files.  Before they werre mapped to service1-service4.  Now they are all mapped to "service" with a different bit mask offset for each input.  This is ok for cfg files, but since ctrlr files don't support bitmasks you can't remap them anymore.  For that matter you can't remap any custom input.  The ctrlr files need to be changed not only to support analog values, but bit offset values as well.  I honestly don't understand why they aren't now as cfg files are basically the same thing and they do.  Regardless, it's a little beyond me in the scope of what I can change in mame.  It's probably something Aaron himself would have to do. 

[krick]

XLS is also compelling.  I looked into it and honestly the syntax to make a template was beyond me.  I got confused and left it.  So good work on that one krick. 

I've got the whole thing working now, except for one game "phrcraze".

One of the dipswitch value names has quotes in it...  Topic "8"  ...GRRRRRR!!!!!!

In XSL, there's a function that replaces any single character for any other character in a string, however, I need to replace a quote character ["] with an escaped quote character [\"], which itself is two characters.

I've found some XSL string replacement code but I'm still trying to understand it.

Incidentally, the whole transformation takes about 3 seconds using the command line tools I referenced in my other post.

[Pi]

I'd like to point out that DatUtil by Logiqx (www.logiqx.com) can transform MAME's XML output to the old datinfo format, keeping most of the attributes. I don't know how well does it currently keep those attributes, but you can try and propose the necessary changes to him.

[krick]

I've finished the first version of the XSL Transformation.
You can grab it with a command line XLST processor from here...

http://mame.3feetunder.com/xml2info/

You run it at the command line like this...

xsltproc xml2info.xsl yourfile.xml > yourfile.info


EDIT:  found a bug already and just uploaded the new version.

[Howard_Casto]

I'd like to point out that DatUtil by Logiqx (www.logiqx.com) can transform MAME's XML output to the old datinfo format, keeping most of the attributes. I don't know how well does it currently keep those attributes, but you can try and propose the necessary changes to him.

Yeah it works, but it seems to only keep data relevant to crc checking.  He's always good about updating it though, the issue comes with the rommanagers themselves, which aren't updated as often. 

[krick]

yeah figured that out and promptly deleted the whole mess as it isn't really needed if you know the xml version. 

Actually, you can't just delete the DTD.  The DTD defines default values for some attributes so they don't need to be explicitly spelled out in the actual XML data.  See "runnable", "status", etc...

[Howard_Casto]

I haven't ran into any issues yet, with or without the dtd. 

I suppose it depends on what you are doing to the data.

[krick]

I've updated my xml2info XSL Transformation again.  This time, it's just a lot of cleanup and re-formatting but it's FAR easier to follow now.  This is what a one day crash course in XSLT will get you.

As usual, anyone who's interested can get it from my xml2info page...

http://mame.3feetunder.com/xml2info/

[Howard_Casto]

Added multiple input controls in -listxml output for games with more
than one type. Added pedal control type. Added more info on analog
controls as defined in the driver: minimum, maximum, sensitivity,
keydelta, and reverse. This required the 'control' attribute in the
XML to be moved into an element. There can now be more than one
'control' element in the input secction. [uRebelScum]

This does sound good

Yes... this is the good news.  And as I'm always one to turn a negative into a positive:

http://fe.donkeyfly.com/forum/index.php?topic=119.msg1342#msg1342

 

[Minwah]

Yes... this is the good news.  And as I'm always one to turn a negative into a positive:

http://fe.donkeyfly.com/forum/index.php?topic=119.msg1342#msg1342

 

Good stuff

Aside from actually parsing the stuff from -listxml, this is gonna slightly mess up my filtering in Mamewah a touch...having multiple control types.  I might just join them into one string for the sake of getting it to work again quickly...

[Minwah]

I've updated my xml2info XSL Transformation again.

Works very well as far as I can tell...thank you

[krick]

Aside from actually parsing the stuff from -listxml, this is gonna slightly mess up my filtering in Mamewah a touch...having multiple control types.  I might just join them into one string for the sake of getting it to work again quickly...

In my XSL transform, I just grab the first display and the first control and ignore the rest because that produces output that matches the old listinfo format.  Of course, if you want to offer proper filtering by number of screens or types of controls, then you'll have to do something different.

The XSL transform is easy enough to modify to output multiple screen items and input items if you want it that way.  Or you can just do the right thing and write the XML parser. 


Oh, and here's a tip for anyone parsing the XML:

If you want to determine if a game is vertical, you can do it like this...

convert the "rotate" attribute to an integer (r), then...

boolean vertical = ( ( ( r / 10 ) mod 2 ) != 0 )

[Howard_Casto]

what I do with the control types is I merge them together and then slap the input element on the end.... the result looks exactly the same as the ol listinfo output except now you cna have strings like

joy8way trackball buttons 2 coins 2 players 2 instead of only the first control


As for display types I just ignore all the data but the first one.  I dare you to find a multi-monitor game in which the secondary monitor runs at a different res.  The vertical and horizontal deal isn't a big thing.  Horizontal games are gonna be 0 or 180 (those odd-ball games that project through a mirror) vertical is everything else.  What does stupify me is the fact that now the dimensions aren't rotated with the native orientation.  I can't see the logic in that one myself. 

[krick]

As for display types I just ignore all the data but the first one.  I dare you to find a multi-monitor game in which the secondary monitor runs at a different res.

MAME currently has a few "multi-display" games where the second and third display are small monochrome LCD screens.  They definitely have different resolutions and I'm not really sure about the refresh rate but I assume it's probably not the same.

In the future, I think you'll see things like single line LCD readouts and seven segment LEDs showing up under the "display" tag too.

[Howard_Casto]

I'm gonna need some examples, cause I haven't run across them.

[krick]

I'm gonna need some examples, cause I haven't run across them.

Thanks to the magic of XSL...

housemnq:
   display  -  width:512  height:224
   display  -  width:480  height:64
   display  -  width:480  height:64
housemn2:
   display  -  width:512  height:224
   display  -  width:480  height:64
   display  -  width:480  height:64
livegal:
   display  -  width:512  height:224
   display  -  width:480  height:64
   display  -  width:480  height:64
bijokkoy:
   display  -  width:512  height:224
   display  -  width:480  height:64
   display  -  width:480  height:64
bijokkog:
   display  -  width:512  height:224
   display  -  width:480  height:64
   display  -  width:480  height:64
mt_beast:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_shar2:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_stbld:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_ggolf:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_gsocr:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_asyn:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_shnbi:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_aftrb:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_tfor2:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_astro:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_lastb:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_wcsoc:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_tetri:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_gng:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_shang:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_gaxe:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_mystd:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_revsh:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_parlg:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_tgolf:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_tlbba:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_cols:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_eswat:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_smgp:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_mwalk:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_crack:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_arrow:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_astrm:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_bbros:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_sonic:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_sonia:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_fshrk:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_gaxe2:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_stf:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_mlh:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_kcham:
   display  -  width:256  height:192
   display  -  width:320  height:224
mt_soni2:
   display  -  width:256  height:192
   display  -  width:320  height:224

...there may be more.  I just printed out the ones where the with on the first display isn't equal to the width on the second display.

[Howard_Casto]

thanks... had the same idea though

yeah i just went through a filter myself and tried every single dual screen game (ugh)

The one's the use the lcds are japanese mahjong porn, so nobody really cares about those.  Also the extremely squased aspect would make it difficult to display on multiple monitors. 


The mega tech are a different res, but they are both stretched to the same aspect. Apparently the mt cab's selection display was smaller, and they didn't see the need to use hi-res output on it. 

One I did run into that seems really odd, even though mame reports it as bothscreens being the same is :

rocknms (Rockin' Mega Session)

 The top monitor appears to be horizontally oriented while the bottom is vertically oriented.  This isn't reflected in the output, probably because a custom layout had to be made to show both screens in the proper layout on a dual screen monitor. 


My point is the display stuff seems to be a tad inconsistant.  It's also rather hard to tell anything from the data.  So for now, until the fields are fleshed out a little more (such as display number or display position) it might be best to leave all but the first (which is always the primary display) out. 

[u_rebelscum]

Coming in late again....

Added multiple input controls in -listxml output for games with more
than one type. Added pedal control type. Added more info on analog
controls as defined in the driver: minimum, maximum, sensitivity,
keydelta, and reverse. This required the 'control' attribute in the
XML to be moved into an element. There can now be more than one
'control' element in the input section.

Do you guys have anything else that you want added to the listXML input controls data set? 

Things I'm considering of:
- Krick mentioned cases where the players have different controls. 
- The (v)2/4/8-way, (v)dial & (v)paddle types handle the single X, single Y, and both X & Y cases, but the rare cases where only one axis of the analog stick or trackball (DOT) is not. (Opps, the vertical dial and paddle types are not checked for, either.)
- Krick also suggested putting the buttons as another control type.  I'm thinking as it's own element, but only if it needs something extra like:

• Show if different players have a different number of buttons. (see examples)
• Show if the start button exists in the driver.  Most games have start buttons, others do not although sometimes mame fakes it by mapping a fake one to (another) the game's button 1.  But then there's the two player alternating games with more starts than players (DOT), hmm...

The players could have the controls item under them, or vice versa.  I'm thinking put the players sub the controls, with the default player to "All", leave the start buttons item out and add the axis item to replace the special "v---" types.  So I'm thinking, as examples:

For normal two player, one button & one 4-way per player:

Code: [Select]

<input players="2" coins="2">
<buttons number="1"/>
<control type="joy4way"/>
</input>
For two player, two buttons & ADstick for player 1 & four buttons for player 2, no start buttons:

Code: [Select]

<input players="2" coins="2">
<buttons number="2" player="1" start="No"/>
<buttons number="4" player="2" start="No"/>
<control type="stick" sensitivity="100" keydelta="10" player="1"/>
</input>
Basebal2 (IMO, the buttons should be emulated more like above example, but that's for another discussion):

Code: [Select]

<input players="2" coins="2" tilt="yes">
<buttons number=6 player="1"/>
<control type="stick" sensitivity="100" keydelta="10"/>
</input>
DOT:

Code: [Select]

<input players="1" coins="2" tilt="yes">
<buttons number="4"/>
<control type="joy8way"/>
<control type="dial" sensitivity="50" keydelta="10" reverse="Yes"/>
<control type="trackball" sensitivity="100" keydelta="10" axis="Y"/>
</input>

An alternative is putting the control sub to the player (Basebal2 example):

Code: [Select]

<input players="2" coins="2" tilt="yes">
<player number="1"/>
<buttons number=6 player="1"/>
<control type="stick" sensitivity="100" keydelta="10"/>
</player>
<player number="2"/>
<control type="stick" sensitivity="100" keydelta="10"/>
</player>
</input>
However, since most games have the same inputs and with the players number defaulting to "All", though, it would be not be nice IMO (DOT example):

Code: [Select]

<input players="1" coins="2" tilt="yes">
<player>
<buttons number="4"/>
<control type="joy8way"/>
<control type="dial" sensitivity="50" keydelta="10" reverse="Yes"/>
<control type="trackball" sensitivity="100" keydelta="10" axis="Y"/>
</player>
</input>Somehow the players attribute in the input could be moved to inside the players element, but I don't know how to put "total number of players" vs "this player number" clearly.


Better suggestions?  Improvements?  Additions?

[SirPoonga]

Heh, urebel, you know everything you mentioned is how controls.dat works   We assume controls are for all unless player2 or greater is specified.  So I would agree with your last two examples.

If you haven't looked at the controls.dat format take a peek, it might give ideas.

[Howard_Casto]

Something that's key that is missing, is the status of play... if it's alternating or not.  This tells you if it's a two player game with one set of controls (like pacman) or a two player game with two sets of controls (like street fighter).  With that, the basic data is there. 


Also I don't think controls.dat specifically is written for this, but this is how I handle those rare instances like dotron you were mentioning in J5:

If all of the controls are used for a control type then the control type constant is used  (like trackball or joy8way) but if some aren't used then each individual input is treated as a control  (control p1_trackball_Y+ and controls p1_trackballY- or whatever)  of course this is a bad example as for analog controls you can just define the axis and up and down is assumed, but for a digital example you would give both directions. 

I really like the new way you've arranged it btw.  Any chance of you fixing the missing elements in the ctrlr files?  Like allowing you to reverse the axis, set the sensitivity or define a bit offset for those odd-ball inputs?

[krick]

Heh, urebel, you know everything you mentioned is how controls.dat works   We assume controls are for all unless player2 or greater is specified.  So I would agree with your last two examples.

If you haven't looked at the controls.dat format take a peek, it might give ideas.

I'd really like to see MAME document the controls better and I think that fits in with MAME's goals.

I looked at the controls.dat XML version and I think that it's a bit "string" heavy but definitely along the lines of what is needed in MAME.  I definitely like each button having its own element so you can attach attributes like "Fire" and "Hyperspace" to them.  Button names/functions should definitely documented in MAME.


The biggest problem I see is convincing the MAME devs to go along.

Check out this thread to see what I mean...

http://www.mameworld.info/ubbthreads/showthreaded.php?Cat=&Number=79110&page=&view=&sb=5&o=&fpart=1&vc=1

It appears that Haze is against it, and R. Belmont seems indifferent.  Based on my experiences with Aaron, I think that he will be for it.  I don't really know enough about the other devs to know how they'd feel.

[Howard_Casto]

Ehh I dunno about the internalizing of controls.dat..... things get corrected and altered all the time.  Plus we get a lot more volunteers due to how easy it is to make a submission.  If someone had to modify the mame source to add and entry, that'd be rough.  Now having mame be able to read an externalized controls.dat into itself and label the buttons in the ui... that seems like a good idea.  It should be fairly doable considering we use the same input constants as mame. 

[krick]

Code: [Select]

<input players="1" coins="2" tilt="yes">
<player>
<buttons number="4"/>
<control type="joy8way"/>
<control type="dial" sensitivity="50" keydelta="10" reverse="Yes"/>
<control type="trackball" sensitivity="100" keydelta="10" axis="Y"/>
</player>
</input>Somehow the players attribute in the input could be moved to inside the players element, but I don't know how to put "total number of players" vs "this player number" clearly.

stupid tabs aren't working but I think you can figure out what I mean...

There's two ways that I think you could go.

The first way has a "player" tag that contains the controls for a player.  If both players share one set of controls, then there is only one "player" group...

For Defender...

Code: [Select]

<input players="2" format="alternating" coins="2">
<player>
<control type="button" name="Start"/>
<control type="joy2way" axis="y" name="Move"/>
<control type="button" name="Fire"/>
<control type="button" name="Thrust"/>
<control type="button" name="Smart Bomb"/>
<control type="button" name="Hyperspace"/>
<control type="button" name="Reverse"/>
</player>
</input>
For Smash TV...

Code: [Select]

<input players="2" format="simultaneous" coins="2">
<player>
<control type="button" name="Start"/>
<control type="joy8way" name="Move"/>
<control type="joy8way" name="Fire"/>
</player>
</input>
For a made up game where player 1 and player 2 have different controls and
the fire button doubles as the start button...

Code: [Select]

<input players="2" coins="2">
<player>
<control type="joy8way"/>
<control type="button" name="Fire"/>
</player>
<player>
<control type="trackball" sensitivity="100" keydelta="10" axis="y"/>
<control type="button" name="Fire"/>
</player>
</input>
Or we could do something completely different and try to just document the
physical controls attached to the machine and use the name to indicate how
they are used by the players:

For Defender...

Code: [Select]

<input players="2" format="alternating" coins="2">
<control type="button" name="P1 Start"/>
<control type="button" name="P2 Start"/>
<control type="joy2way" axis="y" name="Move"/>
<control type="button" name="Fire"/>
<control type="button" name="Thrust"/>
<control type="button" name="Smart Bomb"/>
<control type="button" name="Hyperspace"/>
<control type="button" name="Reverse"/>
</input>
For Smash TV...

Code: [Select]

<input players="2" format="simultaneous" coins="2">
<control type="button" name="P1 Start"/>
<control type="button" name="P2 Start"/>
<control type="joy8way" name="P1 Move"/>
<control type="joy8way" name="P1 Fire"/>
<control type="joy8way" name="P2 Move"/>
<control type="joy8way" name="P2 Fire"/>
</input>
For a made up game where player 1 and player 2 have different controls and
the fire button doubles as the start button...

Code: [Select]

<input players="2" format="simultaneous" coins="2">
<control type="joy8way" name="P1 Move"/>
<control type="button" name="P1 Fire"/>
<control type="trackball" name="P2 Aim" sensitivity="100" keydelta="10" axis="y"/>
<control type="button" name="P2 Fire"/>
</input>

I think that this second option fits more in with what MAME is about: documenting the hardware.

[Howard_Casto]

That creates a ton of bulk though.... again  controls.dat has already dealt with this problem. 

You don't brecket stuff by player.. you read two flags

alternating and mirrored controls. 


Alternating means that players take turns.  Mirrroed means that although each player has his or her own control set, they all have the same control set. 

The best example I can get is tmnt. 

All you need to say is that it's a 4 player simultaneous play game with mirrored controls. 

With this data it's only necessary to post the controls for player 1 (joy8way with two buttons with the labels attack and jump).  Keeps a large list from data from becoming an insainely huge one. 

Again though, I do not think it makes a lot of sense to include this data internally with mame. 

[krick]

Again though, I do not think it makes a lot of sense to include this data internally with mame. 

Most of the data is already in MAME in the form of input definitions.  We're trying to come up with some way of dumping the internal input definitions into the XML so it can be more useful.

For the first pass at this output, we should just concentrate on dumping all the inputs, similar to the way all the dipswitches are dumped.  Then on the second passs, we can look at adding some grouping and/or organization information into MAME to enhance the XML dump.

I'm still partial to listing every separate button as a control.  I'm debating whether we should list each directional input as well and group them with a "joystick" tag if appropriate.  Something like this...

For Defender...

Code: [Select]

<input players="2" format="alternating" coins="2">
<button name="P1 Start"/>
<button name="P2 Start"/>
<joystick>
<direction name="Up">
<direction name="Down">
<joystick>
<button name="Fire"/>
<button name="Thrust"/>
<button name="Smart Bomb"/>
<button name="Hyperspace"/>
<button name="Reverse"/>
</input>
Or maybe...

Code: [Select]

<input players="2" format="alternating" coins="2">
<button name="P1 Start"/>
<button name="P2 Start"/>
<joystick type="2way" axis="y" name="Move"/>
<button name="Fire"/>
<button name="Thrust"/>
<button name="Smart Bomb"/>
<button name="Hyperspace"/>
<button name="Reverse"/>
</input>

[Howard_Casto]

well... actually it isn't in mame.  Mame doesn't use manual/cpo accurate label descriptions. 

Asteriods is a prime example:

First off some of the labels are slightly off from the official cpo labels.  Secondly it's mapped to a joystick, which is wrong as asteroids actually used pushbuttons.  (This is fixed in controls.dat by the direcitonal buttons constant.)

So it's less of a documeting thing in mame and more of a "this description was added because the button mappings are confusing and the full label name was altered to fit in the ui."  Not that they aren't appreciated mind you, I'm just saying they don't have anything to do with the documentation aspect or else they'd be more accurate.   


It's a waste of space to output, say directions for a joystick unless it's a special case.  Simply defining defender as a vjoy2way (which is a mame control constant) tells you that there is an up and down and of course they would be labeled up and down.  There isn't a need to define each direction. 

I'm not sold on the individual button entries either.  Some of the classics have captions for the button function, but most don't.  Around 90% of the listxml file would be "<button name=p1_button1/>" ect....

I actually liked it better early on in the xml cfg file's history when all the possible inputs were printed in the cfg, even ones that weren't remapped.  Made more sense to have it there for me as most (note I said most) people could care less about that data. 

While we are on that tangent...... Somebody really needs to modify the cfg files so that they always print hardcoded remaps even if they haven't been changed by the user.  It was in there for a while and it was great... j5 and those types of viewers potentially were 100% accurate as even keys that were internally remapped from their defaults could be read. 

Makes sense to me.... a user always assumes that button 3 on a game is space and it throws them for a loop when they find out the driver author remapped that button to "a" for this particular game or what have you.  So that kind of craziness definately needs to be documented somehow. 

[Minwah]

I like much of the stuff mentioned here...I think the key is getting as much useful stuff we can, but not being so over complicated that the MAMEdev don't accept it.  IMO the multiple control types is a huge step forward already, combined with no. buttons is pretty decent info for a good number of games.

I like the second system shown by u_rebel...if the issues with the ctrlr files (HC & I have touched on) were sorted too then I think we should be pretty happy fella's

[SirPoonga]

Krick, what do you mean by string heavy?  Just curious as I am always looking for ways to improve the project.

When I designed the xml format I went with the suggestions most books give.  Use attributes as they are faster to parse with some parsers, etc...  times have probably changed and it may need improvement.

Also one thing controls.dat does is assocciate button with a control.  Like the dotron triggerstick has two buttons on it that are player 1's button 1 and 2.  The other thing is there is more resolution on type of controls.  Like you can sort by 360 degree steering wheel or spinner instead of just dial.  That type of stuff I don't see being internalized in mame.  However, I could see that controls.dat be treated like other dat files as an add-on.

HowardC, going to bring up the service button issue? *hint hint*

[krick]

Krick, what do you mean by string heavy?  Just curious as I am always looking for ways to improve the project.

No need to worrry, I didn't make myself clear.  I meant that the content of the controls.dat includes a lot more "text" than the MAME devs would be comfortable with in the -listxml output.  Mainly the "miscDetails" content, but I'm sure we'd get resistance to adding functional labels for the buttons as well. I personally think button functions are a really important part of documentation but Haze (and possibly other devs) worry that all the extra text will bloat the source and make the EXE a lot larger.

I think the ultimate solution for MAME is to have an XML file for each game that defines all the control, display, and dipswitch, roms, etc... information.  These files can be either parsed in at runtime (probably not likely) or translated into actual code at compile time.  If you look at the source for most of the drivers, they look like just a bunch of macros for the most part.  There's no reason that that information can't be generated from XML files.

The upside is that the XML files can include more information than MAME actually uses so it provides a convenient place for further documentation.

[AaronGiles]

I personally think button functions are a really important part of documentation but Haze (and possibly other devs) worry that all the extra text will bloat the source and make the EXE a lot larger.

That's a bit of a mischaracterization. The reason is that right now in many cases there can be one set of input port definitions for multiple games on a system. If you started adding descriptions for the buttons to each port, then you have to create separate input port defs for each game in a driver. So neogeo.c would be polluted with 200+ input port definitions, etc. This also hides all the commonalities of the inputs and makes it difficult to understand how one set of inputs maps differently from others. It really hinges on source code readability.

That said, I think there are ways to fix this. Basically, get rid of the existing way of naming input ports, and have some mapping that says IPT_BUTTON1 = "Jump" for this game. And have that information be separate from the input port definitions. Which is basically controls.dat, as far as I understand it. Whether or not that data should be internal to MAME is a good question. Some of the devs want to push some of MAME's current data out of the EXE and into other files (XML, sorry Howard), while others like to have the "canonical" data stored in the EXE where it is guaranteed to be verified through a known process.

Plus, input port names technically don't matter at the PCB level, which is officially what MAME focuses on emulating. But if we were super sticklers about that, then even joysticks would be mapped as raw switches, the DIP switch information wouldn't be provided, and we wouldn't provide any control mapping. I can hardly claim consistency on anything in MAME (nor would I want to).

[krick]

I personally think button functions are a really important part of documentation but Haze (and possibly other devs) worry that all the extra text will bloat the source and make the EXE a lot larger.

That's a bit of a mischaracterization. The reason is that right now in many cases there can be one set of input port definitions for multiple games on a system. If you started adding descriptions for the buttons to each port, then you have to create separate input port defs for each game in a driver. So neogeo.c would be polluted with 200+ input port definitions, etc. This also hides all the commonalities of the inputs and makes it difficult to understand how one set of inputs maps differently from others. It really hinges on source code readability.

I was just simplifing the issue for purposes of discussion but that's basically what I meant when I said "bloat the source".   The obvious side effect being a loss in readability.

That said, I think there are ways to fix this. Basically, get rid of the existing way of naming input ports, and have some mapping that says IPT_BUTTON1 = "Jump" for this game. And have that information be separate from the input port definitions. Which is basically controls.dat, as far as I understand it. Whether or not that data should be internal to MAME is a good question. Some of the devs want to push some of MAME's current data out of the EXE and into other files (XML, sorry Howard), while others like to have the "canonical" data stored in the EXE where it is guaranteed to be verified through a known process.

Plus, input port names technically don't matter at the PCB level, which is officially what MAME focuses on emulating. But if we were super sticklers about that, then even joysticks would be mapped as raw switches, the DIP switch information wouldn't be provided, and we wouldn't provide any control mapping. I can hardly claim consistency on anything in MAME (nor would I want to).

I'm definitely in the camp that wants to push the extraneous data out of MAME.  All input ports/switches/buttons/etc... should be just numbered/enumerated using some democratically agreed upon system.  Then external XML files for each game could define what these ports/switches/buttons "mean" for that game and how they relate to each other.  The MAME UI can then look at these files to get the text to display when the menus are shown.

If you want, each version of mame can include a zip archive with all the latest XML game mapping files.  Or if you really want to get fancy, you could probably put them in a folder, zlib compress the folder, and imbed them into the exe itself like a resource.  That way, they're not physically mixed in with the source code, but they are still "canonical" data stored in the EXE.

[SirPoonga]

And have that information be separate from the input port definitions. Which is basically controls.dat, as far as I understand it. Whether or not that data should be internal to MAME is a good question. Some of the devs want to push some of MAME's current data out of the EXE and into other files (XML, sorry Howard), while others like to have the "canonical" data stored in the EXE where it is guaranteed to be verified through a known process.

Yep, that's a simple summary of controls.dat.  Not only is it for labelling controls it is for higher resolution sorting of controls.  Like I said earlier, you can sort by 360 steering wheel instead of dial (which would include the spinner games too).

I agree is not putting the info in mame.  I like how history.dat is outside of mame and optional.
It's also makes it easier for non-programmers to add info or correct it.

Krcik, yeah, I figured that was what you meant.  miscDetails is there to be the catch all since games do not use standardize control layouts   Some of the games are even too wordy for my liking but there is no other good way to document the controls in an easy and consistant way to parse.

[Howard_Casto]

You and me both.  Control constants are sometimes "bastardized" for lack of a better word to piece together inputs for odd-ball games with strange control schemes.  It makes documenting our end of stuff very difficult.  On the other hand, I wouldn't want to subject mame users to the horror of opening up the ui only to find "input 1-input 12" and have those guys try to figure out which ones are joystick directions and which ones are buttons.  

If ctrlr files supported bitmasks and all that fanciness it wouldn't be that much of an issue... an external doc could bind itself to the port "address"  and it wouldn't matter if mame called that port "p1_button1" or "p1_joystick_up."


Btw Aaron, while I'm completely in agreement with you on this one, neogeo is a bad example.  A person could document the game's control functions, but what we do (since it is easier to verify and standardize) is to document the official label given on the cpo or if that isn't available the instruction card, manual ect.... in game descriptions of a control are a last resort.  Anyway, the "official" labels for every single neogeo game with the exception of the prototype game with the spinner and irritating maze are A, B, C, D  just like the mvs cpos.  So their label descriptions could be stored at the driver level with over-rides in those few exceptions.  Even neogeo games that don't use all 4 buttons are most likely going to be in a mvs cab, and since the main pcb still has outputs for 4 buttons, the constant still applies.  

Of course there are "classics" drivers that would really be effected.  I driver in which every single game has a 4 way joystick and a single button would now have to have seperate input port definitions.  

Regardless, what you might have been getting at anyway.... a real hassle comes once the listxml is generated and the "bulk" is increased considerably.  Mame is gonna output that A, B, C, D grouping for all 200+ neogeo games.  An external file, like the controls.dat has the benefit of not having to output replicated data.  We don't have entries for clones (except bootlegs that used hacked controls) and when a system has  a universal layout, like the neogeo, we simply make a driver entry and consolidate all of that mess into just a few lines.  Large xml files take a while just to load.  

While the new rom flag output helps a  person that needs just the basics from the entire listxml file is gonna be slowed down.  I don't think anyone would mind this if the data was useful.  In this case, however, I've gotta agree that unless there was some way to toggle the captions on and off, port descriptions generally just confuse things at the mame level, and in the case of documentation it simply isn't possible as a great deal of "official" descriptions won't even fit in the ui screen.   That's why I suggested some sort of external file that could be read in at the ui level only.  People that want to use it could, people that don't won't.

[very slight rant]
Btw krick... I've gotta laugh a little at your string heavy comment.  We are documenting captions and labels.  What did you expect that to be short?  Even the ini version which saves space by not using all the xml tags is still rather huge.  Save maybe removing misc details I don't see how you could possibly make it smaller if you are really trying to accurately document.  Those are the researched official labels and physical controls, there isn't any way to shorten them without making them unofficial and thus you aren't really documenting anything are you?  If anything they are string light.  We could put the driver and clone names in each entry to aid in parsing, but since that can be obtained by asking mame we left them out.  It is the absolute bare minimum format you can document the controls and their labels with.  Actually in some ways it has provento be too minimal and sirp and I are considering expanding the format slightly to add a few misc stuff.  I'm not picking on you, I just wanted to explain.  Controls.dat doesn't use labels that some guy made up that help you understand what's going on in a game, each entry is researched to get the most arcade accurate description of the controls as possible.  If you were to read an entry while looking down at the real dedicated arcade machine, it would precisely describe the panel you are looking at and the labels printed on either the control panel or the instruction card on the bezel.  I'm not saying that the format we have setup would have to be used in mame, but the data would, as that is the actual data.  
[/very slight rant]

[krick]

[very slight rant]
Btw krick... I've gotta laugh a little at your string heavy comment.  We are documenting captions and labels.  What did you expect that to be short?  Even the ini version which saves space by not using all the xml tags is still rather huge.  Save maybe removing misc details I don't see how you could possibly make it smaller if you are really trying to accurately document.  Those are the researched official labels and physical controls, there isn't any way to shorten them without making them unofficial and thus you aren't really documenting anything are you?  If anything they are string light.  We could put the driver and clone names in each entry to aid in parsing, but since that can be obtained by asking mame we left them out.  It is the absolute bare minimum format you can document the controls and their labels with.  Actually in some ways it has provento be too minimal and sirp and I are considering expanding the format slightly to add a few misc stuff.  I'm not picking on you, I just wanted to explain.  Controls.dat doesn't use labels that some guy made up that help you understand what's going on in a game, each entry is researched to get the most arcade accurate description of the controls as possible.  If you were to read an entry while looking down at the real dedicated arcade machine, it would precisely describe the panel you are looking at and the labels printed on either the control panel or the instruction card on the bezel.  I'm not saying that the format we have setup would have to be used in mame, but the data would, as that is the actual data.   
[/very slight rant]

I think my comment was misunderstood by all.  Sir Poonga originally mentioned controls.dat in response to u_rebelscum wanting suggestions on additions to the MAME XML output in the controls area.   I think what is done with controls.dat is awesome, however, I know that if anything was to be added to the MAME source code, it would have to be as minimally invasive and non-bloating as possible in order to be accepted by the devs.  In that respect, controls.dat *is* too heavy.  It doesn't make it invalid or unimportant, just more than the MAME devs would go for if it was incorporated into the source.  External XML files, however, put an entirely new spin on it.

[Pi]

Yeah it works, but it seems to only keep data relevant to crc checking. 

There's an option called -k to (quoting the manual) "keep as much information as possible from the source file". In the past it worked quite well, I'm not sure right now. But as I said, he's available for some requests, if time permits.

In any case, if people really care about xml2info, then someone should take over the source and maintain the project from now on... It'll be much better and cleaner than any other solution, IMHO.

[krick]

In any case, if people really care about xml2info, then someone should take over the source and maintain the project from now on... It'll be much better and cleaner than any other solution, IMHO.

I've been in the xml2info source a lot in the last week or two and let me tell you, it's not a clean solution by any stretch of the imagination.  The original author never anticipated having displays and controls nested the way they are now.  So to get it working at all with the new XML format, I had to resort to some MAJOR hacking and it's still not 100% as I cant' see any way to get the actual control type into the output due to the way the data is parsed.

The cleanest solution, and by far, the easiest to modify for future changes, is to use an XSL Transform:
http://mame.3feetunder.com/xml2info/

[Howard_Casto]

I think my comment was misunderstood by all.  Sir Poonga originally mentioned controls.dat in response to u_rebelscum wanting suggestions on additions to the MAME XML output in the controls area.   I think what is done with controls.dat is awesome, however, I know that if anything was to be added to the MAME source code, it would have to be as minimally invasive and non-bloating as possible in order to be accepted by the devs.  In that respect, controls.dat *is* too heavy.  It doesn't make it invalid or unimportant, just more than the MAME devs would go for if it was incorporated into the source.  External XML files, however, put an entirely new spin on it.

No we got it, but you keep mentioning that you want to add labeling into mame for the sake of documentation.  Well the only possible way you can do that is to get as detailed as controls.dat.  Anything else you could think to replace the current mame labels with isn't documenting anything.  As Aaron pointed out the constants in mame are just there to keep the user from getting confused and at the pcb level they are all just singular inputs. 


Btw, on the xml2info thing......  I've gotta agree, the proggie is dead.  If it isn't going to be officially in mame then it will never be kept up and thus it isn't as useful.  When it gets to the point where you can get more data from one format than the other then it's time to ditch the old format.  Again I don't like it at all, but if it isn't going to be in mame, where it can constantly be updated and have the same data as the xml version then it isn't very useful, with the exception of putting a "band-aid" on older front-ends and applications that aren't going to be updated. 

[headkaze]

This may be a stupid question, but why does Mame.exe generate this data in the first place? Why isn't this data stored in a separate file rather than generated on the fly?

[Howard_Casto]

Because this data already has to be stored for mame.  rom crc's are required, the driver relationship has to be known ect....

Adding some of the stuff others are suggesting would indeed be adding data just for the sake of data. 

[headkaze]

Because this data already has to be stored for mame.  rom crc's are required, the driver relationship has to be known ect....

Adding some of the stuff others are suggesting would indeed be adding data just for the sake of data. 

I still don't get why it's not all stored in an external file, surely Mame could just read an external file instead of having it embedded into the source.

[Howard_Casto]

Because, mame reads this data internally and uses it to run the games.  Games will not function without basic stats like rom crc's and driver, clone and parent relationships.  If it were an external file then people could potentially completely screw up mame.  Since this particaular data MUST be inisde mame for functionality's sake anyway, it might as well be outputted when listxml is called.