NO MORE LISTINFO in 84!

[Lilwolf]

For all frontend authors out there... I hope you made the move to xml a while ago.

 Removed -listinfo option now that xml2info produces fully correct output. [Aaron Giles]

I'm at work right now, so I can't test it... but seems like it might cause some major problems across all frontends.

Luckily I moved to xml a while ago... I was about to move back (for some weird reasons) but now I'm glad I didn't.

[SirPeale]

.84 was released today, for the non-knowing.

[SirPeale]

Just found that you can use the included tool, xml2info to convert that output to a -listinfo standard.

[SirPoonga]

I went to complain on mame.net.  I knew they would tend to lean towards listxml BUT I have found most of the xml parsers are slow.  So you have your choice of two slow ways to get info from mame, -listxml or  -listxml|xml2info.

I figured they'd go with the xml anyway as the listinfo is limitting in some cases.

[Minwah]

That plain sucks!  

[Silver]

argh! I ditched my xml code as it was so slow at parsing the xml (although partly cos I'm not that hot at coding either....)

might have to look at parsing the xml as a txt file as all the info will be there, just in a different format/location.

Sigh. ;-)

[Minwah]

Well at least usually the games list only has to be generated once per MAME version...

[Howard_Casto]

Xml blows!  I don't know why everyone jumped on the xml bandwagon.  

Sure it's easy to parse..... if your using an "internet" based langauge.  Everything else needs a custom routine wrote.  And as there is only one fe wrote in java, the rest of us are pretty much screwed.  
Also xml wastes space...... those hierarchial groupings in ascii text take up a lot more space than simply typing it. (Which is what the listinfo does.)  For anyone using vb and some of the other visual studio languages this is a nightmare as anything ascii reads slow into memory.  

Long story short.... as usual mame devs are completely out of touch with fe developers.  When listinfo first came out we complained the format was illogical and hard to parse... I guess their "solution" was to convert the whole mess to xml, which merely adds another step to the process instead of simplifying it.  

[Daniel270]

So which Frontends are now compliant with Mame .84?

[Minwah]

So which Frontends are now compliant with Mame .84?

Not sure exactly...if I understand SirP correctly, MAMEWAH should still work if you use a DAT File List Generation Method, and create a datfile from using 'mame -listxml|xml2info > mame.dat'.

[SirPoonga]

Minwah, yeah, that's what you have to do.  Annoying isn't it, first the xml file has to be generated, which is larger than listinfo was.  Then it has to be converted.  Time consuming.  An xml parser won't gain you that much time either unfortunately.

[Minwah]

Minwah, yeah, that's what you have to do.  Annoying isn't it, first the xml file has to be generated, which is larger than listinfo was.  Then it has to be converted.  Time consuming.  An xml parser won't gain you that much time either unfortunately.

Thanks for confirming...

IMO The MAME dev's should have left -listinfo in, even if all it did was '-listxml|xml2info'...

[PacManFan]

Yeah, the removal of the listinfo kind of blows. Since Kymaera already uses XML for everything else, it shouldn't be too difficult to get it to parse the listxml output directly.

-PMF

[ultrastyle]

ultrastyle has suffered the same fate as the rest of you. I was baffled for a little while as to why they would remove it and leave it in the supporting txt documents. Just to give us a headache I guess. Do you guys think they will be removing any other switches on us?

Chris

[SirPoonga]

There's alot of stuff that doesn't make it to the docs anytime soon after it happens.  Like for the ctrlr files there is a bunch of stuff that didn't make it to the std.ini file for sometime.

[Howard_Casto]

Dk is now compliant if you use the new listgen I have posted on the site.  I simply have mame convert it for me for the time being since it seems to be the easiest method.  

Ugh... as expected it adds about 2 minutes to the generation process and that's on a fast machine.  I can only imagine how long it takes on a slower one.  

So....

complain complain complain


complain complain

I'll quit when they put it back in!  

[SirPeale]

I must be the only one that is blessed by speed.  a -listxml only takes a few seconds, and conversion only a few seconds more.  Weird, as I've only got an AMD 2400+

[u_rebelscum]

I must be the only one that is blessed by speed.  a -listxml only takes a few seconds, and conversion only a few seconds more.  Weird, as I've only got an AMD 2400+

I wonder what XML parser Roman uses for ClrPro?  XML seems just as fast as listinfo was on my ghetto AMD 1 gHz box.

[Minwah]

Dk is now compliant if you use the new listgen I have posted on the site.  I simply have mame convert it for me for the time being since it seems to be the easiest method.  

Ugh... as expected it adds about 2 minutes to the generation process and that's on a fast machine.  I can only imagine how long it takes on a slower one.  

I just added '-listxml|xml2info' List Gen. Method to MW too....doesn't seem much slower to me.

From what I have done:

'mame -listxml > file' takes ~ the same amount of time as 'mame -listinfo > file'.  'mame -listxml|xml2info > file' takes ~twice as long as either (still only seconds).  From then on MW can't tell the difference, so from there on takes the same time (seconds again).

So it only really adds on a few seconds for me, around 3 or 4 on this P4 2.8 I just tested it on...

[SirPeale]

Here's a question: since a lot of you are adding -listxml|xml2listinfo to your frontends, are your frontends going to be backwards compatible with older versions?  MinWAH, you don't have to answer, I already know yours is.

[Lilwolf]

backwards compatibility is a very nice feature.

I still run old versions waiting for the 2006 rewrite.  Also nice since I have about 30 games that require analog+ or nonamemame... but they aren't updated as often... So I use those ONLY for those games.

(but again... jfront uses xml so it isn't an issue.)

[SirPoonga]

It's not too hard to be somewhat backward compatible.  You could make a config option for mame .84+ and mame 36final-.83.

The last several mame version (at least since listxml was added) has had xml2info.  Not sure how well it works in those version though.

[SirPeale]

I've been thinking of doing an ArcadeOS update, so that was my concern.  Just figuring how I should implement such a beast.

[lokki]

Been thinking about the same thing.
Probably config item.
List generation method default to legacy (if not set)
use -listinfo
else
use -listxml|xml2info

[Minwah]

I've been thinking of doing an ArcadeOS update, so that was my concern.  Just figuring how I should implement such a beast.

Orclord & some others at the ArcadeOS/MAMEWAH board would be VERY happy if you manage to do it...

[lokki]

I've been thinking of doing an ArcadeOS update, so that was my concern.  Just figuring how I should implement such a beast.

Orclord & some others at the ArcadeOS/MAMEWAH board would be VERY happy if you manage to do it...

Just took a quick look at the code. Creating a version that will work with MAME 84 should be pretty easy. Adding support for both the new and the old one will be a little trickier.

[Howard_Casto]

Here's a question: since a lot of you are adding -listxml|xml2listinfo to your frontends, are your frontends going to be backwards compatible with older versions?  MinWAH, you don't have to answer, I already know yours is.

And this is why I keep my fe completely modular.  The listgenerator is a seperate exe from the front end itself so to go backwards you simply use the old listgen.  

Not as big of a deal in my case though.... as my fe is designed for newer machines.  

[lokki]

Friday 9 Jul 2004
ArcadeOS 2.51!
by Peale

Since Mame (as of .84) no longer supports the -listinfo command, changes were made by Carlos Santillan regarding the new -listxml command. There's a new tag in arcadeos.cfg for legacy support:

#Legacy List Generation
legacygen=1

Set to '1' for Mame versions < .84 and '0' for versions > .84

http://www.mameworld.net/pc2jamma/

[Silver]

a quick comment on recent vb/xml fiddling:

I've been messing a fair bit with VB in recent times as I'm writing a little util to generate display files for an LCD screen, showing controls for each game. The info is being pulled from ControlsDAT project and listinfo. I started out on the XML track - partly just to teach myself. It all worked ok - just *horribly* slowly so I ditched it and was heading towards simply speedy txt parsing (with a little help from Minwah).

However, since 84 went xml i went back at it and have found its not that hard to get enormous improvements - i am now parsing the whole listinfo.xml natively, copying a bunch of info from each game into a series of string arrays each around 5000 entries big, all in around 3-4 seconds on a 2gig athlonXP.(My original program parsed the xml file as it went and took around 50minutes to run on a full romset - not pretty). This is not me being particulary clever - its just a matter of using the SAXX xml parser instead of the standard MS DOM one. The SAX parser is implemented by MS in the same bundle - MSXML v3 and up I think. It actually works in a very simlilar manner to parsing txt manually line by line (hence similar speed) except it does it element by element and you can select info based on the tags e.g. "gamename" so you don't even have to bother filtering the text with all the Instr/left/right/len commands.

Anyway, not sure if anyone is fussed anymore by the speed/annoyance penalty of xml2info but seeing as several frontends are VB based I thought I would mention it. If anyone actaully wants to look at the parse code I'm using thats fine - I can post it up or email it (not that there is much beyond a few IF...Then statements, and implementing the MS SAX parser).

u_rebelscum: clrmamepro uses the xerces parser which I believe is also SAX based - and i imagine an even better implementation.

the only penalty for this I can think off (apart from hassle) is that you have to reference the msxml in you vb project so may include the required files count by 1....

Silver

[Minwah]

a quick comment on recent vb/xml fiddling:

I've been messing a fair bit with VB in recent times as I'm writing a little util to generate display files for an LCD screen, showing controls for each game. The info is being pulled from ControlsDAT project and listinfo. I started out on the XML track - partly just to teach myself. It all worked ok - just *horribly* slowly so I ditched it and was heading towards simply speedy txt parsing (with a little help from Minwah).

However, since 84 went xml i went back at it and have found its not that hard to get enormous improvements - i am now parsing the whole listinfo.xml natively, copying a bunch of info from each game into a series of string arrays each around 5000 entries big, all in around 3-4 seconds on a 2gig athlonXP.(My original program parsed the xml file as it went and took around 50minutes to run on a full romset - not pretty). This is not me being particulary clever - its just a matter of using the SAXX xml parser instead of the standard MS DOM one. The SAX parser is implemented by MS in the same bundle - MSXML v3 and up I think. It actually works in a very simlilar manner to parsing txt manually line by line (hence similar speed) except it does it element by element and you can select info based on the tags e.g. "gamename" so you don't even have to bother filtering the text with all the Instr/left/right/len commands.

Anyway, not sure if anyone is fussed anymore by the speed/annoyance penalty of xml2info but seeing as several frontends are VB based I thought I would mention it. If anyone actaully wants to look at the parse code I'm using thats fine - I can post it up or email it (not that there is much beyond a few IF...Then statements, and implementing the MS SAX parser).

u_rebelscum: clrmamepro uses the xerces parser which I believe is also SAX based - and i imagine an even better implementation.

the only penalty for this I can think off (apart from hassle) is that you have to reference the msxml in you vb project so may include the required files count by 1....

Sounds good  To be honest I haven't even looked into the XML route since MAMEWAH doesn't seem to be hit too badly by the latest MAME, but I'd certainly be interested to take a look if I may - I'm going to have to do some pretty major core changes soon so it would be nice to way up my options.  Would you mind emailing me a snippet?

[Aceldamor]

Silver...how's that LCD piece coming? I'd like to test it out.

[Silver]

MinWAH:  I'll get something over to you in the next day or two. And thanks again for your inital help

Aceldamor: Yes I've been meaning to get alpha up for ages. Except I kept fiddling and would end up breaking something or other along the anway. Anyway, it seems to work ok on my system so I'll see if I can post a link up in the next day or two so people can give me some feedback on how it's hosed their system ;-)

...Also if we keep our fingers crossed some FE authors might drop in the code to optionally pass out a txt file on game launch so we won't have to bother with batch files....

[Minwah]

MinWAH:  I'll get something over to you in the next day or two. And thanks again for your inital help

...Also if we keep our fingers crossed some FE authors might drop in the code to optionally pass out a txt file on game launch so we won't have to bother with batch files....

Thanks

...And I hadn't forgotten about the LCD support, been a little tied up with some other stuff but I hope to get something in the next release

[Howard_Casto]

the only penalty for this I can think off (apart from hassle) is that you have to reference the msxml in you vb project so may include the required files count by 1....

Silver

For me that's a huge one.  The reason I switched to a directx based engine is so I wouldn't have to fool with including files anymore.  It's a nasty chore, especially with m$ dlls as they tend to have a seperate version for each version of windows.  

Just something to keep in mind.  

[Silver]

Yes thats a good point.... ah well ;-) M$....

MinWah: I've emailed you the code so you can take a look

Silver

[SirPoonga]

This is not me being particulary clever - its just a matter of using the SAXX xml parser instead of the standard MS DOM one. The SAX parser is implemented by MS in the same bundle - MSXML v3 and up I think.

Yep, it's just a matter of finding the right parser.  I've been wanting to parse xml in php for hte controls.dat project, but the built in xml stuff is slow and resource heavy.  I have been reading to try the PEAR or SOAP parsers.

[PacManFan]

For all you FE authors out there looking to add easy XML support, I would recommend CMarkup, it's a single C++ file that's a complete XML parser. It doesn't rely on a single MS DOM object at all. The entire parser is self contained , and extremely fast and efficient. You can get it from here http://www.firstobject.com/dn_markup.htm

I know most of you are groaning at this point because it's C++ and not VB. I think I'm the only C++ FE developer here. What you can do to use this file is this:

Compile the C++ CMarkup.cpp file as a static library using MSVC, like CMarkup.lib, and then create some VB import functions to use it. I can probably do all this if you'd like and create the lib and a .bas file with the imports.

CMarkup is very simple to use, check it out.

-PMF

[SirPoonga]

I also posted this comment about listxml on mame.net
http://www.mame.net/cgi-bin/wwwthreads/showpost.pl?Board=mamegeneral&Number=151022&page=0&view=expanded&mode=threaded&sb=7#Post151022

Whitespace does make a big difference to some parsers.

[Silver]

Aceldamor :

Well here ya go... pretty much the first program I've let loose to the general public. Please give it a try and let me know if it works for you etc....

Just point it at your roms dir, an empty destination dir, the listinfo.xml (sorry does not autogenerate from mame yet.....) and the controlsDAT xml (or just one of them) and it should do the rest. The defaults are for the PJRC LCD I own, but LCD instructions/headers can be entered manually. Or send me codes that work for you and I will add it to the drop down selection.

Please note this is BETA software and hence will probably wreck your pc/steal your passwords/emails/creditcardinfo/adresss/soul.

Found at:

http://www.silverfoxy.plus.com/

PS: It uses the SAX parser in VB and I've stuck in a counter so you can see how fast it parses the xml's as it goes. To test just set up the xml locations and point it at a dud rom dir and it will parse the xmls.  Its not max speed as its also DoEventing/refreshing the counter everyloop.

EDIT: New Web Address

[Howard_Casto]

On a somewhat related note... the xml conversion inside mame seems to have corrupted some outputs.  Check out alien 3:  the gun to see what I mean.  

Also I've noticed that the conversion is getting slightly faster as the u builds are released.  Perhaps it was buggy?