Lightspeed MameLister© - A New Multi-List Generator (Now with link to download)

[Antropus]

Latest Version: Beta v0.7j
Updated on: 6/14/2016
Self-extract: https://drive.google.com/open?id=0B23vaxsfSk1yWjdXNlBoaTgwcUU
Zip: https://drive.google.com/open?id=0B23vaxsfSk1yaDB1MHFPQjVPVzA

Lightspeed Game List Generator


This program was developed as a token of appreciation towards the retro-gaming community.
Lightspeed is fast and it's free. It's my way of saying “thank you” to a community that helped me many times across almost two decades, directly or indirectly. Thank you and I hope you enjoy it.

Current Features

• Easy to use Wizard interface
• Top menu bar for quick access to any page of the Wizard
• Supports Catver.ini, History.dat, Nplayers.ini, Languages.ini, Series.ini, Mame's output xml file and my own ControlsButtons.ini.
• Built-in updater that will download all those support files and install them for you automatically, so it's a one step solution to get things updated every time a new version of Mame is released. [In Progress]
• Over a hundred individual filters that can be used in combination to produce an incredible amount of possibilities, including filtering by genre, emulation status, decade, clones, control type, number of players, number of buttons, display type, orientation, region, media type, number of screens among others
• You can choose to remove [] and () tags from game descriptions
• You can choose to create CHD dummy roms for those CHD based games that do not have a companion rom, so front-ends can actually import and make those games playable
• You can choose to create [TTL] dummy roms, so those games can also be imported and be playable from a front-end of your choice
• The program will store all your settings on exit, as an ini file, so you don't have to check multiple boxes over and over again, every time you use the program. I'm planning to add “save presets”, where you will be able to save and load presets from.
• No support files are ever merged. All support files are kept as-is and can be easily updated by simply copying newer files over them and any operations requiring access to those support files are handled at memory level, instead of writing files to disk. As a result, the performance is quite good: filtering a whole set of Mame ROMs from about 36,000 ROMs all the way down to a list of 2,600 games containing no Mature, pinball, fruit machines, devices, clones, Bios etc will take about 20 seconds from beginning to end, including saving the final resulting list and with no time spent on merging external files.
• You can either generate “offline” lists, where the program doesn't care if you have roms in your system or you can tell Lightspeed to compare that information to your Mame ROMs folder, so the final list(s) will only contain games matching the .zip/.7z files that you actually have in your system
• Ability to generate lists containing only the games that match your roms, snapshots, flyers and/or videos. This way you can make sure that you only import the games which you have snapshots for, for example. Or lists containing only games that you have videos for, videos+snaps, videos+flyers or any other combination of those. No more missing images/videos if you don't like having missing artwork showing up in your front-end.
• A preset filter for “All Killer No Filler!”, a list of great games compiled by the user “BadMouth”.
• Generate lists by platform using presets for: All Arcade Games, NeoGeo, CPS1, CPS2, CPS3, Cave, PGM, Zinc, Atomiswave, Naomi or Hikaru, individually or in combination
• Generate lists by source driver. Everything supported by Mame is listed. You just need to know which one to pick. Just select a driver from the drop-down menu, “stv.cpp” for example, to generate a list containing only Sega STV games. All other filters affect this list as well.
• Built-in snapshot preview, so you can verify visually the games being imported into your list
• Built-in clone list-view, from where you can selectively add clones to your final output list using a context menu accessible on right-click
• Built-in game launcher. Double click any game in either list-view to launch the game in Mame, a quick way to test games when in doubt if you should or not add them into your exported list
• Built-in move/copy filtered ROMs and CHDs. That means that you can easily create a clean set that precisely matches your exported list, great for populating and synchronizing multiple machines while saving a ton of space since you are only adding the games that you really have, want or actually play

• Support for multiple front-end's game list export formats. The current version supports:
• Launchbox
• Attract-Mode
• Hyperspin
• PinballX
• Maximus Arcade*
• Mala*
• mGalaxy
• MameWah (untested)
• MameUI
• AtomicFE
• Cabrio
• XML
• Rom list
• Description List
• Missing List.

* Maximus Arcade and Mala require third-party programs to convert the lists generated by Lightspeed to their final format.

Exclusive for the Launchbox Frontend

• Quickly and easily imports your filtered list of Mame games into Launchbox. This is a completely automated system that will add all necessary metadata and artwork, while setting up emulators, rom paths and platforms automatically. It will also adds a ton of filterable extra-fields, not originally supported by Launchbox. Run Lightspeed, set everything up, press export and re-open launchbox. It's that simple!
• Imports metadata from History.dat selectively. You can choose what kind of information you want to parse: Bio, Technical, Trivia, Tips&Tricks, Staff, Ports, Scoring and/or Updates.
• When it comes to the “export” versions of the games, History.dat only mentions that for more information you should refer to the original game, the Japanese version more than often. Lightspeed tries, as much as possible, to extract the information from the original game and recycle that information into the other sister games, so your metadata will actually look more complete since “export” games will now contain the metadata from the original game.
• “Prevent Duplicates” prevents the same game from being imported multiple times.

I would like to thank:

• The Mame team first and foremost. Without them none of this would be possible.
• All support files creators and maintainers, especially Alexis Bousiges.
• Every single one of you who create tools to make our lives easier daily
• All involved with creating and cataloging arcade artwork and metadata
• The whole community of generous individuals who had helped me directly or indirectly for almost two decades now
• Jason Carr's Launchbox, an incredible piece of software and a great community
• Those brave enough to try out my betas over the Launchbox's forums, helping me with initial bug tracking and suggestions to improve the program
  
• RomLister for inspiring me in this "art of filtering"

This is a beta, so please test it and report if something is broken. Some features are still in progress, like custom XML file export, CSV and other formats, but I'm working on it.

Cheers,
-Kris

[Antropus]

Attract-mode list format export is now natively supported.
I'll keep adding support to as many front-ends as possible.
I also added an updater that downloads and unzips all necessary support files and can be used every time a new Mame version is released.

I should have a fully functional version for you guys to try in a week or so.

-Kris

[8BitMonk]

Looking forward to the beta version, this type of utility is long overdue and would be greatly beneficial to the community. You should check out the Mawler thread if you haven't already, seems like a similar concept and many feature requests were discussed there.

http://forum.arcadecontrols.com/index.php/topic,136193.0.html

Good luck with development.

[wp34]

Attract-mode list format export is now natively supported.
I'll keep adding support to as many front-ends as possible.
I also added an updater that downloads and unzips all necessary support files and can be used every time a new Mame version is released.

I should have a fully functional version for you guys to try in a week or so.

-Kris

I was going to ask about Attract-Mode.  Thanks for doing this.  I'm anxious to try it out.

[Antropus]

Thanks a lot guys!
That new Mawler project seem pretty awesome! Is he still developing it? Seems like that project is more of a full database though, including individual ROM info and all.

Mine is more focused on the list exporting side of things. The way I made it run quite faster than other alternatives was by stripping out any ROM CRC information from the mame XML file, among other things that are not useful for creating the lists. BUT, if that information becomes necessary at some point, like, if I decide to push this towards a full Mame database, then I might bring that information back.

I'm focusing on speed and accuracy more than anything else. I ran some benchmarks yesterday comparing matching filters between Lightspeed and RomLister, just for the sake of it.
When comparing filtering a large list (over 36000 games) against large list, Lightspeed was about 25% faster to create it. But when comparing lists generated for real-world applications, with no more than lets say 7000 games filtered, Lightspeed was over 160% faster. Also, some random entries were listed under a few lists generated with Romlister, which were not present in Lightspeed.

The cool thing about Lightspeed is that it shows the catver ini info very clearly in the listview, including genre and sub-genre. So if you decide to create a list of driving games only, for example, this should be the only type of genre listed and if an entry doesn't show that genre listed or shows a different genre listed, that means that the filter is not precise enough and needs tweaking, but at least it's a very quick diagnose. I'm testing all filters in complete isolation and also in random combinations and the results look correct as far as I can tell.

Since this is something I'm developing for the community much more than for myself at this point, please feel free to suggest things and please, if your front-end is not yet supported, the best thing you can do to help us all is to post a sample of the list format created by it. You can either post the file structure as a reply to this thread or attach it as a file so I can reproduce that structure and add your favorite front-end to it!

Thanks again,
-Kris

[8BitMonk]

IDS isn't actively developing Mawler anymore, he was going to add to it when his time free'd up a bit but that was probably a year ago. When the mame db structure changed a couple versions ago I ping'd him to see if he could update it but he was still too busy.

I think leaving out rom info is fine, one of the reasons it was included in Mawler was because it was an attempt at recreating the MAWS site where you had a pulldown menu to filter rom based on older versions of mame. Personally I was more interested in the list management features like you're going for.

For list management features, I had two requests:

1.) The ability to make custom meta-data and flag games with it.
2.) The ability to share that custom meta-data flagging with others.

The example I used for custom meta-data was creating a genre that wasn't covered by existing data like say a list of comic or super hero inspired titles (The Punisher and Marvel vs. Capcom) but it could be anything custom. My idea was for the user to be able to create a custom piece of meta-data, say a category named 'comics' and then manually selecting the titles that belong to it. Then in addition to being able to export those selections as a list to also be able to export them to share with others.

I personally use Hyperspin so that list format is primary on my list of FE's. The XML format for the lists is pretty straightforward.

[Antropus]

Thanks 8BitMonk!

Indeed. I thought about this feature as well when I was developing my previous software, which is more robust than this one and also a List Generator, although I shifted focus to this more newbie friendly approach so everyone can use it
I think the concept of having support files publicly maintained is the way to go or else things simply stop being developed. Look no further than the whole controls.xml changing hands for example.
Those things need to be community maintained, with some volunteers to check for inaccuracies and like you said, with the possibility of sharing your own lists.
I could definitely implement something in the sense of you being able to change things and save it to your own xml (or ini) file that could potentially be shared and re-imported by other users.

When it comes to a database though, there are things already being developed that have massive potential. The new Launchbox database for example is supposed to do just that. It's community maintained and moderated by a few volunteers who filter artworks which quality is not good enough. They are adding more and more metadata as well. In the near future people will be able to mass-edit games in Launchbox and update that info to their database, but this is still too specific for their front-end. If they open up for the general public, by creating an independent app to communicate to their database in a two-way mode for example, then the community as a whole would GREATLY benefit from it. I believe the free version of Launchbox will work with it as well, so maybe that's already covered.
Also, a huge feature that we should suggest them is the ability to export lists out of their database. That would be really cool as well.

Check out their database: http://gamesdb.launchbox-app.com/

-Kris     

[nitrogen_widget]

This looks nice.
I've been using RomLister and while i'm sure i'm to blame, it breaks often when it comes to merging all the metadata and generating a list from a search. 

so something along the line's of mawler that is a little harder to break would be very nice.
Looking forward to testing.

[NOP]

I (as the author of ROMLister) welcome the new Lightspeed overlord.
It's been fun, but you're more enthusiastic than I am, so take the job and run with it!

[Antropus]

I (as the author of ROMLister) welcome the new Lightspeed overlord.
It's been fun, but you're more enthusiastic than I am, so take the job and run with it!

I don't post much over here, but I hover over those forums for years now and one thing I can say is that I commend you, good sir, for being incredibly cool and helpful with everyone you interact with.
Your reply is a proof of that attitude. Now that I'm creating this I understand the incredible amount of work and research necessary to create some of those filters, even more so since we are supporting an ever changing environment, which is Mame and the non-stop structural changes when it comes to its xml. Not complaining about it, because those changes are necessary most of the time. Just stating the quick sand we try to stand on when developing tools to be used with Mame

About the enthusiasm, we both know that it comes and goes!
I would say that I'm very excited right now, so if people are looking for features, now would be the perfect time to ask! As a father of two with a full time job, free time comes and goes, but I will try to keep the steam going for as long as I can and I just can't wait to put a working version on your guys' hands.

Thanks again for all your work, inspiration and support over the years.
You rock, dude!

Cheers,
-Kris

[DudeRegular]

Looks like a great project man. I use Attract Mode too, so appreciate that format.

[Slippyblade]

As a ROMlister user, I'm glad NOP is cool with this.  It looks like a great project, good luck!

[seinfeldfan64]

I wish I had something to contribute besides a big thank you! I've used Lightspeed to import into Launchbox and found it very powerful and easy to use. I'm looking forward to trying it out with Attact-Mode and Hyperspin.

[Antropus]

Thank you so much for the support, guys!

It doesn't seem like much, but A LOT is being done under the hood and I got things more optimized and modular now. Still work to do, but getting easier to update things. Some more features:

- Here are the new options I mentioned. If you check all of those last 4 options, the exported list will only contain games that have artwork available for them. I just exported a list and BigBox looked great, with absolutely no missing artwork


The All Killer No Filler filter just got fancier:
1) Firstly, it basically overrides the other filters, so only the games in this list will be imported when you check this box.
2) BUT, you now have the option to choose if the other filters will have influence over this list. If you choose to bypass the filters, then the full list will be generated, including BIOS and all. If you don't choose to bypass the filters, then everything is taken in account, so you can get rid of all BIOS in that list, so the list will contain only the games themselves. Or you can eliminate the CHDs from the list. You can generate an all vertical All Killer shmups for example or any other filters that will be applied to the All Killer list.
3) On the favorites idea, now you can choose to make the games in the All Killer list as favorites, as long as your front-end supports it through the game list. So far it only works for LaunchBox/BigBox, but let me know which other FEs save that in their lists and I can easily implement it. Just mark that checkbox and all games in the list will be favorite'd when the list is exported. If you use Launchbox, another way to do this is to rename the Mame Arcade platform to "All Killer", for example, export those games as part of this platform and in Launchbox you can mass edit all games in it as favorites and then mass edit them again, moving them into your Mame Arcade platform this time:


I started adding a context menu that will grow quite a bit. So far it's pretty basic, but you can already select all, invert selection, delete selected and move selected clones into the output list. I will be adding more stuff soon:


Sorry it's taking a little long to get a new version out. I really need to iron things out some more and there are many ideas lingering around my head now. I had this crazy idea of creating a "GameList Blender". It would work like this:
- Select a list format from the dropdown menu, hyperspin for example, then click import to have your old list imported into Lightspeed. If the information is incomplete, it doesn't really matter right now. This list will be parsed into a "Blender" listview. Now select another format, attract-mode for example, and import your attract-mode list. It will be merged with the previous list and anything already there will be skipped. You will be able to merge as many lists from as many different front-ends as you wish, so you can basically consolidate all lists from all the different FEs you might use into a single list if you wish. But here comes the great part about it: this list will become a filter and all the metadata will be parsed again and anything missing before will now be filled in for you, as long as the exporting format supports all those fields. Now you can simply export this consolidated list to all your front-ends again, super-fast. A little ambitious, but I think I can make it work.

This will turn Lightspeed into an all around importer and exporter for gamelists and can be very useful at times, in my own experience.

Cheers,
-Kris

[Antropus]

I forgot to mention that Lightspeed is being tested with Win7 and Windows 10 only, so I could not say that it will work with windows XP, unless someone tests it.

-Kris

[Antropus]

Weekend update:

1) I changed my code to be faster and more effective when cross-referencing folder contents. As a consequence, you can now also choose to match your final list with the Wheels/Clearlogos that you have.
2) I incorporated the update buttons as a very easy, straight forward way of getting the latest support files available, so Lightspeed can be always up-to-date. The way it works: downloads the files > unzips them into a temp folder > copies the necessary file to the correct folder > deletes the temp folder > reads the uncompressed file to capture its version from the header > updated the interface to reflect that number by the update button side, so you know which versions you are using at all times.


3) AntoPISA did a great thing with his Catver.ini, Series.ini and Languages.ini. He kept the download names consistent every time there's a new version for download, so the updater simply works. Now, with History.dat and NPlayers.ini, the names change every time there's a new version and it's not only an update on the number itself, like "0174", but sometimes "0174a", for example. For that reason I added an inputbox that you can use to enter the new version that you are looking for, with examples you can try. Again, this is only necessary for NPlayers.ini and History.dat


4) History.dat is the only support file that's big. For that reason I added a little progress bar:


5) Because I understand the importance of the work being done by the support files creators/maintainers, after any of those updates you will get a popup window crediting the authors and providing a link that everybody is encouraged to click to support their projects (as I understand, they need traffic and accept donations, so please click the links!).


5b) Same here:


You can now generate/update your Mame xml file via the mame -listxml command line running under the hood:


-Kris

[Antropus]

First post updated. You can now download and test the first *official* version:

Download:
Self-extract: https://drive.google.com/open?id=0B23vaxsfSk1ydFFuclZncld0bmM
7z: https://drive.google.com/open?id=0B23vaxsfSk1yTVo3YVNPc2NSRm8

Please download, install, try it out and report anything broken. A 20+ pages manual comes with it, so make sure you take a look.

Cheers!
-Kris

[gonzo90017]

A small feature request. I would like to see an option to "Move" roms. This way I can have a folder that contains only the roms that are generated from the lists that are created. I know Hard Drives are cheap these days. But i'm using some spare PCs that don't have much hard drive space.

[Antropus]

A small feature request. I would like to see an option to "Move" roms. This way I can have a folder that contains only the roms that are generated from the lists that are created. I know Hard Drives are cheap these days. But i'm using some spare PCs that don't have much hard drive space.

This feature is already present

Check the bottom left corner:

[gonzo90017]

 

[Antropus]

New version coming up tomorrow with some important bug fixes, much improved controllers support and a much better parser for history.dat notes.
As it stands, a lot of games in history.dat are presented as "Export version". When that happens, history.dat normally refers to the original game, usually the Japanese version if you want to read the actual notes, since those are only kept within the original version. Lightspeed does some stuff under the hood to make sure that the most complete notes are always parsed. As a consequence, the notes parsed by Lightspeed are usually more complete than if you are simply parsing straight from history.dat.

Here's a comparison between the data being directly parsed by MAME and my approach, which is better suited for front-ends that support notes, like LaunchBox:

 

[Antropus]

Latest Version: Beta v0.7h
Updated on: 6/9/2016
Self-extract: https://drive.google.com/open?id=0B23vaxsfSk1ydzhZb0NSOHNHMGM
Zip: https://drive.google.com/open?id=0B23vaxsfSk1yNHZ4VzFXN2tBTkk

• Introducing History Reparser^TM, a much better way to extract notes from History.dat. Games marked as “Export version” will now have a much better chance to have their notes extracted from the original games, so, much more complete notes from now on if you use LaunchBox.
• Updated all support files to the latest versions available.
• Improved controllers metadata. Most games now have a controller associated to them.
• Renamed the default platform from “Mame Arcade” to simply “Arcade”
• TTL and CHD “dummy” files generated are now true .zip files instead of text files named .zip.
  
• Improved auto-fill when selecting MAME.exe
• Improved “reset” functions.
• FiltersA, 3rd toggle now ignores “All Killer” filter, since this is a very specific filter that ends up overwriting all others
  Removed the game descriptions in the status bar when parsing lists. This sped up the process by a few seconds. The progress bar is still present.
• Removed some splash screens. All info related should now be presented at the status bar, which is way faster than splash screens (it's all about speed after all).
• Improved attract-mode generated lists.

Enjoy and please report anything funky. Thanks.
-Kris

[Milky]

This program is amazing, great job!  I do have 2 questions and it's probably due to my incompetence, but The Simpsons doesn't seem to come up on the list for me no matter what filters I apply.  I was trying to make a list of 2/3/4 player games, but it wouldn't show up on any of the lists that were generated.  I wasn't sure if it was a bug or something I was doing.

Next, I've been trying to sort using the All Killer filter, but no matter what I try the list never generates as it never finds any games.  Is there a file I need to have the program point to or something else I need to do that I'm not catching? 

Like I said though, amazing program and great work!

[Antropus]

Thank you for trying it out and for reporting, Milky!
The problem is on my side. I changed some filters recently and seem like, for some reason, The Simpsons is falling into the "unknown" category when it comes to number of players, although NPlayers.ini has the information correctly. I will investigate and get that fixed tonight, so no worries!

As for the All Killer list, it's working on my side. Make sure you have a folder called:
.\SupportFiles\All Killer Lists\

You should have a bunch of files in there, separated by genre. To make sure you got the necessary filters on, go to every tab and click "reset", and then check the All Killer checkbox and try to generate your list again.
Let me know if it still doesn't work. If does not work, please post some screenshots of the filters you are using so I can try to emulate your settings on my side.

Thanks a lot!
-Kris

[Antropus]

Latest Version: Beta v0.7i
Updated on: 6/10/2016
Self-extract: https://drive.google.com/open?id=0B23vaxsfSk1yaV9sZU1CMWFldFk
Zip: https://drive.google.com/open?id=0B23vaxsfSk1yNlNXT0lhRHVtRUk

• Fixed a bug that was causing the number of players to be parsed incorrectly (thanks Milky!).

-Kris

[Milky]

Looks like it's catching the game properly.  Also, I must have been missing the support files for the All Killer lists as they seem to be working now!

[Antropus]

Glad to know
Let me know if anything else doesn't look good!

-Kris

[Milky]

I've been toying around with this some more and I was wondering about something.  Is it possible to import a particular XML so that you can constantly edit it without having to recreate a list from scratch?  What I would like to be able to do is generate a list, pull it up in HS, see what games I don't like/don't work, and remove them as I find them.  If it's possible to import an XML file for editing I would be able to constantly pull up the same file and just remove the new games I come across instead generating a full list and having to delete all of the old game all over again. 

I'm not sure if that makes sense or if it's even possible, but I figured I'd check!

[DudeRegular]

Wanted to post some feedback, but understand I may have just not done something correctly causing some of this to be moot. Really liking the program so far. Great work.

1. Love the fact that you can launch a rom right from the final page. Great feature.

2. Would like to be able to highlight just certain games, then hit export to only have the selected in a list.

3. Using the killer lists, it adds it all into one big list at the end. Can you add a column to see which list a title is from? then see point 2.

4. Emulators/platform tab: add supermodel, like the option to go from driver, but that is a big ol list in there, and may be a bit much for most.

5. copy/move is another really great feature.

6. Clones tab: possible to describe some of the clones/hacks in better detail? Not sure if that is present in one of the ini/dat files being referenced or not.

7. resize/maximize the window: seems to be locked to the one size now.

8. in the last tab, i noticed when resizing columns that there are a bunch of columns that appeared from their hiding spots all over. Maybe a right click option there to view/hide columns to this view?

9. "&" Ampersand characters are showing in titles as "&amp"

I've been toying around with this some more and I was wondering about something.  Is it possible to import a particular XML so that you can constantly edit it without having to recreate a list from scratch?  What I would like to be able to do is generate a list, pull it up in HS, see what games I don't like/don't work, and remove them as I find them.  If it's possible to import an XML file for editing I would be able to constantly pull up the same file and just remove the new games I come across instead generating a full list and having to delete all of the old game all over again. 

I'm not sure if that makes sense or if it's even possible, but I figured I'd check!

Good idea. Would make updating a machine easier. I personally would use this in conjunction with my 2 and 3 points above. Say I want to update my fighter list with some new game. Just load it into the app and filter based on the latest roms, add, done.

[DonkyKong]

Antropus Loving your work

Not sure if this a known bug or not?

After 3 years I tried to update my Bartop cabinet with latest MAME the Bartop runs MalaFE while I found out that MalaFE doesn't support the new MAME XML since the merger of MESS, So I tried your great program it filters fine (8762) games, But when I hit the GO! button for the MalaFE output gamelist file it only gets 1/4 through and ends with a total of (1903) games

[DonkyKong]

 

[Antropus]

Thank you so much for the feedback guys!

Based on those feature requests and some other requests I'm gathering from other forums, I think the current format is kinda exhausted, unfortunately. I thought I could keep it simple enough, but we need more features. There are things that I wish I could add here, but the format is not helping. The program was started as a fast way to help Launchbox users to import their collections and because of that it carries a lot of unnecessary legacy code at this point and it's just not as flexible as it needs to be.

For this reason I decided to stop the updates on this format for now and concentrate on merging this program with my previously unreleased one, that in fact carries a lot of the features people mentioned.
For example:
- You can import lists
- You can merge lists
- You can audit many artwork folders
- You have visual feedback for artwork and game status
- It has a built-in rating system.
- It has a more robust, faster way to filter lists
- Built-in, ultra fast search
- Easily merge results from any list into your final list
- Resizable GUI

This transition will take me some time, but I think it's worthy on the long run, because at the end you will get the best solution available, which will still combine the Wizard mentality with a very robust list manager.

For now you can use the latest, which has some bug fixes and if you are patient you will be able to test a very different beast next time:
Latest Version: Beta v0.7j
Updated on: 6/14/2016
Self-extract: https://drive.google.com/open?id=0B23vaxsfSk1yWjdXNlBoaTgwcUU
Zip: https://drive.google.com/open?id=0B23vaxsfSk1yaDB1MHFPQjVPVzA

Attached are some snaps of my older program. The plan is to merge those features into Lightspeed.

Cheers,
-Kris

[Milky]

Just wanted to say you're doing amazing work and the current program is still amazing in terms of ease of use and power!  Great job!

[gonzo90017]

Thank you for hanging in there!

[Antropus]

Thank you so much for your support and kind words, guys. Much appreciated.

I'm slowly but surely making progress on integrating my two programs into an all-in-one solution, while implementing new features and fixing bugs. Some of the new implementations:

- To optimize memory and speed, I decided to generate my own database master file. It will come packaged with Lightspeed, so you don't need to worry about creating it, but every time a new version of MAME is released, you will need to generate a new listxml file anyway, so in that process, a new database file will be created as well. It takes about 20-30 seconds or so. Once it's created, then starting Lightspeed shouldn't take longer than 6-7 seconds.
- Huge speed improvements thanks to the method above. Granted that not all filters are hooked just yet (need to re-write them all...), at this point to produce a fully filtered list of about 7000 games it's taking about 4 seconds, which is more than double the speed of previous methods!
- GUI is now fully resizable, going to as low as 800x600 and as high as your monitor allow.
- "&" Ampersand characters now showing correctly
- There are now 5 different listviews: Full List, Parents, Clones, Search and Output. The first 3 never change and are there as data sources only. You will be able to select any number of rows and sent those games to the output list with a single click. The search listview will contain the results of your searches, initially by: terms in any of the columns, in the description, in rom name, in year, in manufacturer. As with the other lists, you can select any number of games and send those to the output list as you wish. The output list is what you will get when exporting. You can remove games from it or send games from other lists into it at any given time.
- You can launch games from any list. Simply double-click a game and it will launch through MAME, no matter which list you are at.
- Added all other MAME artwork folders. All those folders are being parsed on startup every time you open Lightspeed, so it will list all the artwork you have and everything missing as well, which can be later on exported as missing/have lists. It's a fast process and it's already included in the 6-7 seconds that takes for the whole program to load.
- Added the arcade machine artwork, which supports snap and marquee preview

Things are quite temporary in those screenshots, but at least you have an idea about the progress (see attachments).

More to come and still a LOT left, but I will get there.

Cheers,
-Kris

[DudeRegular]

Kickass man. Keep at it.

[DonkyKong]

WOW! looking good.

[chill]

 
"Ability to generate lists containing only the games that match your roms, snapshots, flyers and/or videos. This way you can make sure that you only import the games which you have snapshots for, for example. Or lists containing only games that you have videos for, videos+snaps, videos+flyers or any other combination of those. No more missing images/videos if you don't like having missing artwork showing up in your front-end."

Excellent utility. I just finished my retropie/Attract Mode build and this was an excellent way to clean my lists that were missing artwork.

[Sorn]

Ahhhh, where were you 6 months ago haha!! I was dying for this utility!!! I ended up spending many many hours hand editing. Thanks for this, it will come in great use eventually and it looks SHARP!

[floob]

Can the tool be independent of MAME? When I try to run it I get a message saying the MAME path cant be blank.
I'll read through your manual in case its in there.

I'd like it in a similar way to Romlister where I can input an xml/dat file and filter based on that alone?

[8BitMonk]

Finally got a chance to take a look at your latest version and this is fantastic! 

2 things:

• v0.7j appears to break Filter B --> Controls, only Unknown will return any results. Ie. selecting just 'Dual 8-Way Joy' won't bring up those games. I went back and tested 0.7h and 0.7i and both returned 41 games for Dueal 8-way joy. Maybe a breakage from when you fixed the # Players for the Simpsons.
• Copy/Move doesn't seem to work for me. Just tried copying (not moving) and it didn't copy files or return any sort of an error.

 
I updated all of my support files and the mame xml to .176, not sure if that affected the functionality, all seemed to update fine.

You've really done a great job so far, thanks for your contribution, I'm sure it's a lot of work. 

[8BitMonk]

Is this still in development or abandoned?

[haynor666]

I think it's still in dev.

http://forums.launchbox-app.com/topic/30530-lightspeed-ultra-fast-mame-importer-tool/?page=22

[Antropus]

Still in development, slowly but surely, since the changes are pretty major!
I just posted a little log progress with many images you can check here:

http://forums.launchbox-app.com/topic/30530-lightspeed-ultra-fast-mame-importer-tool/?do=findComment&comment=210152

Cheers,
-Kris

[Antropus]

Actually, here's a copy of my post over there:

Here's some progress log:

- I got a little fancy with the games tester/launcher, adding some console images and graphics, so it looks less formal.
- Newly added star system. 10 ratings range, in half increments until a total of 5 stars. Those ratings are being stored into an external .ini file that can be later on shared among users. Those ratings will be exported to Launchbox once you export your lists
- Newly added favorite system that works similarly to the above. The favorite info is being saved within the same .ini file as the ratings and will also be reflected once in Launchbox
- You can launch any game, from any soft list or arcade, from any listview with a double click. All command line parameters are populated automatically, so no need to scramble with MESS stuff anymore
- The search engine was improved as well, but I will add some checkboxes to offer better customization on where to look for (year, description, rom name, manufacturer etc)
- All supported software lists are now available as a choice and those will be imported as platforms. Only systems that are arcade machine friendly and are working reasonably well are supported. Not so fun systems were left out as well.
- Added new arcade platforms as available options (Data East, Konami, Midway, Taito etc)

The interface is still messy, with unnecessary buttons, test functions and bad formatting at some places, but this is all being ironed out as I go.
No release date yet, but the project is definitely not dead...

More to come.
-Kris

[haynor666]

Any chance to add number of buttons used besides joystick type ?

[8BitMonk]

No release date yet, but the project is definitely not dead...

Good to know, thanks for the update.

[Antropus]

Any chance to add number of buttons used besides joystick type ?

Already present:

[haynor666]

That's a filter. Actually I was hoping see in mamewah displayed 8 way joystick along with number of buttons.

[fablog]

Thanks for this great app, I like it. I suggest to code some warning messages about absence of Mame.ini or wrong Mame.exe.  I lost maybe 30min trying to understand what was wrong when I was using a fresh downloaded mame. Obviously, mame.ini was missing

[millercentral]

Just started using this, and I'm already blown away. Thanks so much for your efforts here, and I am looking forward to the next version as well.

A couple minor feature requests:

• If I choose to drop the "[]" and "( )" from the title, and also filter clones from the results, it would be nice if the [] and () text was still included in the separate Clones list to make it easier when hunting for a specific version to add to the final list.
• It would be cool if in the right-click context menu of a parent rom I could get a quick pop-up list of just the clones of that parent to more easily add the clone I'm looking for, rather than having to hunt for it in the clones-list view.
• In the current .7 release, it does not appear the window is re-sizable. It would be nice if it was so I could see more columns/rows on my large monitor when making lists.
• It might be interesting instead of starting with the "all-roms-from-mame" listxml source, if you could start from an existing generated list, and then apply filters to that, to make smaller sub-lists from the original. For example, it would be interesting to have the available "all killer" titles within my already carefully curated list of titles as another list.

None of these items are deal breakers. In one day of use this is already my all-time favorite rom-list creator. Thank you!!

[millercentral]

Okay, another question. How do "unchecked" filters work relative to checked filters in other categories?

For example, I tried looking for all vertical games that use dual sticks or a spinner, but leave out any that don't. So I unchecked "2-way Joy", "4-way Joy", "8-way Joy", "Trackball" and a couple of other controls, and also unchecked "Horizontal", but left all the buttons and player items checked.

In the filtered results, I got only vertical roms (good), but the also included unchecked control types (2-Way, 4-Way, 8-Way, trackball, etc) (bad).  Why did those control types get included in the results?

Thanks!!

[ilgrank]

First of all, let me thank you, Lightspeed ROCKS!!

I too noticed that the 7j breaks the filter on the number of buttons. Only "unknown" works.

Apart from this, it would be great if you could add a keyboard shortcut (DEL) to remove games from the list (currently, I had to right-click-delete more than 2000 roms one by one) and CTRL+Z to undo, it would make the pruning of the list way faster

Also, I think it would be great to be able to "expand" the filters. I.e., instead of '70 , '80 , '90 as an example, the 'advaced filter' could allow to input "1985 to 2005". (or a two way slider)
In the same way, the category could have "+" signs to allwo for sub-category. Eg: I want "sport", but I don't want "Football" or "Baseball" and so on
Many, many thanks!

[Antropus]

Just started using this, and I'm already blown away. Thanks so much for your efforts here, and I am looking forward to the next version as well.

A couple minor feature requests:

• If I choose to drop the "[]" and "( )" from the title, and also filter clones from the results, it would be nice if the [] and () text was still included in the separate Clones list to make it easier when hunting for a specific version to add to the final list.
• It would be cool if in the right-click context menu of a parent rom I could get a quick pop-up list of just the clones of that parent to more easily add the clone I'm looking for, rather than having to hunt for it in the clones-list view.
• In the current .7 release, it does not appear the window is re-sizable. It would be nice if it was so I could see more columns/rows on my large monitor when making lists.
• It might be interesting instead of starting with the "all-roms-from-mame" listxml source, if you could start from an existing generated list, and then apply filters to that, to make smaller sub-lists from the original. For example, it would be interesting to have the available "all killer" titles within my already carefully curated list of titles as another list.

None of these items are deal breakers. In one day of use this is already my all-time favorite rom-list creator. Thank you!!

Those are good suggestions, thank you. The program changed quite a bit and the current version has a very fast search engine and the () and [] are only being removed from the output now, so for the purpose of searching you will have no problem since the tags will still be visible in the listview. The clones view is gone in this version and I might add the popup you mentioned to help finding the clones more easily, but honestly, the search can help with that as well.

The window is fully re-sizeable now and both size and position of the window are saved on exit and restored on start. There are also resolution presets. Minimum size is 800x600. Maximum is unlimited.

As for the sources, the databases are fixed and everything will be always available, BUT you will be able to merge lists you already have by pointing Lightspeed to a folder containing as many lists as you wish, for example. In fact, Lightspeed can populate the listview by either:

a) using the regular main sources + filters
b) using a driver as a source (cps2.cpp, for example)
c) using a folder containing any number of roms (correctly named)
d) manually selecting any number of roms (correctly named) from any folder
e) using a folder populated with rom lists saved as text files (one rom name per line for each .txt file).

You can use any combination of the above and simply increasingly add the lists to your final output list.

Thanks!

[Antropus]

Okay, another question. How do "unchecked" filters work relative to checked filters in other categories?

For example, I tried looking for all vertical games that use dual sticks or a spinner, but leave out any that don't. So I unchecked "2-way Joy", "4-way Joy", "8-way Joy", "Trackball" and a couple of other controls, and also unchecked "Horizontal", but left all the buttons and player items checked.

In the filtered results, I got only vertical roms (good), but the also included unchecked control types (2-Way, 4-Way, 8-Way, trackball, etc) (bad).  Why did those control types get included in the results?

Thanks!!

This was apparently broken before. I just ran the very same test in the current version and everything worked accordingly.

[Antropus]

First of all, let me thank you, Lightspeed ROCKS!!

I too noticed that the 7j breaks the filter on the number of buttons. Only "unknown" works.

Apart from this, it would be great if you could add a keyboard shortcut (DEL) to remove games from the list (currently, I had to right-click-delete more than 2000 roms one by one) and CTRL+Z to undo, it would make the pruning of the list way faster

Also, I think it would be great to be able to "expand" the filters. I.e., instead of '70 , '80 , '90 as an example, the 'advaced filter' could allow to input "1985 to 2005". (or a two way slider)
In the same way, the category could have "+" signs to allwo for sub-category. Eg: I want "sport", but I don't want "Football" or "Baseball" and so on
Many, many thanks!

Thanks. Filters are good now.

Shortcuts were added. No Ctr+Z planned yet, but you can delete things already by selecting and then using ctrl+del. It only works on the output list though.

As for the years, I will refer to the new search engine again. You can add any number of arguments and searching by year is one of the options.
In your example, although not ideal, you could use two simple steps: 1) check 90's only then add the results to to the output list. 2) Run a search using the remaining years you want as arguments separated by comma: "1985, 1986, 1987, 1988, 1989, 2000, 2001, 2002, 2003, 2004, 2005" then add the results of this search to the output list again. I could add a search argument option to do this automatically at some point. "1985><2005", where "><", for example, could be the argument for returning a range.

Eliminating sub-categories for genres can be achieved by clicking the sub genre's header in the output list, looking for "Football" and "Baseball" and then deleting those games from the output list. Again, not as straight forward as having a dropdown menu, but the whole process as it is would only consume a few seconds.

Good suggestions. I will keep them in mind for future features. Thanks!

[Antropus]

Here's the current status:

I am back from vacations and back to testing/fixing things! There are some known little bugs that should not be too bad, as long as people are aware of them before hand, until I find possible solutions, but nothing that will stop you from creating your lists at all.

I added many more possible platforms, updated the manual as much as possible and hooked up everything that was disjointed. I will be releasing a version for tests in the next few days and I will really appreciate if people took the time to test it, as this will benefit all other users. The main focus is being Launchbox, because it is by far the most complex of all FEs. In fact, if I removed the support for Launchbox, my code would be reduced to about half.

At this point, I just want to squash as many bugs as possible and get a very stable release out. Then and only then I will contemplate adding new features. One of my priorities is, after the Launchbox module is working nicely, I will be putting some time on adding advanced support to the Attract-Mode and mGalaxy front-ends. At this point lists are being exported to them already, but I would like to add support to creating lists by platform as well and similarly to Launchbox, have a fully automated process, to the point where you would just have to run Lightspeed and open any of those FEs to find everything fully configured and ready to go. But this will come in the future,

For now, stay tuned and a download will become available here in a few days, probably before the weekend!

Cheers,
-Kris

[ilgrank]

Thanks for the update @Antropus!
Can't wait to be one of the beta-testers (If I'm allowed of course!

Again, thanks for your work!

[8BitMonk]

Looking forward to testing, fire away. 

Glad you're focusing on a stable release before going crazy with features, that's smart. 

[Antropus]

Latest Version: 0.181a_RC1 Updated on: 1/13/2017 @ 2:20AM
Here's something for your to try: https://drive.google.com/file/d/0B23vaxsfSk1yQUQ1aXZpSThpbUk/view?usp=sharing

There are still known bugs, but this should be fairly functional and stable at this point. Manual is still in progress but almost complete.
Remember: Lightspeed is designed for people who already have a large collection of MAME ROMS/artwork in their system.

The way I am handling Software Lists at this point is by parsing custom command lines per game. The only FE supported by Lightspeed for SoftLists (MESS) at this point is Launchbox, so if you are planning to export lists to be used within FEs other than Launchbox, please make sure to turn off "MESS (SoftLists)" under the "FiltersA" tab, so only Arcade games will be exported. If other frontends you know have support for a custom command line per game, please let me know and I will do my best to add Software Lists (MESS) support to those FEs as well. For now, try out with Launchbox (the free version) and you will get the benefit of having all Arcade and Software Lists filtered and imported for you automatically.

If you are interested in testing it, please follow this:

7-step guide if you are NOT using Launchbox:

1) Download and install Lightspeed in your favorite folder
2) Run it as Admin (Right-click the main .exe file > properties > Compatibility > check "Run this program as an administrator", to avoid future problems depending on the location where it was installed)
3) If this is a brand new installation (as it should at this point), you will need to run the program once, configure all paths, then close/re-open Lightspeed. This will save the necessary .ini file on exit, which will be used to parse all folders on the next start. This is only necessary the first time you run Lightspeed.
4) Once restarted and before you start playing with the filters, just as a test, turn off the mess filter by unchecking "MESS (SoftLists)" under the "FiltersA" tab, then go straight to the "Export" Tab and click on it. You should now see the listview fully populated. Select what type of list format you want to export using the dropdown menu on the lower-left corner of the GUI, then browse for the destination folder right below it.
5) Click "1 Filter"
6) Click "2 Output" > Add ALL and wait until the process is complete. It should take only a few seconds.
7) Look at the status bar. If the "Total Output" number matches your "Total Filtered" number you can go ahead and press "3 export"

If all paths were configured correctly, you should be able to find your exported list in the folder you selected.

OR

10-step guide if you are using Launchbox:

1) Close Launchbox
2) go to your Launchbox installation folder and backup the entire "Data" folder to a safe place
3) now go to the original Data folder and delete:
Platforms.xml
Emulators.xml
Platforms (folder)

This will guarantee that you are working with an empty library without losing your settings, which is essential for our tests at this point
4) Download and install Lightspeed in your favorite folder
5) Run it as Admin (Right-click the main .exe file > properties > Compatibility > check "Run this program as an administrator", to avoid future problems depending on the location where it was installed)
6) If this is a brand new installation (as it should at this point), you will need to run the program once, configure all paths, then close/re-open Lightspeed. This will save the necessary .ini file on exit, which will be used to parse all folders on the next start
7) Once restarted and before you start playing with the filters, as a test, go straight to the "Export" Tab and click on it. You should now see the listview fully populated. Select what type of list format you want to export using the dropdown menu on the lower-left corner of the GUI, then browse for the destination folder right below it. If you select Launchbox from the dropdown menu, Launchbox root folder will be auto populated (you NEED to point to Launchbox's root folder when exporting lists to it)
Click "1 Filter"
9) Click "2 Output" > Add ALL. This might take some time at this point, as it's adding over 18.000 games to the list, from both Arcade and Software Lists, so be patient and wait until the process is over.
10) Look at the status bar. If the "Total Output" number matches your "Total Filtered" number you can go ahead and press "3 export"

If all paths were correctly configured, including Launchbox's root folder, the list should be exported and you should be ready to reopen Launchbox.

Please test and report any bugs.
I hope things work for you!
-Kris

[Antropus]

 Latest Version: 0.181a_RC3 Updated on: 1/15/2017 @ 2:20PM
Download: https://drive.google.com/file/d/0B23vaxsfSk1yQUQ1aXZpSThpbUk/view?usp=sharing
 
Please test and report any problems you mind find.

Thanks,
-Kris

[Antropus]

I found a bug that was preventing the filters from working correctly.
New version coming today.

[8BitMonk]

Hi Kris,

I know you're working on an updated version but wanted to let you know I had a chance to do some testing over the weekend and below are a couple observations I found in case they're not on your radar.

• I couldn't get the artwork to display. I like to keep my artwork zipped (which still works for displaying in mame) so I thought maybe that was it but I decompressed a couple marquees, placed them in the correct directory and they didn't display over the arcade.
• I had problems with the Favorites/Rated games showing. I would spot add 10 games to favorites or rank them and go to view them in the filtered list and only some would show. Was sporadic and unpredictable
• Minor thing but I noticed it's easy for the Favorites/Rated to get deselected and it becomes confusing. I clicked the All/None at the top of the controls column because I just wanted to test 8-way games and then get any results because I hadn't noticed both Favorites and Not Favorites was unchecked.

Thanks!

[Antropus]

• I couldn't get the artwork to display. I like to keep my artwork zipped (which still works for displaying in mame) so I thought maybe that was it but I decompressed a couple marquees, placed them in the correct directory and they didn't display over the arcade.
• I had problems with the Favorites/Rated games showing. I would spot add 10 games to favorites or rank them and go to view them in the filtered list and only some would show. Was sporadic and unpredictable
• Minor thing but I noticed it's easy for the Favorites/Rated to get deselected and it becomes confusing. I clicked the All/None at the top of the controls column because I just wanted to test 8-way games and then get any results because I hadn't noticed both Favorites and Not Favorites was unchecked.

Thank you so much for testing and reporting, 8BitMonk! This will be very helpful to get the problems solved.
Not sure what's happening with the images not showing, since this is the first time someone reported the problem, but the other issues should be solved by now and wrapped up into the new version coming up in a day or two.

Thanks again!
-Kris

[ilgrank]

Hi
Sorry for the late reply!
I too did step into the filter bug, but I see you already found about it!
This new version of Lightspeed is great!
I tried the CTRL+Del thing, but mostly I select games based on my memories as a child.. and so I briefly look at a screenshot, then delete what I'm not interested in.. in my case, I had to manually delete more than 2000 games one by one.. and being able to use just the canc/del key would mean, for me, saving 2000 clicks
Could just del be implemented? (or key customization?) That would be great!
Many, many thanks!

[ilgrank]

@Antropus
hi!
Edit: Antropus is ok!
Glad to know that he's fine.
For anyone interested in Lightspeed, the thread here:
https://forums.launchbox-app.com/topic/30530-lightspeed-ultra-fast-mame-importer-tool/?page=29
is more active than this one

Thanks Antropus and keep up the excellent work!

..is everything ok?
Just  wanted to ask you is everything is right... haven't seen you in more than a week..
Thanks for your work, and best wishes!

[haynor666]

One small bug (quite old). Every list contains some trash "" at the beginning of game list file. This probably is not a problem for many frontends but mamewah treat first set name as this trash + set name. Also notepad does not show this when editing file list.

flagrall
'96 Flag Rally
1996
unknown

The only option is to copy entire text in notepad and paste again in new window.

[Tetsuo1981]

Hi all.

Downloaded and installed the latest beta of Light speed but when I get to the export section, none of my filtered have been applied so I am unable to generate a list. What am I missing? Any help would be great. Wanting to make lists for Attract Mode FE, last major job computer-wise in project before I start cabinet construction.

Cheers guys

[what2be]

Not sure if this has been fixed yet or not but in version 183.beta 8 there is a issue with Lightspeed exporting the mame.xml correctly. If the Description or the manufacturer has the "&" in the title then it will error out when you go to open it with other 3rd party programs that manage mame.xml files (eg, Filterhyperlist.exe, etc). I had to do a search and replace of "&" and replace it with "&amp;"
After that everthing was ok.

Also I noticed there is a 1kb ratings.ini file in lightspeeds support folder. Are you going to implement that at a future date? I have a complete 0.183 mame.xml with ratings if you would like it. 

[haynor666]

Any news ?

[abispac]

Can you add lakka please.

[haynor666]

New mame change - http://www.mameworld.info/ubbthreads/showflat.php?Number=368244

[shredso]

I just stumbled across this.  I didn't read much of the thread or read any directions.  I finally made a usable MAME game list!  I've not been able to get Romlister to do anything useful since MAME integrated all the gambling machines and garbage.  Thanks! 

[keilmillerjr]

This app looks really cool! The all killer no filler lists lead me to lightspeed. However, it doesn’t work well at vga res.