Main Restorations Software Audio/Jukebox/MP3 Everything Else Buy/Sell/Trade
Project Announcements Monitor/Video GroovyMAME Merit/JVL Touchscreen Meet Up Retail Vendors
Driving & Racing Woodworking Software Support Forums Consoles Project Arcade Reviews
Automated Projects Artwork Frontend Support Forums Pinball Forum Discussion Old Boards
Raspberry Pi & Dev Board controls.dat Linux Miscellaneous Arcade Wiki Discussion Old Archives
Lightguns Arcade1Up Try the site in https mode Site News

Unread posts | New Replies | Recent posts | Rules | Chatroom | Wiki | File Repository | RSS | Submit news

  

Author Topic: MAME Library Builder v1.1  (Read 4266 times)

0 Members and 1 Guest are viewing this topic.

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
MAME Library Builder v1.1
« on: August 18, 2019, 08:27:35 pm »
MAME Library Builder v1.1

Intro:
I have an Arcade Cabinet in my living room running MAME with Maximus Arcade and for obvious reasons I prefer to keep it setup with a subset of ROM's rather then the ridiculous full library. I just moved from MAME 0.156 to GroovyMAME 0.212 so rather then hand pick my roms from my updated master list I decided to script it.

Description:
This is a simple PowerShell script to make creating SubLibraries easier.
The idea is if you keep a master library of MAME ROMs and Artwork you can use this script with a ROMs_List.txt listing all the ROM's and a Folder_List.txt listing which Full Directories to Copy to your Sublibrary.
Then if and or when you update your MAME Library you can quickly copy out your preferred games for Mobile, Handhelds, Cabinets or anywhere else you run MAME.
 
The script will parse a ROMs_List.txt and copy all matching files to the new SubLibrary.
It will then parse a Folder_List.txt used to decide which Full Directories to Copy.

For the script to work it must be placed just outside your MAME Root Folder with a ROMs_List.txt & Folder_List.txt

Note: If you have multiple versions of mame in the same folder it will try to search everyone for the requested files. If some files are only available in certain Builds this will combine them in one folder but it will not overwrite previously copied files so don't expect it to as the final SubLibrary will contain the first copy of each file it found. As such I suggest temporarily renaming other MAME versions even just by removing a letter "MAM", "AMA" etc. 

Example Folder Structure/ Script Location:

Right Click on the "Build MAME SubLibrary.ps1" and choose "Run with PowerShell"

The Script will search for any folder with MAME in the name (MAME 0.212, MAMEUI, GroovyMAME etc.) ignoring capitalization and spaces.

Once it finds your MAME folder it will Create a New Folder named "Building MAME SubLibrary... (*DATE & TIME*)" (Note: MAME is left out of the final name to avoid being read during your next SubLibrary Build but it is used at the beginning as an error check incase your Build is missing some of the EXTRAs Folders)

When it is done the PowerShell window will say "SubLibrary Creation Complete!"  rename the SubLibrary  "Arcade Library (*DATE & TIME*)" and Close automatically after 10 seconds.

Example of Script Running:

Folders Scrapped By Script:
ROMs
Artwork
Cabinets
CPanel
Flyers
Icons
Manuals
Marquees
PCB
Samples
Snap
Soundtrack
Titles
VideoSnaps


These are the Folders the script searches for matching files from your ROMs_List.

I think this Covers all the important stuff but I could add a few more EXTRA's folders upon request.

CHDs Support: CHDs or any folder you would prefer full Copies of (Every file not just ones matching ROMs_List) are supported through the use of a Folder_List.txt

Your ROMs_List.txt should be extremely simple just one line per ROM or BIOS file name with a * wildcard for the extension.

ROMs_List.txt Example:
Code: [Select]
1941u.*
1941.*
arkanoid.*
mslug.*
mslug2.*
mslug3.*
pacman.*
puckman.*

Your Folder_List.txt should be just as simple but can contain Directories within Directories as shown.

Folder_List.txt Example:
Code: [Select]
ROMs\CHDs\area51
Bosses
Ends
Marquees

 I mainly made this for personal use but I realized when I was half done I might as well post it for other as it did take a little time. But it took far less then it would have to hand picking my ROMs from my updated library and It will save you even more!
« Last Edit: August 20, 2019, 02:07:38 am by Dracrius »

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder
« Reply #1 on: August 19, 2019, 04:22:00 pm »
Only a day and I've already got an update. Realized while I was at work there was a fairly simple way to add Folder/CHDs support!

Update v0.2 (Aug 19th 2019)
- Added CHDs support in way of Folder Parser
New Folder_List.txt is used to decide which Full Directories to Copy as well
Folder are copied last so if for instance you'd prefer the entire Artwork folder not just the ones matching your ROMs add "Artwork" to the Folder_List.txt
Subdirectories are supported for instance "ROMs\CHDs\area51" or "ROMs\area51" will both work if added to the Folder_List.txt

Folder_List.txt Example:
Code: [Select]
ROMs\CHDs\area51
Bosses
Ends
Marquees

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #2 on: August 20, 2019, 02:00:58 am »
Update v1.1 (Aug 20th 2019)
- Completely Reworked some sections for stability and bug avoidance in less then ideal MAME folder layouts
- Reorganized and Alphabetized folder creation after ROMs
- Added 8 new Folders to the Scrap list:
Code: [Select]
Cabinets
CPanel
Flyers
Icons
Manuals
PCB
Soundtrack
VideoSnaps

nexusmtz

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 489
  • Last login:June 01, 2022, 03:14:22 am
Re: MAME Library Builder v1.1
« Reply #3 on: August 20, 2019, 03:39:56 am »
Quick thoughts...

I know you're just getting started with this, but I think you'd be better served to put your subfolders in a list and process that with a foreach so you don't have to repeat the code over and over.

I understood v.2 better, when there wasn't a difference between $final_folder and $destination_folder. What's the thought behind two different folders?

I'm glad to see that you've trimmed down the number of references to get-date. The other way was asking for inconsistency. Unless you're timing something, grab the date once at the top of your script, and reference that.

Be careful with globbing your get-childitem -path. A single file matching *MAME* (like readme_mame.txt) at the top level can kill your searches. If you mean to process the first matching path, get-childitem -path . -filter *MAME* and use the first result as the path to each subsequent gci. If you really want all of the results searched for each copy, you can use your method, but be aware of how powershell handles those wildcards in the -path.

Also, you can, if you want, just pipe your get-childitem into copy-item. -path should pick up pspath from the piped objects.

Good luck with your programming!

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #4 on: August 20, 2019, 07:08:54 am »
 Thanks I appreciate it. I roughly touched on the reason I started renaming the folder with #final_destination in my description but I'll elaborate. I started checking for hangups if a folder I'm scraping doesn't exist since I added a bunch of EXTRAs folders. My old code would just hang and never crash. If the folder it should be looking in for your file is missing it doesn't seem to trigger an error to be continued pass as I tried a try catch or telling it to silentlycontinue. So I cheated and made the script start with a MAME based name as it creates all the folders first thing. That way if it makes it to the searches and you don't have an Artwork folder or much more likely VideoSnaps it will find the empty one and be able to skip that step.

I'll admit I'm like 8 years out since I did any serious coding and PowerShell is all new to me I've just been googling and putting it together. I knew especially when I hit that issue that I was doing a bad hack somewhere. I'll try and make heads or tales of your suggestions when I get off work. I'll need to Google's few things haha

I just couldn't do all this manually every update. I'd also prefer to use inis with both list in one file but I was worried thatd over complicate the code and their for blow the reasearch out of my scope. Ideal world

Multiple MLB_*Preferred Library Name*_Settings.ini's
Each Producing its own
*Preferred Library Name* *MAME Folder Name*

So that if you ran it with a MLB_Multicade_Settings.ini, and MLB_Mobile_Settings.ini for instance in a folder with a GroovyMAME 0.212 and say an older MAME 0.156 it would produce 4 folders

Multicade GroovyMAME 0.212
Multicade MAME 0.156
Mobile GroovyMAME 0.212
Mobile MAME 0.156

With an exception when searching to skip any folder with the *Preferred Library Name* in it to avoid chaining them.
« Last Edit: August 20, 2019, 11:28:48 am by Dracrius »

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #5 on: August 20, 2019, 09:55:54 pm »
I wrote up a concept of how I hope to work the next update. I'll be spending the next night or two researching Powershell to translate it to actual functional code since I'm not sure how exactly to do a few things when it comes to referencing an ini for all the settings or how to properly error out with messages but I'm pretty sure it shouldn't be much more complicated then I've imagined. While its not all proper code atm it shortened the script from 206 lines to 75 haha when I'm done I'm guessing it'll be half the length of the original.

Let me know if their are any major logical errors or if you see a part that is easy to explain how to do. I wrote out all the regex expressions for searching the ini's as part of it. I hope they'll work as written.
« Last Edit: August 20, 2019, 10:01:41 pm by Dracrius »

nexusmtz

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 489
  • Last login:June 01, 2022, 03:14:22 am
Re: MAME Library Builder v1.1
« Reply #6 on: August 21, 2019, 03:59:58 am »
As you said, most of your work is related to handling the ini file, but why reinvent the wheel?

Keeping in mind that your ini file isn't standard (because you have no Name = Value pairs in the rom list section), you can still use the method described here to read the file into a hash that you can reference pretty easily. You will, however, have to treat your roms list lines as your default case, similar to the way they treat comments, with an incremental counter.

Note that hashes also take care of the case problems with your search strings, since hash keys are case-insensitive. Plus, the order of the file doesn't matter, since you'll pull what you want, when you want it, and in the sort order that you ask for it.

Here's an adaptation of that web page's code that's more suited to your case. At the bottom are a couple examples of how to get at the data.
Code: [Select]
function Get-IniContent ($filePath)
{
    $ini = @{}
    switch -regex -file $FilePath
    {
        "^(\s+)?;|^\s*$" # Comment or blank/whitespace line
        {
            continue
        }
        “^\[(.+)\]” # Section
        {
            $section = $matches[1]
            $ini[$section] = @{}
            $CommentCount = 0
            $listlineCount = 0
            continue
        }
        “(.+?)\s*=\s*(.*)” # Key
        {
            $name,$value = $matches[1..2]
            $ini[$section][$name] = $value
            continue
        }
        Default
        {
            $value = $_
            $listlineCount = $listlineCount + 1
            $name = "ListLine" + $listlineCount.tostring("000000")
            $ini[$section][$name] = $value
        }
    }
    return $ini
}

$ini=Get-IniContent .\MLB_Arcade_Settings.ini
$root_folder = $ini['settings']['mame root']
$scrap_list = $ini['folders to scrap'].GetEnumerator()|sort -Property name|%{$_.value}

Mike A

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 5884
  • Last login:Yesterday at 01:43:45 pm
  • This plan is foolproof
Re: MAME Library Builder v1.1
« Reply #7 on: August 21, 2019, 06:33:17 am »
This is why my MAME cab has a dozen games on it. :o

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #8 on: August 21, 2019, 07:53:44 pm »
As you said, most of your work is related to handling the ini file, but why reinvent the wheel?

Keeping in mind that your ini file isn't standard (because you have no Name = Value pairs in the rom list section), you can still use the method described here to read the file into a hash that you can reference pretty easily. You will, however, have to treat your roms list lines as your default case, similar to the way they treat comments, with an incremental counter.

Note that hashes also take care of the case problems with your search strings, since hash keys are case-insensitive. Plus, the order of the file doesn't matter, since you'll pull what you want, when you want it, and in the sort order that you ask for it.

Here's an adaptation of that web page's code that's more suited to your case. At the bottom are a couple examples of how to get at the data.
Code: [Select]
function Get-IniContent ($filePath)
{
    $ini = @{}
    switch -regex -file $FilePath
    {
        "^(\s+)?;|^\s*$" # Comment or blank/whitespace line
        {
            continue
        }
        “^\[(.+)\]” # Section
        {
            $section = $matches[1]
            $ini[$section] = @{}
            $CommentCount = 0
            $listlineCount = 0
            continue
        }
        “(.+?)\s*=\s*(.*)” # Key
        {
            $name,$value = $matches[1..2]
            $ini[$section][$name] = $value
            continue
        }
        Default
        {
            $value = $_
            $listlineCount = $listlineCount + 1
            $name = "ListLine" + $listlineCount.tostring("000000")
            $ini[$section][$name] = $value
        }
    }
    return $ini
}

$ini=Get-IniContent .\MLB_Arcade_Settings.ini
$root_folder = $ini['settings']['mame root']
$scrap_list = $ini['folders to scrap'].GetEnumerator()|sort -Property name|%{$_.value}

I'm gonna take a gander at it and see if I can make head or tails of it. My only issue right off is wont this cause issues with the [Folders to Copy] section that is laid out the same as ROMs? Id be able to flip ur example pretty quick except for the fact that from what I can tell wouldn't all the Folders in that section get added to the Default section? I'd also be willing to change the layout of the ini since it was just a thought on how to keep the two lists simple (not requiring Name = for every rom)

nexusmtz

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 489
  • Last login:June 01, 2022, 03:14:22 am
Re: MAME Library Builder v1.1
« Reply #9 on: August 21, 2019, 08:41:35 pm »
Like [folders to scrap], each list is in it's own section, so there's no conflict.
$folder_list = $ini['Folders to Copy'].GetEnumerator()|sort -Property name|select value
$rom_list = $ini['ROMs List'].GetEnumerator()|sort -Property name|select value

Yes, all of them got created by the code in the Default section of the Switch statement, not of the ini. But because we already know the name of the current section, that gets used as the first dimension of the two-dimension hash-of-hashes. The name of the section got set (and, coincidentally, the counter set to zero) when the section header was encountered. So $ini['roms list']['listline000001'] is 1941u.*, and $ini['folders to copy']['listline000001'] is BIOS.
« Last Edit: August 21, 2019, 08:50:00 pm by nexusmtz »

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #10 on: August 21, 2019, 09:08:22 pm »
Oh absolutely perfect thank you for all the help that is a slick solution. I'm really loving PowerShell it has so many shortcut compared to batch scripting. I was just realizing once I get thing wrote out I should probably just try to auto populate the CHDs from the rom list. I can easily search for folders in the ROMs folder that match the rom list and add a second line to check ROMs\CHDs for those that keep them there. Makes it alot easier on the user and will shorten ini files. Feel like I should have though of it earlier though!
« Last Edit: August 21, 2019, 09:11:17 pm by Dracrius »

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #11 on: August 21, 2019, 11:24:33 pm »
I almost had it but its erroring like crazy. It hits the ini function and dies. I read through it and its missing a '}' somewhere but I can't figure it out atm. Unfortunately the method you linked to had the same error of one '}' missing. I'm admittedly overtired (to many 3am nights learning haha) so I'll take a look tomorrow to see if I can fix it but thanks again this got me leaps and bounds ahead of where I was.

nexusmtz

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 489
  • Last login:June 01, 2022, 03:14:22 am
Re: MAME Library Builder v1.1
« Reply #12 on: August 22, 2019, 01:03:17 am »
Get some sleep! The code will still be here in the morning.

The mismatch is a bad copy-paste. If it happened with both the codebox here, and the linked url, your browser or text editor doesn't like the quotes that the author used. They seem ok for me. I can select the codebox, copy, then paste into Windows Powershell ISE without a problem. Look at your lines in the select statement and make sure they look like what you see in the code box of my message.

Function definitions go outside the code. You only define it once, not each time you loop. Pull that whole function definition up to the top of your script.

Glad you had to let go of the globbing on the root. Makes more sense that way, and there were too many caveats to explain with it.
Glad you switched to scrape. Scrap was bothering me, but it's just a word :)

Change both |select value to |%{$_.value} to match the way you are used to using them. (plain text list instead of object array) That's my mistake, not yours.

$scap_destination_folder will cause interesting results.

Think about what's an error and what isn't. To me, not asking to copy a list of folders isn't an error. You'll have to decide if you want to handle there not being a section, or insist that the section be there, even if it's empty. On the other hand, it doesn't make much sense to provide a rom list if you have no folders to scrape, so that error probably does belong.

Those fixes should get you running. From there you can figure out any logic errors.
You're getting there!
« Last Edit: August 22, 2019, 01:18:02 am by nexusmtz »

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #13 on: August 22, 2019, 07:02:22 am »
 :laugh2: I noticed the "Scrap" miss spelling and I was like oh ---steaming pile of meadow muffin--- hopefully noone noticed... Silent fix. Also just before  I crashed I removed the [Folder to Copy] error because your right I honestly don't even care if it's there. Especially when I tack on auto CHDs scraping after this build.

I looked over your code block and their code from my phone, they both look fine. I'll have to see if I was that tired last night or if it was my browser playing tricks on me.
« Last Edit: August 22, 2019, 07:18:14 am by Dracrius »

nexusmtz

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 489
  • Last login:June 01, 2022, 03:14:22 am
Re: MAME Library Builder v1.1
« Reply #14 on: August 22, 2019, 02:29:08 pm »
Look at your lines in the select statement and make sure they look like what you see in the code box of my message.
Correction: (I guess I need sleep too)
Switch statement, not select. It's the quotes on the conditions that didn't copy correctly.

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #15 on: August 22, 2019, 10:46:44 pm »
Sooo weird ya If I copy and paste from the code block the first part of switch is fine but the second two both use odd [“] rather then ["] I can very clearly see it now that I'm looking in Notepad++

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #16 on: August 22, 2019, 11:31:34 pm »
Ok now I've got a run able prerelease v2 but its not treating the ini exactly as I expected. I had to remove the quotations from around the values of the two settings and wild card in the ini doesn't seem to work. It took *MAME* literally and would fail until I change it in the ini to my specific folder MAME 0.212. I'd like the ini to accept wild cards so that the default *MAME* and other combinations could be used to search generically so you wouldnt have to edit the ini every update.

I have loads of other issue to though it seems I want to get it tonight but I should probably sleep again haha Its creating folders but only copied the files in the [folders to copy] folders.

 :banghead: Duh with it taking wildcards as characters its not scraping any files because its looking for "1941u.*" not 1941u.[some extension]. I can think of a quick hack for roms list but is there a way to just change how they are store or presented so that will let wildcards from the ini work?

I also just realized it dumped all the files that should have been in the folders in made in the root with the script. I really dont understand that part.

I'll take your opinion on this next part if you dont mind. My "hack" is to add the .* to the code but would $rom.* even work. And more importantly if I can do it with way do you think I should? It make creating a rom list even easier no need for special extensions but I'm trying to think if there is a negative to removing the extension. I guess if they put anything normal extensions it would break anyways.

nexusmtz

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 489
  • Last login:June 01, 2022, 03:14:22 am
Re: MAME Library Builder v1.1
« Reply #17 on: August 23, 2019, 03:39:31 am »
Good job getting it running. Now you've got the bugs / misunderstandings to clear up.

You can't expect *MAME* to mean "any folder with MAME in it's name" everywhere in powershell. It only means that in get-childitem, because get-childitem allows globbing. Even where it is supported, it's hard to recommend because it's difficult to know what you're going to get for results.

I still recommend doing a get-childitem with the pattern that you get from the ini and selecting one of the results. That way you'll know what to put into $destination_folder, since that can't accept * as part of the name. If you want to give the user a choice of source folder, the laziest way is $selected_object = gci *mame*|select name|Out-GridView -Title 'Choose Source Folder' -PassThru, but -passthrough only works on powershell v3 and above.

At the very least, get things working with one specific folder before you try to add globbing back in.

As for the wildcards in the rom list, why do you believe that they're not being processed as wildcards? You said the files are getting copied to the current directory. That means they're being found.

The reason that they're being copied to the current directory escapes me at the moment. You R going to have to find that error yourself. Yes, that's a hint. On the serious side, if you use Windows Powershell ISE, you can more easily find those kinds of errors, since you can set breakpoints and display variables. When you find out, for example, that $scrape_destination_folder doesn't have a value, you can look back and see why.



« Last Edit: August 23, 2019, 03:42:04 am by nexusmtz »

Dracrius

  • Trade Count: (0)
  • Full Member
  • ***
  • Offline Offline
  • Posts: 64
  • Last login:September 13, 2019, 06:32:53 am
  • Proud Builder of a MAME Cabinet!
Re: MAME Library Builder v1.1
« Reply #18 on: August 23, 2019, 07:12:11 am »
Your to kind! I swear I didn't intend you to solve all my bugs for me but those hints are to good. I'm on my way to work now and I'm pretty sure I know most of what I have to fix.

I'll give in and use PowerShell for my polishing but I doubt I'll give up using Notepad++ for my first drafts and main editing it's just so clean and diverse. I do need to download a PowerShell syntax highlighter file for it (one of the many reasons I love notepad++) especially since even without one it already understands it's code and really I should have seen those quotes for instance because it didn't shade those two lines at all but it does shade anything within "".
« Last Edit: August 23, 2019, 07:15:18 am by Dracrius »