Merge info from two XML lists to rename GameBase roms

[Benzinbruder]

Hi guys,

So here's that: Last night I have finally managed to launch C64 Games from Hyperlaunch via GBLauncher from the GameBase Collection. Boy, oh boy, it was tricky for me.
I still have to directly call the GBLauncher.exe in my .ahk file since HyperLaunch seems to forward the executable variable with indirect paths

Code: [Select]

..\..\GBLauncher.exe
rather than

Code: [Select]

C:\GameBase\GBLauncher.exe
so the appropriate line in my .ahk file is now something like

Code: [Select]

Run("C:\GameBase\GBLauncher db=GBC_v10 fn=" . romName " wait=0"
BTW, anybody here can explain what the . mean?  (Edit: Ah, its a regex for any single character... why would i need that here? It doesn't work without it...)


But here's the thing I have to deal with now:
I have two .xml files at hand, one which I downloaded from HyperList, the other one which I generated by exporting from GameBase and putting it through RomLister (as described in the GameBase wiki).

So, in the HyperList .xml file all the roms are named to correctly match with Hyperspin and emumovies and so on.
In the GameBase .xml file the names of the games match the names in the HyperList .xml file more or less, and it contains the ACTUAL names of the roms that I have.

So how do I rename the thousands of roms to match the rom names in the HyperList .xml file which is the one I have to use to make the videos, artwork, etc. work in Hyperspin?

The problem is that I probably can't use HyperMatch to correct the rom names because it won't find any matches when comparing the game names with the rom names in my collection.
The rom names in GameBase are quite cryptic and I see no way how to do it without having the info that is in the GameBase .xml file.

Does anybody have a clue what I mean? What could I do to rename my roms in an efficient way?

[Benzinbruder]

I should probably make myself more clear .

See, here's a sample for a file entry of the xml file I created witch GameBase:
My roms are named pretty awkward, the GameBase-created .xml file contains entries like this:

Code: [Select]

<game>
        <game name="1942_00016_02">
<description>1942</description>
<cloneof></cloneof>
<manufacturer>Elite/Hit-Pak</manufacturer>
<year>1986</year>
<genre>Shoot&apos;em Up - V-Scrolling</genre>
</game>
However, the HyperList Commodore 64 xml file looks like this:

Code: [Select]

<game name="1942 (1986)(Elite)[cr BAM][t BAM]" index="" image="">
<description>1942</description>
<cloneof></cloneof>
<crc></crc>
<manufacturer>Elite</manufacturer>
<year>1986</year>
<genre>Shooter</genre>
<rating>HSRS - PG (Parental Guidance)</rating>
<enabled>Yes</enabled>
</game>

Do you see my dilemma?
I'd need a tool that would look for matches of descriptions and then renames the roms from e.g. 1942_00016_02.zip to 1942 (1986)(Elite)[cr BAM][t BAM].zip.

[emb]

Tty HyperTools, there is a ROM renamer tool to help. You might also want to play around with the filter settings in there if you are having issues.

I don't support the tool anymore, although there are a few on the HyperSpin forums who could help you if you're having issues.

[NOP]

I kinda think that romlister might help here, but I need your xml files to play around with before I go writing checks my butt can't cash.

can you post them somewhere or zip 'em up and mail them to jeff at silent.net

[Benzinbruder]

I kinda think that romlister might help here, but I need your xml files to play around with before I go writing checks my butt can't cash.

can you post them somewhere or zip 'em up and mail them to jeff at silent.net

As soon as I have access to the files again I'll zip them and send them to the applied address!

It'd be really great if that'd work. I have tried fiddling with another (TOSEC) romset in the meantime, but a lot of games are missing or not working. The GameBase collection seems to be of superior quality. Thank's a lot for looking into this!

One cirtical thing just came to my mind though: I am not sure if GameBase would recognize the renamed roms! If not, all the renaming of the roms would be futile. Oh boy, the longer I think about it, the more I am sure this will be the case... I am pretty sure that the GameBase database relies on the quirky romnames to work. 
So, I have to pass something like

Code: [Select]

Run("C:\GameBase\GBLauncher db=GBC_v10 fn=1942_00016_02 wait=0" as GBLauncher would probably not understand what to do with

Code: [Select]

Run("C:\GameBase\GBLauncher db=GBC_v10 fn=1942 (1986)(Elite)[cr BAM][t BAM] wait=0" because it is not a valid entry in the GBC_v10 database.

So to get this work in Hyperspin, instead of renaming the roms, one would have to use the info in the GameBase xml file and rename the artwork and movies to match the rom names!
e.g. 1942 (1986)(Elite)[cr BAM][t BAM].mp4 must be renamed to 1942_00016_02.mp4

Is this possible? Is Fat Match or Fuzzy Rename capable of doing this?

[NOP]

urgh.
I don't know about fat match, but fuzzyrename definitely will not be able to match some like "1942" to "1942 (1986)(Elite)[cr BAM][t BAM]" - they're just too different.  You might get several hits with other titles, but it would still leave you with a huge pile of work to do.  fuzzy definitely doesn't rename with underscores either, although I kinda like that feature idea...

it's looking more and more like you need a little custom work.  your XML files will be interesting to look at.

[Benzinbruder]

Yup, there's probably no way renaming my files with a "fuzzy algorithm".

But the thing is, the info for renaming the videos is already there: it's included in the xmls!
The task would be:

• Check the first video file in the folder. Oh, it's "1942 (1986)(Elite)[cr BAM][t BAM].mp4" (well, at least if emumovies matches the HyperList file naming).
  
  
• Open the HyperList xml and check the name of the game that matches this (rom) name. Result: It's "1942"
  
  
• Open the GameBase xml and search for "1942".
  
  
• Check what rom name (<description>) is dedicated to the <name> tag for the game "1942". The Result would be "1942_00016_02".
  
  
• Rename "1942 (1986)(Elite)[cr BAM][t BAM].mp4" to "1942_00016_02.mp4".
  
  
• Check the second video file in the HyperSpin media folder.
  
  
• ...
  
  
• Profit!

Is that feasible or not?
[/list]

[Benzinbruder]

Her're the xmls!

One is the .xml I got from HyperList.
It contains the roms named after the HyperSpin / emumovies convention and the according game names, next to a bunch of other information.

The other is the .xml I got from exporting the GameBase Commodore 64 database.
It contains the roms named as in the Game Base collection (i.e. the rom collection I have) as well as the according game names (which should be mostly identical to the game names in the HyperList .xml file).

[NOP]

got 'em.  it'll probably be the end of the weekend before I can get back to you.  busy busy busy!

[Benzinbruder]

Easy!

I am so glad somebody looks into this!
No need to get in a hurry over the weekend. 

[NOP]

ok, this may get you closer...

two step process.  the first is to rename your existing gb files to their descriptions, like this:

gb2desc.txt batch file:
ren "100KPYRA_00001_01.zip" "$100,000 Pyramid, The.zip"
ren "UNKNWN1-_23622_01.zip" "(Unknown) [Preview].zip"
ren "0ANDX_09437_01.zip" "0 and X.zip"
ren "1000KFM_22651_02.zip" "1,000 Kung-Fu Maniacs!.zip"
ren "10PIN_13849_01.zip" "10 Pin.zip"
ren "10THFRAM_00008_01.zip" "10th Frame.zip"
ren "180_00013_01.zip" "180.zip"
ren "19BOOTC0_00014_01.zip" "19 Part One - Boot Camp.zip"
ren "1942_00016_02.zip" "1942.zip"
ren "1943-BOM_00018_01.zip" "1943 - The Battle of Midway.zip"

then rename the description roms into hyperspin names.
desc2hs.txt batch file:
ren "10th Frame.zip" "10th Frame (1986)(Access Software)[cr TSC].zip"
ren "180 - One Hundred and Eighty.zip" "180 - One Hundred and Eighty (1986)(Mastertronic)[cr WCC].zip"
ren "1942.zip" "1942 (1986)(Elite)[cr BAM][t BAM].zip"
ren "1943 - The Battle of Midway.zip" "1943 - The Battle of Midway (1988)(Capcom)[cr Ikari][t +2 Ikari].zip"
ren "1st Division Manager.zip" "1st Division Manager (1993)(Codemasters)[cr SCS].zip"
ren "3D Pinball.zip" "3D Pinball (1988)(Mastertronic)[cr PAR].zip"
ren "3D Pool.zip" "3D Pool (1989)(Firebird Software)[cr REM][Docs][#297].zip"
ren "5th Gear.zip" "5th Gear (1988)(Hewson)[cr Ikari][t +7 Ikari].zip"
ren "Aardvark.zip" "Aardvark (1986)(Bug-Byte Software)[cr TWG].zip"
--------------

I see even from this example there will still be some holes where the description in both xml files don't quite match up. (180 for example)
maybe it won't be too bad to get those by hand. 

To do this I did have to make a slight modification to romlister, but I like it, so I'm keeping it in there.  I added the ability to use %DESC% as another variable when creating batch files.  So it was pretty easy to open each xml file, make an all files list, export that out to a batch file and simply ren "%ROM%.zip" "%DESC%.zip" and then ren "%DESC%.zip" "%ROM%.zip" in the other file.

I haven't released this version of romlister, but would gladly send it to you if you wanted to play around some.
good luck!

edit: crap, this doesn't address your next concern with replacing spaces with underscores in the target filename. 
I think I may have to make another mod to romlister for that...standby!

edit2: ok, I totally trainwrecked this one.  I was still focusing on renaming the roms and you're now looking to rename the movies.  that's probably easy enough to do in though.  In the meantime here's one with underscores.

edit3: right, here's 2 that convert the mp4s from HS format to descriptions, and then from descriptions to GB format.

hs2desc.txt:
ren "10th Frame (1986)(Access Software)[cr TSC].mp4" "10th Frame.mp4"
ren "180 - One Hundred and Eighty (1986)(Mastertronic)[cr WCC].mp4" "180 - One Hundred and Eighty.mp4"
ren "1942 (1986)(Elite)[cr BAM][t BAM].mp4" "1942.mp4"
ren "1943 - The Battle of Midway (1988)(Capcom)[cr Ikari][t +2 Ikari].mp4" "1943 - The Battle of Midway.mp4"
ren "1st Division Manager (1993)(Codemasters)[cr SCS].mp4" "1st Division Manager.mp4"
ren "3D Pinball (1988)(Mastertronic)[cr PAR].mp4" "3D Pinball.mp4"
ren "3D Pool (1989)(Firebird Software)[cr REM][Docs][#297].mp4" "3D Pool.mp4"
ren "5th Gear (1988)(Hewson)[cr Ikari][t +7 Ikari].mp4" "5th Gear.mp4"
ren "Aardvark (1986)(Bug-Byte Software)[cr TWG].mp4" "Aardvark.mp4"

desc2gb.txt
ren "$100,000 Pyramid, The.mp4" "100KPYRA_00001_01.mp4"
ren "(Unknown) [Preview].mp4" "UNKNWN1-_23622_01.mp4"
ren "0 and X.mp4" "0ANDX_09437_01.mp4"
ren "1,000 Kung-Fu Maniacs!.mp4" "1000KFM_22651_02.mp4"
ren "10 Pin.mp4" "10PIN_13849_01.mp4"
ren "10th Frame.mp4" "10THFRAM_00008_01.mp4"
ren "180.mp4" "180_00013_01.mp4"
ren "19 Part One - Boot Camp.mp4" "19BOOTC0_00014_01.mp4"
ren "1942.mp4" "1942_00016_02.mp4"
ren "1943 - The Battle of Midway.mp4" "1943-BOM_00018_01.mp4"
ren "1985 - The Day After.mp4" "1985_00021_01.mp4"

now you've got all the options you need. 

[Benzinbruder]

NOP, your THE MAN! 
That is so awesome!

I feel guilty for not checking the thread earlier as you mentioned it'll take you the weekend to look into it. 
Now I am covered in work and don't see any chance to look into this in detail until the next weekend.

However, I'll gladly dive into it then!! Really looking forward to try your "edit 3"!
Be sure to read my comments here as soon as I get this done!

Thanks man!

[Benzinbruder]

Sunday evening I tried renaming my roms with your batch files and well, it seems it was only partially successful. 
Sadly I have such a of work to do at the moment, that I probably wont have the chance to investigate this before next Sunday...

The current plan is:

• Rename the GameBase roms to HyperSpin convention
• HyperSync the videos and artwork
• Rename the videos and artwork to GameBase convention
• Rename the roms back to GameBase convention
• Profit

I'll keep you updated!

[Benzinbruder]

Just had to notice the that emuMovies naming convention doesn't match the HyperSpin xml file for A LOT of roms. 
I don't know how HyperSync manages this, but after reading this:
http://www.hyperlaunch.net/forum/showthread.php?103-gamebase64-module-support&s=84786fa0dd70382d56e696630cf1721b

I have decided to let go of that. It just doesn't seem to be worth it in the end.
NOP, I am really sorry for you seem to have put quite some work and effort into this. However, there'd still be such a load of manual editing necessary that I just do not think it would be worth the time.

Probably one could shorten this by applying some fuzzy matching in between (fuzzy match emuMovies to HS names, rename to description names, fuzzy match to GameBase description names, rename to GameBase names).

In the meantime, I'll use the publicly available TOSEC romset which seems to work with WinVice and HyperSync well enough.
Maybe I'll add some games manually as well into my list as well.

[NOP]

glad to have tried to help.  i like the feature that is now in romlister, so it was a win on this side anyway.

i haven't read your link yet, but will when i get time, and i bet there is a way to do this through some method somehow- we just have to get more creative.

i rarely give up on a dataset management problem.