Download missing samples & art files (ie. snaps) automatically

[MKChamp]

Here's a little program I made for myself, but thought there might be some of you out there that could use it. What it does is allow you to validate what snaps/cabinets/marquees/cpanels/samples/flyers/pcbs/artworks you have and automatically download the ones you are missing and put them where they belong. You can specify what files you want to validate(I personally don't do flyers and pcbs) and if you want to check for all roms sets or just for the sets of roms you have locally.

It grabs the art files (snaps/cabinets/etc) from the resources at MAWS, grabs the samples from Twisty's and the artwork zips from Mr.Do's...so if any of these sites have a problem with downloading from them, let me know and I will remove this post. Do NOT use this to download full sets because that would suck too much bandwidth from these sites!!!

I wanted to also include video snaps, but I do not know of a source that has the individual avis...just torrents/rars of all of them. If anyone knows of such a place, give me the heads up!

Please let me know of any bugs you may find...I literally threw this together so I call this thing A SAD Program (Automatic Sample and Art Download Program). If you saw the source, you would say "sad". Tested it with XP and Vista...I don't think 98/ME will be a problem...

FYI, your mame version MUST have listfull/listclones/listsamples support.

Lastly, if something like this already exists, I will just consider this a VB learning experience.

04-05-08: Uploaded new version with artwork zip validation (bezel/overlay/etc) and a few other modifications...

04-06-08: Uploaded version 1.0.2...trying to get the reading of listfull files on some computers to work. It now uses the dir the mame exe is in to store/access temp files created....

04-06-08: Uploaded version 1.0.3...GUI changes and work on proper execution on different platforms...

04-06-08: Uploaded version 1.0.4...Created installer so it will run on computers that don't have the needed dlls. Also sets default path in folder section to your mame directory

04-07-08 Uploaded version 1.0.5...Main difference in this version is that it uses an unavailable list...basically a list that has all the files that do NOT exists for download. This speeds up downloading the existing files and also clears up the confusion of why some files still show up as missing in the list.

04-08-08 Uploaded version 1.0.6...Changed the shelling of creating the games/clone/sample list. HOPEFULLY this will fix the issue running on 64 bit systems(I obviously have 32bit). Also, drastically changed the way it downloads...should drastically speed it up and also prevent crashes when trying to download many.

04-09-08 Uploaded version 1.0.7...lucky number seven. Speed up the validation routines...and MOST importantly...should work for everyone now...no more 'fulllist error' crap :-)

04-09-08 Looks like Mameworld is now blocking IPs so this is discontinued!

[crummy]

Wow, that is a very cool tool.

[pie taster]

This worked like a champ!!
 


Fantastic job!!!  Thanks so much!

[Ginsu Victim]

Thanks for this.
The only problem I can see is if people start using this to download everything, rather than getting packs.
You get several people downloading a whole site worth of images all at once, that's quite a bandwidth drain.

[MKChamp]

Thanks for this.
The only problem I can see is if people start using this to download everything, rather than getting packs.
You get several people downloading a whole site worth of images all at once, that's quite a bandwidth drain.

Yes, it should be pointed out to NOT use it to get full sets...but to use to 'complete' your sets. Hopefully, the people that use it won't be lazy and use it for complete set builds.

[Ginsu Victim]

Then you might want to look the other way as people use it for the purpose of getting every single image they don't already have...when they have none to begin with.
The majority HERE might not do it, but there will be a decent number of people doing exactly that.

[keirondrk]

loving the idea!!! hopefully most people will get extra packs and just update from there like I have done myself... although unfortunately the updater seems to keep saying I have the same art missing from what I've noticed even after getting it to download and recheck. strange... will try it again.

luvin the utils!

[MKChamp]

loving the idea!!! hopefully most people will get extra packs and just update from there like I have done myself... although unfortunately the updater seems to keep saying I have the same art missing from what I've noticed even after getting it to download and recheck. strange... will try it again.

luvin the utils!

Glad u like it :-)
If it says you are still missing files after downloading and rechecking, then those files are not available (don't exists)...so at that point you have everything available from MAWS...

[unclet]

I like the idea of having a tool like this to allow people who have none of this information to obtain it all.  However, you should probably update the program to have a maximum download limit imposed so if people who have none of this information decide to leech "everything" from the web then they will still be able to do so but it might take a couple weeks (or longer) to get everything they need since you would limit the number of downloads allowed per day (or something like that)

Just a thought. 

[DaOld Man]

Great job!!
 
I suggest that you put a "# of files left" box to let user know how many more its got to go..

Neat utility.. I like it!

[Dariusz]

Hi,

I need some assistance. I get the following message when using the app. I using mame 0123 from the offcial web site, and runing the mame.exe.

Thanks

[MKChamp]

Hi,

I need some assistance. I get the following message when using the app. I using mame 0123 from the offcial web site, and runing the mame.exe.

Thanks

Make sure the directory(folder) the app is in is not read only...right click the folder and select properties and uncheck the read only option. If that doesn't work and you are running on vista, try running it as administrator. Let me know if that works...

[MKChamp]

Great job!!
 
I suggest that you put a "# of files left" box to let user know how many more its got to go..

Neat utility.. I like it!

Uploaded updated version of the app. 3 things added:

1) Displays number of files left to download
2) Added artwork validation (zip files with bezel/overlay/etc)
3) DISCLAIMER displays when app is run to NOT use to download full sets!!

[Dariusz]

Hi,

I need some assistance. I get the following message when using the app. I using mame 0123 from the offcial web site, and runing the mame.exe.

Thanks

Make sure the directory(folder) the app is in is not read only...right click the folder and select properties and uncheck the read only option. If that doesn't work and you are running on vista, try running it as administrator. Let me know if that works...

Thanks for the reply,

I do have Vista. I have updated the software, and the recomendation to disable read access was attempted. But for some reason windows does not allow me to disable the read access of the folder & files ? I do have admin control, and it look like windows is dis allowing me to do so on all my drives? I have no idea why windowes is not allowing me to disable the read access of the folders/files? Any idea....

Thanks

Dariusz

[massive88]

Im running windows XP with Mame 0.117 and I get the same listfull error.

I tried downloading both 0.124 and 0.121 from the official website and pointing the program to those binaries, and I still get the same error...

Any idea whats going on or how to fix it?  I am using the most recent version of your program, downloaded this morning.

[MKChamp]

Ok, guys...try to download ver. 1.0.2. I changed the location of the temp files created to the directory the mame executable is in and changed 'file detection' in the app. I tested it back to mame 103 and it worked on my computer.

Dariusz and massive88...let me know if that works. If it doesn't and you are interested in helping this get fixed, I may send you a 'special' version that creates a log file with debug statements in it. I'd have to put the debug statements in and I am lazy...so hopefully the version I uploaded works <crosses fingers>!

[ScooterMAC]

Cool program Champ, But I'm having some problems. When I click on the download button it downloads it, but to where I don't know. When I run the check again, it still says I'm still missing the flyer's I just DL'ed. I'm using Mame32 114 if that help. Thanks.

[MKChamp]

Cool program Champ, But I'm having some problems. When I click on the download button it downloads it, but to where I don't know. When I run the check again, it still says I'm still missing the flyer's I just DL'ed. I'm using Mame32 114 if that help. Thanks.

It downloads the files to the path specified. So, for example, if you have your flyer path as c:\mame\flyers, then the flyer files will be downloaded there. The label that states "downloading XXX' is a little misleading....it means it is going out (it goes to MAWS for flyers) to download that particular flyer...HOWEVER, if that particualar flyer doesn't actually exists, then there's nothing to download...so everytime you rescan, those flyers will still be missing. So, basically at that point you should have every flyer that is currently 'available'.

[Dariusz]

MJChamp,

Version 1.02 works for me now. thank you works great.

Dariusz

[ScooterMAC]

Cool, thanks for the info.

[nuka1195]

Very nice, thank you.

One note you have a weird "Norton Systemworks" type scrolling animation with your browse dialogs when you click the scrollbars. They always scroll down. I've only seen similar animation with Norton tools.

[headkaze]

Hey MKChamp great to see more coders on these forums Can I make a few suggestions if you don't mind?

Just with the layout organisation of the GUI. First thing of note is it's a bit cluttered. This tool will primarily be used by cab users. People who run cabs often run their machines in a low resolution. Your window is 653 x 639 pixels which is too big to fit on a 640 x 480 and still quite awkward on 800 x 600. Can I suggest you use tab pages to break the GUI up into parts. You could have the main tab "Output" which has a textbox that displays each file as it's processed and the appropriate buttons to download each type or artwork. The next tab could be "Folders" where you can set your artwork paths. Since you only need to set your paths once you could go to that tab when it's first run (ie. If MKSupport.cfg is not found). As a general rule-of-thumb try not to create programs with a window larger than 600 x 400.

Other minor things of note
- You have a custom "folder browser" dialog. It's much nicer using Window's built in folder browser. Although there is no common control in VB6 to do this you can use the SHBrowseForFolder API function.
- Not sure how your outputting info but consider appending text to a textbox. Adding a line for each ROM as it's being processed.
- Consider using a statusbar to indicate the mode the program is in Eg. "Idle.", "Processing.", "Error." etc. You need to add "Microsoft Windows Common Controls 6.0" component" to get this control.
- Most important point, consider downloading Visual Studio 2008 Express and write your program in VB.NET. If makes life a whole lot easier and you can do more in less time using the magic of .NET.

BTW A bug I found is that it doesn't seem to work using Mame64 on my XP 64 machine. The dialog "Unable to create game list. Please check your mame version has listfull support". My Mame version is 0123.

[MKChamp]

Thanks for the input Headkaze...
I've shrunk it down about the most I can...it got it under 640x480. I've broke it into tabs. Hopefully, they make sense. I didn't add the roms to a text box as they were being processed because it seemed to make the processing considerable slower and I felt speed would be more important. I changed the folder selection box to what you suggested and although I didn't add a 'statusbar', I tried to include more status's in my status label at the bottom.

Most importantly, I added more OS's to the identify routine so if you could try it again on your xp 64 and let me know if it works, that would be great.

Once again, thanks a ton for all your suggestions and keep 'em coming :-)

Oh...I don't have 2008 but I had downloaded 2005 and looked at doing this in that, but I am pretty decent at vb6 and felt lost in 2005. Since I was originally doing this just for myself, I wanted to throw something together fast without having to learn new functionality,etc....yea...I've said it before...I am LAZY.

[joebells]

Hi,

I'm running vista x64 here and it won't run it says its missing comdlg32x.ocx or one of its dependencies?

[massive88]

Heyas.  1.0.3 worked for my mame 0.117, but i tried first using just the fresh downloaded binaries for 0.124 and 0.121, and it didnt work.  I pointed it to my 0.117 though and away we go.

Odd that it didnt work with the other binaries, but it worked with my main one, so good enough for me!  Thanks a bunch for the tool!

Let me know if you want me to trouble shoot the other two for ya.

[MKChamp]

Uploaded 1.0.4. This one is in an installer. This should fix your issue, joebells.

Massive88...glad it worked for your 'main' version. Could you try moving the 121 and 124 mame exe into the same dir as your 117, then point to them and run it and see if it works? Also, it you don't mind, try it with the one I just posted.

Thanks!

[joebells]

yep thats working now, thanks, I am missing more artwork than I thought I was. I had what I thought was a prety full set from 123 and just moved it over to 124 and on snaps it found 670 or so, and for cabinets it found over a thousand and so on down the list. It was able to pull in 370 snaps. Its still running through.

Thanks. Is there a way to get it without an installer in the future? Not trying to be rude in anyway, just prefer not to install what I don't have to, so figure it can't hurt to ask.

Thanks this saves me from manually going and downloading a bunch of seperate files.

Well it just crashed in the middle of the cabinets. I restarted and am doing them one by one now. No biggie just thought I'd let you know.

[keirondrk]

thanks matey for the 1.4 version.... 

will give it a try... even the first version has worked fine for me on a vista ultimate 32bit pc..
straight outta the zip..

just confused me when it kept saying it was downloading things again and again.. thanks for explaining why hehe, in reality I don't think it ever downloaded anything as I had them all in the first place but I can see it works and when updates appear it will make the mini updates.. 

[Dariusz]

Hi MKChamp,


I have a new problems which I hope you can assist me with. I thought earlier te program was working until I got inconsistant results. The results being (including your current release  v4):

Mame 123 I have missing always 134 snaps.
Mame 124 I have missing always 180 snaps.

Does that sound right?

The app does indicate its downloading the missing file above when run, but then re-ranning the app, it gives me the same result as above again, consistanlty.

Your help again appreciated.

Thanks,,
Dariusz

[joebells]

maybe the server doesn't have them? I am missing 300 snaps now I believe after this tool was able to pull down around 300 or so(mame124). From the other categories like cabinet and such I am missing thousands of them after running this. I am thnking that they just aren't out there. You probably see it saying its downloading all the images at the bottom but I think thats just the program requesting them from the server. The program doesn't know what files the server has so it requests them all and accepts whatever ones the server gives it. Thats just my guess though. I could be wrong.

[Teknique]

Hi,

I need some assistance. I get the following message when using the app. I using mame 0123 from the offcial web site, and runing the mame.exe.

Thanks

Make sure the directory(folder) the app is in is not read only...right click the folder and select properties and uncheck the read only option. If that doesn't work and you are running on vista, try running it as administrator. Let me know if that works...

Not working for me so far.
Im using XP pro- and taking the read-only atribute off folders is something that I have thus far been unable to do.  I used MS database to change a registry entry for the issue- no good.  Tried using the attrib command at a command prompt- no good, tried unchecking the greyed out read-only box to set properties- no good, tried setting Protected Files service to manual and stopping the service- no good.  Tried sharing or unsharing the folder, tried resetting the default folder parameters, tried to make a new folder and remove the read only attribute before putting your app inside, tried placing the app and related files all on desktop to run- all no good.

Nothing seems to remove the read-only attribute, trying all these still gives me the 'make sure your version has listfull' error.

Any suggestions?

Tek

[nuka1195]

put the app in the same folder as mame.exe.

[Teknique]

put the app in the same folder as mame.exe.

Didnt work with .123i686 and tried also .124 regular.

Also tried to un-checking the "Remember each folder's view settings" in
the Folder Options View dialog.  No good.

Possibly I am doing something else wrong and am focusing on the read-only thing too much.  I'll play around a bit more.

Thanks

Tek

[MKChamp]

Uploaded 1.0.5. The main thing this one has is an unavailable list it now uses. This list contains the files that do not exist for download. So, when you run your validation, it will only show you your missing files that CAN be downloaded (which should consist of all that exists). This should speed up the download part and clear up the confusion of thinking you downloaded files when really you didn't because they don't exists.

Teknique,
Not sure what could be causing your issue...have you downloaded the latest one? It may have been an issue with 64 bit systems...Headkaze had the same issue and I put a fix in...haven't heard if it fix it or not.

[Teknique]

Uploaded 1.0.5. The main thing this one has is an unavailable list it now uses. This list contains the files that do not exist for download. So, when you run your validation, it will only show you your missing files that CAN be downloaded (which should consist of all that exists). This should speed up the download part and clear up the confusion of thinking you downloaded files when really you didn't because they don't exists.

Teknique,
Not sure what could be causing your issue...have you downloaded the latest one? It may have been an issue with 64 bit systems...Headkaze had the same issue and I put a fix in...haven't heard if it fix it or not.

Tried again with v1.5  no luck.

I do have a dual core machine - Athlon 64 4200+ X2 if that makes a difference.

Tek

[jasonbar]

Hi-

I tried running your program on my mamepp v.115 & mame32 v.115u2 & got the listfull support error.  Is there some list somewhere that can tell me what MAME versions have the listfull support?

Thanks,
-Jason

[Dariusz]

MKChamp,

I've done some more test but this time with version 1.2 and 1.5. Using the example of the snap directory, with V1.2 I have missing 180 snaps, but when I ran version 1.5, I have 48 missing. But if I run again v1.2 (freshly installed), I still get 180 snaps missing. But with 1.5 is re-run, I still get 48 missing.

Could there be a problem with the app? as I do not know what the true outcome is. Or is v1.5 filtering somthing which I unaware of.

Just some feedback.  Keep up the good work.

Dariusz

[keirondrk]

  the new 1.5 is working good with the available file detection  now stating I do not need any updates apart from on the artwork button... so giving that a go and will rescan.

[MKChamp]

MKChamp,

I've done some more test but this time with version 1.2 and 1.5. Using the example of the snap directory, with V1.2 I have missing 180 snaps, but when I ran version 1.5, I have 48 missing. But if I run again v1.2 (freshly installed), I still get 180 snaps missing. But with 1.5 is re-run, I still get 48 missing.

Could there be a problem with the app? as I do not know what the true outcome is. Or is v1.5 filtering somthing which I unaware of.

Just some feedback.  Keep up the good work.

Dariusz

Yes..1.5 is only displaying the missing files that ARE available for download (according to the list it downloads to check for availability). All previous versions before 1.5 would just show all your missing files regardless if they exist to download or not. This seemed to be confusing alot of people because they would think they downloaded them, yet they were still showing up as missing. Thanks for 'debugging' the app! Keep it up ;-)

[keirondrk]

all worked perfectly got 107 artworks I didn't realise I needed and rechecked with nothing needed getting!!!.

[jasonbar]

Hi-

I answered my question--MAMEPP 115 does have listfull support, which I verified from the command line.

I made sure that my mame folder (and everything underneath it) was read/write.

I tried putting the ASAD executable in the same directory as my mame app & got the same error.

I'm running XP Home.


Any suggestions?


Thank you,
-Jason

[MKChamp]

Uploaded 1.0.6. I am HOPING this fixes the creation of the game/clone/sample files that is causing some people to get the "listfull support" error. Please try it out and let me know if it does. I don't have a 64 bit OS, so it is impossible for me to test.
Also, it this release, the downloading part is completely revamp. Should drastically speed things up...test it out!

[Dariusz]

MKChamp,

Thank for the reply. Re my last post, and you feedback. If v1.5 is saying I'm missing 48, and have the 48 available for download, why then after downloading the 48 snaps, and re-running the app, why does it still give me a result of 48 missing. I would have thought the missing 48 files which would have been downloaded if it reporting that the files are available for download?

Thanks,

Dariusz

[Teknique]

Sorry MK,

1.6 didnt fix my problem.  It did seem to get further though before giving me the same listfull error.  Status said 'creating games list' then 'creating clones list', then 'creating samples list', then it errored out.  Prior to this version it would error almost immediately.

Id be happy to help debug this for you if you wanted to send me a logging version of the program.

macantrell@hotmail.com

Tek

[MKChamp]

Ok OK....
Uploaded 1.7. Thanks to Teknique debugging some stuff for me...this one SHOULD work for everyone! :-) No more 'error creating game list' crap. Problem was for paths that had a space in the name...like yours Teknique.."mame .123".  Validate away!

Also, this version speeds up the validation part...minor if you couldn't validate anyways!

I plan on updating the unavailable list for every major(non u) release of mame to try to stay current...so if you are validating again a u release, it may state there are files missing that you won't be able to download.

Anyway...let me know if this release has any bugs.

[jasonbar]

Bam!  1.7 worked!  I did have a space in one of the folder names in my path--I guess that was my problem.

Very very cool application.  The download was pretty quick.

I'm really impressed by people who come up w/ things like this (I'm a mechanical engineer--software is over my head...).     The only constructive criticism I could give is that my progress bar & the text underneath was going by so quickly that I really couldn't tell what my actual progress was.  It didn't really matter to me, but I thought it was fun trying to guess how far along I was... :]


Thank you!!!
-Jason

[Dariusz]

Hi MKChamp,

I've tried 1.7, and I get the message when downloading 'cannot connect'. It may just be the server that's down when I'd tried  connecting. I will try  again later.

Thanks,

Dariusz

[arzoo]

Just tried 1.7. First off - great idea - thanks MKChamp! 

Here's how it worked (and didn't work for me)... After validation, I was missing a couple dozen files in all the groups with the exception of Artwork which was missing around 250 files. I ran 'Download All' and the status bar indicated it was downloading the artwork files. After the artwork download was complete, the status changed to 'Finished'. I immediately ran the validation again and this time it indicated I was missing approximately 250 files in all the groups - odd. So I checked the actual folders, sorting by timestamp. The artwork folder had about 220 new zip files; the other folders had none or one new file.

I ran ASAD again, validated, and this time the missing file list went back to indicating a couple dozen missing for each folder with artwork missing 209. So I tried downloading the folders individually. Each one indicated a status of "Couldn't Connect". Maybe the source sites are down, but that seems unlikely.

So it seems I was able to pull down about 220 new artwork files and that's about it. 

[MKChamp]

Ummmm....mameworld (where maws is hosted) is currently DOWN. I HOPE this has nothing to do with people using this thing...if it does...it is GONE.
The 'couldn't connect' is displayed when either the site is down or the file it is trying to get isn't available. Basically, if you want to attach your missing.txt file (generated when maws IS up) I can add those files to the unavailable list so it has all the files that do not exist.

[MKChamp]

Ok...doing a little investigating...looks like mameworld is blocking IP's....so....it was fun while it lasted. Thanks for all the input you guys gave. Learning experience = over :-)

[arzoo]

Ok...doing a little investigating...looks like mameworld is blocking IP's....so....it was fun while it lasted. Thanks for all the input you guys gave. Learning experience = over :-)

Bummer