Build Your Own Arcade Controls Forum

Main => Software Forum => Topic started by: Chris on December 30, 2006, 02:03:18 pm

Title: MAME Benchmarks
Post by: Chris on December 30, 2006, 02:03:18 pm
The question often comes up "What processor is good to run x?" and the response is usually ends up pointing to the MAME benchmarks page at http://benchmark.mameworld.net/ .  However, there really aren't that many benchmarks in that database, and lots of the ones that are there are for games that almost no one has any trouble running.

Yes, this request is selfish, 'cause I'm about to upgrade my cabinet... but if everyone could take a moment and add a few benchmarks to the database, I think it would be a great help for all of us.  I'm adding in data for an Athlon 64 at 1800 and 2000 MHz right now, and I'm going to go into my office later where I have access to my test lab and get some on some other machines.  Let's fill that database up!  :)

--Chris
Title: Re: MAME Benchmarks
Post by: Chris on December 30, 2006, 08:09:46 pm
Here's my additions for the evening:

   ROM      MAME Build      CPU      MHz      FPS   
   area51      0.111      AMD Athlon 64      1800      110   
   area51      0.111      AMD Athlon 64      2000      128   
   blazstar      0.111      AMD Athlon 64      2000      178   
   calspeed      0.111      AMD Athlon 64      1800      14   
   calspeed      0.111      AMD Athlon 64      2000      18   
   carnevil      0.111      AMD Athlon 64      2000      34   
   carnevil      0.111      AMD Athlon 64      1800      31   
   cbaj      0.111      AMD Athlon 64      2000      61   
   cottonbm      0.111      AMD Athlon 64      1800      41   
   crusnusa      0.111      AMD Athlon 64      2000      42   
   crusnusa      0.111      AMD Athlon 64      1800      39   
   cryptklr      0.111      AMD Athlon 64      1800      67   
   cryptklr      0.111      AMD Athlon 64      2000      75   
   daytona2      0.111      AMD Athlon 64      1800      10   
   ddra      0.111      AMD Athlon 64      1800      79   
   ddra      0.111      AMD Athlon 64      2000      88   
   gforce2      0.111      AMD Athlon 64      2000      97   
   harddriv      0.111      AMD Athlon 64      2000      63   
   hthero      0.111      AMD Athlon 64      2000      189   
   hyperath      0.111      AMD Athlon 64      2000      70   
   hyperath      0.111      AMD Athlon 64      1800      62   
   irobot      0.111      AMD Athlon 64      2000      221   
   kof2002      0.111      AMD Athlon 64      2000      162   
   openice      0.111      AMD Athlon 64      2000      84   
   openice      0.111      AMD Athlon 64      1800      74   
   p47aces      0.111      AMD Athlon 64      2000      87   
   pacman      0.111      AMD Athlon 64      1800      648   
   pblbeach      0.111      AMD Athlon 64      1800      60   
   pblbeach      0.111      AMD Athlon 64      2000      67   
   propcycl      0.111      AMD Athlon 64      1800      14   
   propcycl      0.111      AMD Athlon 64      2000      15   
   ridgerac      0.111      AMD Athlon 64      2000      12   
   roadblst      0.111      AMD Athlon 64      2000      180   
   rvschool      0.111      AMD Athlon 64      1800      32   
   rvschool      0.111      AMD Athlon 64      2000      39   
   sfex      0.111      AMD Athlon 64      2000      51   
   simpsons      0.111      AMD Athlon 64      1800      250   
   sonic      0.111      AMD Athlon 64      1800      89   
   souledge      0.111      AMD Athlon 64      2000      55   
   souledge      0.111      AMD Athlon 64      1800      49   
   starblad      0.111      AMD Athlon 64      1800      44   
   starblad      0.111      AMD Athlon 64      2000      50   
   stunrun      0.111      AMD Athlon 64      2000      91   
   stunrun      0.111      AMD Athlon 64      1800      83   
   surfplnt      0.111      AMD Athlon 64      2000      20   
   swa      0.111      AMD Athlon 64      2000      102   
   sws95      0.111      AMD Athlon 64      2000      127   
   sws99      0.111      AMD Athlon 64      1800      35   
   sws99      0.111      AMD Athlon 64      2000      36   
   swtrilgy      0.111      AMD Athlon 64      1800      9   
   tekken      0.111      AMD Athlon 64      2000      55   
   tekken      0.111      AMD Athlon 64      1800      50   
   tekken2      0.111      AMD Athlon 64      1800      50   
   tekken2      0.111      AMD Athlon 64      2000      57   
   tetrisp      0.111      AMD Athlon 64      1800      107   
   tetrisp      0.111      AMD Athlon 64      2000      122   
   timecris      0.111      AMD Athlon 64      2000      15   
   timecris      0.111      AMD Athlon 64      1800      13   
   toutrun      0.111      AMD Athlon 64      2000      121   
   vbowl      0.111      AMD Athlon 64      2000      74   
   vbowl      0.111      AMD Athlon 64      1800      65   
   vf      0.111      AMD Athlon 64      2000      71   
   vf      0.111      AMD Athlon 64      1800      64   
   vfkids      0.111      AMD Athlon 64      1800      28   
   vformula      0.111      AMD Athlon 64      2000      75   
   vfremix      0.111      AMD Athlon 64      1800      37   
Title: Re: MAME Benchmarks
Post by: Chris on January 02, 2007, 06:26:59 am
I just finished running some benchmarks using my P4 3.2GHz at work... I was surprised by the results.  On the newer 3D systems the P4 outperformed the Athlon 64 2800+ (1.8GHz, some tests overclocked to 2GHz), as expected.  But on older 2D systems, the Athlon was usually faster, sometimes by a wide margin!

Example: Soul Edge
Athlon 64 at 1800 MHz: 49 fps
Athlon 64 at 2000 MHz: 55 fps
Pentium 4 at 3200 MHz: 68 fps

Example: RoadBlasters
Athlon 64 at 2000 MHz: 180 fps
Pentium 4 at 3200 MHz: 96 fps

And here's the real surprise...

Example: Pac-Man
Athlon 64 at 1800 MHz: 648 fps
Pentium 4 at 3200 MHz: 87 fps

 :o

Both tests run using the same version of MAME .111b , the default download.  I'm at a loss to explain the P4's aversion to Pac-Man.

--Chris


Title: Re: MAME Benchmarks
Post by: releasedtruth on January 02, 2007, 10:43:33 am
Very interesting results I would say. I should do some benchmarking of my own. I have 3 machines to try things out on, but my P4 2.2 machine that runs the cab isn't exactly modern by any stretch.

GT
Title: Re: MAME Benchmarks
Post by: Chris on January 02, 2007, 10:55:30 am
Quote from: releasedtruth on January 02, 2007, 10:43:33 am
my P4 2.2 machine that runs the cab isn't exactly modern by any stretch.
This isn't necessarily a bad thing, as many of us use "surplus" hardware for building cabs...
Title: Re: MAME Benchmarks
Post by: Taborious on January 02, 2007, 11:24:57 am
What are you doing the benchmarks with? and how easy is it to use. I haven't done a lot of benchmarking but would like to start...
Title: Re: MAME Benchmarks
Post by: Chris on January 02, 2007, 11:32:47 am
There's a batch file on that site to run the test.  I had to tinker with it a tad to take out options that aren't in MAME anymore.  The one I used is:

mame %1 -noafs -nothrottle -ftr 10000 -r 1280x1024x32 -nowaitvsync -norc -nosleep -effect none -nobezel -nooverlay

I saved that as bench1280.bat, so to test, say, Pac-Man, I just do:

bench1280 pacman

on the command line.  It'll run through 10,000 frames as fast as possible and exit back to the command line showing the FPS.  For games like Pac-Man, 10,000 frames will be over in seconds; for a 3D game that's taxing your system, it could take 15 minutes for the benchmark to run, although theoretically once it's looped through the attract sequence at least once you should be able to exit and still have a reasonable result.

For a couple of the games that have to be calibrated or have a flash RAM prepared, I ignored the first result and ran it again after the calibration or preparation was complete.
Title: Re: MAME Benchmarks
Post by: Taborious on January 02, 2007, 02:48:31 pm
You're running the test at 1280 resolution? I'm only running 800x600 I think, so what, if any, is the logic behind the resolution your test at; just whatever one you want?
Title: Re: MAME Benchmarks
Post by: Chris on January 02, 2007, 02:51:52 pm
Quote from: Taborious on January 02, 2007, 02:48:31 pm
You're running the test at 1280 resolution? I'm only running 800x600 I think, so what, if any, is the logic behind the resolution your test at; just whatever one you want?

That's to match the other entries in the chart.  The site says to use 1280 for PC monitor output and 800 for a TV or arcade monitor setup.  I'll bet the resolution only makes a big FPS difference on vector games.
Title: Re: MAME Benchmarks
Post by: Chris on January 02, 2007, 05:41:12 pm
Looking at NewEgg, I'm strongly thinking of getting an Athlon 64 4000+ (San Diego core at 2.4GHz), maybe mildly overclocking it to 2.6.  I can get the processor and motherboard for $137 total at NewEgg.  The potential competition is the Pentium D 805 (Smithfield core at 2.66GHz); motherboard and processor total $127 at NewEgg.  Decisions decisions.  The Pentium D is a dual core, but MAME won't use the extra CPU... maybe Windows will?  Neither one of these CPUs is on the benchmark table yet...





Title: Re: MAME Benchmarks
Post by: Taborious on January 03, 2007, 02:22:47 pm
I have always wondered if turning of hyperthreading helped any of the benchmarks? Anyone run benchmarks with hypertherading on and then off?
Title: Re: MAME Benchmarks
Post by: Chris on January 03, 2007, 02:30:00 pm
Now that you mention it, I have seen reports that turning off hyperthreading can help, but I haven't tested it.  Maybe that would explain some of the poor results in Pac-Man.  I'll try it without the hyperthreading when I get a chance and see if that improves things.

--Chris
Title: Re: MAME Benchmarks
Post by: Chris on January 03, 2007, 02:42:56 pm
Nope... I tried Pac-Man and Soul Edge really quicky and got exactly the same frame rates with HyperThreading turned off.
Title: Re: MAME Benchmarks
Post by: Lilwolf on January 03, 2007, 03:27:23 pm
I've always thought it would be better to rank all games on one piece of hardware (ok maybe a few, AMD, Intel, ect).  And not emphasize the frame rate, but how they rate compared to each other.

Why?  So when you buy a new piece of hardware, you can run down the lost and move up / down the list until you find the slowest games you can run at near / full fps... And only add those to your cab.

This way you don't have to look around and try them all... or worse yet... run a game for the first time when others are over... and make your cab seem broken.
Title: Re: MAME Benchmarks
Post by: u_rebelscum on January 03, 2007, 06:06:30 pm
Quote from: Lilwolf on January 03, 2007, 03:27:23 pm
I've always thought it would be better to rank all games on one piece of hardware (ok maybe a few, AMD, Intel, ect).  And not emphasize the frame rate, but how they rate compared to each other.

Why?  So when you buy a new piece of hardware, you can run down the lost and move up / down the list until you find the slowest games you can run at near / full fps... And only add those to your cab.

This way you don't have to look around and try them all... or worse yet... run a game for the first time when others are over... and make your cab seem broken.

Not bad idea for comparing general game play. 

People won't be satisfied though, since different games like different hardware err... differently. 
You'd need a list for at least a64, p4, & core2 CPUs, and possibly for core, pm, p3, semperon, & athlon(32); game P(sx) might run slower than game Q on cpu A(md), but faster on cpu I(ntel). 
Add that game X might be okay at 90% fps while game Y sucks at steady 95% fps. 
Or game Z averages 97% fps but goes to 50% at some key point vs that game W that never goes below 90% averaging 95%. 
And newer versions of mame effect different games more than others (such as adding dynamic recompilation, or better emulation) possibly moving games up or down the list. 
And there are 6000+ games (3000+ parents); by the time you run them all, the next major version will be out. ;)

Not that that game relativity lists are a bad idea; they would make it easier for people with slightly different hardware than tested to dial in on which boarderline games to test, and which are hopeless.  I just don't think they'll be any easier to keep up to date.


Quote from: Chris on January 03, 2007, 02:42:56 pm
Nope... I tried Pac-Man and Soul Edge really quicky and got exactly the same frame rates with HyperThreading turned off.

Chris, did you enable mame's multithread option (-mt) with hyperthread on?  Looking at the mame32 benches (http://mame32qa.classicgaming.gamespy.com/Bench.htm), John noticed a difference.  Compare the d3d@3.6, d3d/MT@3.6, and nv/ns@3.6 columns.  (-mt = multithread, nv/ns = no video, no sound).  He used a core 2 duo, though.
Title: Re: MAME Benchmarks
Post by: Chris on January 03, 2007, 06:39:18 pm
Quote from: u_rebelscum on January 03, 2007, 06:06:30 pm
Chris, did you enable mame's multithread option (-mt) with hyperthread on? 
No, I hadn't.  It didn't really make a difference with Soul Edge (70 with -mt vs 68 without), but it made a HUGE difference with Pac-Man (156 with -mt vs 88 without).  Still way, way behind the Athlon, though.
Title: Re: MAME Benchmarks
Post by: Chris on January 27, 2007, 11:55:41 am
I added benchmarks for an Athlon 64 3700+ San Diego (2200 MHz) last night and I'm doing an Athlon 64 4000+ San Diego (2400 MHz) right now.
Title: Re: MAME Benchmarks
Post by: Buddabing on January 27, 2007, 03:16:49 pm
Aaron Giles did a bunch of benchmarking recently. Look on his page (http://www.aarongiles.com).