Part of it is what the compiler can do.  Actually, I think mame compiles better than pentium pro.  It's compiled with the I686 flag set to on.  That's 686, pentium III class I believe.
Anyway, it's also in part what the compiler supports too.
IIRC:
486 = i486 

pentium = i586
pentium MMX = i585 + MMX code
pentium pro = i686
pentium II = pentium pro w/ different cache + MMX
pentium III = pentium II + SSE
pentium IV = "i786" (includes MMX, SSE, and SSEII)
If you notice, pentium pro, II, and III are all i686 chips, with the only changes are the cache and new math stuff (+ Mhz, but we're talking about compiling).  The rest of the chips' timings, strengths and weaknesses are about the same; so optimizingfor them are about the same.  
Pentium IV is the first chip with changes other than adding math stuff.