Has there ever been a documented reverse engineering of an arcade game?

[jimmy2x2x]

Like a complete breakdown of all code, documenting subroutines, methods, input/output parameters - that kind of thing.

Just wondering if it had been done at any time

[TopJimmyCooks]

Isn't that what MAME is?  or is intended to be by the mamedev's?

[jimmy2x2x]

Not really what I was meant.

I was thinking about reconstructed source code that could be assembled into a working rom image, with all the code seperated from the data and documented to some degree.

[TopJimmyCooks]

The documentation issue- not sure about that, but theoretically the roms are from the chips and can be used to reprogram chips that could be used on the PCB boards for the game. 

[ark_ader]

Not really what I was meant.

I was thinking about reconstructed source code that could be assembled into a working rom image, with all the code seperated from the data and documented to some degree.

Unless it was for security improvement, wouldn't that be against the DMCA?

[newmanfamilyvlogs]

http://home.comcast.net/~jpittman2/pacman/pacmandossier.html

This is the only one I know of off the top of my head, it's probably pretty close to what you're thinking of.

[DaveMMR]

http://home.comcast.net/~jpittman2/pacman/pacmandossier.html

This is the only one I know of off the top of my head, it's probably pretty close to what you're thinking of.

I clicked on this link to check it out - still reading it an hour later.  Interesting stuff.

[jimmy2x2x]

Same here!

Nicely written and researched article, about as close as I am going to get I reckon

[Gray_Area]

That's not really what you're looking for. This question would, I'm betting, be more likely answered at MAMEWorld.

[Howard_Casto]

I'm no expert on the history of arcade games or anything, but I can give you the short answer.... no.

When source is compiled, especially on arcade machines, it gets compiled to some form of assembly.  All comments are removed.  Heck, even the names of the variables are removed!  It saves space that way.  Once you de-compile, you can reverse engineer and make some new comments based on how you think things are working (see mame) but once it's gone, it's gone.  Mame rarely does that much actually.  Think of the game's program chips as the harddrive of a computer.  Mame doesn't emulate the harddrive... it emulates the entire pc, which can play the contents of the harddrive.  So it isn't necessary to understand completely what is going on in the program roms, so long as the emulated hardware is setup correctly.

If any source code still exists to some arcade games, it would be rare if the public ever gets to see it.  The games are still copyrighted afterall. 

That isn't to say that there might be a few cases out there, but in general no.... you aren't going to find any source code.

[MonMotha]

Reverse engineering even relatively simple software (say, a page of C) to the degree you speak of is mind-numbingly boring and generally has to be done by somebody who is knowledgeable enough to have something much more interesting to be working on.  I've done similar things on key sections of programs to do things like emulate an IO board, and it can take weeks just to fully reverse engineer maybe 5-10 functions.

It's not necessary, anyway.  You can fully emulate the system without ever even looking at the program (source or compiled binary).  You can make small patches to the program with a disassembler, hex editor, and maybe a debugger (this is how all the "infinite lives" stuff is done).

That said, a lot of the source code is still out there.  I know there are people that still have the source to some mid-90s Midway games, for example.  I suspect most companies that are still alive have the source laying around somewhere, and for companies that have been liquidated, somebody probably ended up with the source.  Whether they bothered to keep it or not is up to them, of course.  Many of those buyers are "IP warehouses".  They only care about enforcing the copyrights they've purchased, not the artistic value (ya know, the stuff that "promote(s) the Progress of Science and useful Arts".  I'm sure plenty of source has been lost to the ages, but nobody would have ever been under obligation to release it, anyway.

[ChadTower]

Didn't they decompile Pacman back in the day in order to document the ghost behaviors?  I'm pretty sure I remember reading that was done via a real decompile and not just basic observations.

[jimmy2x2x]

I'm no expert on the history of arcade games or anything, but I can give you the short answer.... no.

When source is compiled, especially on arcade machines, it gets compiled to some form of assembly.  All comments are removed.  Heck, even the names of the variables are removed!  It saves space that way.  Once you de-compile, you can reverse engineer and make some new comments based on how you think things are working (see mame) but once it's gone, it's gone.  Mame rarely does that much actually.  Think of the game's program chips as the harddrive of a computer.  Mame doesn't emulate the harddrive... it emulates the entire pc, which can play the contents of the harddrive.  So it isn't necessary to understand completely what is going on in the program roms, so long as the emulated hardware is setup correctly.

If any source code still exists to some arcade games, it would be rare if the public ever gets to see it.  The games are still copyrighted afterall. 

That isn't to say that there might be a few cases out there, but in general no.... you aren't going to find any source code.

I really dont think most arcade games would be coded in any language that needed to be compiled, assembled seems a lot more likely to me.

The content of the roms would be mainly data in most cases, leaving a relatively small code area.. I just thought it might be interesting to have a look at how some of the classics where coded, how they actually did things internally. Due to the nature of proms, variable tracking should be a bit easier than some other systems.

I am surprised there isn't any leaked source code for anything after all these years

Did anyone else remember a story about Taito losing the source for Bubble Bobble?

[jimmy2x2x]

Didn't they decompile Pacman back in the day in order to document the ghost behaviors?  I'm pretty sure I remember reading that was done via a real decompile and not just basic observations.

Thats more like it!  Pretty sure it would have been done at some point.

[MonMotha]

The early stuff was probably written in assembly.  However, when you run it through the assembler, you still lose all the variable names, label names, function names, etc.  Imagine trying to read a program where the everything was just named "x", "y", "z", etc. and all the jump points were just labeled "addr0", "addr1", "addr2", etc.  It's surprisingly difficult to figure out wtf is going on.  Add in the fact that the programmers are free to pick wild calling conventions and even vary them between functions, and it can be a mess to read.  Usually the only saving grace is that there's just not much code, and the assembly output will (due to the way assembly works) be identical in structure and flow to the input, which tends to leave some human-friendly idioms intact that compilers often destroy.

Some REALLY early stuff was probably written directly in "machine code".  This is basically the same as assembly, but rather than being run through an assembler, that translation is done by hand and entered using a hex editor, card punch, etc.  Even then, most programs would have had lots of documentation that essentially comprises the "source".  It's essentially impossible to maintain programs written directly in "machine code", even if you're the person who wrote it just a few hours ago!

I'd guess that by the mid-late 80s, most stuff was being written in C.  Compilers were readily available for all the common CPUs (68k, z80, etc.), and they were reasonably good.  You could run them on a PC and get output that you could dump to your parallel port connected EPROM programmer for testing.  The longest part of the dev cycle at that point was waiting for a batch of ROMs to erase (takes about 20m - most companies would have about 10-20 full sets of ROMs for each developer so they could do rapid cycle development with about 2-5min cycle times, then "batch erase" 10-15 sets while continuing development on the remaining stuff).

Even before then, I'd have expected to see at least some FORTRAN.  You might also see some PASCAL starting in the very late 80s.

If you've ever looked at the output of even a non-optimizing C compiler, it can be a bit tough to follow.  In addition to having no variable names, label names, function names, etc., the compiler also has to translate the common flow control structures (for loops, while loops, do-while loops, if statements, switch/case statements, etc.) into compare and branch instructions.  Essentially, all loops become do-while (which is one of the least used kinds, when writing C) and if statements become a hodgepodge of goto (which are avoided because they're hard to follow and maintain).  The only real saving graces are that the calling convention is consistent, and the basic program flow stays roughly in-order.  Optimizing compilers remove that last nicety (for the reverse engineer) and can produce output that's actually quite difficult to follow as it drastically differs from how one would normally write the input C code or even how one would write it by hand in assembly.

Reverse engineering just one section of the program is quite reasonable, though.  Someone who's good at it can probably do it in a couple days, but it'll depend on complexity and the size of the "section".  "Decompilers" produce very rudimentary output.  It's often better to just pore over the disassembly and start scratching down psuedocode in a text editor.

And yeah, people lose source all the time.  A lot of old mega-corporate and bank applications either have no known source available, despite at one point being written in COBOL or similar, or the binary running on the system has been patched directly so many times that the source is out of date and really only useful as a vague reference (knowing that the program that's actually running and that you need to potentially work on may have been altered from it).  You might be surprised how much source is still floating around on some old floppy, though, and how often somebody knows where and what it is.

[CheffoJeffo]

I seem to recall some reverse-engineered and documented source for early Williams games, but I can't recall whether it appeared on RGVAC or was something that Mike Ranger (who posted as WilliamsGuru) told me about when I bought his Joust cocktail.

As a piece of arcade trivia, I guess that Tim Skelly was the first to do this when Larry (last name escapes me) left Cinematronics and took all of his notes and all of the hardware. IIRC, Tim rebuilt the software and was able to reconstruct the hardware based on a hex dump.

[Howard_Casto]

MonMotha:

Did you not see my post?

I liked your explainations, but you basically just repeated what I posted in long-hand. 

[jimmy2x2x]

I'm still not really convinced that arcade game code would have been written in C or any other high level languages in the mid to late 80's. With most games running around 60 frames a second, binary code size and chip prices at the time, speed of the CPU in many arcade games, non mature nature of compilers back then... seems like a stretch to me.

[MonMotha]

I'm still not really convinced that arcade game code would have been written in C or any other high level languages in the mid to late 80's. With most games running around 60 frames a second, binary code size and chip prices at the time, speed of the CPU in many arcade games, non mature nature of compilers back then... seems like a stretch to me.

C compilers in the late 80s were already getting fairly good.  UNIX was written in it, after all, and it was nearing 10 years since the publication of the K&R book and nearly 20 years since initial development.  Figure most PC games were written in C and other "high level" languages, so why wouldn't arcade games be?  The maintenance is just so much easier, and you can easily knock 50% off of the software dev cycle, which was a major deal in the arcade industry.

By 1990, people would pretty much assume you were insane if you wanted to write an entire program of any real complexity entirely in assembly without a really, really good reason.

If you know what you're doing, it's trivial to write pretty tight C.  Honestly, I find it easier to write "efficient" (both space and size) programs in C than assembly, even allowing for extra development time for the assembly variant.  It's just so much easier to make modifications to improve performance when you don't have to worry about every tiny little detail.  You can always inspect the output assembly and tweak either your input or drop to inline assembly, if necessary.  The optimizers are better these days, but it was not hard to write "small" and efficient C programs back in the day, even with fairly immature compilers.

C isn't a crazy bloated language.  The language itself is extremely lightweight, and the standard library is small, reflecting the storage/computing capabilities of when it was developed (late 70s).  printf and malloc are generally some of the biggest functions in the library, and you'll often burn a whopping 20-30k to include both of those, which was getting small even for ROMs by the late 80s.

FWIW, I'm working on a project right now using GCC for ARM, and the total binary output (including all the init and library code) is...40348 bytes, about half of which is a somewhat suboptimal (it handles floats, which I don't need and are complicated) implemenation of printf and some bitmap font data.  I haven't even really been trying to make it small.  The input is 17 mid-size (typically 100-250 lines) source files plus a libc.

I regularly program little 8-bit micros that have less than 1k of RAM and 8k of FLASH for program code and run at 8-16MHz (performance comparable to a 20-32MHz M68k) in C without any real issues performance-wise.  I'll admit I'm running on somewhat newer compilers, but even disabling optimization entirely generally leaves me with something that works just fine, even in time critical applications.

Many arcade games also ran at only 30fps, not 60.  Each frame was just output to the monitor twice while the system worked on the next frame.

Then again, if you want to know, just grab a program ROM image and run an appropriate disassembler on it.  It's usually fairly easy to tell if it was hand written or compiled.  Heck, you can often tell what language it was written in and sometimes even which compiler was used, if you know what to look for.  Honestly, excepting cases where inordinate attention was paid, the compiled code is often quite a bit more efficient than the hand-written code.  Like I said, you might see a few sections in a program written in a high level language (C, Pascal, etc.) that were hand tuned assembly for performance reasons, but often the "base" for that code was the output of a compiler.  Easier to know that it's "right" to start with, that way.

[MonMotha]

MonMotha:

Did you not see my post?

I liked your explainations, but you basically just repeated what I posted in long-hand. 

I think it got posted while I was composing my post, so, while I saw it, I figured I'd already written everything up and might as well post it anyway.

[Howard_Casto]

I got ya. 

I felt kind of neglected there for a sec, but I didn't think it was intentional so I thought maybe I wasn't explaining it right. 

I have sensitive feelings you know. 

[newmanfamilyvlogs]

If you dig a little deeper on the Pacman Dossier there's a link to a commented decompiliation of the Pacman code in assembler:

http://www.vecoven.com/elec/pacman/pacman.html

[ChadTower]

Ah, there it is, I knew I had read through that someplace in the past.  Thanks!

[Gray_Area]

I got ya. 

I felt kind of neglected there for a sec, but I didn't think it was intentional so I thought maybe I wasn't explaining it right. 

I have sensitive feelings you know. 

No need. You gave a general explanation. I read it. But MonMotha's post was like being in a cozy lecture hall with someone who knows how to speak and tell stories. I love that ---steaming pile of meadow muffin---.

[jimmy2x2x]

Many arcade games also ran at only 30fps, not 60.  Each frame was just output to the monitor twice while the system worked on the next frame.

Maybe this is my point of confusion, I thought the bulk of arcade games ran at 60fps, updating each frame.

Do you know a few well known titles of the era we are talking about that run at 30fps?

[MonMotha]

I can't name any specific examples from the era you're probably thinking, but some notable late-80s ones ran at even less (e.g. Hard Drivin').  Having talked to some ex-Midway people, I know it's at least reasonably common.  Most games were still locked to the vertical refresh, they'd just evenly divide it down if they couldn't keep up.  I was told (by a guy who worked on it) that this is one reason MK2 runs at such a weird refresh rate.  The hardware couldn't quite keep up with 60Hz, but they didn't want to drop all the way to 30 since that's a bit slow for something as action-sensitive as a fighter, so they just increased the total video resolution until they hit ~53Hz, which everything could keep up with.

Doing this does potentially require more RAM, though.  You can't generate the graphics and spit them out on the fly if you can't draw them in real-time to start with.  You have to have not just one but TWO framebuffers (one to operate on and one to scan out to the monitor).  This would have made the practice somewhat uncommon until the mid to late 80s when RAM started getting cheap-ish.  Some really old games didn't have a framebuffer at all: they really did generate the graphics right as they were about to be spit out to the monitor.

Of course, this doesn't stop you from having the game's event loop run slower (and just drawing the graphics twice based on old state data), but the event loop isn't normally the blocking factor in terms of complexity for most (but not all) games.  The event loop (and surrounding code) is what we're talking about reverse engineering, here.

I wouldn't be surprised if some games that ran at a lower graphical framerate actually ran their event loop faster just not drawing graphics based on the interim states, but that's conjecture.

I remember reading somewhere that the design guides for getting your game licensed on one popular console (I think it was either the original Xbox or one of the Playstations, but this sounds like a very Nintendo thing) required that you always run the game at the same framerate, and it had to evenly divide the output rate.  That is, you couldn't allow your game to degrade to 30fps (NTSC land) momentarily - it had to always run at 30fps unless it could consistently manage 60.

If you do want to reverse engineer a game, the Laserdisc stuff is probably some of the simplest.  Not much there besides a timer, a cuesheet for the laserdisc, a list of "correct" moves and their timing, and some input handling.  If you have a target for what you want to reverse engineer (e.g. "How do the ghosts behave?" or "How does this IO board work?") things are usually a lot easier.  I took the question to be "Has anybody ever completely reverse engineered a game to the point where one can compile, from reverse engineered source exactly mimicking the structure and functionality of the original, a total workalike using the original graphic, sound, etc. assets?", and the answer seems to be "it's pretty unlikely", since there's relatively little reason to do so.  You might as well just make your own similar game: it's generally a lot easier.

[jimmy2x2x]

Thanks for the wordy reply, I appreciate it.

Polygon based games of course ran at much lower frame rates back in that era and I wouldn't be surprised at all if they where written entirely in C or another non assembly language. Very interesting background information on the Mortal Kombat games.

Yes, the original question was about a complete reconstruction at source level and this has been partially answered in the pac man dossier, lots of fascinating information there.  The reason for the question was more about curiosity than anything else, I just wondered how they did things and how much support the games had from the hardware, collision detection, hardware limitation dictating the games design, underlying logic for when events are triggered - that kind of thing.

[lilshawn]

I'm still not really convinced that arcade game code would have been written in C or any other high level languages in the mid to late 80's. With most games running around 60 frames a second, binary code size and chip prices at the time, speed of the CPU in many arcade games, non mature nature of compilers back then... seems like a stretch to me.

one has to realize that when it came to early videogames, you didn't write the code for the computer. you built the computer for the code.

they wrote a program built some computer to support it...coded some more... added to the computer. when it was all said and done, you have a very specialized computer running tailored software with tailored hardware.

sloppy coding aside, you don't need to drop a V8 engine to win the race. lots of amazing things could be done with a z80 mainly because alot of needlessly repeating things can be eliminated from code.


[EXAMPLE]

my friend and I both had laptop computers running basic. we both wrote the same identical program to pick 2 random numbers between 1 and 1,000,000 and compare them... if they where different it picked 2 new ones. if they matched end program. easy peasy.

here's the kicker. My laptop was a Toshiba T1200 80c86 running at ~10mhz. His was a Macintosh Powerbook with a 68030 25mhz so his computer was approximately 2.5 times faster right?

my computer crunched the numbers WAY faster. I had several hundred thousand results before his hit even 3000. How could this be?

we found one reason (among many) was the 80c86 was capable of automatically processing repeating operations receives the instruction once and spits out multiple results... where the 68030 had to process and reprocess and reprocess over and over and over... wasting valuable clock cycles doing the same thing over and over again.

of course his powerbook could do lots of things way better and faster than my Toshiba... but it shows that code optimized for the hardware is capable of many feats.

[/EXAMPLE]

[jimmy2x2x]

A point well made, do you think that the kind of optimization you are referring to indicates that assembly language would or would not be best suited to this kind of exacting machine code flow?

EDIT: I am basing my thoughts on this around the gaming computers of the time, 8 bit and 16 bit machines such as the amiga and st.  If you wanted optimal speed and size you had no other viable choice other than assembly language.

If you are writing something where frame rates are irrelevant such as adventure or management games, that is a different argument altogether.

I feel there is a cultural divide here with American programmers adopting high level languages much sooner that Euro coders, IMO that's the main reason that Euro coded games of that era (on home machines at least) run rings around the US offerings (performance and storage wise).

For example: How many silky smooth platform games or shmups (50 or 60fps) came out of the States on the Amiga vs how many came out Europe?  Different ideas on what makes a great game.

[lilshawn]

don't forget that arcade games are VERY short, often only a few levels at most. the difficulty increases greatly by level. Game makers made games nearly impossible in higher levels in some cases.

you have to remeber too that the Z80 (the most popular processor used throughout the 70's and 80's has a very intuitive instruction set. Allot of tiny simple instructions that otherwise had to be done in programming through several steps could be executed in a single command with the Z80.

in fact the 80c80 series processors where a predecessors of the Z80 processor and use a very similar instruction set (hence the great increase in random number generation and comparison as shown in my last post)

As well as allot of complex memory operations (moving bits/changing bits) can be executed with a single (or way fewer) commands without having to control the processor and tell it step by step directly. this eliminated seperate memory control hardware. When it come to computers, the more you can get the processor to do without having to wait for stuff outside the CPU to do it, the faster things can happen. Memory at this time was very slow (by todays standards) eliminating an extra clock cycle to not wait for a ram refresh could potentially DOUBLE processing speed.

to reduce program size LOTS of operations where abbreviated so single letters, so instead of "MOVE byte ptr" it was simply "LD".moving a piece of information in memory used to be "MOV byte ptr [DI+01],02" now it was "LD (IY+01),02"

remember LOGO? probably not.  for a square it was "forward 10 right 90 forward 10 right 90 forward 10 right 90 forward 10 right 90" or "forward 10 right 90 repeat 4" we reduced our code by a factor of 4 by using a shortcut. how about a circle? "forward 1 right 1 forward 1 right 1....(repeat 360 times) or instead "forward 1 right 1 repeat 360" now we have reduced our program size by a factor of several hundred.

just think of it more like stock car racing - the cars are very specialised machines built for one purpose. turning left. ever seen those cars try and turn right or go straight? it's not very effective. as the machine has been built around only having to turn left. (suspention, tire alignment, etc etc)

NA designers sort of got stuck in the thinking of having the processor do everything. The Asians realized we don't need to waste time trying to deal with audio so we will add a chip to do that. that way we can use a few clock cycles to send it a command, while we go do this other stuff while the support the chip pulls the sound out of ram and sends it to the D/A converter...by the time the graphics are ready to get refreshed to the screen the audio is on it's way. and with PRC and Hong Kong at the forefront of chip manufacturing, making/finding chips to do unusual things was allot easier for them. need a chip to take bits, invert the bits and spit them out in hexadecimal, no probrem! </racisim>

as a side note, you will notice desktop computers in the 2000's started a movement... designers realized that even though 3.5ghz was fast there was still issues with having to waste those clock cycles waiting for thing like RAM. RAM was fast? what's the big deal? problem? the memory controller is on a bus that operates at 10X 20x 30x slower than the processor. By moving the memory controler to the CPU they gained a huge speed increase without really making anything faster. The same kids of things where happening with game boards.

Game makers started installing chips that soley deal with video (the first GPU if you will.) it involved a little bit of re learning how to code but they took alot of load off the CPU because now it didn't have to deal with the video. and using another Z80 to deal with audio. it went on an on. the multiple automated instructions that could be executed with the Z80 made it very quick and efficient. hell they used versions of the z80 in the gameboy, colecovision, sega neogeo everything... for years, because they didn't need a huge complex code set to to complex things.

[MonMotha]

No compiler can match properly hand tuned assembly.  In some pathological cases, even modern compilers can't even come close.  This is especially true or some esoteric hardware like SIMD units (SSE, Altivec, etc.), DSPs, heavily parallel environments, etc.  Someone who's willing to spend a day or two tweaking a small loop can often run circles around even a pretty good compiler if the code being optimized has any real complexity.

HOWEVER, if one has a fixed time budget, one can be a LOT more productive in a high level language (e.g. C).  That seems obvious, and it probably is.  That's generally the point of higher level languages: they speed development and make things easier to maintain.

What may not be obvious is that even a mediocre compiler will often beat the pants off of a naive first pass at doing something in assembly, especially if the programmer is hurried.  There's a couple reasons for this:

1) it's often easier to see algorithmic level improvements in the high level languages, and, since they're faster to work in, the programmer is more likely to make such improvements.  Going from O(exp(n)) to an O(log(n)) algorithm will improve performance (assuming comparable constant factors) on any non-trivial size workload any day, even if the former has been optimized to the limit by hand while the latter is built with a non-optimizing compiler from 1985.

2) Less of a factor back then, but certainly so these days, the compiler generally knows the CPU better than most programmers.  Properly optimizing something in assembly by hand requires a LOT of work.  I've worked with a DSP that could pull off 2 multiply-accumulates per CPU cycle, including loading and storing the data, but you had to tune the loop JUST right.  Pulling out a few extraneous instructions was just the start.  You had to have everything in exactly the right order, sometimes a non-intuitive one, to ensure that the pipeline would never stall and all execution units were always busy.

In general, I've found that gcc (an easily obtainable benchmark of a compiler that I'll reference specifically here as having a reasonable optimizer but not as good as some commercial compilers like the one from Intel) will generally outdo me on most architectures unless I call up the reference manual for the specific CPU in question and spend a good day or so working on a single function (longer for really complicated CPUs like modern Intel processors, shorter for stuff that's strictly in-order and single issue like an ARM Cortex-M or Atom).  Of course, if the program spends 99% of its time in that function, that may be worth it, and if you happen to have someone on staff who knows the CPU *really well*, they can probably shorten that a bit.  On the flip side, if the program spends 99% of its time in one function that comprises 1% of the code, that's a pretty quality candidate for writing the program in a high level language and just manually optimizing that 1%.

As another development model, apparently Wolfenstein 3D was written in C but "compiled" almost entirely by hand.  Carmack is a bit crazy like that, though.  This is for those who want the development advantages of C and other high level languages, including the ability to rapidly cycle the development (and maybe run it on a faster dev system or at reduced speed) with the performance advantages of hand tweaked assembly and have "all the time in the world."  This model would mean that C source exists, and presumably the code that ends up "shipping" is fully equivalent right down to the structure and calling conventions.

[jimmy2x2x]

It sounds like you might be basing your development examples more in the present day, not some 20-25 years ago - this is the crux of the argument.

Things where very different back then, very limited workspace, processing power and storage - combined with immaturity of compilers in that time.

To rethink "one can be a LOT more productive in a high level language" statement, back then this would have held true up to a point until resources started running out - then the assembly option looks a lot more sensible.

"a mediocre compiler will often beat the pants off of a naive first pass at doing something in assembly, especially if the programmer is hurried" - today I am sure this will hold true for the most part. Back then - maybe if the programmer was working on his first project on a new platform. If the programmer has more than one project already done for the current platform a lot of intimate processor and hardware knowledge and resources from the last project would be available to him.  The new development cycle would offer time to further refine already efficient working code, this principle does not usually apply to higher level development.

I'm thinking about things like self modifying code, dynamic stack relocation, efficient zero page usage, sub routine register return values instead of stacked return values, unrolling loops, dynamic routine entry - low level coding that you rarely see outside of native assembly.

Like home computing, arcade hardware evolved too. Of course there would be a point where resources became less of an issue and compilers where performing in an intelligent and efficient way.  Seems like its a question of when this became viable and was widely adopted.  Mid to late 80's seems too early to me, well at least for games where pushing the hardware and frame rates where valued.

After looking a quite a few games from that era I am certain that the majority do not update at 30fps and draw twice, they have true 60fps updating.

timeframe + hardware of the day + high level language = 60fps ? Still not convinced, sorry.

[MonMotha]

It sounds like you might be basing your development examples more in the present day, not some 20-25 years ago - this is the crux of the argument.

Things where very different back then, very limited workspace, processing power and storage - combined with immaturity of compilers in that time.

To rethink "one can be a LOT more productive in a high level language" statement, back then this would have held true up to a point until resources started running out - then the assembly option looks a lot more sensible.

So you take the path of "write it in C then start optimizing the sensitive parts in assembly as it becomes necessary".  The computer (and I don't just mean PC) guys had been doing it for years, basically since the dawn of usable compilers.  I also think you underestimate the maturity of compilers by the mid 80s.  UNIX was quite popular and was written in C at that point.  The language was essentially a decade old.  In general, they do no worse than what a typical "first shot" would be by someone writing by hand in assembly, but I'll admit compilers of the era would often do no better.  C is not a complicated language to write a non-optimizing compiler for that behaves properly - such a task is often given to sophomore or junior university computer science students.

"a mediocre compiler will often beat the pants off of a naive first pass at doing something in assembly, especially if the programmer is hurried" - today I am sure this will hold true for the most part. Back then - maybe if the programmer was working on his first project on a new platform. If the programmer has more than one project already done for the current platform a lot of intimate processor and hardware knowledge and resources from the last project would be available to him.  The new development cycle would offer time to further refine already efficient working code, this principle does not usually apply to higher level development.

I'm thinking about things like self modifying code, dynamic stack relocation, efficient zero page usage, sub routine register return values instead of stacked return values, unrolling loops, dynamic routine entry - low level coding that you rarely see outside of native assembly.

Some of these are indeed "rarely seen outside native assembly" like self modifying code (though the IOCCC people do pull some crazy stunts), but register returns are standard on many archs with alternate calling conventions available from many compilers, especially old ones (hell Microsoft STILL has fastcall, stdcall, and cdecl - for grins guess which one is actually the industry standard) back when stuff like that really mattered.  I've seen people pull crazy stack tricks in C (perhaps with a little inline assembly), and almost all compilers (or, usually, linkers) have supported forcing certain things into certain locations since the beginning allowing you to tune your code/data locality.  People also used to use setjmp/longjmp (you'd be crucified if you did that at most companies, these days).  Most compilers supported pinning globals into a register, and people often used such tricks for multiple register return values.  You can, and many people did, treat C as a form of "architecture independent assembly" that allowed you to more easily express complex things if you wanted to while still sticking close to the hardware and having control over the generated output.

Loop unrolling in C was in fact so popular that compiler makers are STILL counseling people to NOT do it, since modern compilers usually do a better job on the loop than the unrolled code.  People would frequently re-use local variables because the old compilers weren't smart enough to re-use registers or space on the stack until something fell out of scope.  The register keyword actually had relevance, and people used it.  Old code is littered with macros to ensure inlining where modern code would use an "inline function" (not available back then).  Heck, look up duff's device.  People used to do that crap in C all the time.  They don't any more because 1) the compiler figures it out, and 2) there's little reason, but back then, heck yeah.

Especially back then, using C was not a substitute for the programmer knowing what they were doing.  The compilers weren't that good, just good enough to let you do what needed to be done.  The difference is that where C lets you get done what needs to be done, assembly FORCES you to think about every nitty gritty little thing in order to get things done.  I'm well aware that there were (and still are) some great assembly hot shots around, but even the best of them generally admit that they're much more productive in a high level language with usually little downside in terms of either performance or code size.

Also, keep in mind most arcade games used "cleaner" architectures like the 68000, not x86, so they didn't have all the weird memory segmentation tricks to deal with (68000 has a flat address space) that made C a pain on DOS and the like.

There are plenty of ways to be smarter than the compiler without dropping all the way to assembly.  Look up Carmack's floating point inverse square root trick for a nifty example (note: totally obsoleted by SSE on x86).

I do doubt that you'll find any arcade game from the 80s or even early 90s where EVERYTHING was written in a high level language and just run through a compiler without giving it any further thought.  They were asking too much of the hardware for that.  Carmack and friends were still hand optimizing things well into the late 90s (and probably still do one some occasions).

I just think it stands to reason that, given the common availability of reasonably robust compilers (again, UNIX was written in C and very popular, as I think was VMS, which begot OS/2 in 1987 and eventually Windows NT), especially for the popular M68k, that it would make sense to concentrate optimization efforts on the parts of the program where it matters.  Arcade games were usually high budget for their time, but they also had tight timetables and somewhat limited programming staff.  It just doesn't make sense to have a bunch of people hanging around banging out assembly all day for weeks on end when they could hammer it out in a few days in C, test and debug it, and optimize the parts that matter.  Especially as the games got more complex, writing the bulk of the game in a faster to develop, more maintainable language would make sense.  It might even let you re-use some boilerplate code across multiple games, even as the hardware changed.

I'm actually really kinda curious when the switch actually occurred (I know for a fact that almost all arcade games are nearly pure C/C++ or sometimes even Java, .NET or even Flash, now - I've seen a fair bit of the source, and the disassembly is often quite telling with C++).  My guess would be mid-late 80s (some time around 1988-1989) with perhaps some early forays in the early-mid 80s (like 1983-1984) and a near total conversion (all but very "hot" sections of code) by 1995 or so.  Your guess is apparently later (maybe early to mid 90s?).

I should try to get back in touch with the former Midway guys I know and see what the answer to that is.  I'm sure it varied with game house (e.g. Midway was one of the first to go to PCs while Konami was one of the last).

timeframe + hardware of the day + high level language = 60fps ? Still not convinced, sorry.

I don't see why you couldn't do it on a lot of things like side scrolling beat 'em ups or platformers that had simpler scenes and behaviors.  The graphics stuff like tile/sprite addressing and priority was often partially accelerated in hardware by then, and separate sound processors were common, too.  Honestly, I kinda wonder what percentage of the time the main CPU on your average arcade game from 1988 spent doing nothing (just waiting on the vertical sync interrupt to run the next iteration).

[jimmy2x2x]

If you could get in touch with the former Midway guys that would be very interesting, would love to hear some of their stories and development hell scenarios!

My first encounter with commercial game development was at a small UK software house, doing a lot of conversion work sometimes from arcade to home computers.

We had specialists for each processor family, 6502/6510, Z80, 68000 and x86 - this was further fragmented by guys who would favour certain platforms. High level development was purely reserved for prototyping and development tools - map editor, sprite editor, compression that kind of thing.

Another thing to consider would be programmers experience and having them learn dual disciplines, why ask someone to learn something new when they had been banging out solid conversions in pure Z80 in a 3 month cycle - everyone was already happy.

This was 1987 to 91.

[RayB]

Guys, even coding for for the 16bit systems was generally done in Assembly. We're talking 1mhz machines. You had to squeeze out every bit of juice possible and manipulate data directly at the bit level.

[jimmy2x2x]

Guys, even coding for for the 16bit systems was generally done in Assembly. We're talking 1mhz machines. You had to squeeze out every bit of juice possible and manipulate data directly at the bit level.

Not to undermine anyone else, but you are talking a lot of sense here and this is my impression of that era too.

[lilshawn]

see, everyone is stuck in "today thinking".

back in the day they just wrote the programs and controlled the CPU directly. there is no coding in c+ and compiling it, there is no FORTRAN...nothing. no compiling there is no assembing of laguage at all. Everything is/was straight machine language. you needed information from a register, you sent the command and the CPU spit out the contents of the address.

inherently machine code makes absolutely no sense... this is why there is no reverse engineering of the original code. it becomes a mish mash of commands randomly reading and writing to registers. it's all there but only once the program runs does the outcome show whats going on. In the Z80 the machine code 00000101, which causes the CPU to decrement the B processor register, would be represented in assembly language as DEC B. the 00000101 gets lost in the stream of other commands and makes it impossible to "see"

i'd like to compare it akin to a plain box full of puzzle pieces but even that doesn't seem fitting.

[Gray_Area]

Following up this topic, I came across this:

Anyhow, you might be curious about how I actually went about developing this. Seeing as I'm not a CAVE employee, I don't actually have the source code. Instead, I relied on my competancy as a reverse engineer to find all of the things I needed to change in order to make the game mechanics how I want. Now, you might assume I used a compiler, or even an assembler. You'd be dead wrong! I wrote out the assembly code I wanted to insert, then I referred to the M68000 User Manual to manually assemble every instruction.

http://dodonpachi.daifukkat.su/ddpa/

[pwrx]

Ancient post but just seeing this site...  There are just a few 'fully' reverse engineered arcade machines.

By this I don't mean selective commenting on sections of the source code, rather the purpose of every last byte 100% known, all interaction with routines, memory locations documented with variable names, internal state machines documented - pseudo intenal languages known with tools to compile/decompile that too.  Result being code that is largely relocatable, and easily modifiable but initially can produce byte-matching roms.

I spent a huge amount of time working on Gauntlet II and only got about 25% through.

I did go all the way with Omega Race some time ago.  This one is without a doubt written in assembly and for the most part every byte / cycle counted.  It was pretty clear though that they had a sound effect team and programming team that didn't spend enough design time together.  Lol.