Can someone explain this whole SmallCHD / BigCHD thing to me?

[JoyMonkey]

I've noticed a lot of people talking (or arguing) about SmallCHD and BigCHD on alt.binaries.mame, but I seem to be completely ignorant to these programs. I've picked up little bits of information on them here and there, but don't fully understand what's going on



According to the included txt file, it's:

Simple tool to remove the redundant bytes from your MAME CHD files.
The latest version of this program can be downloaded from:
http://home.comcast.net/~eevert/

So it makes CHD files smaller, right?
And MAME still runs them like normal?
Will ClrMame still recognise a CHD if SmallCHD has crunched it?
How do I know if a CHD I've downloaded has been through it already?

Can someone spell it all out in plain English for me? I'm sure I'm not the only one that's a bit confused.

[Howard_Casto]

the option to run small chds was removed from mame as of last version.  Apparently it's a cheap hack that causes issues.  If mame won't run the chd then it isn't "small".  

[u_rebelscum]

CHD = compressed hard drive.  Each sector is compressed using an open source zip algorithm, each and every one of all these sectors are stringed together in order on the HD into one PC file, with a TOC/header in the front so when the game needs the info in sector 51, mame knows that sector 51 starts at byte 12345.  (This is needed because the different sectors compress to different sizes.)

BigCHD:  See above.

SmallCHD: Some sectors in a hard drive might containt the same info (before and thus after compression).  The idea behind smallCHD is, unlike the normal CHD, keep one and only one copy for each group of sectors that are the same, and have the TOC point to that one copy.  This results in a smaller file, but can mix up the parts of an area of a HD to be stored in various parts of the file.


Example:
First two rows is the TOC, third row the sector number, forth row the data in that sector.

Normal CHD
abcdefghj
123456789
1 2 3 4 5 6 7 8 9
A B C A D A E E F

SmallCHD
abcdefghj
123141556
1 2 3 4 5 6
A b c d E f

The small'ed CHD removed two "extra" copies of secotrs containing the data "A" and one of "E".  However, if part of the game originally read c-d-e in that order, the original HD and CHD would read sectors 3 4 5, but with the smallCHD, it now has to read 3 1 4, which might mess up caches and other stuff, thus slowing down the reading of the data.

There also was the whole arguement on what if the games changes one of the duplicate parts and not another (not going into that).  

Then the person who designed the CHD "format" for mame desided SmallCHD wasn't good enough for mame, and made it so mame does not construct smallCHDs.  However, AFAICT, mame can still read and use smallCHDs for playing.  If you run into any problems with them, though, too bad since mame does not officially support them.


The consense I've read is: play with the full size CHD (aka bigCHD), but backup and distribute in smallCHD.

[lokki]

Hi,
I think another isse with the big CHD vs Small CHD. is that if the game tries to write to the disk. On a small CHD the write operation could write a sector and overwrite other sectors that are compressed as the same.