Main > Main Forum
I would like to use a compact flash as a hard drive in a bartop cabinet, but...
(1/7) > >>
jukingeo:
Hello All,

I have some questions that I been mulling about in my head as a result of some on-line reading in regards to using a Compact Flash as a compact solution for storing an embedded operating system on for the use in various home arcade devices (and also other projects).  

Why CF?

While I am into the speed gains when going with a form of ssd, I am also into the convenience of being able to remove the CF card and put it on my main computer to update it, or for that matter making a copy of a system to create another system using an identical motherboard.

Test bed:

My current test bed that I am using is an old Compaq Evo D51s computer in which I removed the hard drive and installed a Transcend 16gig Compact Flash card using a Syba Dual CF IDE adapter card.  The computer has a 1.8ghz Intel processor with 256meg of ram.

The OS:

Currently I have a stripped down version of Windows XP Professional loaded on to the Compact Flash card.  I stripped Windows down using nLite.   The reason why I have chosen Windows XP is that it is the most current OS that I have been working with and just about all the projects I would like to work on use Windows XP.   Also certain programs, such as the Hyperspin Front End only can work with Windows XP.   Windows XP also has full support for both USB AND Firewire in addition to the ability to use parallel and serial ports (most of my projects need some form of I/O capability for interfaces).  

I currently have Windows XP set up on the compact flash with several programs that I would use in a stand alone situation.  For the most part, the compact flash performs just as well as a hard drive.

OK, Now for the "But..."

The "BUT" lies in the fact that my OS choice of using Windows XP is a double edged sword.  In my recent readings I have found out that Windows XP is notorious for writing constantly to the hard drive.   Now for a normal mechanical hard drive, this isn't a problem as a hard drive stores data magnetically, you can read AND write data until the cows come home or until the drive fails (whichever comes first).    BUT (there is that word again), with a compact flash, this is not the case.   The act of erasing and re-writing data will eventually take its toll on the compact flash card.   I know that this is on the order of 100k to 1m writes and that would normally seem like a trivially large figure, but given the amount of r/w operations Windows XP does, those numbers could add up very quickly.

Moving forward...

Ok, so I have looked into several things that cause Windows to write to the hard drive repeatedly, such as the restore feature, the Indexing feature, the paging file (Virtual Ram or Swap File) as well as other things mentioned on this web page:

http://www.jasonn.com/turning_off_unnecessary_services_on_windows_xp

The good thing was that many items mentioned on the list were not applicable because I had not installed them via the use of nLite.

Where I stand Now...

Now after I turned off just about everything mentioned in that link that I didn't need, I disconnected the network cable and sat there watching the hard drive light to see if it would blink.   Sure enough about 30 seconds pass and the light flashes once.   The timing isn't consistent, but every now and then, the light does blink when the system is sitting idle.   So now I am wondering what else could be still running that is causing Windows to still access the drive?  Being that the light shows all drive activity, I have no way of knowing if what is happening is a read or write operation.   Now I am second guessing using Windows XP as the operating system for this project.

What are my alternatives?

Being that I am worried about ruining the compact flash, I am starting to think of alternatives.

1) Change the OS.

Alright, I CAN do this, but my choices then would be Linux, Windows 98SE, or DOS.  As far as I know Win2k (while a great OS) still shares the same hard drive accessing issues as Windows XP.

Of the three operating systems Linux would be the most compatible with modern motherboards, but most of the programs I would like to use don't run under Linux, or if they do, they run too slow.

With Windows98SE or DOS, I wouldn't have the problem with constant access to the compact flash, but then I run into the problem of getting drivers for the motherboards I would like to use.

While DOS (FreeDOS to be exact) is extremely small and is a very good candidate for compact flash use, the problem I run into with this OS is finding drivers for the on-board sound on the motherboards I would like to use.  Also with DOS I run into problems with USB support.   Only MAME and DWJukebox are two programs that can run in DOS.  The good news is that am mostly basing my projects on these two programs.

2) Another storm of the brain...

So my next bright idea which I would like to swing by everyone here is as follows:   Bouncing back to using Windows XP,  with a modern motherboard, I can load it up with 4gig (or more) of memory and have the OS load up entirely in ram.   In this case, it wouldn't matter the number of read/write cycles.   I would set up the compact flash as a read only device.

Now, can this be done?  If not then would there be a solution in using another OS?

I turn the floor over to you.

Thank You,

Geo


nox771:
First, regarding write endurance I think this comes down to whether your CF cards have wear leveling or not.  CF is commonly used as a HD replacement in embedded applications, and I think there are ones out there which do have wear leveling circuits.  If you obtain specific cards which do, and you don't fill them to max capacity (assuming dynamic not static leveling) then this is a non-issue, you could write to them full out for years (decades?) and never hit the cycle limit.

I use USB flash drives extensively at my work, and I've run full Linux installs off them for years at a time with no issue.  I think most controllers these days do dynamic leveling, but I did have one drive which did static leveling (it was unbearably slow).  Currently I have an Arch linux image I run off a USB flash drive (it is a virtualbox image inside a truecrypt container), no problems yet.  My concerns with flash, esp MLC cells, are always much more with regard to bit rot than wear leveling though.  I don't know if there are any studies of MLC flash reliability with regard to environmental factors (ie. heat), but having designed flash cells in the distant past I know leakages are a major concern (and manufacturers are reluctant to disclose mitigating circuitry, such as ECC implementations, probably due to a lack thereof - it makes it very difficult to gauge reliability).

Regardless of all this, I think the goal of having a never fail CF card is the wrong way to go about it.  For a game machine the install is essentially static (aside from maybe high scores).  Small capacity flash is cheap these days, just get a couple cards and make one an offline backup.  If you are constantly tweaking the machine just refresh the backup every 2-3 months or so.  This is actually the way I go about it myself - I put all my files inside a truecrypt container on the flash device, and every day/week/month/whatever I drag and drop the whole container to a backup HD.  Since the HD is much larger than any flash you can keep multiple timestamped images.

Your other idea is intriguing though - running it off a ramdisk.  I've never gotten around to it myself, but there is something you could try here:
http://memory.dataram.com/products-and-services/software/ramdisk

I think if you can get your image small enough to fit inside a ram disk, plus allow enough working ram for the thing to function, then that is a great idea.  I remember running a Puppy Linux live CD a long time back, and it would load itself entirely into ram, and it was blazing fast - programs would open instantly.  I've had that sort of thing on my to-do list for a long time, so if you do that I would be interested in knowing your results.

Actually this gives me another idea - if you use a ramdisk to load the OS and emulator and then add in a wireless card and a NAS, you could source the ROMs wirelessly from the NAS so you would have no capacity issue.
newmanfamilyvlogs:
Why not use a proper ssd with a sata-to-ide adaptor?
http://www.newegg.com/Product/Product.aspx?Item=N82E16820139
MonMotha:
With proper wear leveling, "wearing out" a large flash devices is not really a concern.  I wouldn't recommend using them as swap, though not just for reasons of flash endurance: it'll be slow as hell, too.  High end CF cards should all have decent wear leveling.  Low end ones sometimes seem to have none (!!) and sometimes even seem to lack usable block replacement (!!!!).  Get a quality card from a reputable vendor like Sandisk.  Be aware that most CF cards in TrueIDE mode aren't capable of anything near the performance of even a low end consumer SSD.


As to environmental, my understanding from my technical reps is that elevated temps are the only real concern aside from things that cause corrosion or mechanical wear such as extremely high humidity.  Elevated temps (still within spec'd operating range) will cause premature margin degradation on the flash cells.   Data retention is usually spec'd as 10 years minimum at room temp for high density, non-industrial devices.  Elevated temps might reduce this to a few years, though you'll typically see much more in most cases.  Frequent reads from a reasonably smart card will keep this from being a problem: the device will notice a mis-read, correct it with ECC, then erase/rewrite the data and entire mark the block as bad (pessimistic) or attempt to erase and test it for reuse (optimistic).

I had a talk with a TI rep regarding one of their microcontroller products with integrated flash.  They were supporting 125C operation for a client, but the usable lifetime was only about 3 HOURS!  At 85C, they spec'd 10+ years minimum.  I'm guessing the relationship is pretty steep on certain parts of the curve.


Note that keeping the card "only partially full" doesn't work unless the card and your OS/filesystem support TRIM.  I'm not sure that many CF cards support this, but some may.  Windows XP doesn't support TRIM at all.  Windows 7 should.  Newer versions of Linux should.  DOS will not at all.  Using an extremely large (much bigger than your dataset), making sure it's fresh (new or recently "secure erased"), and limiting writes may help with this, but it's a one-time deal.  Once a logical sector on the card has been written once, it never knows if it's actually logically free again unless TRIM is used.

If you must run WindowsXP, there are some tricks, but I think MS only officially supports them on the "embedded" version.  You can have it do copy-on-write to a ramdisk.  This effectively "freezes" the system partition as any changes you make exist in RAM only and go away when you reboot and everything loads back off the mass storage.  This avoids the overhead of a full ramdisk while still removing all write activity to the system partition.  There's a "thaw" command if you want to make changes and commit them.  I don't know anything about how to do this, only that it exists.  A similar trick can be done on Linux via a variety of methods, or it's possible to just run Linux with everything mounted readonly, if you're careful.
nox771:

--- Quote from: MonMotha on June 17, 2011, 11:44:29 pm ---Note that keeping the card "only partially full" doesn't work unless the card and your OS/filesystem support TRIM.  I'm not sure that many CF cards support this, but some may.  Windows XP doesn't support TRIM at all.  Windows 7 should.  Newer versions of Linux should.  DOS will not at all.  Using an extremely large (much bigger than your dataset), making sure it's fresh (new or recently "secure erased"), and limiting writes may help with this, but it's a one-time deal.  Once a logical sector on the card has been written once, it never knows if it's actually logically free again unless TRIM is used.

--- End quote ---

I don't think this is exactly right.  Since flash erases in blocks, on modification normally unmodified sections would need be read prior to block erasure such that they could be combined with the new data and written to a new block, only then could the original block be erased.  TRIM is just a way to preemptively mark a section as completely ready to overwrite, it negates the overhead of reworking the whole block when you are modifying just a bit of it.  However regardless of TRIM support, erased blocks on USB flash and CF flash can be overwritten - the device will not completely fill up with deleted blocks.  I know I've worked with many flash devices running at 90% capacity with no issues.  That said, barring static wear leveling the dynamic wear leveling normally used can only operate on unused blocks, so running at that 90% capacity is a bad idea with respect to cell wear.
Navigation
Message Index
Next page

Go to full version