USB drive on server?

[Ted Marshall]

----- Original Message ----- 
From: "John Austin" <ja at jaa.org.uk>
Newsgroups: gmane.linux.redhat.fedora.general
To: "For users of Fedora" <fedora-list at redhat.com>
Sent: Wednesday, October 24, 2007 6:15 AM
Subject: Re: USB drive on server?


> On Tue, 2007-10-23 at 16:32 -0700, Ted Marshall wrote:
>> From: "John Austin" <ja at jaa.org.uk>
>> Newsgroups: gmane.linux.redhat.fedora.general
>> To: <tim at birdsnest.maths.tcd.ie>; "For users of Fedora"
>> <fedora-list at redhat.com>
>> Sent: Tuesday, October 23, 2007 9:40 AM
>> Subject: Re: USB drive on server?
>>
>>
>> > Dup from other thread
>> >
>> >> Over 10 year life time
>> >
>> > http://www.corsairmemory.com/_faq/FAQ_flash_drive_wear_leveling.pdf
>>
>> The problem with the statistics given for Dynamic Wear Leveling is that 
>> the
>> flash controller must be able to know which blocks are not in use by the
>> filesystem.  AFAIK (and this is very difficult to verify one way or the
>> other), consumer grade devices (at least) are only aware of FAT 
>> filesystems
>> and only if there is only one partition on the device.  If you use ext2/3 
>> or
>> have multiple partitions (both typical for Linux) the controller cannot
>> track blocks being freed by the filesystem and so any block ever written 
>> by
>> the host will looks busy to the flash controller.  Therefore, writes will
>> only rotate through the (small) pool of extra blocks maintained by the 
>> flash
>> controller.
> This is where I hope you are wrong !!!!!!!! See below
>>
>> Also, AFAIK, few if any consumer-grade and, I suspect, few
>> professional-grade flash devices do Static Wear Leveling.
>>
>> The result is that the flash drive will burn out much faster than the
>> statistics quoted.  How much faster, I am unable to compute.  It may 
>> still
>> be adequate for your needs.  I don't know.
>>
>>
>
> You have made me have a very careful read of the the Corsair info
> and to query my understanding!!
>
> Assuming the Corsair info true then my understanding is as follows
>
> The Flash Voyager GT uses Dynamic Wear Levelling
>
> I am not convinced that it knows about VFAT, NTFS, ext2 ...
>
> The stick does not know about which is static data and which is dynamic,
> all its knows is that if it is written to then it has to find
> somewhere to put it.
>
> The Corsair info suggests that there is more memory on the stick than
> you can get at, See Fig 3, I do not know if this is true.
>
> The stick does not need to know which bits are freed by the
> operating system, only which bits have been freed by the
> wear levelling look up table.
>
> When the operating system deletes a file then it will free up
> a certain number of cylinders/tracks/sectors as "it" understands them.
> At a later time the operating system will write to those 
> cylinders/tracks/sectors again.
> At that time the stick will remap them to different locations on the 
> stick,
> freeing up the previously used physical memory.
>
> My F7 stick layout is shown below.
>
> I conclude that there is approx 6GB of "Static" data and 2GB of
> Dynamic data (Free Space as far as Linux is concerned).
>
> Thus if there is 2GB "free" as understood by the operating system then 
> this
> is "almost" equivalent to 2GB being "free" to be mapped by the stick.
>
> The bit that gets the hammering will be the superblocks - every time
> the inode access time is updated for either read or write.
> The superblock data will be moving around on the stick as
> a result of the wear levelling, however the wear levelling
> will only be applied over 2GB instead of 8GB.
>
> If I remember correctly then the superblock (or hopefully only part of it)
> will be written every 30 seconds.
>
> http://www.storagesearch.com/siliconsys-art1.html
> http://www.stec-inc.com/downloads/AN-0702_STEC_SMALL_CARDS_WEAR_LEVELING_LIFETIME_CALCULATOR.pdf
>
> both give a formulae to compute the lifetime of wear levelled devices.
>
> Filling in the first as follows gives a very large number of years!!
> The 100,000 write cycle figure comes from the Corsair info
> A bit of superblock = 10kB ????????
>
> [(8000 - 6000)*100,000*(1 - 0)]/[(0.01*2 writes/min)*525,600]
> = 19,026 years.
>
> I conclude:
>
> You should NEVER use the stick for long periods when it is
> close to full as the wear levelling will only be able
> to act over the remaining memory available at that time
> and will become ineffective.
>
> I would be very grateful for feed back on this !!
>
> I still have a 10 year warranty to fall back on !!
>
> As an aside
> Is it a good idea to run "/" mounted without "atime" ??
>
> John
>
>
>
> [root at corsair ~]# df
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/sda2              7549152   4869528   2296148  68% /
> /dev/sda1                41053     19301     19632  50% /boot
> tmpfs                  1684780         0   1684780   0% /dev/shm
> [root at corsair ~]#
> Disk /dev/sda: 8287 MB, 8287944704 bytes
> 241 heads, 32 sectors/track, 2098 cylinders
> Units = cylinders of 7712 * 512 = 3948544 bytes
>
>   Device Boot      Start         End      Blocks   Id  System
> /dev/sda1               1          11       42400   83  Linux
> /dev/sda2   *          12        2000     7669584   83  Linux
> /dev/sda3            2001        2098      377888   82  Linux swap / 
> Solaris
>
>
>
>
>
> -- 
> fedora-list mailing list
> fedora-list at redhat.com
> To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
>

Blocks that have been freed in the filesystem are still static data as far 
as the flash controller is concerned, unless the controller can understand 
the FS metadata.  Yes, when they are reallocated by the FS, the controller 
will be able to move that to another free flash block, freeing the old 
block.  However, as long as they just sit unused by the FS, they are 
unavailable to the flash controller for wear leveling unless "static" wear 
leveling is used.  As I see it, the worst case is where the filesystem has 
touched all the blocks that the controller makes visible and then one 
single-block file (or the superblock) is constantly rewritten in place.  The 
controller will only be able to wear level between that block and its 
spare-block pool.

So the question is "how big is the spare-block pool"?

http://www.sandisk.com/Assets/File/OEM/WhitePapersAndBrochures/RS-MMC/WPaperWearLevelv1.0.pdf 
was published by Sandisk 4 years ago.  Here's an early paragraph:

"Each memory chip is divided into blocks. A block is an array of memory 
cells organized as
sectors. The number of blocks and sectors vary from product to product. The 
minimum unit for a
write or read operation is a page (or sector). The minimum unit for an erase 
operation is a block.
Physical blocks are logically grouped into zones. For the current 
technology, a typical zone size is
4 MB. However, this may change from product to product. Wear leveling is 
done within a zone.
The current firmware does not spread the wear across the capacity of the 
card. Each zone has
about 3% additional "spare blocks" beyond what is assigned to meet the 
logical capacity of the
flash card. This group of blocks is commonly referred to as the "Erase Pool"."

On the other hand, your Corsair PDF talks of a 2GB device having a write 
page size of 2KB, an erase block size of 128KB (64 pages) and a leveling 
management unit of 128MB (1024 blocks).  Unfortunately, it does not tell us 
how many spare blocks are in a leveling unit.  I'll assume the 3% from 
Sandisk and this gives us about 32 spare blocks.

Next, we have to ask what happens if only one page is changes in the block? 
I'm going to assume the worst case that this causes the whole block to be 
moved to a new block and the old one freed.  Corsair states that they use 
SLC technology for 100,000 erase cycle life instead of MLC with 10,000 so I 
will use this assumption.  (I have read specifications for other devices for 
10,000 cycle lifetime so I'd but that the cheaper devices d ouse MLC).

Ok, so assume that the superblock does get rewritten every 30 seconds (I do 
not know this for a fact).  The lifetime of the wear management unit 
containing the superblock is 32 * 100000 writes or 3 years.  OK, that may be 
ok for you.  On the other hand, if the device uses MLC technology, devide by 
10 and you only get 3.6 months!!

Now it may be that some devices use better algorithms and will give you a 
longer life.  (I would expect that a US$5000 "sold-state disk" will have 
implemented things better and would have a much longer life.)  On the other 
hand, if you run out to Fry's and buy a $10 1GB USB stick, don't expect it 
to last vey long.

One problem I've seen is that no manufacturer gives you all of the 
information you need.  Also, what is true for one device may not be true for 
another.

It seems to me that one way to extend the life is to use ext3 with a large 
"ordered" journal that will spread out the writes to the superblock 
throughout the journal (I assume the superblock is journaled, anyone know 
for sure?).  Also, turn off atime on the filesystem.

What we need is a version of JFFS that can be used on an external device. 
Assume that the device does not do wear leveling and do it in the OS.