Linux I/O Scheduler Comparison On The Linux 3.4 Desktop

[kruegsch]

It's nice to see, that this topic has still a scientific relevance. Unfortunately most of the results don't find the way to us :/
The paper about the "Selective Timeout Bundling" scheduler for example was very exciting to me.

Seungyup Kang, Hyunchan Park, Chuck Yoo, "Performance Enhancements of I/O Schedulers for Solid State Drive", 2011 IEEE International Conference on Consumer Electronics (ICCE), Jan. 2011.(PDF)

I'm using now the deadline I/O scheduler for my Samsung 830 SSD.

[del_diablo]

Re: Responsiveness

For years I've had problems using the desktop while a intensive IO operation is running. Even when simply transferring file to pendrive.

https://bugzilla.kernel.org/show_bug.cgi?id=12309

Some other site covered the issue.
I don't think we will see anything like it on Phoronix, but framejitter and uneven load is fairly important issues.

[cyring]

I don't really know, but I have read this:

I did once a secure erase, however my SSD is dual booting Linux and Seven. This last one is a pain to backup/restore keeping a right alignment while Linux is a piece of cake
Beside package management, I don't do so much writings on the SSD, throwing much of them on a ram fs. I also keep around 18% of unused space for wear leveling.

I have followed ArchLinux's template for SSD benchmarking and here are my published results, selecting noop as an I/O scheduler.

[ChrisXY]

I don't really know, but I have read this:

On occasion, users may wish to completely reset an SSD's cells to the same virgin state they were at the time he/she installed the device thus restoring it to its factory default write performance. Write performance is known to degrade over time even on SSDs with native TRIM support. TRIM only safeguards against file deletes, not replacements such as an incremental save.

[cyring]

Ok, thanks. I'll leave it this way until I can at least test with SATA3.

And sorry for hijacking the article thread.

Your benchmarks seem so random. It seems to be coincidence what data it writes fast and what it writes slow. Was your SSD used much before these benchmarks and have you tried a "factory reset" (however unhealthy it may be)?

Nop since a long time : more than 2 years a go I did a firmware update which refreshed the SSD

I wonder if the fstrim command is enough to keep it in a good state ?
Whenever I enter this it replies immediately with a

$ sudo fstrim -v /
/: 17685495808 bytes were trimmed

[ChrisXY]

Your alignment sounds OK

Ok, thanks. I'll leave it this way until I can at least test with SATA3.

And sorry for hijacking the article thread.

Your benchmarks seem so random. It seems to be coincidence what data it writes fast and what it writes slow. Was your SSD used much before these benchmarks and have you tried a "factory reset" (however unhealthy it may be)?

[cyring]

That output doesn't really match up. Fdisk says my sector size is 512 byte and the partition 1 begins at sector 2048. And 512 * 2048 / (1024*1024) = 1.

How does parted think that this is at 1049 kB? And why does it still think it is optimally aligned?

 % sudo parted /dev/sda align-check opt 1
1 aligned

I also have the same issue

$ sudo parted --list

Model: ATA CORSAIR CMFSSD-6 (scsi)
Disk /dev/sdb: 64.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      [COLOR="#FF0000"]1049kB[/COLOR]  25.8GB  25.8GB  primary  ntfs
 2      25.8GB  52.2GB  26.4GB  primary  ext4         boot

Notice that my first part is NTFS and not Linux.
I mainly use fdisk and don't bother with parted.

Your alignment sounds OK

[cyring]

hdparm -I should give you more detail information.

for instance , mine (sdb) which is a Corsair 64GB SSD with a Samsung firmware, outputs the following :

$ sudo hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
        Model Number:       CORSAIR CMFSSD-64GBG2D
...
        Firmware Revision:  VBM19C1Q
Standards:
        Used: ATA/ATAPI-7 T13 1532D revision 1 
        Supported: 7 6 5 4 
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  125045424
        LBA48  user addressable sectors:  125045424
        [COLOR="#0000FF"]Logical/Physical Sector size:           512 bytes[/COLOR]
        device size with M = 1024*1024:       61057 MBytes
        device size with M = 1000*1000:       64023 MBytes (64 GB)
        cache/buffer size  = unknown
        Nominal Media Rotation Rate: [COLOR="#0000FF"]Solid State Device[/COLOR]
...

However to be sure of the size, I have checked @ Corsair product support which confirms that the physical size is actually 512

Using fdisk -l /dev/sdb gives the same. May come from the same source that hdparm.

$ sudo fdisk -l /dev/sdb

Disk /dev/sdb: 64.0 GB, 64023257088 bytes
255 heads, 63 sectors/track, 7783 cylinders, total 125045424 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/[COLOR="#0000FF"]physical[/COLOR]): 512 bytes / [COLOR="#0000FF"]512[/COLOR] bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1687aa3a

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            [COLOR="#008000"]2048[/COLOR]    50333695    25165824    7  HPFS/NTFS/exFAT
/dev/sdb2   *    [COLOR="#008000"]50333696[/COLOR]   101963775    25815040   83  [COLOR="#0000FF"]Linux[/COLOR]

$ sudo fdisk -l /dev/sda

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xca100020

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048   100665343    50331648    7  HPFS/NTFS/exFAT
/dev/sda2       100665344   257039999    78187328   83  Linux
/dev/sda3       257040000  1953525167   848242584    5  Extended
/dev/sda5       257040063   484681049   113820493+  83  Linux
/dev/sda6       484681113  1953525167   734422027+  83  Linux

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
[COLOR="#0000FF"]rootfs           25G  8.1G   16G  35% /[/COLOR]
/dev            3.0G     0  3.0G   0% /dev
run             3.0G  316K  3.0G   1% /run
/dev/sdb2        25G  8.1G   16G  35% /
shm             3.0G   76K  3.0G   1% /dev/shm
tmpfs           2.0G  105M  1.9G   6% /tmp
tmpfs            64M  412K   64M   1% /var/log
/dev/sda2        75G   45G   27G  63% /vm
/dev/sda5       109G   71G   33G  69% /users
/dev/sda6       700G  441G  225G  67% /data
tmpfs           1.0G  1.1M 1023M   1% /home/cyril/tmp

Because I had doubts about Linux alignment, I did this way :

1. Backup my root FS with the tar command ( as show above my root FS is onto the SSD, the others on the HDD )
2. Prepared all partition with Seven but without formatting them, at least for the Linux one
3. Tagged the Linux part as Extended FS using fdisk
4. Formatted it with mkfs.ext4 , because Ext4 seems to be the only one to offer the discard option to trim SSD
5. Restored the root FS from the tar.gz backup

Btw, I'm also doing I/O Scheduler benchmark and I have eratic results with the SSD !
They change all the time whatever the scheduler is noop , deadline or cfq

Fyi, I'm simply benchmarking with the dd command

dd if=/dev/zero of=./outfile bs=[I]NNNNN[/I] count=100000 oflag=noatime,direct

so far the SSD with a bs=32768

noop     = 51 MB/s
[COLOR="#008000"]deadline[/COLOR] = [COLOR="#008000"]83[/COLOR] MB/s
cfq      = 55 MB/s

just after executing

fstrim -v /

noop     = 132 MB/s
[COLOR="#008000"]deadline[/COLOR] = [COLOR="#008000"]143[/COLOR] MB/s
cfq      = 96 MB/s

with a bs=16384

noop     = 99 MB/s
deadline = 79 MB/s
[COLOR="#008000"]cfq[/COLOR]      = [COLOR="#008000"]102[/COLOR] MB/s

still bs=16384, right after triming

[COLOR="#008000"]noop[/COLOR]     = [COLOR="#008000"]104[/COLOR] MB/s
deadline = 94 MB/s
cfq      = 78 MB/s

with a bs=8192 and 'nocache' rather than 'direct' in oflag

noop     = 164 MB/s
deadline = 153 MB/s
[COLOR="#008000"]cfq[/COLOR]      = [COLOR="#008000"]170[/COLOR] MB/s

while the HDD is constant with a bs=32768

noop     = 107  MB/s [30.5302s]
deadline = 107  MB/s [30.7194s]
cfq      = 107  MB/s [[COLOR="#008000"]30.5122s[/COLOR]]

for now I choose noop for the SSD and cfq for the HDD

That's what I meant. What is a "physical sector" size in a SSD?

Would resizing the beginning of the partition to the correct "sector" be the right thing to do or do I have to recreate the whole partition (and/or table)?


Right, english is better:

 ~ % sudo LC_ALL=C fdisk -u -l /dev/sda

Disk /dev/sda: 240.1 GB, 240057409536 bytes
64 heads, 32 sectors/track, 228936 cylinders, total 468862128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1d172924

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   419432447   209715200   83  Linux


 % sudo LC_ALL=C parted --list
Model: ATA SanDisk SDSSDX24 (scsi)
Disk /dev/sda: 240GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End    Size   Type     File system  Flags
 1      1049kB  215GB  215GB  primary  ext4         boot

I have read that 64 heads and 32 sectors align to 1 MiB EBS blocks (64 * 32 * 512 = 1048576 byte = 1 MiB) so I tried that. I did this because the original default alignment of util linux fdisk 2.20 seemed to be wrong but it's more probable I only thought it was wrong when it was in fact correct.

Hopefully 1049kB (not 2049) is actually 512 kbyte * 2.

But it doesn't matter that much since I don't have too much bandwidth on this notebook anyway (will get replaced in the not so far future):

 % sudo LC_ALL=C hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 776 MB in  3.00 seconds = 258.59 MB/sec

[ChrisXY]

64 heads, 32 sectors/track, 228936 cylinders, total 468862128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1d172924

 Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   419432447   209715200   83  Linux

.....

Number  Start   End    Size   Type     File system  Flags
 1      1049kB  215GB  215GB  primary  ext4         boot

That output doesn't really match up. Fdisk says my sector size is 512 byte and the partition 1 begins at sector 2048. And 512 * 2048 / (1024*1024) = 1.

How does parted think that this is at 1049 kB? And why does it still think it is optimally aligned?

 % sudo parted /dev/sda align-check opt 1
1 aligned

[Sadako]

Bellow, some of my readings

http://en.wikipedia.org/wiki/Noop_scheduler

http://grzen.blogspot.fr/2009/05/ssd...-linux_18.html

http://ubuntuforums.org/showthread.php?t=1178209

According to Wikipedia :

• " NOOP scheduler is best used with solid state devices such as flash memory or in general with devices that do not depend on mechanical movement ... "
• Deadline: " The kernel documentation suggest this is the preferred scheduler for database systems ... "

In Kernel doc /usr/src/linux/Documentation/block/cfq-iosched.txt says :
" That means by default we idle on queues/service trees. This can be very helpful on highly seeky media like single spindle SATA/SAS disks ... "


I'm looking for a way to specify the kernel which respective scheduler to use with SSD & HDD, using boot command such as : elevator-DEVICE=<scheduler> rather than dealing with /sys/block/DEVICE/queue/scheduler (which happens after boot, ie. rc.local)

TYVM for the links, I didn't really come across anything sugesting that deadline or noop would result in less writes, and the reason I'm so curious is because I've seen suggestions that noop can actually increase the number of writes, because it doesn't delay writes to "group" them like deadline and especially cfq, meaning more smaller writes, possibly being less efficient on SSDs with relatively large write block sizes.

Don't have any links for this though, but it does kind of make sense to me...

Also, somewhat off topic, but I'm adding an SSD to my desktop and considering putting the journal for the SSD's ext4 filesystem on a partition at the very start of a hard disk in the system, should reduce writes somewhat, with hopefully not much of a performance impact, any thoughts?