Ahh, interesting, this time my post doesn't wait for approval - so this is a benchmark after how long my post gets deleted - reposting my original message sent 4 days ago:
I'm really fascinated by you "benchmarks" - in a negative way though I must admit. For instance your "infamous" sqlite insert test - what you do is a complete nonsense. Without looking at the benchmark code, I can confidentaly say that you do inserts without enclosing them in a transaction. The result is that what you benchmark is "really bad programmer's code" performance rather than something that any reasonable program would do. Of course you _would_ know this if you knew what you are doing (which is the biggest problem of your benchmarks that just show some random charts without any context). For instance, this is from SQLite FAQ:
Now look at the results of your insert benchmark - does it look familiar? I bet it does - few dozen inserts instead of tens of thousands of inserts. Is your benchmark useful? Not at all - nobody will be so stupid to do such a thing in his program. (What your benchmark does for rotational hard drives is that it measures the number of rotates per second).
(19) INSERT is really slow - I can only do few dozen INSERTs per second
Actually, SQLite will easily do 50,000 or more INSERT statements per second on an average desktop computer. But it will only do a few dozen transactions per second. Transaction speed is limited by the rotational speed of your disk drive. A transaction normally requires two complete rotations of the disk platter, which on a 7200RPM disk drive limits you to about 60 transactions per second.
Transaction speed is limited by disk drive speed because (by default) SQLite actually waits until the data really is safely stored on the disk surface before the transaction is complete. That way, if you suddenly lose power or if your OS crashes, your data is still safe. For details, read about atomic commit in SQLite..
By default, each INSERT statement is its own transaction. But if you surround multiple INSERT statements with BEGIN...COMMIT then all the inserts are grouped into a single transaction. The time needed to commit the transaction is amortized over all the enclosed insert statements and so the time per insert statement is greatly reduced.
Another option is to run PRAGMA synchronous=OFF. This command will cause SQLite to not wait on data to reach the disk surface, which will make write operations appear to be much faster. But if you lose power in the middle of a transaction, your database file might go corrupt.
One more example would be your article "The Performance Of EXT4 Then & Now" which was supposed to demonstrate how the performance of ext4 evolved. Well, you did _not_ measure that at all. The fact that some disk benchmark is slower under kernel A than kernel B doesn't mean that it's because of ext4 - there have been quite dramatic changes in IO scheduler towards lower latency, which I think is the main cause of the performance changes. Also the per BDI flusher threads had definitely significant impact on IO performance. See 1.1 and 1.5 here
ext4 is relatively stable and there are not many changes these days that would influence performance dramatically. However, 2.6.33 contains a large number of CFQ changes, which I'm quite sure contribute to performance differences in benchmarks much more than the modifications of filesystems.
If you want to get filesystem performance history, you should take several filesystems (ext3, ext4, XFS, reiser) and measure their performance for all the kernel version. If all of them have some performance in kernel version A and lower performance in kernel version B, the reason will most probably be that something else than the filesystem has changed that influenced the performance (e.g. IO scheduler) [Of course, some IO scheduler change can influence one filesystem more than the other filesystem so this is not very exact either]. Now concerning the performance drop between 2.6.30 and 2.6.31 - are you sure that Ubuntu didn't start using CFQ scheduler then instead of anticipatory IO scheduler? (I really don't know - just a wild guess, probably totally wrong. What I want to say is that when you perform a benchmark, you do it for the whole kernel - and there are pretty many things that might influence the result.)
I'm really sorry if I sound too offensive but I just can't understand your attitude to the benchmarking. Less is more sometimes. Instead of having tens of more or less garbage benchmarks a few benchmarks where you know what you are doing and where you can interpret the results somehow is _much_ more useful. It's a real pity - I can imagine you spend a lot of time by preparing everything for your web page but the result is something that nobody can take seriously. How about re-prioritising your work and start learning what you are doing? ;-)