Facebook's Flashcache For The Linux Kernel

[ernodv]

In light of OCZ releasing their new Synapse drive with a proprietary Windows-only caching software (review), I started to look for a Linux alternative for using a SSD as a block-level cache for a HDD and stumbled upon bcache. (Yes, bcache has been mentioned earlier in this thread too). At first I was a bit disappointed because there seemed to be some major tradeoffs, e.g. lacking TRIM. Thankfully only the bcache wiki was out of date and support for TRIM has been completed. The code has been rebased for 3.1 and bcache looks really promising. The lkml response hasn't been very warm so far (surprise ), but this thread is worth checking out anyway. Hopefully Michael will also test bcache and maybe give it some publicity in form of a news posting.

[dibal]

Powersaving

This kind of stuff would be nice for powersaving of home used NAS devices. But there must be some more smarter control of the power control for the rotating disks.

[madbiologist]

I would like to see tests of Flashcache compared to the other options mentioned above, and also compared to Windows ReadyBoost and/or ReadyDrive.

[Haven]

I've been running flashcache for a while now on both SSD and ZRAM devices. There's definitely a big improvement under the right situation. One of the biggest improvements I've seen so far has been running it against an ext4 LVM2 partition running a minecraft server:

0 4194304 flashcache stats:
reads(1553408), writes(11806595)
read hits(986945), read hit percent(63)
write hits(10820089) write hit percent(91)
replacement(524380) write replacement(963051)
write invalidates(0) read invalidates(1)
pending enqueues(0) pending inval(0)
no room(0)
disk reads(566463) disk writes(10820089) ssd reads(986945) ssd writes(12373057)
uncached reads(1) uncached writes(0), uncached IO requeue(0)
pid_adds(0) pid_dels(0) pid_drops(0) pid_expiry(0)

If you get high enough cache reads and writes then it makes a world of difference.

Would be nice to see some hard and fast numbers though.

I also replaced tmpfs with zramfs a while ago on and saw substantial improvements there as well.

[Otus]

Results for Flashcache by itself (or only in comparison to uncached hdd) wouldn't be terribly exciting, but if you add in bcache and/or a hybrid drive I'd like to see it.

[V!NCENT]

Every time I see the word "FlashCache" I can't help but remember that stripper in Montreal so many years ago who used to open up her raincoat to get her g-string stuffed with 20's.....

Have you, by any chance, heared about the GIMP?

[mcdebugger]

Of course it will be interesting!
Thanks a lot!

[deanjo]

Every time I see the word "FlashCache" I can't help but remember that stripper in Montreal so many years ago who used to open up her raincoat to get her g-string stuffed with 20's.....

[NoEffex]

Yes. I manage several MySQL database servers for a large forum (~360k members, thousands online at once, ~1.5m posts all within the past 9 months) and a service that is incredibly database intense. What we're doing right now is we have a read-only slave that is clocked super high and has a reasonable amount of cores (hex-core hyperthreaded I believe) for low-latency transactions and a read-write master with 32 physical cores. They both have SSDs.

To put it in perspective, APC (A php opcode caching module) reduced our PHP load on the PHP servers by about 3/4. That's just PHP. If this FlashCache kernel module has a reasonable impact I'm very very interested in compiling into the database servers because it would save quite a bit of money as we wouldn't have to get another 32-core database server and organize real database clustering so soon. The reason I can't just do it is because it's the database master that would probably benefit the most from it, and that can't be taken offline anytime but the middle of the morning and for a very short period of time.

If anybody hosting very large websites is interested in how my boss and I deal with attacks (*very* large HTTP floods, bandwidth floods), we use Varnish's inline C support to determine when a cllient is not a user by doing stuff like honeypotting to see if a certain IP makes an impossible amount of connections, whether a client loads a certain page for an infinite amount of time, whether a client switches useragents more than a reasonable amount of times, etc. and if it finds that a user is not a user it gives them an Error 418 (I'm a teapot lolololol but that can only be seen from an application) and if they continue to violate the rule they get blackhole'd. We have not had a successful HTTP flood since that was put in place.

[snadrus]

Tiered data

Spinning drives are so much slower than physical ram, I like the idea of having an intermediary (compressed ram (compcache?), SSD, etc) that I don't manage, but simply install and forget about. Then the best uses (tmp, startup, cache) occur without configuration. I could see room for recommending both since SSD is fast permanent storage while compcache may benefit large data processing.