Linux Device Mapper Adding An "Emulated Block Size" Target

[Zan Lynx]

I can see uses for it. Think you have imaged a disc that file system was installed expecting the drive to be 512 bytes and the drive you are now using is 4k. This would be a data recovery/transfer usage. Of course I would like to see this for SMR drives and it will need hell lot larger block size.

Virtual machines images are another likely thing to have been created 512 and are now on a 4kb drive asking to-do things that don't exactly fit any more.

Its not always just update the software its a lot more tricky when it update a complete disc image to be formatted different in a hurry just to be drive compatible..

A disk image is read by operating system functions which hide the difference in block sizes by going through the filesystem cache. A virtual machine or loopback mount has no idea of what the actual block size is. It issues a command to read block 27 and the operating system figures out where that is and what to read.

Direct IO and direct block device access is pretty much the only Linux IO that needs a block size.

I suppose that some sort of database server VM might rely on direct hardware access for the best performance.

But of course, whatever is doing this direct access for performance is going to give it all up as soon as this 512 to 4K block emulation goes into place. At that point they'd be doing just as well to use the regular filesystem.

I suppose I shouldn't be surprised that RedHat is doing weird compatibility hacks like this. Their customers are in love with things like Docker which mean they never have to update their software. I bet some companies have lost the build environments and can't update their containers.

Heck, I once had to rewrite a NodeJS C++ plugin because it had gotten so bad that they couldn't update past an ancient 0.5 release version and none of the existing code would build on new Node versions. Management only agreed as a sort of last resort. They'd probably still be happily running prelease beta versions of Node otherwise.

[oiaohm]

A disk image is read by operating system functions which hide the difference in block sizes by going through the filesystem cache. A virtual machine or loopback mount has no idea of what the actual block size is. It issues a command to read block 27 and the operating system figures out where that is and what to read.

That is not true in fact.

https://jc21.com/2015/01/format-4k-ext4-hard-drives-properly.html

There is a reason you need to format ext4 for 4K if you don't it can perform badly. Yes annoy right the way file system is formatted tells the OS file system driver this is either 512 or 4K. File systems drivers are not the brightest tools on the shelf they don't check what kind of hard drive this is instead expect the information in the way the partition was formatted..

Direct IO and direct block device access is pretty much the only Linux IO that needs a block size.

The file system driver filling page cache does Direct IO for items not in the page cache. Linux has a hell load of file system drivers.

But of course, whatever is doing this direct access for performance is going to give it all up as soon as this 512 to 4K block emulation goes into place. At that point they'd be doing just as well to use the regular filesystem.

It will all depend how it done for file systems if this device mapper results in you request a 512 the 4K is placed in page cache instead this will remove drive from having to emulating 512.

Stupid as it sounds there are a lot of cases that people are doing direct access for so call performance and it would have been faster to use the page cache. So depending on work load this will help some and hurt others. But using 512 on a 4K driver will generally hurt more as you increase processing in the controller on the harddrive. The harddrives are slow enough without the controller on the harddrive getting bogged down in excess processing yes this is way worse than pagecache overhead.

Yes the fun part of Linux is that pagecache is the block layer cache as well so it is perfectly legal for the block layer to populate the page cache.

[starshipeleven]

Red Hat is a funny company. They made a demo for us, to showcase their "Satellite" technology. I'll spare some details to protect collateral damage. Anyway, this wasn't some small demo. There were about 70 people present. They were telling us how we could manage updates to server farms. They first sold us on how simple and robust their solution was. Then came the demo. Nothing worked. They had create a virtual cluster of five servers, and told us they were going to send an update. Nothing worked. One failure after another. Catastrophic failures. Then they assured us that it the real world we would never encounter these problems. We had plenty of free beers and chicken wings, and I even got a free tshirt. But I left with TMI. Nothing good has ever, nor will ever, come out from Red Hat.

Look at all the problems the linux ecosystem has, everywhere it's broken beyond recognition, and you'll find Red Hat is behind it.

Even assuming this does not come straight out of your ass (and I'm being generous here), you are still taking the failure of a single local branch or team and assuming it's all like that, so you are still most likely trolling.

[starshipeleven]

I suppose I shouldn't be surprised that RedHat is doing weird compatibility hacks like this. Their customers are in love with things like Docker which mean they never have to update their software. I bet some companies have lost the build environments and can't update their containers.

The more likely one is they bought a software without the source code and the provider either disappeared or discontinued it. Server applications are B2B and more often than not they are NOT opensource or free, and are custom-made or customized for that specific company.

Happens all the time with Windows stuff, and is one of the backbones of companies like VMWare, you gotta keep running Windows server 2003 or something to keep using that ONE critical application the whole company depends on. Bigger companies have more VMs with more critical applications.

[starshipeleven]

unapproved post for Zan Lynx above this