GNU Coreutils 9.0: cp Now Enables CoW By Default + Copy Offload, wc Now Uses AVX2

[milkylainen]

So basically - for lack of a better term - preemptive dedup.

No. Not lacking a better term. It's Copy-on-Write.

[darkbasic]

"cp now enables copy-on-write (CoW) support by default..."

What means this? I mean, these gnus ain't stupid and don't go around gratuitously breaking things. So what happens e.g. when rsnapshot uses cp to backup files from one file system to another? And what does sync do?

It means --reflink=auto by default. Works very well.

[uid313]

Netcat is not a part of Coreutils, and iirc most of the major implementations of netcat have not been updated in 10 years or more since Netcat is relatively stable.

Thanks for clarification!
That's unfortunate that they haven't been updated for so long, especially considering how user-unfriendly the OpenBSD version is.
Netcat is pretty amazing, its like a Swiss army knife of networking, so it's quite unfortunate that it is not in a better state.

[F.Ultra]

Thanks for clarification!
That's unfortunate that they haven't been updated for so long, especially considering how user-unfriendly the OpenBSD version is.
Netcat is pretty amazing, its like a Swiss army knife of networking, so it's quite unfortunate that it is not in a better state.

there is also ncat, a fork by the nmap project. Available in Ubuntu.

[Adarion]

Is there any way to influence CoW?
I mean, I got that this is basically neat for many operations.
But: If I do make a copy of a file (on the same file system) on purpose, I actually WANT it to be a duplication. Because a) it might be a backup once I am going to modify things (so then in that moment it would suddenly start crunching on the HDD / or silent-crunch on flash ) and then do the ctual copy? And b) I want the space to be allocated. I mean, what if I have just little space left, but then am going to modify and it runs out of space? Or will CoW preallocate space and "symlink" the file and metadata there and just not do the very user data copy?
And will CoW stop on FS boundaries, speak: if I cp something from /dev/sda4 to /dev/sdc1 it will actually backup my data and not simply "symlink" metadata and if my media should fail (god forbid) I'm in the tar pit with just a kind of symlink "backup" to a broken media?

[mystiquewolf]

So, this is good for BTRFS for example?

Yes, it tries to do it and if the filesystem does it because it can, it's done. If the filesystem denied to do it because it can't do it, it's done the classic way.

[JanC]

Is there any way to influence CoW?
I mean, I got that this is basically neat for many operations.
But: If I do make a copy of a file (on the same file system) on purpose, I actually WANT it to be a duplication. Because a) it might be a backup once I am going to modify things (so then in that moment it would suddenly start crunching on the HDD / or silent-crunch on flash ) and then do the ctual copy? And b) I want the space to be allocated. I mean, what if I have just little space left, but then am going to modify and it runs out of space? Or will CoW preallocate space and "symlink" the file and metadata there and just not do the very user data copy?

You can use the --reflink and --sparse options to control how/what data is copied, space allocated, etc. See the manpage or info doc for more information.

[JanC]

That's unfortunate that they haven't been updated for so long, especially considering how user-unfriendly the OpenBSD version is.
Netcat is pretty amazing, its like a Swiss army knife of networking, so it's quite unfortunate that it is not in a better state.

There are many, more modern, alternatives to netcat, like socat, ncat, etc.

[jbean]

I know I'm late to the party on this one but I'll throw in my complaint to this change. I don't like the default to COW. I think it's going to break expected behavior. Users expect copies to be duplicated with cp, not snapshotted, which is effectively what a reflink is, a snapshot of a file. If they want this, they can specify the flag! It's like setting --sparse=always as default, it's asinine. This is expected behavior for some time now and so scripts and programs are accustomed to it.

This change then causes really weird behavior when you want to do cp --sparse=always

When cp defaults to reflink, your sparse copy request is straight up ignored. The copy is made "instantly", as one expects with reflink copies since it's just a metadata operation, and du will indicate the file size is the same (even though it should be sparse, that what we asked for) since that's how reflinks work... and of course, you get no change to your actual disk usage since what can't be sparse isn't actually duplicated. The sparse flag is just straight up ignored now. So in the current state now, if you really want a sparse copy, you need to do cp --sparse=always --reflink=never , even though in times past, if you scripted this or expected it to behave properly, it would have.

But then, what makes this worse is that some people just won't notice this. If you're using ext4, you can't use reflinks anyway, so it's like nothing changed. If you're using XFS, you *might* notice it, if it's not an old XFS filesystem created before it supported COW (or if you didn't disabled it at mkfs time).

I would prefer this change just straight up reverted, but at the very least, if the sparse flag is specified, it should default to never.