The New NTFS Driver Looks Like It Will Finally Be Ready With Linux 5.15

[PeterB]

Any sign of an NTFS fsck/chkdsk for Linux?

[birdie]

Michael

You may want to check it: https://lkml.org/lkml/2021/8/3/1304

[calc]

Okay. I've ordered the parts for a new build. First in ten years so I might be a bit stale. I'll need to pick some filesystems. The new machine will be 5950x and initially have a 2TB SDD plus 8TB rotating rust for backup and storage. I use rsnapshot for backup. Fedora is my daily driver. I'm open to suggestions, but I'm thinking
/dev/sda:
1MB grub2
256MB UEFI fat16 or exFAT WHICH???
2GB boot
64GB ext4 Fedora
64GB ext4 CentOS9 when released
128GB linux swap
1.74TB <rest>

No need for a grub partition/area with UEFI/GPT as grub installs into the ESP which should be FAT32.

Unified Extensible Firmware Interface Specification - Page 504 - 13.3 File System Format

[Slartifartblast]

I hope you have a robust backup system

That goes for any filesystem, nothing is foolproof. EXT4 with journaling isn't going to protect you from burglars swiping your precious PC.

[oiaohm]

It's 1) not the default 2) it severely degrades performance 3) it increases fragmentation 4) it breaks some applications 5) It relies on your HW actually physically writing data out which is not guaranteed on consumer devices 6) I don't know anyone who uses it.

Birdie point 3 is in fact wrong. Ext4 using journal mode results in lower fragmentation to clean up in the filesystem. Writing the data to journal gives a delay before final placement so reducing fragmentation by the final placement being better. This lower fragmentation does not matter that much when you have Ext4 online de-fragmentation that can deal with cases of fragmentation that comes up with a lower IO cost. data=journal does decreases fragmentation of the ext4 but the IO cost todo it is quite excessive because the cost is on every write operation be it one that will fragment or not. So it works out cheaper in IO to write to the ext4 at times fragmented and let a background auto ext4 defrag fix the mess(yes this is higher fragmentation at times than the data=journal would have had).

Yes you magically don't get more IO operations.

data=journal was more of a question in the ext3 days when you did not have online defragmentation as part of ext4. Yes defraging ext3 when it got fragment was fairly much copy everything from partition 1 to partition 2. Ext3 had decent fragmentation resistance without journal mode but it was not impossible to fragment the crap out of it.

That was when using the ext3 file system driver. Please remember that ext3 file system driver is these days with most distributions is using the ext4 driver that has auto defrag. That change over from using the ext3 file system driver to the ext4 file system driver end most of the advantage of data=journal because of the effect of the auto defrag.

[dragon321]

I would assume a fair bit of work would need to be put into grub to support the different NTFS features.

You're right but you can workaround this by having separate boot partition with supported file system. I'm not sure how this driver will deal with Linux permissions. Not a big deal in exchanging data between Windows and Linux but pretty important thing in case of Linux installation.

[birdie]

The data=journal mode absolutely protects from data loss

It's 1) not default 2) it severely degrades performance 3) it increases fragmentation 4) it breaks some applications 5) It relies on your HW actually physically writing data out which is not guaranteed on consumer devices 6) I don't know anyone who uses it.

data=journal        All data are committed into the journal prior to being
            written into the main file system.  Enabling
            this mode will disable delayed allocation and
            O_DIRECT support.

You used a non-standard, rarely if ever used mode of journalling to counter my statement which renders your argument 100% invalid. Lastly I'm not even sure other journaled filesystems (e.g. XFS, JFS, Reiser) have this mode of operation.

[birdie]

exFAT is lousy for file archiving.

The problem is that ext(x) filesystems allow any character in a filename except for NULL and forward slash, which is a superset of the allowed characters in the exFAT definition:

https://docs.microsoft.com/en-us/windows/win32/fileio/exfat-specification#table-35-invalid-filename-characters

For example: Asterisk; Less-than sign; Greater-than sign; Colon; Question mark; Vertical bar; and Back slash are not allowed in exFAT filenames (plus sundry control codes)

I've been caught out by this and had to rename a non-trivial number of files.

Similar restrictions apply to NTFS: https://docs.microsoft.com/en-us/win.../naming-a-file - note that if NTFS uses the POSIX namespace any Unicode character except / and Null are is allowed. Note that the POSIX standard for portable filenames is considerably more restrictive.

Non-Microsoft drivers can put non-standard characters in filenames used in exFAT, but obviously this will generate undefined behaviour if accessed by a Microsoft driver, or one written to abide by the Microsoft specification.

I'm not going to hold up the relatively liberal approach of many Linux filesystems towards filename standards as being perfect. It is subject to well argued criticisms: https://dwheeler.com/essays/fixing-u...filenames.html

I'm still looking for a good cross-platform approach for file archiving. exFAT isn't the solution, and neither, unfortunately, is UDF (in any of its revisions), which is a shame.

I don't understand your point. Who or what makes you create files with "bad" characters in Linux? In my 30+ years of using PCs I've never had such troubles and never had the need to create files containing them. Lastly you can use sed, tr, awk to remove them if you like.

[timofonic]

Good news.

Will they do the same to their HFS+ implementation?

Maybe they will focus on APFS and release a ReFS implementation to get profit from it.

Anyway, they are expanding their market to forense computing.

[intelfx]

Sad to see the code GPLv2 with all the *BSD projects wanting to avoid any new GPL encumbered code.

Ha. They can wish for whatever they want, why would Linux ecosystem bow to it?

Say no to cuck licenses.