Paragon Sends Out Updated NTFS Driver They Want To Mainline For The Linux Kernel

[starshipeleven]

Ok, but maybe those filesystem specific tools like ntfstools could use this driver in the future when it moves data around the drive like when you resize a NTFS partition and it says that this operation could take a long time because it relocates data.

Afaik for most filesystems the tools are working at a lower level and moving blocks manually and then adjusting the index tables to reflect the new locations, while the partition is unmounted and the driver isn't active.

Afaik the only filesystems where the filesystem driver is involved in these operations are the next-gen filesystems like ZFS and Btrfs, where you can issue requests to relocate blocks to the driver and it will actually do the leg work while the filesystem is still online and usable by other applications/users. (used by balance, scrub, remove device, and shrink partition size)

[ALRBP]

Most existing smart TVs support fat and ntfs only. exFat is still relatively new and its support leaves a lot to be desired.

exFAT is supported by a lot of Android devices (maybe all after mainlining) while NTFS is not. This is why I stopped using NTFS for my "multi-system" USB key, in favor of exFAT. That said, I still have a FAT32 "all terrain" key ; FAT32 is still the only thing you are sure to be working everywhere and will probably remain for long, until all non-exFAT supporting devices are gone (notably, Linux-based pre-exFAT mainlining systems, since a lot of them do not include FUSE-based exFAT support). NTFS is too complicated and will never be an "all terrain" FS, unlike exFAT. While NTFS was previously the only way to share >4 GB files with Windows, and thus adopted by some old TVs, it has been obsolesced by exFAT for this application.

I personally care even less about NTFS mainlining than exFAT mainlining. The main advantage of exFAT mainlining, with MS removing the patent threat, is that now, Linux based (notably Android) devices are more likely to include exFAT support, without fearing a trial from MS if they do not pay a license. For NTFS, well, all modern systems supporting it will also support exFAT, some systems will likely only support exFAT (Linux post-exFAT but pre-NTFS or just devices wanting to support no more than SD specification, with mass storage devices formatted in exFAT by default) and NTFS-3G is sufficient for Linux desktop anyway (especially if you just want to share with your Stone Age smart TV that you should anyway pair with a modern (far cheaper than a new TV) Android TV box if you want to read modern codecs, like H265 or AV1).

[mifritscher]

Regarding partition tools: I would love a "real" successor to Parititionmagic. parted had removed almost all filesystem code, so most operations are very basic. Not even efficient move operations, but also no conversions, merge/split of partitions with data etc. Partition Magic allowed almost everything one would dream to do with partitions. And it was very reliable.

A starter would be a collections of libraries (one per filesystem) which can
* check the file system
* do a full file+metadata <-> sector mapping (+ reverse operation: get free sector regions)
* move files to given sector regions
* add files by name and sector list
* needed offset+alignment for new files
* remove files
* change start + end of a file system by only moving metadata (+ a "dry run" which get the statically needed sector regions for metadata if any (super block, FAT etc.)
* create filesystem, with a dry run to get the statically needed sector regions
* align metadata to a new physical/logical/cluster sector size + change sector size (data must be moved beforehand) (physical: only optimization, logical: the new size is strict, cluster: can be greater than a sector (e.g. for fat))
* change properties from a filesystem, directory and file (at least at the first version: no abstraction needed)

All write operation should support a list of "forbidden" sector regions to e.g. ease conversions or merge operations to avoid overwriting data which aren't bound to the filesystem yet (ie: not added to its file structure). For filesystem creation: static metadata could be temporary redirected to other sectors to ease conversions - with a function to move them to the final destination.

This would allow the most needed extended operations, leaving as much as possible work in the "higher level" (e.g. allocation/move strategies).

Version 2 could support "versioned files" - ie. snapshots. Things like datasets (from zfs) could be seen as an additional directory layer at the API side (at least at the first version), zvols could be handled as files. Datasets could be an additional hierarchy type between the whole filesystem and directory structures as well or later.

Version 3 could be handling of filesystems which have more than one continuous sector regions (jbod, raidx etc.).

Version 4 would an extension to the driver so some operations could be done online ;-) For this, another function would be useful:
* reserve a sector region - permanent (if possible) or temporal (until umount)

[Danny3]

this is a filesystem driver, so no it won't. Gparted is just calling filesystem-specific tool applications to do the actual job. For NTFS it's ntfstools from the ntfs FUSE project.

Afaik they plan to opensource tools as well, but it's not part of the current code.

Ok, but maybe those filesystem specific tools like ntfstools could use this driver in the future when it moves data around the drive like when you resize a NTFS partition and it says that this operation could take a long time because it relocates data.
If they release some open source tools as well that would be great also, but if not, I appreciate anyway what they are doing.

[starshipeleven]

I would expect exFAT to be supported by most somewhat recent TVs, as it's becoming the de-facto standard for large external hard-drives and SD cards.

It IS the standard for SD cards bigger than 32GB. It's in the acutal specification

[starshipeleven]

I'm now curios if Gparted could also benefit from this when creating NTFS partitions or resizing them which sometimes have to move some data around.

this is a filesystem driver, so no it won't. Gparted is just calling filesystem-specific tool applications to do the actual job. For NTFS it's ntfstools from the ntfs FUSE project.

Afaik they plan to opensource tools as well, but it's not part of the current code.

[starshipeleven]

Does the code drop include utilities like CHKNTFS?
If not can we expect them as well?
An utility to check the integrity of the filesystem and fix errors is what i miss most for NTFS.

the developer said that they planned to opensource mkfs and hopefully also fsck utility.

[dragon321]

ExFAT should also be a good option for this (and for external device support).

In some (or even most) cases yes but lack of journaling can be quite problematic sometimes. When NTFS support will land in kernel, you will get journaled file system that is supported natively by both operating systems. It would be nice choice for storage.

[LinAGKar]

Most existing smart TVs support fat and ntfs only. exFat is still relatively new and its support leaves a lot to be desired.

I would expect exFAT to be supported by most somewhat recent TVs, as it's becoming the de-facto standard for large external hard-drives and SD cards.

[uid313]

I almost never need to access any NTFS disk partition, and when I do the FUSE driver works just fine, so I don't really need this.
It feels like this is a bit late, I wish this was available over a decade ago.

By the way, is anyone working to mainline support for Apple's APFS in the Linux kernel?
I know there was some code on GitHub for APFS support in the Linux kernel, as well as a FUSE driver.