Announcement

Collapse
No announcement yet.

Paragon Sends Out Updated NTFS Driver They Want To Mainline For The Linux Kernel

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • drSeehas
    replied
    Originally posted by Old Grouch View Post
    UDF is also full of quirks between versions and platforms. It is a mess.

    https://en.wikipedia.org/wiki/Universal_Disk_Format

    Quoting the following (I recommend reading the whole thing) - https://github.com/JElchison/format-udf
    First of all, I made my post as an answer to the claim "NTFS was previously the only way to share >4 GB files with Windows".
    And this is not true.

    I am using UDF 1.50! formated Flash drives for many years now without any problems. I format them with Windows (/FS:UDF /R:1.50).
    I guess, the problems are mainly historic.

    Try it out yourself!

    Leave a comment:


  • Old Grouch
    replied

    Originally posted by drSeehas View Post
    ???
    What about UDF?
    Originally posted by starshipeleven View Post

    lack of fsck tools, funky handling if the partition is bigger than 1TB
    UDF is also full of quirks between versions and platforms. It is a mess.

    https://en.wikipedia.org/wiki/Universal_Disk_Format

    Quoting the following (I recommend reading the whole thing) - https://github.com/JElchison/format-udf

    Why?


    format-udf was created to address some OS-specific quirks that prevent a naively-formatted UDF device from working across various operating systems. Here are some of the complicating factors, which format-udf aims to abstract away:
    • Different operating systems support different versions of the UDF specification. Some OS versions only offer read-only support.
    • Windows seems to only mount UDF devices if the file system block size matches the device's logical block size
    • Different operating systems (like Windows XP) only attempt mounting UDF file systems with a hard-coded block size
    • Windows does not support hard disks without a partition table. (This is strange because Windows does not apply the same limitation to flash drives.)
    • macOS seems to only mount UDF file systems that utilize the full disk (not just a partition)

    At first glance, these constraints appear to be in partial conflict. The solution, as suggested by Pieter, is to place a fake partition table (via MBR) in the first block of the drive, which lists a single entire-disk partition. This works because UDF (perhaps intentionally) doesn't utilize the first block. Unfortunately, there has been no easy way to do this, while juggling all of the other variables (such as device logical block size). format-udf writes such a fake MBR for added compatibility on Windows. If this is not what you desire, you can disable the MBR with -p none.

    Leave a comment:


  • ALRBP
    replied
    Originally posted by mishakov View Post
    I have never seen a device which would support exFAT but can't at least read NTFS
    You have not used many Android phones with USB keys then.

    Leave a comment:


  • uid313
    replied
    Originally posted by bemerk View Post

    The github stuff is far from ready as it is a spare time reverse engineering effort.
    Paragon could speed that up a lot
    My impression is that the stuff on GitHub have accomplished read-only support for APFS. But maybe it was just read-only without decompression of transparent compression. I don't know if Paragon is involved in APFS, I just know they do NTFS.

    Leave a comment:


  • bemerk
    replied
    Originally posted by uid313 View Post

    By the way, is anyone working to mainline support for Apple's APFS in the Linux kernel?
    I know there was some code on GitHub for APFS support in the Linux kernel, as well as a FUSE driver.
    The github stuff is far from ready as it is a spare time reverse engineering effort.
    Paragon could speed that up a lot

    Leave a comment:


  • starshipeleven
    replied
    Originally posted by drSeehas View Post
    ???
    What about UDF?
    lack of fsck tools, funky handling if the partition is bigger than 1TB

    Leave a comment:


  • mishakov
    replied
    Originally posted by ALRBP View Post

    exFAT is supported by a lot of Android devices (maybe all after mainlining) while NTFS is not. This is why I stopped using NTFS for my "multi-system" USB key, in favor of exFAT. That said, I still have a FAT32 "all terrain" key ; FAT32 is still the only thing you are sure to be working everywhere and will probably remain for long, until all non-exFAT supporting devices are gone (notably, Linux-based pre-exFAT mainlining systems, since a lot of them do not include FUSE-based exFAT support). NTFS is too complicated and will never be an "all terrain" FS, unlike exFAT. While NTFS was previously the only way to share >4 GB files with Windows, and thus adopted by some old TVs, it has been obsolesced by exFAT for this application.

    I personally care even less about NTFS mainlining than exFAT mainlining. The main advantage of exFAT mainlining, with MS removing the patent threat, is that now, Linux based (notably Android) devices are more likely to include exFAT support, without fearing a trial from MS if they do not pay a license. For NTFS, well, all modern systems supporting it will also support exFAT, some systems will likely only support exFAT (Linux post-exFAT but pre-NTFS or just devices wanting to support no more than SD specification, with mass storage devices formatted in exFAT by default) and NTFS-3G is sufficient for Linux desktop anyway (especially if you just want to share with your Stone Age smart TV that you should anyway pair with a modern (far cheaper than a new TV) Android TV box if you want to read modern codecs, like H265 or AV1).
    I had actually found exFAT (or any FAT*) filesystem(s) to be terrible for file sharing between Linux and Windows due to the lack of journaling, case sensitivity, symlinks, compression and ACLs (in Windows) and was much less excited about exFAT mainlining than about NTFS now. You would never know what would break when using it on Linux (because of the case-insensitivity), and after recently giving it a try, I had reformated my external drives back to NTFS (and BTRFS/XFS on my Linux NAS) after one of them had exFAT go irreparably corrupt (something I've never seen on journaled filesystems, but have (unsurprisingly) witnessed many times on FAT*). I have never seen a device which would support exFAT but can't at least read NTFS, but a lot that only supported the latter and I'm not expecting new devices to give up supporting NTFS due to its wide adoption. And with these patches, it will now perform as good as other Linux filesystems and fully support journaling, IMO possibly making it a perfect choice for the drives to mostly be used on Linux systems, but which could also be easily accessed on Windows (without fiddling with drivers) and read by pretty much any other OS/device. And although ntfs-3g might be sufficient for some, it is much slower, especially on low-end linux machines (e.g. Raspberry Pis) and isn't very reliable due to it not supporting journalling.

    Leave a comment:


  • sinetek
    replied
    Paragon wrote they would release the fsck and mkfs utility at a later point when they've been over it more thoroughly for a gpl release.
    I want to see this merged in, I think Paragon is doing a great thing here and the code's good.

    Leave a comment:


  • drSeehas
    replied
    Originally posted by ALRBP View Post
    ... NTFS was previously the only way to share >4 GB files with Windows, ...
    ???
    What about UDF?

    Leave a comment:


  • starshipeleven
    replied
    Originally posted by mifritscher View Post
    Regarding partition tools: I would love a "real" successor to Parititionmagic. parted had removed almost all filesystem code, so most operations are very basic. Not even efficient move operations, but also no conversions, merge/split of partitions with data etc. Partition Magic allowed almost everything one would dream to do with partitions. And it was very reliable.

    A starter would be a collections of libraries (one per filesystem) which can
    * check the file system
    * do a full file+metadata <-> sector mapping (+ reverse operation: get free sector regions)
    * move files to given sector regions
    * add files by name and sector list
    * needed offset+alignment for new files
    * remove files
    * change start + end of a file system by only moving metadata (+ a "dry run" which get the statically needed sector regions for metadata if any (super block, FAT etc.)
    * create filesystem, with a dry run to get the statically needed sector regions
    * align metadata to a new physical/logical/cluster sector size + change sector size (data must be moved beforehand) (physical: only optimization, logical: the new size is strict, cluster: can be greater than a sector (e.g. for fat))
    * change properties from a filesystem, directory and file (at least at the first version: no abstraction needed)

    All write operation should support a list of "forbidden" sector regions to e.g. ease conversions or merge operations to avoid overwriting data which aren't bound to the filesystem yet (ie: not added to its file structure). For filesystem creation: static metadata could be temporary redirected to other sectors to ease conversions - with a function to move them to the final destination.

    This would allow the most needed extended operations, leaving as much as possible work in the "higher level" (e.g. allocation/move strategies).

    Version 2 could support "versioned files" - ie. snapshots. Things like datasets (from zfs) could be seen as an additional directory layer at the API side (at least at the first version), zvols could be handled as files. Datasets could be an additional hierarchy type between the whole filesystem and directory structures as well or later.

    Version 3 could be handling of filesystems which have more than one continuous sector regions (jbod, raidx etc.).

    Version 4 would an extension to the driver so some operations could be done online ;-) For this, another function would be useful:
    * reserve a sector region - permanent (if possible) or temporal (until umount)
    sounds like a good weekend project for the next few decades

    Leave a comment:

Working...
X