DragonFlyBSD 5.0 Released With Initial HAMMER2 Support, Support For 900k+ Processes

[Guest]

Then I guess Netflix, Facebook and whatsapp are stupid to use FreeBSD?

Facebook is mainly using Linux. When comes to big irons there's no BSD in SAP.

[Guest]

Serving 100Gbps from one server?
https://medium.com/netflix-techblog/...e-cdb51dda3b99

And you failed to show how it is advantage over Linux. BSD is a toy when comest to serious computing. Take a look at this:

https://www.numascale.com/world-record/

A Numascale shared memory system targeted for big data analytics reported a world record run of the Stream Benchmark. The system reached 10.06 TB/s for the Scale function, lifting the record with 53%.

https://www.computerworlduk.com/infr...linux-3244936/

The London Stock Exchange has said its new Linux-based system is delivering world record networking speed, with 126 microsecond trading times.

I'll tell you something in secret: netflix will switch to Linux in few years.

[Guest]

Serving 100Gbps from one server?
https://medium.com/netflix-techblog/...e-cdb51dda3b99

And no, Linux is faster:

https://community.mellanox.com/docs/DOC-2294

FreeBSD result from your link: 90Gbps.

Linux result: 98.9Gbps.

[alexcortes]

And no, Linux is faster:

https://community.mellanox.com/docs/DOC-2294

FreeBSD result from your link: 90Gbps.

Linux result: 98.9Gbps.

Your link is pointless. Unless I missed something it does not tell in what hardware the test was made.

So, it can be anything, ever a four socket machine.

For the record, the Netflix one was using a single-socket Xeon E5–2697v2, as pointed in the link.

[aht0]

Guest Idle question, if I may. Did you even notice, before storming in to "prove supremacy of the Linux" that this thread is talking about DragonFly, not FreeBSD?

It's pretty fucking funny, watching you panic and posting "proof" ASAP. And start attacking completely different OS. DragonFly is not FreeBSD. I doubt they have much common code left, besides license headers. Dillon rewrote pretty much the whole kernel single-handedly.

Anyway, your activity is complimentary, with a shitty OS, you would not see reason to bother.

[aht0]

Oh come the fuck on, isn't "as long as you don't need to do anything fancy or modern" caveat obvious enough? Their sound infrastructure is better than plain ALSA, but is frozen in 90's as far as features go, so you don't have 5.1 audio, bluetooth audio, and more advanced stuff.

Both 5.1 and 7.1 are usable in FreeBSD.
OpenBSD, same. It's sound is more functional though.
No clue about DragonFly's sound sub-system. Haven't used this that much yet.
General difference from Linux: Audio get's processed on kernel-level.

Off the top of my head (I don't know how much these apply to DragonflyBSD, I know more FreeBSD and derivatives):
-better ZFS support/integration
-kernel supposedly better for bigger iron systems (many many cores/CPUs)
-their firewall is better/easier to use
I don't use other features like jails or whatever, so I don't know.

DragonFly does not have ZFS. This release brought HAMMER 2 though
DragonFly should do okay up to 256 CPU's/cores (per machine). Can't tell how linear the scaling would be. No clue nor access to such multi-socket hardware.
DragonFly has OpenBSD's version of PF firewall.
jails are present in DragonFly

[gilboa]

Serving 100Gbps from one server?
https://medium.com/netflix-techblog/...e-cdb51dda3b99

Without getting into to many details (proprietary software), we have no issues shoving 160-200Gbps into a dual / quad Xeon v3/v4 socket machine running (Fedora) Linux. (Half duplex). Throwing out raw packets at 100-200Gbps is even easier.
When dealing with multiple 40Gbps NICs, the main limit is a (relatively) slow 8x PCI-E.

BTW, Intel's own DPDK library can also handle multiple 40 and 100Gbps NICs. (Never used it myself, but I believe they are capped at 360Gbps full-duplex).

- Gilboa

[gilboa]

Off the top of my head (I don't know how much these apply to DragonflyBSD, I know more FreeBSD and derivatives):
-permissive license (closed-source-product friendly)
-better ZFS support/integration
-kernel supposedly better for bigger iron systems (many many cores/CPUs)
-their firewall is better/easier to use
-better sound system, as long as you don't need to do anything fancy or modern.

I don't use other features like jails or whatever, so I don't know.

At least on 4 socket Xeon machines, Linux is definitely faster. Especially once the process count goes 5 figure.
Though I must admit that we spent far more time optimizing code for Linux compared to FBSD.

- Gilboa

[aht0]

At least on 4 socket Xeon machines, Linux is definitely faster. Especially once the process count goes 5 figure.
Though I must admit that we spent far more time optimizing code for Linux compared to FBSD.

- Gilboa

Sigh, DragonFly is Not FreeBSD. How fucking hard is to realize that. It's not even using same model of kernel that Linux and FreeBSD are.. Yeah. It was forked from FreeBSD, ages a go. By now it has hybrid kernel (as opposed to monolithic Linux and FreeBSD kernels) and not much left of it's FreeBSD origins, internally.

KERNEL


Please keep in mind that major modifications have been made to nearly the entire DragonFly kernel with regard to the original FreeBSD-4.8 fork. Significant changes have been made to every kernel subsystem, as a consequence this list is constrained to the largest, most user-visible changes unique to DragonFly.

• The scheduler abstraction has been split up into two layers. The LWKT (Light Weight Kernel Thread) scheduler is used by the kernel to schedule all executable entities. The User Thread Scheduler is a separate scheduler which selects one user thread at a time for each CPU and schedules it using the LWKT scheduler. Both scheduler abstractions are per-CPU but the user thread scheduler selects from a common list of runnable processes.
• The User Thread Scheduler further abstracts out user threads. A user process contains one or more LWP (Light Weight Process) entities. Each entity represents a user thread under that process. The old rfork(2) mechanism still exists but is no longer used. The threading library uses LWP-specific calls.
• The kernel memory allocator has two abstracted pieces. The basic kernel malloc is called kmalloc(9) and is based on an enhanced per-CPU slab allocator. This allocator is essentially lockless. There is also an object-oriented memory allocator in the kernel called objcache(9)which is designed for high volume object allocations and deallocations and is also essentially lockless.
• DEVFS - is the DragonFly device filesystem. It works similarly to device filesystems found on other modern UNIX-like operating systems. The biggest single feature is DEVFS's integration with block device serial numbers which allows a DragonFly system to reference disk drives by serial number instead of by their base device name. Thus drives can be trivially migrated between physical ports and driver changes (e.g., base device name changes) become transparent to the system.
• VKERNEL - DragonFly implements a virtual kernel feature for running DragonFly kernels in userland inside DragonFly kernels. This works similarly to Usermode Linux and allows DragonFly kernels to be debugged as a userland process. The primary use is to make kernel development easier.
• NFS V3 RPC Asynchronization - DragonFly sports a revamped NFSv3 implementation which gets rid of the nfsiod(8) threads and implements a fully asynchronous RPC mechanism using only two kernel threads. The new abstraction fixes numerous stalls in the I/O path related to misordered read-ahead requests.

EXTREME SCALING


DragonFly will autotune kernel resources and scaling metrics such as kernel hash-tables based on available memory. The autoscaling has reached a point where essentially all kernel components will scale to extreme levels.

• Process and thread components now scale to at least a million user processes or threads, given sufficient physical memory to support that much (around 128GB minimum for one million processes). The PID is currently limited to 6 digits, so discrete user processes are capped at one million, but the (process x thread) matrix can conceivably go much higher. Process creation, basic operation, and destruction have been tested to 900,000 discrete user processes.
• File data caching scales indefinitely, based on available memory. A very generous kern.maxvnodes default allows the kernel to scale up to tracking millions of files for caching purposes.
• IPI signaling between CPUs has been heavily optimized and will scale nicely up to the maximum hardware thread limit (256 cpu threads, typically in a 128-core/256-thread configuration). Unnecessary IPIs are optimized out, and the signaling of idle cpus can be further optimized via sysctl parameters.
• All major kernel resource components are fully SMP-aware and use SMP-friendly algorithms. This means that regular UNIX operations that manipulate PIDs, GIDs, SSIDs, process operations, VM page faults, memory allocation and freeing, pmap updates, VM page sharing, the name cache, most common file operations, process sleep and wakeup, and locks, are all heavily optimized and scale to systems with many cpu cores. In many cases, concurrent functions operate with no locking conflicts or contention.
• The network subsystem was rewritten pretty much from the ground-up to fully incorporate packet hashes into the entire stack, allowing connections and network interfaces to operate across available CPUs concurrently with little to no contention. Pipes and Sockets have also been heavily optimized for SMP operation. Given a machine with sufficient capability, hundreds of thousands of concurrent TCP sockets can operate efficiently and packet routing capabilities are very high.
• The disk subsystem, particularly AHCI (SATA) and NVMe, are very SMP friendly. NVMe, in particular, will configure enough hardware queues such that it can dispatch requests and handle responses on multiple cpus simultaneously with no contention.
• The scheduler uses per-cpu algorithms and scales across any number of cpus. In addition, the scheduler is topology-aware and gains hints from whatever IPC (Inter-Process Communications) occurs to organize active processes within the cpu topology in a way that makes maximum use of cache locality. Load is also taken into account, and can shift how cache locality is handled.
• The kernel memory manager is somewhat NUMA aware. Most per-cpu operations use NUMA-local memory allocations. User memory requests are also NUMA aware, at least for short-lived user programs. Generally speaking, the scheduler will try to keep a process on the same cpu socket but ultimately we've determined that load balancing is sometimes more important. CPU caches generally do a very good job of maximizing IPC (Instructions Per Clock). Because memory management is fully SMP-aware, a multi-core system can literally allocate and free memory at a rate in the multiple gigabytes/sec range.
• Generally very high concurrency with very low kernel overhead. The kernel can handle just about any load thrown at it and still be completely responsive to other incidental tasks. Systems can run efficiently at well over 100% load.
• Supports up to 4 swap devices for paging and up to 55TB (Terrabytes) of configured swapspace. Requires 1MB of physical ram per 1GB of configured swap. When multiple swap devices are present, I/O will be interleaved for maximum effectiveness. The paging system is extremely capable under virtually any load conditions, particularly when swap is assigned to NVMe storage. Concurrent page-in across available cpus, in particular, works extremely well. Asynchronous page-out. Extended filesystem data caching via the swapcache mechanism can operate as an extended (huge) disk cache if desired, and/or used to increase the apparent total system memory.

[gilboa]

Sigh, DragonFly is Not FreeBSD. How fucking hard is to realize that. It's not even using same model of kernel that Linux and FreeBSD are.. Yeah. It was forked from FreeBSD, ages a go. By now it has hybrid kernel (as opposed to monolithic Linux and FreeBSD kernels) and not much left of it's FreeBSD origins, internally.

I believe you are barking at the wrong tree.
As far as I understood the OP was taking about FreeBSD and not DF-BSD. So was I.

We only ported the software we are using to FBSD, so I cannot compare the performance to any other BSDs.

- Gilboa