FreeBSD Working On A New Installer, Updates To Their Linux Compatibility Layer

[dragon321]

Of course not what?
The fact that they can doesn't mean that they should.

To be clear, I'm just saying that the ability to run binaries from other OSes is always good

Of course not that having such features is bad for OS. I don't consider having compatibility with another OS as a bad thing.

[k1e0x]

Thanks for the response. To be clear FreeBSD as you have rightfully stated has definite technical advantages/improvements over Linux but from what I have noticed using TrueNAS on a surface level there are also a lot of usability and even some other stability issues, I will list them here

1. FreeBSD is still using standard init system, which means that booting is really slow when you have large disks because its done in serial rather than parallel. Ontop of this, the web ui for TrueNAS doesn't boot until your Zpools are mounted (talking about the non-root Zpool here), even though technically speaking nothing is stopping the UI from starting if the pools are still waiting to mount
2. I have had the network stack crash a couple of times on of my NAS due to some IPFW error
3. I had a case where one of my zpool's became corrupted. Thankfully i managed to fix it, but I had to hack around it by disabling truenas because truenas would automount the zpool even when I didn't want it to because mounting the corrupted ZFS pool caused a kernel panic (and in reality I wanted to recover the partition using -T/-F flags).
4. When using bhyve for archlinux based virtual machines, the virtual machines would crash. This wouldn't be a big problem if it was just the VM crashing, but it also caused problems on the NAS which forced me to restart (I am now using Ubuntu which for some reason seems to be more stable). in both cases I am using VirtIO for NIC/disk.
5. FreeBSD is also pickier about hardware which although may not be an issue in datacenter for commodity/homelab scenarios its more problematic.

So I have faith in TrueNAS, there is a reason why I picked (as you said its the most stable ZFS environment so far) but the fact that IXsystems is building TrueNAS scale already hints that they see a future in Linux and its only a matter of time until ZFS on Linux is also stable.

TrueNAS is something sort of totally different.. I've evaluated it for enterprise use and found a lot of the features of it non-functional or lacking so I didn't really see a benefit of using it in production. (for small nas storage systems) over FreeBSD Vanilla. I think it's a good system for home users who just want something to work but a lot of the enterprise features are lacking so.. whats the point vs vanilla?

1. FreeBSD uses RC, it is it's own implementation and not a sysvinit descendant that caused so much pain in Linux. TrueNAS uses OpenRC.. OpenRC (ovis RC inspired) has a lot of advanced features like systemd and I would say it's "modern" where are RC is traditional. -- Honestly.. nobody cares how long a server takes to boot in a work environment. Some of these systems take 2-3-5 minutes to post. Laptops yes, Servers no.-- That being said, RC has some advantages in it's simplicity over an advanced init. It's much easier to adjust or do custom stuff with. See below about FreeBSD's Response to systemd. (99% of FreeBSD's startup configuration is done in one file rc.conf, this is kind of nice, least when small, I've seen some over 200 lines.. ouch)

2. Odd yeah not sure. It def shouldn't be crashing. This might be due to the fact TrueNAS follows FreeBSD head (unstable) and not release (LTS). If you want a nice home nas with a web frontend and you are not willing to go full Vanilla FreeBSD, you might try XigmaNAS.

3. Oh no! I've had ZFS become corrupt on desktop virtual machines before that cache it in a way that breaks ZFS, for performance.. gotta go fast. (TURN WRITE CACHES OFF OMG!) I've never seen a on-metal pool at work become corrupt though. And I've run pools as large as 48 spindles ever since it was released in Solaris 10. what.. 20 years ago now? Take it from me, ZFS is extremely reliable as opposed to most of the other stuff out there.. Netapp, Dell EMC, DDN, etc. For enterprise on the OS level nothing in Linux is acceptable. btrfs? No, nobody trusts it. Stratus? No, it's vaporware atm. ZFS? no not really, Ubuntu is closest but the server LTS does not have it on root yet. So you end up with ext4 + LVM and compared to ZFS that is primitive.. every time I use LVM to extend a disk I feel like I'm using Dos, mucking about with loopback devices and partitions and such. Yuck. In ZFS you expand it by "zpool online -e pool" done.. no need to think about the lower mechanics no need to remount anything, more space just magically appears.. like putting in a larger ram stick.


4. bhyve is pretty good for what it is but it's a lot different than KVM. They serve different roles really. bhyve is a simple small hypervisor with no emulation at all. Where as KVM emulates everything under the sun with qemu. It's good to have both out there and I've used bhyve for projects and found it to be fairly good but simple. Ordinarily though if I want a VM usually I'd use either VMWare or KVM.. If I want a "VM" on FreeBSD usually I use a Jail. Take a look at the Bastille jail manager. https://bastillebsd.org/

5. Yes, very true. Most Intel servers you get you won't have to worry about and FreeBSD will always focus on the server market and that is where they shine but for commodity home hardware yeah things can be problematic. You can see this in what they focus on. They have an extremely advanced network stack but their wifi is 802.11n at best.. why? Servers don't need wifi so they don't focus on it. One thing they did do with their wifi though is they built it into ifconfig.. because on Unix ifconfig is the tool to manage network interfaces so they improved it to manage wifi network interfaces, where as in Linux, they didn't.. you use some other 3rd party wifi network manager (like NetworkManager etc, the tool they use changes almost by-weekly tho distro to disto). And actually they even replaced ifconfig with ip.. dur.. That would never happen in FreeBSD, if it was broken or needed improvements they would fix ifconfig. Being one team with one source base they can do things like that.

So systemd, what is FreeBSD doing about systemd? You can trust that FreeBSD won't be adopting systemd or anything monolithic like it but they are working on providing some of the features systemd does in a different way.

For an init system there has been some talk about adopting OpenRC. It has the right license and originally was created on NetBSD. It already runs on it, as shown by TrueNAS and GhostBSD and provides service watchdog and parallel start functions and seems to be a very fast init system that is -just an init- and nothing more. But systemd does a lot more than init and is a full system layer so what about those things?

For a messaging system like dbus they want to create one that is kernel based. This has advantages in security over dbus because if it's in the kernel you can trust the process sending the message is what it says it is. So you're going to see a lot of the other things systemd does on Linux being implemented in the FreeBSD kernel level they will be small, they will do one thing and one thing well and they will be glued into the system and it's other features (Jails, PF, ZFS, VNET etc) like everything else in FreeBSD. They will work with the other things and be very Unix'ey you can trust.

You won't see features in systemd like homed, oomd, (FreeBSD's kernel OOM already works, unlike Linux's) or dns or logging or some of the other cruft being implemented. No officed here.

Both FreeBSD and Linux are good things and competition makes everyone better. I also mentioned Illumos before.. Take a look what OmniOS can do. https://omnios.org/ - ZFS, KVM AND/OR bhyve, Zones including Linux compatible containers, PF, Crossbow and all kind of other advanced stuff. It really is the best of breed.. at least for enterprise.

[mdedetrich]

Hi mdedetrich,

Can you expand on this some? What do you mean by server management? I ask because I am a system administrator and I work on many Linux and Unix systems all day. I tend to find FreeBSD to be nicer and easier to work with than Linux.

Far as I know TrueNAS uses OpenRC that does have the parallelized start functions similar systemd.. so it's probably a design choice by TrueNAS to wait for the pool to mount first and not a limitation of the system.

ZFS also isn't FreeBSD's only feather in it's cap vs Linux. Right now today if you want ZFS in enterprise you kind of have to use FreeBSD because it's been in it a long time and is very stable and built in. In Linux LTS versions it's still a add on or bolt on component. (an example of this is boot environments just work in FreeBSD) - Right now in enterprise today, Linux really has nothing in the way of advanced storage management.

That being said FreeBSD has a lot of other things about it that are very nice and not available on Linux.

DTRACE is better than anything on Linux making debugging performance issues easier on FreeBSD. (performance issues are the worst kind)
Jails and containers are better than Docker, LXC, whatever. It's baked in better. (top, pkg, pf, zfs all work with jails)
Kqueue is superior at event notification than Epoll, this has performance benefits.
Capsicum is superior to SELinux. Who doesn't just set permissive? Am I right? .. well you don't have to.
Blacklistd is far better than fail2ban. Blacklistd does not parse logs to block.
It has a greater degree of tuning in sysctl functions. In Linux many of these options are compile time flags.

PF is better than any firewall implementation on Linux. It has simpler rule structure and it does not need scripts and front ends to use. In Linux the firewall is controlled exclusively by the kernel and has to have 3rd party tools to control.. and there are many. In FreeBSD you have userland software tools to help control and manage it. It's all built in. (pfctl)

Software management on it is easier. The base OS is separated from the packages, this is really nice because you know every single install of the same version of the base OS is 100% identical and no packages need to be changed to make them identical.. nor do you need to update any packages to update the OS.

It also has two software management systems, pkg and ports, giving you source build like Gentoo OR binary (or both!) It backports old software versions so you are less often in a situation where you can't update the version of it because it's running old software .. and if you really really need to update it "and can't" because of an old app.. you are not stuck on FreeBSD 8. You can tar it up and run FreeBSD 8 software stack and apps in a jail on a FreeBSD 13's kernel. RedHat has super PIA problems with updates usually requiring a full redeployment of the server between major versions. I've been bit by this in RedHat so many times. (the latest moving from CentOS 7 to 8 then back to 7, .....yeah) In FreeBSD you don't need to redeploy.. you can update the base OS without fear, tar up /usr/local, create an old version jail and extract the tarball into it.

It's one single project from one single vendor. It causes them to build solutions that last and not changing the way the OS works every few years. Different companies like RedHat etc are not competing over what technology gets adopted in it that they can up sell to enterprise. It allows them to make sane decisions and chose to do what is best for the OS itself.

And above all it's a far more simple system. Simple gives you flexibility and options.. Maybe you should take a deeper look at it but I am curious about your opinions here.

(and actually.. I think Illumos is really the Apex predator of the *Nix world.. the things that OS can do are actually next level.. but it does not have a lot of acceptance.. yet..)

Thanks for the response. To be clear FreeBSD as you have rightfully stated has definite technical advantages/improvements over Linux but from what I have noticed using TrueNAS on a surface level there are also a lot of usability and even some other stability issues, I will list them here

1. FreeBSD is still using standard init system, which means that booting is really slow when you have large disks because its done in serial rather than parallel. Ontop of this, the web ui for TrueNAS doesn't boot until your Zpools are mounted (talking about the non-root Zpool here), even though technically speaking nothing is stopping the UI from starting if the pools are still waiting to mount
2. I have had the network stack crash a couple of times on of my NAS due to some IPFW error
3. I had a case where one of my zpool's became corrupted. Thankfully i managed to fix it, but I had to hack around it by disabling truenas because truenas would automount the zpool even when I didn't want it to because mounting the corrupted ZFS pool caused a kernel panic (and in reality I wanted to recover the partition using -T/-F flags).
4. When using bhyve for archlinux based virtual machines, the virtual machines would crash. This wouldn't be a big problem if it was just the VM crashing, but it also caused problems on the NAS which forced me to restart (I am now using Ubuntu which for some reason seems to be more stable). in both cases I am using VirtIO for NIC/disk.
5. FreeBSD is also pickier about hardware which although may not be an issue in datacenter for commodity/homelab scenarios its more problematic.

So I have faith in TrueNAS, there is a reason why I picked (as you said its the most stable ZFS environment so far) but the fact that IXsystems is building TrueNAS scale already hints that they see a future in Linux and its only a matter of time until ZFS on Linux is also stable.

[JackLilhammers]

Of course not. Both platforms can live without it.

Of course not what?
The fact that they can doesn't mean that they should.

To be clear, I'm just saying that the ability to run binaries from other OSes is always good

[dragon321]

I agree, but that doesn't mean having those solutions is bad for a platform. Quite the contrary

Of course not. Both platforms can live without it.

[k1e0x]

I have also tried FreeBSD recently (mainly due to ZFS with TrueNAS). There are some things that I like about it, but the lack of something like systemd is really showing when it comes to server management. At least with TrueNAS, it automounts your zfs even if its not your boot partition, this basically means when you start TrueNAS then your webserver doesn't get started until your main pool gets mounted (which if you have a big pool can take minutes). Something like systemd would have easily parallelized this.

On another note, FreeBSD really needs to find a niche where it can compete against Linux. ZFS isn't gonna cut it anymore because tbh distros get around the licensing issues without any problems, and the main technical advantage of packaging both userland and kernel at once (so the tools are completely consistent) isn't massive perk that will bring people over. In the old days FreeBSD had a far superior code quality (especially their network stack) but most of this has been already copied over so its not so much of an advantage anymore.

Hi mdedetrich,

Can you expand on this some? What do you mean by server management? I ask because I am a system administrator and I work on many Linux and Unix systems all day. I tend to find FreeBSD to be nicer and easier to work with than Linux.

Far as I know TrueNAS uses OpenRC that does have the parallelized start functions similar systemd.. so it's probably a design choice by TrueNAS to wait for the pool to mount first and not a limitation of the system.

ZFS also isn't FreeBSD's only feather in it's cap vs Linux. Right now today if you want ZFS in enterprise you kind of have to use FreeBSD because it's been in it a long time and is very stable and built in. In Linux LTS versions it's still a add on or bolt on component. (an example of this is boot environments just work in FreeBSD) - Right now in enterprise today, Linux really has nothing in the way of advanced storage management.

That being said FreeBSD has a lot of other things about it that are very nice and not available on Linux.

DTRACE is better than anything on Linux making debugging performance issues easier on FreeBSD. (performance issues are the worst kind)
Jails and containers are better than Docker, LXC, whatever. It's baked in better. (top, pkg, pf, zfs all work with jails)
Kqueue is superior at event notification than Epoll, this has performance benefits.
Capsicum is superior to SELinux. Who doesn't just set permissive? Am I right? .. well you don't have to.
Blacklistd is far better than fail2ban. Blacklistd does not parse logs to block.
It has a greater degree of tuning in sysctl functions. In Linux many of these options are compile time flags.

PF is better than any firewall implementation on Linux. It has simpler rule structure and it does not need scripts and front ends to use. In Linux the firewall is controlled exclusively by the kernel and has to have 3rd party tools to control.. and there are many. In FreeBSD you have userland software tools to help control and manage it. It's all built in. (pfctl)

Software management on it is easier. The base OS is separated from the packages, this is really nice because you know every single install of the same version of the base OS is 100% identical and no packages need to be changed to make them identical.. nor do you need to update any packages to update the OS.

It also has two software management systems, pkg and ports, giving you source build like Gentoo OR binary (or both!) It backports old software versions so you are less often in a situation where you can't update the version of it because it's running old software .. and if you really really need to update it "and can't" because of an old app.. you are not stuck on FreeBSD 8. You can tar it up and run FreeBSD 8 software stack and apps in a jail on a FreeBSD 13's kernel. RedHat has super PIA problems with updates usually requiring a full redeployment of the server between major versions. I've been bit by this in RedHat so many times. (the latest moving from CentOS 7 to 8 then back to 7, .....yeah) In FreeBSD you don't need to redeploy.. you can update the base OS without fear, tar up /usr/local, create an old version jail and extract the tarball into it.

It's one single project from one single vendor. It causes them to build solutions that last and not changing the way the OS works every few years. Different companies like RedHat etc are not competing over what technology gets adopted in it that they can up sell to enterprise. It allows them to make sane decisions and chose to do what is best for the OS itself.

And above all it's a far more simple system. Simple gives you flexibility and options.. Maybe you should take a deeper look at it but I am curious about your opinions here.

(and actually.. I think Illumos is really the Apex predator of the *Nix world.. the things that OS can do are actually next level.. but it does not have a lot of acceptance.. yet..)

[aht0]

it's not case in point, it's the only example freebsd has because everyone else uses linux(and netflix is not the top streamer btw). netflix uses freebsd only because it was started by freebsd guy, there are no other reasons

They literally stated in public paper that they chose freebsd because it offered superior performance considering their use case.
Do I need to take your hand and point you to the article and quote it or you perhaps still have ability to use Google and read something not fitting your prejudice?

[aht0]

you can't fix it that way, there are substantial differences. linux and freebsd are unix systems, windows is separate world with most programs requiring complete rewrite. windows(ms) is abusing its market dominant position(to the point of being fined for billions), linux and freebsd are in the same boat.
if freebsd was good, linux users would just switch to it like they switch distros. but freebsd is not good
i do, i.e. i sometimes deal with it

Linux ain't no unix system. It's wannabe-minix-clone at best with recent aspirations to become clone of windows.

If FreeBSD wasn't good, it'd be dead already. Linux's reasons of success ain't technical superiority really. There's also lots of luck involved
a) court case in early 90's against BSD giving Linux advance start by few years, b) FreeBSD fucking up it's SMT implementation in FreeBSD 5, c) market inertia.

Torvalds has said that if there was free BSD version back in the time, he wouldn't bothered with starting on Linux project.

Now, unless Linux fucks up somewhere royally (example: major security issues abusing some designed-wrong-and-hard-to-revert-nuance, hitting like sledgehammer blows compromising most Linux systems), it will retain it's secure market position after windows server and has nothing really to fear even with coming more convoluted, complicated, bloated...yadayada by every day.

[aht0]

FreeBSD had one of the most toxic community I've ever seen with insane dislike of Wayland and absolutely broken ports with embarassing dependencies.

I liked what DFLYBSD has done with their kernel threads and HAMMER2, but they share their ports with FreeBSD + GCC (I don't want to compile my own packages).

Both OS'es have binary packages - you dont need to fuck with compiling at all. Unless you want it. Or need specific feature flag in some port package not enabled/disabled by default.
When you do fuck with port options, it becomes little like Slackware - all bets are off, it's really easy to screw things up but it also gives you lots of control. I've used synth to let it sort out customized port options and their inherent new dependencies automatically. Pain goes away.
Synth is also present in DragonFly, it's actually written for it, originally, later Dillon re-wrote it from Ada to C for DFly..

[pal666]

How so? Plenty of FOSS appliances are built on FreeBSD. PFSense/OpnSense, FreeNAS/TrueNAS, m0n0wall. And there's the big commercial projects based on FreeBSD as well, BlueCoat, CheckPoint, NetScaler, Dell Compellent, Cisco Ironport, Juniper JunOS, NetApp Filer, QNAP appliances, Sony PS3/4, and many more.

that's not plenty, that's almost zero. "commercial projects" select it based on license, which while valid reason for a company, is completely invalid for users

The intelligent engineer selects the best tool for the job

lol. freebsd is selected either when engineer doesn't know linux, or when management is afraid of gpl.

. For a *nix workstation/desktop, Linux wins. Virtualization hypervisor, Linux wins. But for network and storage focused appliance roles, FreeBSD has substantial advantages.

both network and storage are fairytales, linux is superior in both.

Case in point, Netflix uses FreeBSD to serve the content you watch, because they were able to serve far more video throughput on FreeBSD than on Linux. Netflix did some substantial NUMA optimization to get even more throughput. Other companies like NetApp are doing the same on the storage side. IMO FreeBSD has a much more performance optimized LAN and Storage stack than Linux does at this point.

it's not case in point, it's the only example freebsd has because everyone else uses linux(and netflix is not the top streamer btw). netflix uses freebsd only because it was started by freebsd guy, there are no other reasons

FWIW I'm an RHCE and have made a career of Linux since the 1990's, but I run FreeBSD for my home storage server, and on my home firewall (pfsense). For any given problem, there is an ideal solution - and it isn't always Linux.

fwiw i met freebsd in 1995 before meeting linux and i still work with it, but i would never make decision to put freebsd on my server because it makes no sense