KLANG: A New Linux Audio System For The Kernel

[ninez]

anecdotal evidence, but OSSv4 once served me well. I had a low end C-media discrete board with terrible driver in ubuntu 10.04 and maybe 11.04. cracks, bursts of garbage into the speakers. OSS did support it much better (ALSA did too with a later kernel in a later distro)
the card itself was flawed maybe, decent output but "brittle", maybe drivers were of low quality both in ALSA and OSS.

it was convenient to have things in a single place (volumes and application mixing) though pulseaudio would be more powerful (I heard pulseaudio would be especially relevant when adding usb microphone, bluetooth audio etc. which I've never done yet)

I can't comment on your C-Media card, but didn't 10.04 use PA? it's no secret that older versions of PA had the very issues you describe above. Did you try 'plain alsa'? I've owned (and do own) various (low-end) consumer and proaudio grade soundcards, none of which i've had big issues with ALSA (or Jack). I don't use PA because i don't require being able to switch to a usb/bluetooth mic/speakers, HDMI, etc ~ but that is really where PA shines and one of the main issues it tackles. However, i think some of the work PA has set out to fix, should've been done in ALSA, for example the DLL approach of coreaudio, which Paul pointed out that 'glitchfree' in PA was sort of attempting/doing - but really it might be better to improve ALSA in this way, rather than trying to do it in a layer above.

now KLANG promises to be JACK compatible. this is interesting. the problem with JACK is, you install it with a few applications and nothing happens. dead silent. or Ardour complains things are'nt set up and shuts down when you launch it. so, how the hell do you set everything up, I don't know. I know my way around the command line and do a bit of network admin but don't know how to get started on this.

oh, there are THREE audio systems, somewhat overlapping on each other. well easy thing is getting rid of pulseaudio first (crippling a bit your computer) and see if you can understand something about your empty QjackCTL.
that's just a big barrier to entry, especially as I merely want to launch software and see if it works, in the remote hope I can get friends who tinker with music to use linux and Free software, Free plugins and synths and stuff etc.

PA isn't going anywhere and it does address some 'real world' issues for users. What more likely needs to happen is improvements/modernization in ALSA and better integration and interoperability between ALSA and PA. On top of this, a more seamless experience (again, integration/interoperability) for the end user between Jack and PA...

KLANG would not fix any of your jack problems, anymore than using OSS, ALSA, Coreaudio, FFADO and portaudio backends would address them. they are backends - they don't directly interact with Jack's routing and clients/apps. As far as not knowing how to get started, maybe jackaudio.org and linuxaudio.org might help. There are a ton of tools, apps and options for jack and also documentation. Jack it not that hard to setup and there are many good apps/tools for managing it. My first experiences (back 5 years ago) were pretty good, i did have to read a little doumentation - but that should be expected, when we are talking about proaudio systems (analog or digital). After getting started, i haven't looked back.

I'm totally isolated, as everyone who uses a computer for music purpose is on Windows, where everything works by double-clicking setup.exe (or maybe use a Mac). time is wasted apt-getting non working or garbage quality software in the repository as well, even when they use ALSA and/or pulseaudio and thus should be easier to get working in theory. Actually I can't even play doom with a midi soundfont, on an OS which has doom and quake as the only games. (with timidity and prboom, a combination that worked on windows).

so, KLANG pretends it can do everything in a single place, be both OSS and JACK compatible, midi, do pulseaudio things. (route things into other things, so maybe I can have a software audio compressor that works, on my global output? something to prevent screaming ads or youtube from firefox, weird filter tricks, music player output to a second stereo output etc.

it entirely depends on what you mean by 'music purpose' - making music, is far different than playing music. With playing music, yes 'double-click' method makes sense, but in the case of things like Jackd (on windows or mac), ReWire, ASIO, etc - there tends to be more setup, as these systems inherently have much more complexity (for good reason). That being said, if jackd is configured and running, after installing xyz program it should be as simple as starting the app, making sure it is using Jackd (usually in preferences, somewhere) and then making sure the app/clients outputs are connected to system outs. ~ and this is not much different than using ASIO in windows, except that Jackd gives you control of the routing too.

as far as using DSP processing of some kind of global outputs, i do that right now (and have been for years) using Jack. I use Jackdbus as my soundserver (no PA) and manage my sessions via Gladish. In my case, i own a nice commercial mastering suite (for windows/mac) called 'T-Racks', which i run via Wine + FSThost. My setup is overkill for a general desktop user, but i am a musician and thus want to be able to sit in front of one of my midi keyboards and be able to play music, at any moment in time... maybe take a quick piano lesson online, as well as having access to multi-gig sample libraries (think pianos) as well as top-notch VSTi/plugins (Organ3, Absynth, etc). but even without such high demands (my system is low-latency, RT kernel 64-128frames / 96khz multi-in/out), i could run Jack with a slightly couple of LV2 plugins on the master and play with FOSS synths/apps. ~ which the reality is, if using linux for proaudio - you are going to be using jack, not ALSA or PA.

but even with PA - couldn't someone just write a compressor/limiter/leveling amplifier for PA? (in some mixer utility) this hardly sounds like a case, where we need a complete re-write of both kernel and user-space sound APIs to accomplish... if i remember correctly, isn't it also possible to do this with ALSA/dmix too?

[grok]

thanks a lot for your lenghty comment.

but even with PA - couldn't someone just write a compressor/limiter/leveling amplifier for PA? (in some mixer utility) this hardly sounds like a case, where we need a complete re-write of both kernel and user-space sound APIs to accomplish... if i remember correctly, isn't it also possible to do this with ALSA/dmix too?

at a time I was in a quest for a global equalizer, and ended up installing an equalizer for pulseaudio. a little "cheating" to get a bit more bass and slightly less aggressive treble. it was not very easy to find and went unmaintained by its author, made the sound a bit "dirty" too. (similar to "issue with older versions of PA" you described).

your experience and advice about Jack is comforting. only I can't exactly find "Jack output" as an option in Firefox, VLC, etc. so this mean maintaining a PA/Jack/Alsa triad if you want to do everything.
maybe common software outputs to pulseaudio, whose output connects to jack (I'm looking at the description of pulseaudio-module-jack package in ubuntu/mint). then Jack layer is the "new pulseaudio" for programs that talk to it. then a global output goes to Alsa (or maybe to a network audio protocol, recording to file, output as a "web radio" etc.)

ideally, things should go wrong only if my Alsa driver is bad. ideally I could be using OSSv4 instead of Alsa and nothing changes (not only the OSS driver could be good, but OSS is available on other operating systems as well)

ah, I've noticed Jack is an option in Audacious, and in gnome-mplayer. not too bad

a strong issue is, it is hard to know what to blame when you fail at doing something or have a problem. is because of Alsa, PA? bug, configuration or bad quality driver? plus the common silly problems such as forgetting to select an input in a program or a volume set at zero.

[grok]

btw I have the timidity synthesizer, and an already huge 256MB soundfont (SGM v2.01, free of use)
this is great midi playback for free and without effort. it was for playing doom 1 and 2, a stupid bug prevents me from doing it but I can still play the songs in the form of standalone midi files. I didn't get DosBox midi working either. I could do everything in windows XP. I want to blame the applications, or maybe it's Alsa's midi support.
I have VMPK as a working piano toy, I'm only lacking a physical keyboard and that would more than enough for just learning to play the piano.

(apt-get install vmpk sets everything up then a line in /etc/timidity/timidity.cfg loads the soundfount. then you have to set vmpk output to timidity else you get no result and can give up thinking it's not working )

[ninez]

thanks a lot for your lenghty comment.

no problem

at a time I was in a quest for a global equalizer, and ended up installing an equalizer for pulseaudio. a little "cheating" to get a bit more bass and slightly less aggressive treble. it was not very easy to find and went unmaintained by its author, made the sound a bit "dirty" too. (similar to "issue with older versions of PA" you described).

I think i 'vaguely' remember this plugin, but like i said i don't really use PA on my system(s), since i pretty much only use Jack.

your experience and advice about Jack is comforting. only I can't exactly find "Jack output" as an option in Firefox, VLC, etc. so this mean maintaining a PA/Jack/Alsa triad if you want to do everything.
maybe common software outputs to pulseaudio, whose output connects to jack (I'm looking at the description of pulseaudio-module-jack package in ubuntu/mint). then Jack layer is the "new pulseaudio" for programs that talk to it. then a global output goes to Alsa (or maybe to a network audio protocol, recording to file, output as a "web radio" etc.)

ahh, for firefox (ie: adobe flash, etc) there are 2 options. You can either use alsa-to-jack plugin, which isn't hard to setup, but has it's annoyances - such as the jack client not being persistent, but does work, okay. or the slightly newer (and imo, better way) is to setup an alsa-loopback device and use a nifty little commandline utility called a2j-zita, which comes with a manual. you simply attach the a2j-zita to your loopback device, and have ALSA-only apps use that device. a2j-zita will do all of the lifting (of resampling, etc) and make this device into a jack client (which you can than route anyway that you like).

i've only used pulseaudio-module-jack once. (for a few months). I had a part-time job, doing some web-development for a company that had a virtual-office using skype. I had issues with ALSA/jack + skype but was able to get it to work using pa-to-jack. But i wouldn't refer to jack as 'the new PA', instead i would recognize that PA and jack are both user-space sound APIs that both run on top of ALSA (or other backends).

ideally, things should go wrong only if my Alsa driver is bad. ideally I could be using OSSv4 instead of Alsa and nothing changes (not only the OSS driver could be good, but OSS is available on other operating systems as well)

yes... and this is why i say the work should be done on ALSA, PA and Jack. if ALSA needs some modernization, even if slightly invasive and a fairly big undertaking, that is still where the work should be done as it will improve PA and Jack also. ripping out the whole kernel (and user-space) audio systems is not going to improve anything. Making changes in an above layer (like PA is trying to do with 'glitchfree) while noble in it's cause, still seems like a band-aid, and not addressing the underlying issues.

ah, I've noticed Jack is an option in Audacious, and in gnome-mplayer. not too bad

gstreamer also allows you to switch the audiosinks to jack;

https://wiki.archlinux.org/index.php/JACK_Audio_Connection_Kit#gstreamer

so any app that can use gstreamer can use Jack. (and as a side note, there is some good general configuration tips in this wiki, in general.

a strong issue is, it is hard to know what to blame when you fail at doing something or have a problem. is because of Alsa, PA? bug, configuration or bad quality driver? plus the common silly problems such as forgetting to select an input in a program or a volume set at zero.

Well, i've botch configurations before and been subject to bugs in drivers too. But there is often logging info than can help assist in figuring out what the problem is and obviously other avenues such as forums, bug trackers and mailing lists. these issues aren't just sound specific in linux though, i've bumped into similar issues on all platforms at one time or another, it's just with linux there is a lot less hand-holding to help you through. Human error is obviously another issue.

I know in my case, since i use session management in Jack, more specifically in my case jackdbus/Ladish, my sessions are saved and automated. So for example, if i want to take a piano lesson or 2 on youtube - i simply launch that particular session; which will consist of linuxsampler(piano) with some processing/plugins and firefox. both will be routed jack_mixer (16 stereo channels), then from jack_mixer's main outs to T-Racks then to speakers, So it's a no brainer. The same is true of if i want to do some rough recordings, i just have templates that will have a DAW (with tracks ready to go), various plugins, mixer, instruments, etc setup with routing in place.

but even if i wasn't using Ladish/jackdbus, i could do the same thing with Jack2 (or jack1) and a few commandline utilities that can accomplish the same goals, and then there is also jack_session.

[multi_io]

apparently, you're not familiar with the linux engineering dictum that the kernel provides mechanism not policy. putting mixing, sample rate conversion and sample format conversion into the kernel would completely violate that policy.

What? Has Linux turned into a microkernel recently? Do you have to use a userspace daemon to enable multiple programs to send data over the network via same network card simutaneously? I didn't know that. :-D

[F.Ultra]

Hm, ok, let me approach it from the other side. I'm not trying to teach you, just sharing thoughts.

The discrete approach, contrary to analog, has finite states. 1 or 0. You can't really chop 1 down to 0.9, same with zero. This is distinct feature, not bug, of discrete - to withstand chaos, science requires precision.

You can, however chop analog in half, so much you need. 0,5; 0,25; 0,125 ... ∞
However, the disadvantage is that analog is completely unstable to noise and "going down the road" you will encounter higher noise levels, which do sum up in operations, and will limit the amount of real granularity. Pretty sure everyone knows it. Musicians value analog for "chopping" and use good cables to protect from noise.

Well, you may call it "analog model nested on discrete approach", or "analog emulation on discrete".. it is the floating point variables.

I use the term "fake analog" for digits that exist and are processed using discrete approach (granular one),
but have the analog nature (of inifinite "going down the road") inside.

Because they are based on discrete, they are protected from noise and chaos which you would encounter in analog wire. They are precise here for.
Because they follow analog model, they can be chopped and chopped, every time correctly in half.

But the disadvantage of comes from their discrete basement - they can only be chopped down up to certain volume, limited by their mantissa.
This does not happen with true analog approach - at all (read above).

After they (float) run out of space, they will "approximate" by cutting down, rounding. All this imprecision is cased by lack of volume, by lack of processing power. And of course by some specific operations, that are only applicable to analog values that is - such as "rounding up" (mod), for example. Pretty same happens to discrete integer, if it runs out of "volume" - it will overrun or clip the "less insignificant part".

I think the engine, proposed by datenwolf will have problems processing audio volume (audio fidelity) and will require much higher bit capacity to overcome than he proposed, granted it will have zero imprecision. 64bit, 128bit?

Well what you kind of miss here is that even though binary data has finite states, the real world difference between these states are magnitudes lower than what you can create an analogue circuit for. For example if you use 32-bit DACs then the output difference beteen a single bit is lower than 0.000000001v, good luck implementing an analogue circuit with that precision (or finding a 32-bit DAC that can even produce such precision). You will not find a single amplifier in the world with a SNR that is even close to that value, especially less so if you consider speakers and the listening room.

There is this great myth that analogue like vinyl is so much better than digital, but that is only because so many people do not understand that analogue and digital works in completely different realms, knowledge in one cannot be automatically applied to the other. Add to that the fact that most samplers tend to display the PCM values for digital audio (which produces steps that makes people believe that digital is sharp/edgy/bumpy) while that is not what will come out of a DAC due to (among others) the sinc function. If you sample any vinyl out there and play it back on a cd-player, no one will be able to distinguish that from the original vinyl in a true double blind test.

And regarding your "fakeanalogue", that is completely false. The reson one uses floats is not to make the signal handling more analogue. It's because if you want to mix 256 channels of 32-bit samples then you have to operate with 287-bit integers. Doing that at 192000 times a second over 256 channels is far from reasonable for a sound system (unless you really crave that precision). While you might think that the floats allow you to represent fractions of a bit (0,5 0,25 as in your example) this is simply a false understanding of how it works. What you do is that you scaled the 32-bit integer sample into a 1,0 to -1,0 float, perform the mixing, and after that you have to rescale it to a 32-bit integer again in order to feed it to the DAC and imaging that our floats would have infinite precision the result would be exactly like the result from the 287-bit integer calculation.

[varikonniemi]

My problems related to Linux audio is restricted to the occasional "jump/scratch" in audio while system is under load, and a few wine games.

If i listen to music loudly from my relatively small speakers, the audio glitch sounds like it could damage them as being the opposite to a pure waveform!

This is something that never happened on windows, no matter what.

I think KLANG proposal is excellent especially since it has backwards compatibility with programs. The only thing i fear is having such a piece of code in the kernel (a new attack vector?). How advanced stuff are we talking about here? Is it straightforward code that will not contain exploitable functions after a few kernel hackers have gone through it and accepted it?

[datenwolf]

*sigh* ? so many misconceptions about floating point vs. fixed point here. Again, this is to be addressed in the detailed design document, I (yes it's still not out) have to publish.

Okay, here it goes, a 101 in number representation. So how do floating point numbers work? Essentially you have to numbers in one: The mantissa, and the exponent. The mantissa gives you the digits, the exponent defines the value range. In the case of binary float:

x = m * 2^E

The important thing is, that you always have the very same number of digits. Digits are what give you precision. A 32 bit floating point number gives you 23 bits of precision (1 bit for the sign, 23 bits for the mantissa, and 8 bits for the exponent for the value range); note that this is already less than good audio ADC do give you (24 bits). Nobody can actually hear the 24 bits, but they're important if you apply some gain on the signal in, lets say, some live recording where each instrument has its own microphone.

Say your regular signal full scale goes from [-1, 1], i.e. the exponent is 0, then you have 2^23 states inbetween. This gives you about 138dB dynamic range full scale ? if the exponent is 0. Now lets consider what happens if your signal goes beyond the full scale, and be it only one bit. With a floating point number, the only thing that can be done is increment the exponent by 1 and shift the digits one to the right, i.e. truncating the least significant bit. Whoopsy, now you've lost one bit in precision. And something interesting happened: While your floating point number still has a total dynamic range of 138dB, the full scale dynamic range now sees only the reduced precision, and that's -6dB, i.e. 132dB. Every time your value range reaches a power of 2 boundary you're loosing 1 bit of precision!

Let's see the reasoning behind using floating point. The original reason for floating point numbers was, that people wanted to be able doing numerics without caring too much about the value range of the numbers they feed it, with the tradeoff, of loosing precision. So what reason is there for audio to use float?

The obvious reasoning is, that you no longer have to care about clipping/wrapping/saturation. But does the use of float make sense? The typical audio floating point number format is 32 bits wide as stated above, with 23 bits of precision. Those give you a total value range of -2^127 to 2^127. To put this into perspective, that is twice the size of the range covered by IPv6 addresses. But only 2^23 of those 2^256 values can be actually reached. In audio terms your full dynamic range would be 1541dB BUT with only 138dB precision your quantization noise level is only that far away, so if you'd really make use of that range your noise would be 1403dB? which is not very usefull. You need to be at least 1.6dB over the noise level to actually deliver some information (Shannon theorem); given the fact that most audio sources are well below 100dB you can see the problem.

Now lets see how 32 bits of fixed point numbers behave. First, instead of only 23 bits of precision your have the full 32 bits at your disposal, which gives you a total dynamic range of 192dB, yet keeping your 24 bits full scale dynamic range of 138dB. In other words you got 8 bits == 48dB of headroom. Let's say you've got a number of audio sources, which make all use of exactly the 24 bits full range (at the same time); how many of those can be mixed without saturation? 2^8 == 256 times. To put this into perspective, this is how 256 audio sources look like







and actually another half of that picture. Of course in your typical audio mixer you have some decent headroom four your individual sources, usually around 12dB, i.e. 2 bits, i.e. you can mix up to 64 audio signals without clipping or saturation while not loosing a single bit of precision.

With 32 bits fixed number precision you can mix up to 64 signal sources, each exceeding the full scale by 12dB without clipping and without loss of precision.

In practice you'll never find that large number of sources playing at full scale at once. Why? Because this would be just noise and not something you'd like to listen to. In your typical audio setup most sources will be somewhere around -20dB FS, i.e. -32dB VR. And not even all of them play at the same time. Of course when you attenuate those signals, you're going to loose precision if your value range makes full use of the all the lower bits. This is where 24 bit ADCs/DACs enter the picture. Due to their larger dynamic range, but the limited range we humans can make of, a good 24 bit audio conversion will place the RMS signal level not at -20dB FS but at -40dB FS so you have some 4 bits (~24dB) head- and footroom for the signal within the 24 bits FS originally given to it. So you can safely add gain and attenuation within reasonable limits without loosing precision. Namely those +/- 24dB; now looking back at floating point, where a mere 6dB gain lost you some bits (any probably even less) looks laughable. And remember, we're still within the 32 bit number size, we also had with floating point. 16 bits ADCs you will just pad to those 24 bits to match and get similar behavior.

So far this is the low profile of KLANG signal processing, meant for embedded devices. The desktop profile introduces some additional 8 bits of precission (footroom) and the studio profile augments this with further 8 bits headroom.

ITT somebody compared floating point with analogue signaling as over a wire. This person neither understood the concept of analog nor of floating point numbers. If you'd compare number formats with wires, then floating point would be an active cable with some attenuator at the front (attenuating in steps of n*6dB) and a complementary amplifier on the other end, and the SNR on the wire between was always 138dB for a 32 bit float. The complementary active attenuation and gain at front and back do increase the (quantization) noise by exactly those n*6dB, but at the same time impair the signal's fidelity.

Fixed numbers on the other hand are the true digital equivalent to analogue wire.

TL;DR: Just with everything in life, just because a lot of people do it, it doesn't make it right. Floating point numbers are a poor choice for digital audio mixing and transmission and fixed point numbers do the job just fine. Oh and BTW, due to request, KLANG will also feature support for floating point endpoints to user space processes, yet still does all internal audio processing fixed point (it's trivial to convert floating point to fix point, other way round is a bit tricker though, because it involves a log2, which is however efficiently done by some bit twiddling).

[grok]

My problems related to Linux audio is restricted to the occasional "jump/scratch" in audio while system is under load, and a few wine games.

If i listen to music loudly from my relatively small speakers, the audio glitch sounds like it could damage them as being the opposite to a pure waveform!

This is something that never happened on windows, no matter what.

oh, yes, this happens even when you get a great sound card. potentially damaging your speakers, I'm glad to hear it said out loud.
on windows, the drivers are professional and the sound system doesn't move, and program authors test their stuff or are less willing to release bad quality software.

when I start vmpk, a piano program for instance, there's a strong glitchy sound in my right speaker that seems to damage it or is tuned to the slight degradation it developed. but no screechy sound on my left speaker. it's scary.

I really wonder why does that piece of software has to dump garbage to my speakers on start up, it's like flushing a buffer down a toilet seat, except the toilet is your speaker.

sound garbage, I got that last time too when attempting to run some silly thing in wine. or maybe still, small clicks when changing sound volume in vlc, which is high profile kind of software, a staple.

[ryao]

1. Unless he wants to only support integer sample formats, this will never be allowed in the kernel, because Linus doesn't allow floating point in the kernel. I guess he'd have to resample any inbound floating point audio from programs in userspace before passing it to the kernel. But since the proposal is for directly accessing a character device, that sounds impossible. Good luck with that.

That is fine. This is in no way Linux-specific.

2. The OSS API? Seriously? It's not even an API; it's just a character based interface. That's not very portable, you know. Even ALSA is more portable than that (you could in principle implement libasound2 backend on Windows and compile it and it should work). Also, OSS is a terrible API from the 90s that has no concept of modern ideas like power savings. The problem is that a "character device" is a concept unique to *NIX operating systems, so any OS that doesn't implement that concept is kinda screwed. A "C" API is pretty much a universal concept; all but 1 or 2 exotic OSes support that. So you can't talk about portability while saying "forget all OSes that don't derive their core design from UNIX".

You can also write a library to simulate character devices on Windows. As far as I know, ALSA is Linux-specific while OSS is available on a wide variety of platforms. Portability-wise OSS is better. Are you volunteering to address that?

3. We don't need another solution. Period. We have too many already. It's way past the point where introducing new solutions is even remotely plausible. All this can possibly do is bring further fragmentation and brokenness to the Linux desktop, and add more headaches for application developers trying to support every system. Hopefully it does not gain any traction whatsoever.

The same could have been said when ALSA and PulseAudio were made.

4. Using the justification of being "annoyed" at PulseAudio is not a reason for starting a completely new project. Instead, improve on the things about PulseAudio that annoy you. Although I can't imagine what; I haven't even thought about sound infrastructure on my system for more than a year. I start up apps and they play sound. I don't see what there is to be annoyed about. Everything "just works" with no glitches/dropouts/etc, exactly like it was promised when PA started back in '06 / '07.

It is a perfect reason.

5. It'll take you a decade to develop all the hardware support needed to be even competitive with OSS4, let alone ALSA. Audio drivers can't ever be pure and simple; they necessarily have to come with tons of ASIC-specific hacks and workarounds, because hardware manufacturers like to add little tweaks to break your driver, like changing pinouts, etc. It's part of the territory. It takes a ridiculous amount of manpower to develop and maintain all those workarounds. One person isn't going to cut it. All the traction is already with ALSA. All the success is already with ALSA.

ALSA is an API. if sound drivers require it, then they are poorly designed.

This thing is just stupid. It's like going on a 500 mile trip in the car, traveling 498 miles to your destination, then say "oh $@(# I meant to take a train, not a car, this sucks", then drive all the way back home and get on a train to go there again. What a terrible waste of manpower.

The only stupid thing here are the people that try to tell others how to spend their time.

Contribute to ALSA or PulseAudio (or both) instead, ya dolt!

The same could have been said for OSS when those were in development. Where were you then?