Google Posts Initial Code For Lyra Speech Codec

[davidbepo]

google reinventing the wheel again...
opus is great for this, no need to make a new one

[carewolf]

I am not getting along with Google's build system: bazel, which is needed to compile this. Feels like somebody wanted a non-Java maven, and this got slapped together. The current Lyra source requires a specific checkout of clang and is Ubuntu-aligned for building. I'm going to go use podman with Ubuntu 20.04 on my Fedora 34 to try to play with this.

Google's definition of open source, means providing sources that doesn't build, and a binary they promise is build from it.

[Mathias]

google reinventing the wheel again...
opus is great for this, no need to make a new one

At these tiny bitrates, OPUS is not very suited for speech. The question is, is there a reason to go that low since UDP/TCP overhead is on the same level as the lyra bitstream...

[cl333r]

It's not for the Moon, because we didn't go there. So... Cui bono?

No, we have been there, Alex Jones has confirmed.

[juarezr]

From the github project page:
Please note that there is a closed-source kernel used for math operations that is linked via a shared object called libsparse_inference.so. We provide the libsparse_inference.so library to be linked, but are unable to provide source for it. This is the reason that a specific toolchain/compiler is required.

If it is just a single kernel, it shouldn't be too hard to replace it with some calls to openBLAS or something like that, won't be as fast though...

Also from the github project page:

Lyra harnesses the power of new natural-sounding generative models to maintain the low bitrate of parametric codecs while achieving high quality

For adhering to the principles of Open/Libre Source, they could also provide the source code and the assets of these "generative models" jointly with the tooling and the knowledge needed to build and improve them.

Also, the audio source samples used to validate the performance and accuracy of the codec are absolutely indispensable for the codec development.

Without these parts, this release consists of a partial source code release because one is only able to build and integrate it. Improving and constructively criticize it is practically denied to very talented people existing in the open-source community.

I hope that this situation would improve as the developers have more time to evolve their code, the math, and the assets that are not included in the Github repository.

[juarezr]

google reinventing the wheel again...
opus is great for this, no need to make a new one

At these tiny bitrates, OPUS is not very suited for speech. The question is, is there a reason to go that low since UDP/TCP overhead is on the same level as the lyra bitstream...

Good questions!

Opus already was developed as a "combo" from other 2 codecs:

• Skype SILK codec for low bitrates and voice
• Xiph.Org’s CELT codec for music and high-quality sound needs.

I know that Lyra is a different kind of codec based on speech synthesis instead of the traditional DCT/Quantization/Entropy Coding scheme. It is similar to Jean-Marc Valin's LPCNet codec.

What I'm thinking and wondering is:

1. If the machine learning models or the audio assets used to create and train Lyra couldn't be used to further improve Opus?
2. If a parametric codec like Lyra, WaveNet, or LPCNet couldn't be mixed in the Opus combo for achieving even lower bitrates?

[microcode]

google reinventing the wheel again...
opus is great for this, no need to make a new one

Opus works at 3k, but it's not spectacular, and you have to increase the frame size to get good quality at that rate, which cuts against use in voice.

[microcode]

What I'm thinking and wondering is:

1. If the machine learning models or the audio assets used to create and train Lyra couldn't be used to further improve Opus?
2. If a parametric codec like Lyra, WaveNet, or LPCNet couldn't be mixed in the Opus combo for achieving even lower bitrates?

I think the answer on the second one is maybe.

The first, the audio assets may be useful for developing codecs generally, but probably the models wouldn't be directly useful to Opus enhancements.

[Alex/AT]

If it is just a single kernel, it shouldn't be too hard to replace it with some calls to openBLAS or something like that, won't be as fast though...

Would possibly be a wasted effort because you can expect to hit a patent or two on the way.

[hotaru]

Lyra can function at just 3kbps.

so it's not as good as Codec 2, which can function at just 450bps?