Mozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizations

[Quackdoc]

Nah. Failure to read again. Go back and see what was asked for.

You seem confused on what that means. Community doesn't mean you make arbitrary demands of pet features from any open source projects.

I know very well what community means, Community means a large group of people centered around something. In this case, JXL is one of the more requested features on both firefox's idea's page (6th behind another real banger for bringing back PWAs) as well as it being the most upvoted feature request for web compat. Firefox pretends to be a browser that "prevents chrome from dictating the direction of the web". It pretends to be a browser, and I quote " People-driven, The things we create prioritize people and their privacy over profits." and yet they can't even be bothered to merge PRs for one of their most requested features.

[swastika]

It pretends to be a browser, and I quote " People-driven, The things we create prioritize people and their privacy over profits." and yet they can't even be bothered to merge PRs for one of their most requested features.

Still very confused. One has nothing to do with the other. None of your pet features add any privacy.

[Draget]

I am impressed how quickly the thread derailed to something completely unrelated, literally not taking a single post before it was offtopic. xD


That being said: The whole 'single file LLM' idea is... meh. There are advancements in interference and advancements in the models, artificially coupling those two into one file seems kind of pointless to me.
I'd rather have a single-binary LLM interference tool that eats most of the common formats and 'just works' with most of the non-CPU accelerators. Having to pass a single file as an argument is not difficult and would be much more useful.

[qarium]

that would be more work then what mozilla is willing to put into firefox. I never understood why people will still defend mozilla when they beg for donations and still refuse to implement features other people have put the work into.
You say "show me the code" why? firefox won't do anything with it. This https://phabricator.services.mozilla.com/D119700 PR is part of a series that was made in 2021. These series of patches make JXL completely usable in firefox. Proper progressive decoding, animation, color management etc. and firefox's response was and I quote (highlighting mine for emphasis)
This is an attempt to pretend that the people can contribute to firefox. we can't. we cannot add the features we want to firefox, they will be rejected, even if the feature is extending something already inside of firefox, that is locked behind nightly.
EDIT: this is the same issue with image-rs when me and eugenevert tried to get jpeg-xl decoding into image-rs, it was rejected outright.

but AVIF is clearly the better option from a technical standpoint and the reason is acceleration hardware

right now there is no acceleration hardware who can perform JXL its pure software codex right now.

on any SOC and cpu and gpu who already has AV1 decode and encode ASIC hardware you can abuse this acceleration hardware to decode and encode AVIF...

from this point of view JXL makes no sense.

[qarium]

the issue is that
A) this is a nightly feature so there is no security burden.
B) these patch sets are very minimal.
C) Jpeg-xl is one of the more highly requested features, even bloody apple supports it.
you asked for my skills in getting the features I want into firefox, unless you have no concept of context.
no. If mozilla want's to pretend they are "for the community" when they both refuse to implement highly asked for features that even other web browsers now have like JXL decode going so far to even reject contributions that make these features useful, as well as keep begging for donations when they already have loads of funding they refuse to allot to firefox, Im going to keep talking shit about them, because people keep defending their abhorrent behavior.

all the opensource and free software people are behind AV1 and AVIF

all the enemies of open-source like apple they support jXL...

from technical standpoint jXL makes no sense at all because there is no acceleration hardware in most hardware

with AV1 and AVIF you can use the AV1 decode and encode ASIC hardware in most socs and APUs and gpus.

[Quackdoc]

but AVIF is clearly the better option from a technical standpoint and the reason is acceleration hardware

right now there is no acceleration hardware who can perform JXL its pure software codex right now.

on any SOC and cpu and gpu who already has AV1 decode and encode ASIC hardware you can abuse this acceleration hardware to decode and encode AVIF...

from this point of view JXL makes no sense.

hwaccel doesn't make sense for avif, its legitimately faster to swdec the single frame.

[qarium]

hwaccel doesn't make sense for avif, its legitimately faster to swdec the single frame.

hardware acceleration for AVIF is not only used to display or encode one single frame.
of course if you only want to display one image you can do software encode
but what if you want to do same on million and billion of pictures because you need this pictures to train an AI ?
and then you need the same in encode mode with the same AI again to putput a movie or million of different pictures.

your idea that you only need to display 1 single picture and thats why software decode is all you need is complete naive.

[Quackdoc]

hardware acceleration for AVIF is not only used to display or encode one single frame.
of course if you only want to display one image you can do software encode
but what if you want to do same on million and billion of pictures because you need this pictures to train an AI ?
and then you need the same in encode mode with the same AI again to putput a movie or million of different pictures.

your idea that you only need to display 1 single picture and thats why software decode is all you need is complete naive.

just checked, neither firefox nor chrome use vaapi for avif decode. so this is indeed a null point. JXL is a royalty free standard just like AVIF/AV1, so there is no debate there. JXL decodes faster then AVIF on software.

From a technical standpoint, AVIF makes no sense since it's inferior in decode/encode times, doesnt support progressive decoding, has a relatively small max resolution limit. colorspace options are limited to CICP, the AVIF+AV1 spec are massively long to read even if they are free, are much harder to read then just using libjxl.

The only technical benefits that AV1 has over JXL is animation technology. that's it.

[qarium]

just checked, neither firefox nor chrome use vaapi for avif decode. so this is indeed a null point. JXL is a royalty free standard just like AVIF/AV1, so there is no debate there. JXL decodes faster then AVIF on software.
From a technical standpoint, AVIF makes no sense since it's inferior in decode/encode times, doesnt support progressive decoding, has a relatively small max resolution limit. colorspace options are limited to CICP, the AVIF+AV1 spec are massively long to read even if they are free, are much harder to read then just using libjxl.
The only technical benefits that AV1 has over JXL is animation technology. that's it.

does it even matter if chrome and firefox use hardware acceleration for AVIF decode ?

the performance to display one picture is not the relevant point here.

why should you use JXL in the web and chrome/firefox is this then tank the performance of AI workloads who do training or output in this file format without hardware acceleration?...

"JXL decodes faster then AVIF on software."

something like this does not matter at all...

also keep in mind chrome/firefox only does it in CPU because old cpus need many cycles just to send the data to the PCIe dedicated gpu and send it back to the cpu this way its faster to calculate it on the cpu.

but this is no longer true for modern CPUs like a AMD ryzen 7950X3D the cpu is a APU/SOC and has a iGPU and this iGPU has decode and encode hardware for AV1/AVIF...

this means only because old system with dedicated gpus has high latency going from cpu to gpu over the PCIE bus and by this fact alone the CPU software rendering is faster does not mean that this is true for future designs.

the classical desktop/workstation with CPU and dedicated GPU over PCIe is more or less death..

and MEGA SOCs likc Apple M4 and AMD Strix Halo rule the world... in 2-3 years nearly no one will buy classic computers with "CPU and dedicated GPU over PCIe" anymore.

"AVIF makes no sense since it's inferior in decode/encode times, doesnt support progressive decoding, has a relatively small max resolution limit. colorspace options are limited to CICP, the AVIF+AV1 spec are massively long to read even if they are free, are much harder to read then just using libjxl.
The only technical benefits that AV1 has over JXL is animation technology. that's it"

your proclaimed software rendering numbers will not matter in the future because every hardware even CPUs will be APUs and will have ASIC for AVIF/AV1

"doesnt support progressive decoding"

tell me why do you need clind side progressive decoding if AV1 supports server side based delivery of lower resolution out of the same file on the server... means you can deliver only 480p to the clind even if the server file is a 8K video.. and same with AVIF...

progressive decoding is a old technology comes from GIF and PNG who low internet speed and low performance of the decode hardware did make proressive decoding a valueable feature to display a unfinished picture long time before the download was finished ...

in modern world this makes no sense. AV1 supports server side down scalling of the resolution means you have a huge full fat 8K video file on the server and the clinds only get small 480p video if you want to.

"has a relatively small max resolution limit."

why do you think you need much more than 8K ??? ,..,. (AVIF has an image resolution limit of 65,538 by 65,536 pixels)

to the rest you have to unterstand that it is strategic decision they plan to do a AV2 code and AVIF2 picture format.

AV2/AVIF2 again will have hardware acceleration in all SOCs and APUs and GPUs and AI NPUs

[Quackdoc]

snip

if you are going to speak nonsense im not going to bother. you clearly can't stay on point.