Linux 5.9-rc1 Kernel Released - Line Count Dominated By AMD Radeon Navi 2 Additions

[phoronix]

Phoronix: Linux 5.9-rc1 Kernel Released - Line Count Dominated By AMD Radeon Navi 2 Additions

As expected Linus Torvalds has christened Linux 5.9-rc1 to mark the end of the merge window and new feature development of Linux 5.9 as another featureful update that will debut as stable this autumn...

http://www.phoronix.com/scan.php?page=news_item&px=Linux-5.9-rc1-Released

[marlock]

One thing I don't quite understand is the need for so many of those autogenerated lines of code.

Intuitively I always think that if it can be autogenerated it could also be replaced by smart code that does the same thing in a more on-the-fly manner or dinamically on compile time?

Obviously this is a non-expert view and there must be a ton of perfectly reasonable motives for it, I'm just curious to how that goes and haven't bumped into a for-noobs version of the explanation yet.

Is this initially auto-generated then gradually tweaked, is there a limitation in the programming language in this case because it's about headers, would it kill performance if done on the fly, is it impossible to autogenerate on compile time?

[AJenbo]

https://www.youtube.com/watch?v=miS5lh9xq_g

[AJenbo]

One thing I don't quite understand is the need for so many of those autogenerated lines of code.

Intuitively I always think that if it can be autogenerated it could also be replaced by smart code that does the same thing in a more on-the-fly manner or dinamically on compile time?

Obviously this is a non-expert view and there must be a ton of perfectly reasonable motives for it, I'm just curious to how that goes and haven't bumped into a for-noobs version of the explanation yet.

Is this initially auto-generated then gradually tweaked, is there a limitation in the programming language in this case because it's about headers, would it kill performance if done on the fly, is it impossible to autogenerate on compile time?

It can vary a lot by the case, but in my time as a developer, I have often found autogenerated code to have a lot of drawbacks, and often when encountered it turns out that is overused even when somewhere done for valid reasons. Yes, language verbosity can be a reasonable argument for it, but often a better understanding of the language and tools can solve this is a better way as you hinted at your self. But, I would trust that AMD has fairly talented developers and that whoever is doing the review for the kernel has also considered the pros and cons to this, so probably this is a reasonable case for using autogenerated code.

If done right, autgoenerated code can make things more maintainable, if done wrong it can make it much less maintainable.

[muncrief]

I just compiled and tried it for giggles. There weren't any boot dmesg failures or errors, but my main KVM VM, with GPU/SATA/USB passthrough failed on the first passthrough device with the good ol' "VFIO_MAP_DMA failed: Cannot allocate memory" error. So I just booted back into 5.8.1 for now.

I don't know if the security setup/permissions have changed, or if it's a bug but it's late on Sunday night and I don't have time to mess with it now. I just wanted to give anyone that cares a heads up

[intelfx]

One thing I don't quite understand is the need for so many of those autogenerated lines of code.

Intuitively I always think that if it can be autogenerated it could also be replaced by smart code that does the same thing in a more on-the-fly manner or dinamically on compile time?

Obviously this is a non-expert view and there must be a ton of perfectly reasonable motives for it, I'm just curious to how that goes and haven't bumped into a for-noobs version of the explanation yet.

Is this initially auto-generated then gradually tweaked, is there a limitation in the programming language in this case because it's about headers, would it kill performance if done on the fly, is it impossible to autogenerate on compile time?

The autogenerated code is not exactly code. It's definitions and magic constants that depend on the hardware. They are generated on AMD side, presumably from some machine-readable hardware reference manual; you cannot pull them out of thin air by some sort of algorithm.

[marlock]

I was imagining something a little more like boilerplate code, but that makes a lot more sense! Thanks!

[acobar]

NFS extended attribute protocol patches were merged on Linus branch, fingers crossed !

https://lkml.org/lkml/2020/8/15/44

Looks like 5.9 kernel will be a really good version.

[JustRob]

One thing I don't quite understand is the need for so many of those autogenerated lines of code.

Intuitively I always think that if it can be autogenerated it could also be replaced by smart code ... is it impossible to autogenerate on compile time?

It is generated before compilation, by running ./configure - that then assesses which OS, Compiler version, libraries, many, many, things, you are using and does the magic necessary for the code base to work with the largest number of different setups. A small amount of source code can remain unchanged and configuring can substitute OS and Library differences permitting different systems with adequate overlap in their capabilities to run the same code (often with a difference in the feature set).

There's not really a 'noob explanation', but here's the Docs: https://www.gnu.org/software/autogen...l#Generalities

[bridgman]

I don't think we have any auto-generated code in the driver other than register definition headers. What they contain is:

- for each register in the GPU, register name and offset from start of register BAR (xxx_offset.h)
- for each field in each register, shift and mask values to access that field (xxx_sh_mask.h)

The files are big because the chips have a lot of registers. We used to only IP review and release register info for the registers we were actually using (since IP review was so painful in the early days of the open source effort) but now IP review and release info for essentially all of the registers from the start, so that if the driver code needs to access a new register we don't have to go through another round of IP review. It does make the files bigger though.

Register header files are provided for each version of an IP block, but if different chips use the same IP block version we also re-use the header files, so as an example NAVY_FLOUNDER added only a couple of hundred lines of header file info. Organizing the driver code around IP blocks and block versions was one of the big changes going from radeon to amdgpu.

The files are in drivers/gpu/drm/amd/include/asic_reg if you want to take a look.

The main remaining inefficiency that would be nice to eliminate is that in cases where we have multiple instances of the same block (I think this only applies to SDMA) we also have multiple instances of the header files. The files aren't that big though...