Linux 5.17 To Continue With I/O Optimizations, 5~6% Improvement Pending For NVMe

[coder]

What you mention: "TFLOPS" - floating point operations - are algorithms. This is software, but often implemented in hardware. Likewise texture filling are algorithms also usually implemented in hardware. Once a better algorithm comes out you get more performance on the same hardware.

I chose those examples because GPUs implement both floating point operations & texture sampling in hardware. Doing them in software will lose you so much performance that no algorithmic tweaks will get you a net gain.

But how would you explain the increase of IPC on the same type of hardware? (e.g. Intel 14nm technology)

I don't understand your point, here.

Everything is software today, especially when it comes to memory. The best, fastest and most shiny Optane cells are not worth a dime without the correct software. Jens Axboe's work is a prime example on how software increases the usefulness of hardware. From that perspective andrei_me 's dream will remain a dream forever.

There's a theoretical limit to how fast you can write those cells. So, the goal is for software on the host CPU to have a performance and structure that allows us to hit the limit of the hardware's native performance. Whether higher-layers in the software stack are doing anything clever is beside the point. Delivering the hardware's native performance is the best that the bottom layers in the software stack can do.

[lowflyer]

I chose those examples because GPUs implement both floating point operations & texture sampling in hardware. Doing them in software will lose you so much performance that no algorithmic tweaks will get you a net gain.

The term "implemented in hardware" is not black magic or the ultima ratio of hardware performance. Even if it is "in hardware" it is still an algorithm. As such it may be better or worse depending on the quality of the algorithm - not so much the hardware technology. The Intel Handbook (page 3-367) for example shows the instruction FPREM which calculates the remainder of a floating point division. Although "implemented in hardware" it is an algorithm and Intel provides pseudo-code for this instruction.

I don't understand your point, here.

IPC stands for instructions per cycle/clock. This tells you how many things a CPU can do in one cycle. (from Tom's Hardware)

The same processor instruction can have very different timings depending on the algorithm. It is very understandable when it is from a different vendor. But this is also the case when looking within a single vendor. Even on the same basic hardware technology.

Resources:

• Stackoverflow
• Agner's document
• instlatx64

There's a theoretical limit to how fast you can write those cells. So, the goal is for software on the host CPU to have a performance and structure that allows us to hit the limit of the hardware's native performance. Whether higher-layers in the software stack are doing anything clever is beside the point. Delivering the hardware's native performance is the best that the bottom layers in the software stack can do.

With flash memory the issue is even worse. I understand your "theoretical" limit as the real physical limit. You would not be able to enjoy the speed of a "directly wired" flash memory (even if it carries an Optane flag). Flash memory is fundamentally unreliable and only gets half-way usable with significant support of software.

Resource:

• CCC

[coder]

The term "implemented in hardware" is not black magic or the ultima ratio of hardware performance. Even if it is "in hardware" it is still an algorithm. As such it may be better or worse depending on the quality of the algorithm - not so much the hardware technology. The Intel Handbook (page 3-367) for example shows the instruction FPREM which calculates the remainder of a floating point division. Although "implemented in hardware" it is an algorithm and Intel provides pseudo-code for this instruction.

The basic operations measured in FLOPS are pipelined to one-per-cycle. CPU designers could surely find more/less-efficient ways to implement them, depending on whether the priority is power or area, but the performance is at the limit of what the overall chip can support, and the precision should be at IEEE-754.

IPC stands for instructions per cycle/clock. This tells you how many things a CPU can do in one cycle. (from Tom's Hardware)

Also Inter-Process Communication, but that wasn't the point of confusion.

The same processor instruction can have very different timings depending on the algorithm.

The algorithm isn't the only factor, as I was saying above.

It is very understandable when it is from a different vendor. But this is also the case when looking within a single vendor. Even on the same basic hardware technology.

Okay, so I think the detail you're missing is everything to do with circuit design and SoC design. Intel's "14 nm" is not a static thing. They were continually improving it, which means improving cell libraries and layout tools to match what the fab was able to deliver. This enabled tweaks in the logic design. These iterations of improvements were denoted by the "+", as in "14 nm++".

As their 14 nm technology matured and more production lines were added, the yield improved and they could afford to make bigger chips. This allows reductions in timing, by trading off some amount of perf/area. Finally, it enabled more L3 cache, and successive "14 nm" chips were able to support faster memory. All of those things improve "IPC" in the coarse way that people seem to measure it (hint: it's not really IPC, but rather the amount of useful work done per cycle).

There are further factors, such as power limits and turbo-boosting behavior, which can appear to boost "IPC", depending on exactly how one normalizes for clock speed. Since Intel's first 14 nm CPU, Intel has been turning up the juice on these CPUs for just about every new CPU generation.

Finally, we know Intel has designed some mitigations against side-channel attacks into some of their more recent CPUs, which means comparable software mitigations needn't be run. This also affects the apparent IPC.

I think the real issue is that you made a conceptional leap from software -> algorithms, without realizing they're at different levels of abstraction. When it comes down to it, some things on a chip are actually hard-wired. Although you can describe their behavior using an algorithm, they fit every definition of hardware. Other operations are performed using something like instructions running on programmable hardware, which we can describe as microcode or firmware.

[lowflyer]

As a reminder, we are "fighting over Cesar's Beard" here: The original statement of andrei_me simply stating "software "isn't" the bottleneck anymore".

I tried to use the matter of IPC to illustrate why I say "software will always be the problem". While my lengthy explanations didn't seem to impress you, you quickly jumped into details about IPC calculation. I do not disagree with you here. IPC is a complicated matter and dependent on more than one factor. But that whole matter is a tangent.

I disagree with you here:

the performance is at the limit of what the overall chip can support

It only takes one brilliant mind to come up with a superior algorithm.

The algorithm isn't the only factor

Funny. You seem to agree with me that algorithms are a factor.

Okay, so I think the detail you're missing is everything to do with circuit design and SoC design.

No, I'm not missing these details.

I think the real issue is that you made a conceptional leap from software -> algorithms, without realizing they're at different levels of abstraction.

Well, here we see the root of our disagreement exposed. Let me rephrase that from my perspective:

I think the real issue is that you made a conceptional leap from algorithms -> hardware without realizing they're just different levels of abstraction.

Your point of view seems to be: "Once algorithms are implemented in hardware they actually "are" hardware."
While my point of view is: "Although implemented in hardware, algorithms still "are" software"

Can we agree to disagree here?

[coder]

The original statement of andrei_me simply stating "software "isn't" the bottleneck anymore".

I disagree with you here:

It only takes one brilliant mind to come up with a superior algorithm.

The point was FLOPS. The FLOPS are fundamental to the hardware. That's why it's a spec that's used to characterize the hardware. If you can make a clever algorithm for using the hardware that's less FLOPS-dependent, then maybe you can get more useful work out of it, but not because you're making it deliver more FLOPS.

This was essentially the original claim - that it's nice to see when the software is optimized to the point that it's as fast as the hardware will let it go. Taken somewhat out of context, but still equivalent at some abstract level.

Funny. You seem to agree with me that algorithms are a factor.

Algorithms are relevant in many places, but they're not synonymous with software. You seemed to be arguing that everything is really software, simply because it can be expressed in the form of code and characterized using algorithms. This is not so. There is a level at which real hardware exists. Sometimes, we treat a thing as hardware, even when what we're interacting with is really firmware that's running inside a black box. However, when you can see inside that box, there's always some point at which you run into actual fixed-function units/circuits that meet every definition of the term hardware.

No, I'm not missing these details.

Then it's not at all clear why you decided to seize on the changing IPC of 14 nm Intel CPUs, or what you were attempting to prove in doing so.

my point of view is: "Although implemented in hardware, algorithms still "are" software"

They're not software, because they're physical and immutable. That's what makes them hard. If you remove those from the definition of hardware, then the distinction between software and hardware virtually disappears.

It's almost like you're arguing that: because the atmosphere has some water vapor, that it can be considered water. And therefore, the term "sea level" has no meaning, because there's no dividing line between water and the atmosphere.

I don't deny that one can play semantic games to make words increasingly vague and distinctions increasingly meaningless, but the point of words is to have meanings, so that different concepts can be communicated. If we eliminate hardware from the English language, then we have to perform verbal gymnastics and say hideous phrases like: "the type of software that's synthesized into fixed-function circuits and fabricated on silicon wafers".

Nobody wants to say that, instead of "hardware". So, let's keep hardware meaning the parts that are physical & immutable, while "software" can mean the machine-interpreted instructions that's potentially upgradable.

Can we agree to disagree here?

I can accept that you'll not be convinced, but I don't agree that there's no distinction.

If you declare a draw, without positing any further claims than what's already been introduced, then I'm willing to let it rest.

[lowflyer]

I see you're not willing to let it go. So then ...

The point was FLOPS. The FLOPS are fundamental to the hardware. That's why it's a spec that's used to characterize the hardware. If you can make a clever algorithm for using the hardware that's less FLOPS-dependent, then maybe you can get more useful work out of it, but not because you're making it deliver more FLOPS.

This was essentially the original claim - that it's nice to see when the software is optimized to the point that it's as fast as the hardware will let it go. Taken somewhat out of context, but still equivalent at some abstract level.

Two times no.

No, FLOPS is not fundamental to hardware. It is used to characterize "systems" -> hardware in combination with software. Nowhere is it defined as a means to exclusively "characterize hardware". It is always defined like "method of measuring a processor's processing speed" (Techterms ) or "measure of computer performance" (Wikipedia) or "speed of a computer" (Freedictionary).

And no, the original claim was about I/O operations - not about FLOPS.

Algorithms are relevant in many places, but they're not synonymous with software. You seemed to be arguing that everything is really software, simply because it can be expressed in the form of code and characterized using algorithms. This is not so. There is a level at which real hardware exists. Sometimes, we treat a thing as hardware, even when what we're interacting with is really firmware that's running inside a black box. However, when you can see inside that box, there's always some point at which you run into actual fixed-function units/circuits that meet every definition of the term hardware.

We almost agree here. However, I've never stated that algorithms and software are the same. But are you really arguing that algorithms and hardware are the same? I think you miss one thing: algorithms - even when baked into hardware - retain some of the algorithms properties. Like:

"a better algorithm will render better performance".

That's about the only point I tried to get across. But you seem to argue that "since floating point operations are now in hardware, no further improvement is possible ever." But please note that when implementing algorithms, "hardware" will inherit properties of algorithms. Some - not all of them. (to be more precise)

Then it's not at all clear why you decided to seize on the changing IPC of 14 nm Intel CPUs, or what you were attempting to prove in doing so.

Do you understand what the term "illustration" means? (consider the possibility that I suck at illustrations)

They're not software, because they're physical and immutable. That's what makes them hard. If you remove those from the definition of hardware, then the distinction between software and hardware virtually disappears.

You actually said it! "algorithms are hardware". We "virtually" agree here. These distinctions are disappearing. This is a real world fact. From which point onwards does a VHDL program cease to be "software"? on the hard drive of the developer? on the FPGA? or on ASIC? (another illustration)

It's almost like you're arguing that: because the atmosphere has some water vapor, that it can be considered water. And therefore, the term "sea level" has no meaning, because there's no dividing line between water and the atmosphere.

That's a quite blatant misrepresentation of me. A blatant misrepresentation of your statement would be: "because water can solve some of the gases of the atmosphere, all gases can be considered water". (are these illustrations too?)

I don't deny that one can play semantic games to make words increasingly vague and distinctions increasingly meaningless, but the point of words is to have meanings, so that different concepts can be communicated. If we eliminate hardware from the English language, then we have to perform verbal gymnastics and say hideous phrases like: "the type of software that's synthesized into fixed-function circuits and fabricated on silicon wafers".

Nobody wants to say that, instead of "hardware". So, let's keep hardware meaning the parts that are physical & immutable, while "software" can mean the machine-interpreted instructions that's potentially upgradable.

The "semantic game" already started with andrei_me 's statement. I fully acknowledge your concerns about the language. But to ignore that technology changes does not help. The distinction between hardware and software is vanishing. Nowadays it is unthinkable to have hardware without software. Event the libreboot guys say about CPU microcode: "That makes it software".

I can accept that you'll not be convinced, but I don't agree that there's no distinction.

If you declare a draw, without positing any further claims than what's already been introduced, then I'm willing to let it rest.

I never said there's no distinction. There used to be. Both concepts "hardware" and "software" were probably coined in the 50's of last millennium. From the very beginning there was ambiguity between them. The technology these words describe has since evolved, further blurring the distinction. To remain concise in our language we have to either go through language gymnastics or invent new words. Insisting on the use of words in the old meaning while ignoring that the things they describe have changed will render the language unintelligible - not more concise.


Your turn.

[coder]

I see you're not willing to let it go. So then ...

You can't expect one party to withdraw when new claims are introduced by the other. As long as you keep countering and introducing new claims, you really haven't dropped the matter. Don't expect from me what you're unwilling to do, yourself.

Two times no.

No, FLOPS is not fundamental to hardware. It is used to characterize "systems" -> hardware in combination with software. Nowhere is it defined as a means to exclusively "characterize hardware". It is always defined like "method of measuring a processor's processing speed" (Techterms ) or "measure of computer performance" (Wikipedia) or "speed of a computer" (Freedictionary).

By digging up archaic definitions, you're just playing semantic games. When something like a GPU is rated in terms of FLOPS, that's characterizing the peak computational throughput at a hardware level. That's why people use it as a hardware metric. The fact that you're even debating this point says you're more invested in winning the argument than getting to any sort of reasonable position.

And no, the original claim was about I/O operations - not about FLOPS.

Yes, GPU FLOPS was used as an example.

We almost agree here. However, I've never stated that algorithms and software are the same.

Not in so many words, but that's the equivalence you seemed to draw.

But are you really arguing that algorithms and hardware are the same?

No, of course not.

I think you miss one thing: algorithms - even when baked into hardware - retain some of the algorithms properties.

An algorithm isn't "baked into the hardware". It's a model of what the hardware is doing, just as one can model the tensions and stresses of a building's superstructure. You can design an algorithm and then base some hardware design on it, as an implementation of that algorithm, but the algorithm remains an abstract and incomplete description of what the hardware is doing.

But you seem to argue that "since floating point operations are now in hardware, no further improvement is possible ever."

Basically. Nothing that software running on it does can extract more operations/sec from the hardware than it's natively capable of performing. All you can do is be more efficient in the way you use those capabilities.

Do you understand what the term "illustration" means? (consider the possibility that I suck at illustrations)

Well, I think it doesn't illustrate what you claimed it does.

You actually said it! "algorithms are hardware".

No, I didn't.

These distinctions are disappearing. This is a real world fact.

Not as much as you think. The original 8086 CPU had micro code, and the original IBM PC had microcontrollers in its peripherals. They didn't have FPGAs, though.

From which point onwards does a VHDL program cease to be "software"? on the hard drive of the developer? on the FPGA? or on ASIC? (another illustration)

It's not the code that makes it hardware vs. software. If you design a circuit schematic and run it in a simulator, it's effectively software. I already said what's the ultimate dividing line for hardware, and it's what's physical and fixed-function. Why is that so hard to understand?

Even a FPGA is itself hardware, even if the gate configuration you're running on it isn't. This is knowable by the FPGA's physical limits. It has only so many gates and constraints on how they can be connected. No matter how you program it, you can't exceed those limits.

That's a quite blatant misrepresentation of me.

It's how your argument seems, to me. You seem to be arguing that hardware doesn't exist, which would be an exercise in the absurd. We all know it exists. A child knows it exists. The only point that can possibly be debated is where one draws the dividing line.

The "semantic game" already started with andrei_me 's statement.

No, it didn't. That statement seemed perfectly reasonable to me. I've already explained it enough times.

I think you just seized on it to test your thesis. Well, you tested it and I rejected it.

to ignore that technology changes does not help.

The details change, but there are fundamentals which don't. We've long had microcontrollers with embedded PROM, sequentially-executed machine code, and horizontally-executed microcode. FPGAs are just another variation on the same themes.

the libreboot guys say about CPU microcode: "That makes it software".

Yes, there's indirection between the CPU's ISA and its actual ALU. However, its ALU does exist as real fixed-function hardware, and that imposes fundamental limits on the capabilities of its microcode. Microcode is nothing new.

I never said there's no distinction. There used to be.

You're effectively saying that, by arguing only that it existed in the past.

[lowflyer]

The fact that you're even debating this point says you're more invested in winning the argument than getting to any sort of reasonable position.

Propose a "reasonable position" on how to conclude this argument.

[coder]

Propose a "reasonable position" on how to conclude this argument.

Though spirited, I think it was a debate in good faith. You raised an interesting perspective that I feel was worth exploring. Thanks for keeping it above board.

I hope you got something out of it, even if only to sharpen your thinking. Even when I don't learn anything that moves my position, arguing a point often serves to clarify my own thinking on a matter.

I wish the best for you & your family, as we enter the holiday season.
: )

[lowflyer]

Though spirited, I think it was a debate in good faith. You raised an interesting perspective that I feel was worth exploring. Thanks for keeping it above board.

I hope you got something out of it, even if only to sharpen your thinking. Even when I don't learn anything that moves my position, arguing a point often serves to clarify my own thinking on a matter.

I wish the best for you & your family, as we enter the holiday season.
: )

Thank you for your wishes and I wish you and your family also the best for the holiday season and beyond.

I'm actually still waiting for a proposal of a "reasonable position".