Rust GCC Code Generator "rustc_codegen_gcc" Can Now Bootstrap Rustc

[ssokolow]

[...]

Again, you steer it back to operating systems. This has nothing to do with replacing syscalls or changing the calling convention for libc.

This is about creating something that is to WebAssembly Interface Types's IDL as something like fastcall is to cdecl... a more portable alternative to in-process COM that's less complex at runtime.

...and with the work on WebAssembly, I can see it being very viable for people to Cunningham's Law their way to breaking this stalemate.

([just does the FFI by embedding a WebAssembly runtime and compiling the other languages to WASM.] "Fixed it!" "What? It runs at half the speed of native code and my binaries are bloated out with a WASM runtime? That's your problem. It's fast enough for me and I've got work to do. Don't like it? Then build something better for me to use.")

...and if they don't break the stalemate, Embedding WebAssembly runtimes in native apps may be the next systemd or PulseAudio or Electron... an ill-fitted solution that prompts much gnashing of teeth because tons of people declare it "good enough" because nobody who hates it is willing to put their money where their mouth is and write a better solution.

[oiaohm]

([just does the FFI by embedding a WebAssembly runtime and compiling the other languages to WASM.] "Fixed it!" "What? It runs at half the speed of native code and my binaries are bloated out with a WASM runtime? That's your problem. It's fast enough for me and I've got work to do. Don't like it? Then build something better for me to use.")

No its not fixed. This is not learning from history we have .net and java bytecode and others before this pulling this stunt. Please note this route that WebAssembly, .,net and JVM solutions use don't get the language developers to agree. They replace the machine code/assembly section of the build of the language. Of course this has performance problems because hardware optimised instructions are not built into cpu to process this stuff quickly.

You just wrote "but sometimes" yes this

"Then build something better for me to use."

is "but sometimes". Yes if something does make something better you will go this will take me too much work to port my code right. So WASM is going to fragment question is more when then if and yes you can look at the prior examples with JVM and .net with the minor miss matches between implementations that cause those developing programs headaches.

Please note WASM, .net and Jvm is not all of them either who took the route of provide their own basic bytecode/assembly as the solution all of them have had performance problems leading to more cases of "but sometimes" forks .

Remember Microsoft and people working in java have gone to the complete effort of making complete operating systems in .net and java. Lots of developers who are not you said stuff having the poor performance and refused to port to .net as well.

The reality here the problem is cursed. You are not going to get those developing programming languages to 100 percent agree on foreign function interface standard and stick to it due to the "but sometimes" that effect performance. You are not going to be able to keep agreement with something like JVM, .net or WASM over the long term due to the "but sometimes" problems that will come up and resulting in people attempting to make better and people saying put causing fragmentation. Yes you just made the same sales pitch that Microsoft made when .net was released for WASM.

There are such things as unfix-able problems that are just the nature of the beast.

...and if they don't break the stalemate, Embedding WebAssembly runtimes in native apps may be the next systemd or PulseAudio or Electron... an ill-fitted solution that prompts much gnashing of teeth because tons of people declare it "good enough" because nobody who hates it is willing to put their money where their mouth is and write a better solution.

PulseAudio is being worked on being replaced by Pipewire and now we have fragmentation there were some are wanting to stay on Pulseaudio and others are moving to Pipewire. That gnashing of teeth is always a ticking timebomb to fragmentation. Remember you said nobody who hates it is willing to put their money where their mouth is and you included Pulseaudio that has pipewire where someone has put their money where their mouth is.

This is the problem the stalemate problem is not solved by systemd, pulseaudio or Electron... and people declaring solution "good enough" only gets you a defacto standard those always end up fragmented.

ssokolow the foreign function interface problem is so far stalemated its not funny. People keep on attempting to solve the foreign function interface problem and just create more fragmentation and has been that way for over 50 years. Biggest problem here is no party wants to admit they are in a total stalemated location.

The solves to the stalemate location are kind of savage. Take systemd the Linux work init system was a fragmentad mess so systemd goes for a almost total replacement this is a elimination move. May not 100 percent fix the stalemate problem but will make it less complex. Yes the chess method of reducing the stalemate to the least number of pieces. This is where gcc and llvm coming the dominate compilers is good as this is a elimination move. Pulseaudio did not fix the problem but it was a elimination move resulting in reducing in sound servers.

At this point agreement is not possible between programming languages for foreign function interface but the question should be what elimination moves can be done at worst will make the problem less complex at best a solution will come viable but only after elimination is done making the problem less complex.

[ssokolow]

No its not fixed. This is not learning from history we have .net and java bytecode and others before this pulling this stunt. Please note this route that WebAssembly, .,net and JVM solutions use don't get the language developers to agree. They replace the machine code/assembly section of the build of the language. Of course this has performance problems because hardware optimised instructions are not built into cpu to process this stuff quickly.

*chuckle* That's the thing though. "No its not fixed" is what tons of people said about systemd and PulseAudio and using Electron for portable app development. Did that change anything? No.

They considered it "fixed for me" and the people who were complaining didn't put their money where their mouth was and produce a more appealing alternative.

Hell, C was a "fixed for me" thing relative to raw assembly language that punted on the "but sometimes" by supporting linking against hand-written assembly.

My point is that the demand is growing and things like WebAssembly are reaching a point where they might get shoehorned into being a "good enough" hack.

Heck, once WebAssembly Interface Types is complete enough and mature enough, I could easily imagine someone patching the language compilers which implement it to support for non-WebAssembly targets as a lower-level analogue to how GObject Introspection gets so many languages and libraries to agree on how to communicate with each other.

Also, lumping WASM in with JVM and .NET isn't quite correct since, while they are all bytecode+VM systems, JVM and CLR never had the degree of push WASM is getting to make it easy to compile things to WASM and embed WASM runtimes in things.

(eg. Emscripten, Wasmer's support for 17 host languages and counting, various languages' native support for targeting WebAssembly, etc.)

The reality here the problem is cursed. You are not going to get those developing programming languages to 100 percent agree on foreign function interface standard and stick to it due to the "but sometimes" that effect performance.

If C++ can care so much about not breaking ABI that it's faster to spin up a copy of PHP than to use std::regex, I think having some kind of extern "FFI-IDL" is attainable.

PulseAudio is being worked on being replaced by Pipewire and now we have fragmentation there were some are wanting to stay on Pulseaudio and others are moving to Pipewire. That gnashing of teeth is always a ticking timebomb to fragmentation. Remember you said nobody who hates it is willing to put their money where their mouth is and you included Pulseaudio that has pipewire where someone has put their money where their mouth is.

...and I wouldn't expect it any other way. Pipewire is young and still developing. In fact, I think it's good. KDE 4.x and PulseAudio and systemd were all rushed into production use prematurely.

[oiaohm]

Also, lumping WASM in with JVM and .NET isn't quite correct since, while they are all bytecode+VM systems, JVM and CLR never had the degree of push WASM is getting to make it easy to compile things to WASM and embed WASM runtimes in things.

https://en.wikipedia.org/wiki/Common_Language_Runtime CLR stated goal is basically the same.

(eg. Emscripten, Wasmer's support for 17 host languages and counting, various languages' native support for targeting WebAssembly, etc.)

https://en.wikipedia.org/wiki/List_of_CLI_languages

JVM and CLR also supports a lot of different languages. JVM and CLR did at different times had the degree of push WASM has now but it faded.

WASM is a bytecode by the way and to be usable it either needs a AOT/JIT of some form so a VM.

https://github.com/WebAssembly/design/issues/1397

Like or not there are already issues turning up in the same places they did with JVM and CLR when porting applications written for native to wasm.

ssokolow the reality is there has to be a bytecode breaking change in wasm future to solve particular problems already documented and if its not solved in wasm soon or later its going to fragment. The know faults it just a question of time until the current wasm fades.

History loves repeating self. Yes the issue that turn up are very much the same. This is the problem here with these attempts as universal bytecode for multi programming languages fairly much none of them study the parties that come before them for what went wrong. Yes there are universal bytecode attempts for multi programming languages before Java and CLR/net exist as well.

ssokolow in reality I am not seeing a long term solution. Something like wasm might be able to reduce the complexity for while but its not the final solution so its going to fragment. The reality here just suxs the FFI problem is not a simple fix.

The higher the push something has more likely people will attempt ignore fundamental flaws were are seeing that with WASM now and we saw this with early .net as well.

Remember Microsoft singularity OS where everything was to be CLR based lot of people thought this was going to be a good idea so a lot of effort went to what was a failure due to fundamental problems and one of those fundamental problems is foreign function interface with the what different languages expect this includes memory layouts of data.

This is just a cursed problem. Java, CLR, Parrot.... I could list the attempts but its 60 odd with different levels of backing. All run into the same set of fundamental problems and that includes wasm.

Its the good old pick fast or stable/safe. This is a shade of grey scale problem. Different programming languages pick safety vs speed differently this results in different FFI requirements. Yes the pick of fast or stable/safe effects memory layout like the choice of 128 bit integer format.

Yes wasm solution is like the java solution or the CLR solution lets attempt to have a square peg in round hole and expect it to still hold exact alignment(that its never going todo) Yes some of the languages that have wasm backend you see this problem where they have lost protections they have when built native.

.
Yes and all of this starts with "but sometimes" we have X issue so we have to solve it. "but sometimes" can lead you into a infinity sized problem. Wasm has the weakness that it software so if people don't like it they can change it without the cost of fab time and possible gain themselves either security or performance while breaking the ABI.

[NobodyXu]

ssokolow in reality I am not seeing a long term solution. Something like wasm might be able to reduce the complexity for while but its not the final solution so its going to fragment. The reality here just suxs the FFI problem is not a simple fix.

There are two fixes for this using wasm:
- We can simply compile all the languages to wasm
- We can embed the wasm runtime (interpreter/jvm) into the application

The first option is supported by C/C++, C#/. NET, Rust, Java, Python, Elixir, Go
The second option is supported by all languages which supports C/C++ API/ABI and Rust.

Remember Microsoft singularity OS where everything was to be CLR based lot of people thought this was going to be a good idea so a lot of effort went to what was a failure due to fundamental problems and one of those fundamental problems is foreign function interface with the what different languages expect this includes memory layouts of data.

Is there any tool to compile C/C++/Rust/Java/Go to CLR?

Edit:

It looks like C++ can be compiled to CLR by using microsft's compiler:

• C++/CLI: A version of C++ including extensions for using Common Language Runtime (CLR) objects. Provides full support for .NET Framework and library only support for .NET Core. Produces mixed-mode code that produces native code for C++ objects. The compiler is provided by Microsoft.

But that is still no where near the support of compiling C++ to wasm.

You can just use clang to compile C++ to wasm, without having to resolve to any proprietary compiler.

And notice that it can only compile "A version of C++" to CLR, while clang can literally compile any valid C++ code to wasm.

Yes wasm solution is like the java solution or the CLR solution lets attempt to have a square peg in round hole and expect it to still hold exact alignment(that its never going todo) Yes some of the languages that have wasm backend you see this problem where they have lost protections they have when built native.

What protection does the languages lost when compiled to wasm?

If anything, they actually gain more protection as wasm is designed to easily sandbox any software.

For example, wasm has capabililty based filesystem API, which makes sandbox much easier while still making the software reusable.
The creator of Docker even said if there was wasm, then they won't create Docker and would just use wasm.
It is trivial to control which part of the filesystem a wasm program can have access to and what privileged operation can it do.

[ssokolow]

https://en.wikipedia.org/wiki/Common_Language_Runtime CLR stated goal is basically the same.

There's a big difference between a stated goal and an achieved reality. The CLR was always a Microsoft endeavour, while WASM is a multi-party effort that's getting plenty of buy-in from people both in and outside the Microsoft ecosystem.

Also, CLR was fundamentally hampered by being a closed-source browser plugin with no official Linux option and no support on things like iOS, while WASM is a web standard supported by basically every browser.

WASM is a bytecode by the way and to be usable it either needs a AOT/JIT of some form so a VM.

That's misleading at best, and, given that WASM is descended from Google's PNaCL, which is descended from NaCl, which is descended from using LLVM IR as a bytecode format, it's like saying that LLVM Clang is incapable of producing native binaries that run without a VM.

Yes, WebAssembly uses a VM... but AOT-compiled WASM is more like Windows binaries running in Wine inside a cgroups sandbox than what people usually think of as a VM. It's not the machine opcodes that are being virtualized, but the platform APIs... which means that, theoretically, someone could write a Visual J++ to WASI's Java SE... it's just not a very useful thing to do.

https://github.com/WebAssembly/design/issues/1397
Like or not there are already issues turning up in the same places they did with JVM and CLR when porting applications written for native to wasm.

ssokolow the reality is there has to be a bytecode breaking change in wasm future to solve particular problems already documented and if its not solved in wasm soon or later its going to fragment.

I fail to see how this is a counterexample to my point that, if nobody else comes up with a better IDL for cross-language interop, people may start using WebAssembly. It's like saying that the transition from libstdc++5 to libstdc++6 proved the existence of KDE was impossible. No... it just meant that, if you wanted things to interop within the same process, they had to be built to the same ABI version.

The know faults it just a question of time until the current wasm fades.

Given how many browsers WASM is baked into and how many sites are already using it, I think that the only thing that'll cause WASM v1.0 to fade is WASM V2.0... in the same way that HTML5 caused HTML4 to fade.

ssokolow in reality I am not seeing a long term solution. Something like wasm might be able to reduce the complexity for while but its not the final solution so its going to fragment. The reality here just suxs the FFI problem is not a simple fix.

I never said it was going to be the only solution... just that it's likely that people will get sufficiently fed up with the "C ABI" non-standard and having to bundle a C compiler for two non-C languages to talk to each other that they'll look elsewhere, and that I anticipate WASM will be "good enough" for a lot of them.

The way you're fixating on the straw man of "WebAssembly will be the universal solution" rather than "WebAssembly is a likely candidate that many people who are fed up with 'C ABI' will turn to" almost feels like the same mindset as religious people who think that, if they can find evidence of living dinosaurs, it'll discredit evolution, somehow discredit all of science, and prove that Christianity is true.

Remember Microsoft singularity OS where everything was to be CLR based lot of people thought this was going to be a good idea so a lot of effort went to what was a failure due to fundamental problems and one of those fundamental problems is foreign function interface with the what different languages expect this includes memory layouts of data.

Given the mismatch between what I said and how you've responded, that's directly equivalent to saying "You can't write an OS kernel in Electron, therefore Electron apps like Discord don't exist."

As I've said a million times, this isn't about a universal solution. This is about the potential for people getting fed up with their less stringent needs being held back and saying "Good luck with your mess. I'm going to use WebAssembly as my FFI layer. It's Good Enough™ for me."

I'm tired of talking in circles with you ignoring me, so I'm out. Feel free to continue to miss my point to your heart's content.

[ssokolow]

What protection does the languages lost when compiled to wasm?

If anything, they actually gain more protection as wasm is designed to easily sandbox any software.

For example, wasm has capabililty based filesystem API, which makes sandbox much easier while still making the software reusable.
The creator of Docker even said if there was wasm, then they won't create Docker and would just use wasm.
It is trivial to control which part of the filesystem a wasm program can have access to and what privileged operation can it do.

While it doesn't look like what oiaohm is fixated on, you did stumble on something WebAssembly regressed on:

• WebAssembly doesn’t make unsafe languages safe (yet)
• Everything Old is New Again: Binary Security of WebAssembly

[NobodyXu]

While it doesn't look like what oiaohm is fixated on, you did stumble on something WebAssembly regressed on:

• WebAssembly doesn’t make unsafe languages safe (yet)
• Everything Old is New Again: Binary Security of WebAssembly

I agree that wasm cannot make unsafe memory operations safe, it cannot prevent use after free, buffer overflow, etc.

But for filesystem related io operations, wasm indeed makes it safer and easier to sandbox, with it capability based API.

[ssokolow]

I agree that wasm cannot make unsafe memory operations safe, it cannot prevent use after free, buffer overflow, etc.

But for filesystem related io operations, wasm indeed makes it safer and easier to sandbox, with it capability based API.

Definitely. The point being made by those articles, however, is that WebAssembly has reintroduced vulnerabilities like "Because 0 is a valid memory address, dereferencing a null pointer won't crash the program".

[NobodyXu]

Definitely. The point being made by those articles, however, is that WebAssembly has reintroduced vulnerabilities like "Because 0 is a valid memory address, dereferencing a null pointer won't crash the program".

They could definitely do better on that one, maybe providing a new mode similar to pointer tagging in ARM (sorry I forgot the name), which ises 128 bit to store address, where 64 bit is used to encode a tag?

They could also try embedding the size of allocation in the pointer as an extension.