Rewriting Old Solaris C Code In Python Yielded A 17x Performance Improvement

[linuxgeex]

Or you can choose an optimized C library for those matrix operations. The reality is Python is often chosen for these tasks due to rapid development capability! Sometimes that is throw away code. I know that my limited programming efforts these days is done on Python for that very reason. Well that and code readability.

For FOSS, less lines of code = less effort expended by the magnitude of eyes the community has focused on it, which means it's more likely to be bug-free and optimised.

The corner cases these days are for tools where C actually will outperform scripting languages, given the standard development constraints Good = featureful and performant, Correct= bug-free, and Fast being on time and on budget. Those corner cases are getting fewer all the time.

The real gains for C are in reduced memory usage, which helps with resource-constrained scenarios like embedded, device drivers, kernel, bootstrap. Languages like Rust and Go are starting to make inroads into those arenas now. Hell even the kernel has eBPF for reducing the kernel code while increasing performance for dynamic problem solving scenarios which are hard to code efficiently in AOT-compiled languages like C.

Brendan Eich semi-seriously joked that kernel mode Javascript is coming, because moving it to the kernel would provide the ultimate performance level and once it's there, it becomes a compilation target for all other languages lol, becoming the single unifying platform-independent runtime that sysadmins have been dreaming of for half a century (which was DaoOS's goal, Java's goal, C#'s goal, Rust's goal, so why not Javascript? lol!)

Assembly, HDL/Verilog, and hard wiring still own the real-time, deterministic operation arena, for microcontrollers, FPGAs, CPLDs, board logic, and very high frequency operation. But Python and Javascript have even made inroads into microcontrollers in recent years. Clearly you're not going to bit-bash an I2C interface with 2 GPIO pins in any scripting language, regardless of how fast it is, because it won't be deterministic enough to produce the correct waveforms. But you can call a compiled-in C I2C library asymptotically as fast from Python and JS as from C.

[tuxd3v]

Programming languages have all strong points and weak ones..
Python,Lua,Ruby, even Java( forgetting the tons of verbose code you write in it.. ).. are great and fast prototyping languages, they also serve a valid purpose..and in any case, JIT compilers can help in a lot of cases( and that language could be faster than C )..

Other languages could have faster run-times, even more predictable runtimes, but of-course are slower to program with..

Two Languages that are trying to bring the best of C language( simplicity, predictable speeds.. ), and adding to that higher level programming functionality, is the Terra Language, the other one is Pallene Language.
Both work around Lua types,
Terra seems more close to C, simplicity in mind, Pallene seems more close to Lua, higher level.. But both have ugly parts also these are what prevents me from going into them right now..lets "wait and see"..

[tuxd3v]

(...)
Brendan Eich semi-seriously joked that kernel mode Javascript is coming, because moving it to the kernel would provide the ultimate performance level and once it's there, it becomes a compilation target for all other languages lol, becoming the single unifying platform-independent runtime that sysadmins have been dreaming of for half a century (which was DaoOS's goal, Java's goal, C#'s goal, Rust's goal, so why not Javascript? lol!)

There are already Lua into NetBSD

[linuxgeex]

Programming languages have all strong points and weak ones..
Python,Lua,Ruby, even Java( forgetting the tons of verbose code you write in it.. ).. are great and fast prototyping languages, they also serve a valid purpose..and in any case, JIT compilers can help in a lot of cases( and that language could be faster than C )..

Java's RAD days are long gone. The JRE is so massive that compiling takes far longer than C, even when JITting with jar files. That doesn't stop you from throwing 99% of the JRE in the garbage and using the true core language in a much lighter manner (like Google did by creating Dalvik), but then you discard the entire community around the language. And in the case of Google, they got sued for re-implementing the core APIs, and some idiot judges actually sided with Oracle because they don't understand the concept of an open specification. And they don't understand the costs of implementation. Using the same API isn't derivative in any way. The work to precisely match an existing API actually costs significantly more than creating your own from scratch. It was an extra cost to Google, in the spirit of cooperation, and the morons are looking only at the small body of the interface definition files and claiming that since those were "copied wholesale" therefore the work is derivative. Each of those judges should be sentenced to 1 year of systems programming with zero chance for parole, and then revise their findings.

[F.Ultra]

Modern C ain't that modern. Because of the severe limitations of the language, you don't get a hash set or a tree set in the standard library (from the JFC category).

So storing whole objects instead of pointers are now some holy requirement for implementing a hash or tree set?

edit: Nevermind, having thought some more about this you are probably thinking about the key and not the value, so yeah that would make a generic set quite hard to implement.

[Delgarde]

Java's RAD days are long gone.

They were never there... Java has never been a RAD language. Sure, some tool providers tried to build VB-like tools around it, but it's never really fitted that niche.

[vladpetric]

So storing whole objects instead of pointers are now some holy requirement for implementing a hash or tree set?

edit: Nevermind, having thought some more about this you are probably thinking about the key and not the value, so yeah that would make a generic set quite hard to implement.

Large hashes are enemies of caches. You really don't want to traverse an extra link (dereference) in a hash access. Two back-to-back misses are considerably worse than one, particularly if you end up going all the way to memory (which takes hundreds of cycles, for a single instruction ...).

So yeah, storing the value as opposed to a pointer is pretty important, whether it's the key or the value.

[F.Ultra]

Large hashes are enemies of caches. You really don't want to traverse an extra link (dereference) in a hash access. Two back-to-back misses are considerably worse than one, particularly if you end up going all the way to memory (which takes hundreds of cycles, for a single instruction ...).

So yeah, storing the value as opposed to a pointer is pretty important, whether it's the key or the value.

Well it also depends on the size of the object stored. Even if we don't consider the cost of copy memory around we also face the problem of fitting the set itself in the cache so we are not hit by cache misses when traversing the set. Now of course we can hopefully trust that the cpu will prefetch in a sane manner here (which is no certainty, especially if the lookup is not linear). Sometimes I miss the more simpler days when I could remember the exact cycle count for each instruction and just look at two pieces of code and immediately see which one fast faster...

[linuxgeex]

They were never there... Java has never been a RAD language. Sure, some tool providers tried to build VB-like tools around it, but it's never really fitted that niche.

You've forgotten how much the definition of RAD has evolved.

In 1991 IBM's published the RAD development model. Before that it was largely unheard of, though some development teams were doing some loose form of Spiral/Agile as a knee-jerk reaction to customer's needs for short turnarounds... but tools and processes were non-existent and poorly defined at best.

In 1992 Borland released the popular Turbo Vision and Object Windows Library (OWL) with TCC 3.1, which were freely reusable OOP Visual Application Development Frameworks for DOS and Windows.

In 1993 Microsoft created a product to compete with TCC+OWL: Microsoft Visual C++ 1.0.

In 1995 Sun release Java, which borrowed the concepts of prototyping, OOP, Visual GUI toolkit, streams, auto-allocation of objects, a large convenient standard library derived largely from C++, and fully automatic memory management, which freed developers from a ton of boilerplate around memory management and cross-platform compatibility between Windows, Unix, OS/2, and VMS. It was by far the best RAD language of choice for years with its only significant competitors being MSVC, MSVB, Turbo C++, Delphi, MS Access, and Lotus123. When it got a JIT in 1997 and WebSphere was launched, developers in all languages were filling their shorts as Java became THE *IT* thing in the minds of corporate management worldwide. And that's what killed it, lol.

Of course, TCL/TK/Wish/Guile and Oberon/Modula predated those, but they never achieved the mindshare of Java as RAD development languages, let alone that of PHP, Ruby, Python, Perl, and now Javascript.

[ermo]

Of course you could do it faster in *insert pet compiled language here* if you needed to; the real win was clearly that Python has a built-in set operation that made the logic trivial to implement and maintain going forward.

That the new implementation showed no run-time performance deficit for sysadmin purposes was merely an unexpected -- but no less welcome -- bonus.