OpenJDK 15 To Have Better Out-Of-The-Box Performance

[HarlemSquirrel]

Ugh I just realized that's a very old version of Rails so I think that benchmark stuff is gone.

[caligula]

Don't go for Oracle's line. Java's performance has been slowly decreasing since at least JDK 6. And it's got nothing to do with default memory region sizes or GC. It's become also slower to start and, to some degree, in operation. I can see some of this as being caused by the more advanced features added to the JVM, but I don't think that explains all the performance decline I've seen.

Didn't notice any slowdown when running 'hello world':

$ set-jvm openjdk9
OpenJDK Runtime Environment (build 9.0.4+11)
$ hyperfine "java hello"
Benchmark #1: java hello
Time (mean ± σ): 73.4 ms ± 0.9 ms [User: 93.0 ms, System: 16.1 ms]
Range (min … max): 71.2 ms … 75.6 ms 39 runs

$ set-jvm openjdk10
OpenJDK Runtime Environment 18.3 (build 10.0.2+13)
$ hyperfine "java hello"
Benchmark #1: java hello
Time (mean ± σ): 61.7 ms ± 1.4 ms [User: 79.1 ms, System: 15.8 ms]
Range (min … max): 59.4 ms … 68.2 ms 46 runs

$ set-jvm openjdk11
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
$ hyperfine "java hello"
Benchmark #1: java hello
Time (mean ± σ): 60.2 ms ± 1.8 ms [User: 72.8 ms, System: 10.9 ms]
Range (min … max): 58.4 ms … 66.0 ms 48 runs

$ set-jvm openjdk12
OpenJDK Runtime Environment (build 12.0.2+10)
$ hyperfine "java hello"
Benchmark #1: java hello
Time (mean ± σ): 32.3 ms ± 3.5 ms [User: 35.8 ms, System: 9.5 ms]
Range (min … max): 29.9 ms … 41.0 ms 92 runs

Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

$ set-jvm openjdk13
OpenJDK Runtime Environment (build 13+33)
$ hyperfine "java hello"
Benchmark #1: java hello
Time (mean ± σ): 31.1 ms ± 1.5 ms [User: 34.1 ms, System: 9.6 ms]
Range (min … max): 30.1 ms … 42.5 ms 90 runs

Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

$ set-jvm openjdk14
OpenJDK Runtime Environment (build 14-ea+30-1385)
$ hyperfine "java hello"
Benchmark #1: java hello
Time (mean ± σ): 28.1 ms ± 0.5 ms [User: 31.4 ms, System: 9.3 ms]
Range (min … max): 27.5 ms … 30.9 ms 96 runs

$ native-image hello
$ hyperfine ./hello
Benchmark #1: ./hello
Time (mean ± σ): 0.8 ms ± 0.2 ms [User: 0.8 ms, System: 0.7 ms]
Range (min … max): 0.5 ms … 2.3 ms 946 runs

[Veerappan]

Don't go for Oracle's line. Java's performance has been slowly decreasing since at least JDK 6. And it's got nothing to do with default memory region sizes or GC. It's become also slower to start and, to some degree, in operation. I can see some of this as being caused by the more advanced features added to the JVM, but I don't think that explains all the performance decline I've seen.

This may go unnoticed on >4GHz desktops with heaps (pun intended) of RAM, but on more resource constrained hardware, it can kill Java as a viable option.
Measuring startup time (in both server and client mode) is probably not Michael 's cup of tea, but maybe he can find some time to fire this up on a RaspberryPi or some dual/quad core setup with 2-4GB of RAM.

Isn't that most of the point of Project Jigsaw (starting in java 9)? Slim down the java library so that it's more appropriate for running on smaller embedded systems?

https://openjdk.java.net/projects/jigsaw/

[loinad]

Don't go for Oracle's line. Java's performance has been slowly decreasing since at least JDK 6. And it's got nothing to do with default memory region sizes or GC. It's become also slower to start and, to some degree, in operation. I can see some of this as being caused by the more advanced features added to the JVM, but I don't think that explains all the performance decline I've seen.

This may go unnoticed on >4GHz desktops with heaps (pun intended) of RAM, but on more resource constrained hardware, it can kill Java as a viable option.
Measuring startup time (in both server and client mode) is probably not Michael 's cup of tea, but maybe he can find some time to fire this up on a RaspberryPi or some dual/quad core setup with 2-4GB of RAM.

That's plain incorrect, IMHO. You can easily find independent testing around that shows otherwise and corroborates the improvements I've seen in both client and server/container machines in production.

https://cl4es.github.io/2018/11/29/O...hrough-11.html
https://blog.qfotografie.de/2019/01/...s-application/
https://optaweb.org/blog/2019/01/17/...rIsJava11.html

You should probably try gathering and showing your own data if you really believe it's become slower, because it isn't just Oracle who says it's been getting faster.

[loinad]

Isn't that most of the point of Project Jigsaw (starting in java 9)? Slim down the java library so that it's more appropriate for running on smaller embedded systems?

https://openjdk.java.net/projects/jigsaw/

Yes. It brought improvements and regressions (which eventually got fixed in Java 10 and 11).

https://cl4es.github.io/2018/11/29/O...hrough-11.html

[stiiixy]

Sooo, I recently had a little play of a java game called Towns (broke my leg, sooo bored).
Will these updates improve performance? AI in that game is atrocious and I would like to increase the node to 6 (games max from default, 2), which usually comes at CPU cost.

[andrei_me]

Michael I would pay for premium again, but with the covid crisis, the unfavorable exchange rate become unbearable, to US$ 1 => BR$ 5.30 from US$ 1 => BR$ 4.00 ( in early 2010's it was around BR$ 2.20 ~ BR$ 3.20)

I'll make some savings and when possible/feasible I will do my contribution 👍🏻

[Ealrann]

Michael Could you add the Eclipse OpenJ9 vm to your java benchmark please ? It targets fast start, and lower memory footprint (for containers for example) :
https://www.eclipse.org/openj9/

[cynic]

Phoronix: OpenJDK 15 To Have Better Out-Of-The-Box Performance

It turns out our recent OpenJDK 8 through OpenJDK 14 benchmarks caught some on Oracle's Java team by surprise. But they were able to replicate the outcome and as a result OpenJDK 15 will be seeing better out-of-the-box performance...

http://www.phoronix.com/scan.php?pag...er-Performance

Well, Oracle should consider making a (recurring) donation to Phoronix for the great hint it provided them (not only this time, btw).
What for them is "small" could be a big help for Phoronix.

If there are Oracle employee reading this, please talk to your company about this!

[bug77]

Isn't that most of the point of Project Jigsaw (starting in java 9)? Slim down the java library so that it's more appropriate for running on smaller embedded systems?

https://openjdk.java.net/projects/jigsaw/

Yes, that was the point, but it fell flat on its face. Most third party libraries have not been repackaged as modules, so nobody bothers deploying customized runtimes.
Still, modules only address startup performance by reducing the overall runtime size. Their main goals was to reduce download sizes and reduce the attack surface, not performance improvements.

I also like how a guy thinking he's measured startup performance with a "hello world" application get several upvotes around here

loinad The benchmarking I did was for work (and as I said, not on typical desktops, but on more limited hardware), I'm not at liberty to disclose results. However, looking at the links you posted, I realized another thing: two of those links only discuss OpenJDK. Up until version 11 there were differences between Oracle and Open implementations, with Oracle usually being the better performer. It's possible if you only look at OpenJDK, you'll see the performance has been steadily improving. However, jumping from Oracle JDK 7 or 8 to OpenJDK comes with performance regressions. Unfortunately, I haven't looked at the situation with JDK 11, it would seem performance issues have been ironed out since. But I have no idea whether we got back to Oracle JDK 8 or 6 levels of performance.

Edit: And then there's also x86 vs ARM. At least on the OpenJDK side, Oracle has strangely decided not to offer ARM binaries since JDK 8.