Announcement

Collapse
No announcement yet.

Fedora's Java Packages Have Fallen Into Rough Shape

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #71
    Originally posted by oiaohm View Post
    Also this increase in python has seen increase in usage of items like https://cython.org/ yes python to C. So you don't code C instead code in python and covert to C for performance. So C usage in the backend of cloud is not as rare as it use to be. But will you be coding in C directly more often than not no instead you will be coding in a different language and converting to C. For the reason you stated with security and memory vulnerabilities risk coding straight in C is normally not worth it. But the performance of C cannot always be ignored either.
    Using Python is kind of retarded because it's currently among the slowest scripting languages. I know it can be further optimized and there are other implementations than CPython, but I'm just saying it's currently maybe the slowest language there is. PHP is much much faster. So is Julia for data science. I also know this performance doesn't always matter and you can glue together efficient C libraries, but it's still stupid to choose the shittier platform when there are faster options. It's also not the only downside. The C code that is being glued together with Python suffers from impedance mismatch between Python and C (OOP single/multi inheritance and more advanced strong types like optional, enums, variants etc.), relatively weak typing in Python, and also very weak typing and general lack of thread / memory safety in C. There are multiple linters and analyzers that attempt to improve the safety of C code, but overall the end result is just pure shit compared to Rust, C#, Java, Wasm etc. Python also doesn't appeal to all users of curly brace languages and there are still many Python 2.x projects out there, especially in production environments. Also the variety of pip, virtualenv etc. environments introduce some deployment issues.

    Comment


    • #72
      Originally posted by reba View Post
      Why we chose Java for our High-Frequency Trading application - Oct 21, 2020 - 8 min read
      Originally posted by mdedetrich View Post
      I actually interviewed someone who used Java for HFT, and yeah it was interesting. Basically they turned off the GC in the JVM for the slight time window when the trading happened and then turned it on again after the time window.

      Java is not as slow as people mistankingly think it is, if you code in Java in a very similar way to C (i.e. very iterative, keep abstractions to a minimum) the performance is quite competitive.
      There is more than 1 catch here. I have sat down with those who use python in HFT. One of the first things you learn is that don't use python stock alone.

      Originally posted by mdedetrich View Post
      The issue with Python (and I had to deal with this in many companies) is that its only performant if you have a C library, if you don't then you have to write one yourself which brings us back to square one. The python developers even kept the python interpreter slower by generating bytecode that is easy to read/understand at runtime but slower to execute (i.e. it deliberately doesn't do many stack based optimizations). Also note that python has a GIL (global interpreter lock) which makes multithreading in Python terrible
      That is true if you are using Cpython the core python the HFT using python don't use that version. Cython that is your python to C and they also will use python to java bytecode solutions as well.

      Python as a language is not restricted to the Cpython implementation. Yes the problems you listed as the Cpython implementation. "global interpreter lock" .


      mdedetrich you don't in fact need to write a C library. You have cython and nuitka to take your python code covert it to C and build. Yes nuitka it can nicely automatically turn that python library area into c then into loadable dll and .so without you as programmer having to touch a line of C.


      Yes interesting enough you can over come pythons GIL. Yes using numba package to add extra JIT magically 99.9% of python multi thread problem pulls the disappearing act without needing to make .so or .dll files.

      Numba & Cython can significantly speed up Python code. Numba can speed up Python code by 1 Million time by simply adding a single word.


      I would love if nuitka and numba got into mainline python.

      mdedetrich and reba the reality here is python code does not have to be as slow as it first appears. You statement mdedetrich your statement that python is only performance if you have a C library is not in fact true. numba has the java jit advantage of improving over time. Stock python is slow. Stock python + different addons for performance like nuitka, numba and cython is not that slow.

      Yes items numba, nuitka and cython is doing stack based optimisations. Java vs python with addons for speed don't result in java winning often. Yes I would love to see the speed addons to python move into mainline python. Yes the idea that python is lower than php is also false once you have python with the different speed addons.

      Comment


      • #73
        Originally posted by oiaohm
        Sorry but a lot of ways Java is slowly losing market share. Oracle vs Google over license has not done java any favours. Oracle vs Google result in the loss of items like gcj that java to native binary.

        Java might be more modern than C and C++ but without a native binary production tool it does not end up as high performance as python that has python to C or the other languages like rust and go that can also make native binaries. Like or not JIT brings some horrible overheads.

        Yes Oracle vs Google fight has put java in a bad location leading to slow but progressive market loss due to lack of performance. At this stage the biggest winner of that market loss is python. Is it possible java to turn around at this stage yes it is. But it would require people making java to admit native binary/AOT feature is important.
        No it isn't lol. There is no alternative to Java that is not also making use of the JVM, and nobody who is currently using Java is thinking "man I can't wait to get back to C++". Java can match C/C++ performance and does not require you to deal with manual memory management, which is a huge win in terms of reducing complexity and error. JIT also brings some advantages over native binaries, like simpler allocation. There are tradeoffs of course, the main one being latency and the minor one being increased memory usage, but these improve constantly. When you say "performance" I take it to mean you don't really use or like Java, or you are repeating something that you heard from 1996.

        Comment


        • #74
          Originally posted by oiaohm View Post


          mdedetrich you don't in fact need to write a C library. You have cython and nuitka to take your python code covert it to C and build. Yes nuitka it can nicely automatically turn that python library area into c then into loadable dll and .so without you as programmer having to touch a line of C.


          Yes interesting enough you can over come pythons GIL. Yes using numba package to add extra JIT magically 99.9% of python multi thread problem pulls the disappearing act without needing to make .so or .dll files.

          https://www.pickupbrain.com/python/s...thon-vs-numba/

          I would love if nuitka and numba got into mainline python.

          mdedetrich and reba the reality here is python code does not have to be as slow as it first appears. You statement mdedetrich your statement that python is only performance if you have a C library is not in fact true. numba has the java jit advantage of improving over time. Stock python is slow. Stock python + different addons for performance like nuitka, numba and cython is not that slow.

          Yes items numba, nuitka and cython is doing stack based optimisations. Java vs python with addons for speed don't result in java winning often. Yes I would love to see the speed addons to python move into mainline python. Yes the idea that python is lower than php is also false once you have python with the different speed addons.
          I am sorry but none of these solutions are used in any serious scale and this is from speaking with countless Python developers. Even when multi-million corporations re-implement interpreted languages to be faster (i.e. PHP's hack or Dropbox's Pyston) have completely failed in getting any traction (Dropbox actually pulled the plug on Pyston and instead started using Go).

          The biggest issue with these implementations is they create severe incompatibilities with in the ecosystem, i.e. they only work well with a very small subset of libraries and/or language features at which point you may as well pick a better language for the job (which in pythons space is Go, Rust or Scala depending on what you are talking about).

          Ironically this is something the JVM excels in because its an excellent platform to build languages ontop while allowing those languages to interopt with eachother. A good example of this is JRuby which may have reduced in popularity now, around half a decade before actually saw a huge amount of traction.

          JVM is a juggernaut for typical backend/server applications and thats not going away any time soon.

          Originally posted by krzyzowiec View Post

          No it isn't lol. There is no alternative to Java that is not also making use of the JVM, and nobody who is currently using Java is thinking "man I can't wait to get back to C++". Java can match C/C++ performance and does not require you to deal with manual memory management, which is a huge win in terms of reducing complexity and error. JIT also brings some advantages over native binaries, like simpler allocation. There are tradeoffs of course, the main one being latency and the minor one being increased memory usage, but these improve constantly. When you say "performance" I take it to mean you don't really use or like Java, or you are repeating something that you heard from 1996.
          To be clear, in typical code Java is 90%+ as performant as C++, there are still areas where C++ will beat out Java especially if you aggressively optimize C++ code. The point is that 90%+ is more than good enough for almost every web application out there and is MUCH faster than Python unless you are using Python as a nice API wrapper ontop of C libraries like Numpy

          Not having memory safety issues as well as much better support for concurrency (which is critical in web applications that scale with cores) is why its so strong.

          And yeah, I have never heard of a case of someone using Java wanting to go to C++ because there is a reason why they are using Java in the first place. If anything they go to Kotlin/Scala or Go, but not C++.
          Last edited by mdedetrich; 01 October 2021, 04:57 PM.

          Comment


          • #75
            Originally posted by krzyzowiec View Post
            No it isn't lol. There is no alternative to Java that is not also making use of the JVM, and nobody who is currently using Java is thinking "man I can't wait to get back to C++". Java can match C/C++ performance and does not require you to deal with manual memory management, which is a huge win in terms of reducing complexity and error. JIT also brings some advantages over native binaries, like simpler allocation. There are tradeoffs of course, the main one being latency and the minor one being increased memory usage, but these improve constantly. When you say "performance" I take it to mean you don't really use or like Java, or you are repeating something that you heard from 1996.
            Please read my stuff again. I did not say go back to manual memory management or manually coding in C or C++. You use python code solutions that generate C and C++ for performance you are not coding C and C++ the automated tools are. The reality is you coded in python and generated C and C++.

            Originally posted by krzyzowiec View Post
            There is no alternative to Java that is not also making use of the JVM,
            This is not true. There are a few different JIT aned AOT for python that don't make use of JVM. You also have them for PHP and other languages.

            Python as a language brings a huge reduction in complexity and error as well. So java is not the only language that reduces how often your coders screw up.

            krzyzowiec something to beware of here is python also gives simpler allocation of memory. Even when you are using the different JIT options for python that are not JVM the memory usage does not explode. The java bytecode to native code is not that memory effective.

            numba is a JIT that is not JVM same with PYPY and quite a few others with python. Python you have a mix of options from AOT to JITs to slow cpython.

            The reality I would say that you are saying stuff that is not current. Java does not have the advantages it use to. The memory usage increase of java is not exactly minor either. JVM memory increase is very high once you start comparing it to python jit options.



            Stuff like numba does not require you to learn c or c++. JIT its not just the domain of java. Python you have a mix options without change language able to use what ever option is more suitable for the problem at hand.

            Python is growing because of its flexablity. Yes core python does still have some major problems.

            A tiny GIL-less python implementation (based on TinyPy), fully upgraded from C to C++11, and optimized for speed and portability to mobile platforms.


            I have not been going into the specialist pythons. I have been sticking to the pythons addons that you keep over 90% of the full standard python language.

            krzyzowiec something that should be of interest a lot of the python jit and aots avoid language bytecode as in straight to machine code or straight to likes of llvm bytecode that is platform particular. There may in fact be a fundamental flaw in the JVM idea.

            Comment


            • #76
              Originally posted by mdedetrich View Post
              I am sorry but none of these solutions are used in any serious scale and this is from speaking with countless Python developers. Even when multi-million corporations re-implement interpreted languages to be faster (i.e. PHP's hack or Dropbox's Pyston) have completely failed in getting any traction (Dropbox actually pulled the plug on Pyston and instead started using Go).
              Picking up where Dropbox left off, a new development team has released a fresh version of the just-in-time optimizing Python runtime, compatible with Python 3.8


              The interesting point is pyston did not get zero traction.
              We’re excited to announce that the Pyston team has joined Anaconda to continue the open-source development of their optimizing Python interpreter. Kevin Modzelewski and ‪Marius Wachtler will join Anaconda’s team of funded OSS developers, alongside our contributors to Dask, Numba, Bokeh, and many…

              At the 2021 Python Language Summit , Guido van Rossum gave a presentation about plans for making CPython faster. This presentation came rig...


              mdedetrich the reality here python has been re-implemented may times. Now those multi-million dollar companies that have done those re-implementations are looking to push that work mainline python. This is without going the JVM route. So jVM will have to face off against competition.

              Comment


              • #77
                Originally posted by oiaohm View Post
                mdedetrich the reality here python has been re-implemented may times. Now those multi-million dollar companies that have done those re-implementations are looking to push that work mainline python. This is without going the JVM route. So jVM will have to face off against competition.
                Yet many companies use the slowest main implementation of CPython which kind of proves they don't really care about performance, at all. They'd happily deploy faster Python runtime, but they don't have any skills nor resources to actually improve the code base. In fact, they could just as well work with any other language, but they've just wanted to come up with bogus claims for marketing reasons.

                Comment


                • #78
                  Originally posted by caligula View Post
                  Yet many companies use the slowest main implementation of CPython which kind of proves they don't really care about performance, at all. They'd happily deploy faster Python runtime, but they don't have any skills nor resources to actually improve the code base. In fact, they could just as well work with any other language, but they've just wanted to come up with bogus claims for marketing reasons.
                  There is something else here. Python is a very strict language if you don't format you code at least somewhat reasonable the code will not run.

                  Not having the skilled to improve the code base is not 100 percent the case. Python you see a lot lot developers from companies come out when there is a security fix. Not so much for performance fixes. So not having the skilled to improve the code base is not in fact true.



                  There is a lot of studies on software developer happyness and lack there of. The more unhappy developers are they more likely they will leave the company. What is the point of having the fasts possible solution if you have no developers to work on it.

                  Sorry to say you arguement it could be any language does not 100 percent hold up. Some of the reason why python is so high even that is so bad performing is how hard to is to write hard to read code in python. So on average python code is less bad to maintain. So a lower developer quit rate.

                  The problem here is if python does gain speed default Java not exactly in the best location.

                  Comment


                  • #79
                    Originally posted by oiaohm
                    krzyzowiec something that should be of interest a lot of the python jit and aots avoid language bytecode as in straight to machine code or straight to likes of llvm bytecode that is platform particular. There may in fact be a fundamental flaw in the JVM idea.
                    Do you know what JIT means? It’s when language bytecode is converted to machine code. When the JVM JIT is in action, you are not dealing with interpreted bytecode any longer, you are dealing with optimized machine code.

                    That’s how Java can perform on the same level as C/C++.

                    Comment


                    • #80
                      Originally posted by krzyzowiec View Post
                      Do you know what JIT means? It’s when language bytecode is converted to machine code. When the JVM JIT is in action, you are not dealing with interpreted bytecode any longer, you are dealing with optimized machine code.
                      Except with java that not 100 percent true. Its the problem with the java bytecode being part of the define language even that you have produced a machine code equal you cannot be 100 percent sure you will not have some query into the java bytecode different forms of java reflection actions expect this.

                      The way java bytecode is in fact used that information from it has to be accessable at runtime may explain some of the higher memory usage. Python when you are looking at JIT and AOT forms you are seeing that python is basically able to 100 percent ditch the source files and operate with the machine code alone. C, C++, Rust, Go. something interesting here they all have the means to ditch the source files.

                      Basically there is converting to machine code and ditching the source source files and there is converting to machine code and by design of your language you have to keep the source file around(java is the latter). This could be truly a fundamental flaw in the JVM design.

                      The reason why the bytecode was kept around at first with java was for runtime profiling and optimisation. That feature is better served by llvm bytecode or equal that is platform particular.

                      krzyzowiec there is more than 1 way to do JIT and java language design may be counter to use the best possible JIT options. If that is the case that Java design is counter to using the best JIT options then java does truly have a fundamental design flaw.

                      Comment

                      Working...
                      X