HOPE: The Ease Of Python With The Speed Of C++

[alcalde]

c++'s core spec is about two times bigger than c. and same as c# and java, so please stop spreading bullshit
there was good parts book about javascript earlier and somehow millions of not really programmers are using it
python is designed from ground up for slowness

It's not "bullshit", it's the consensus opinion. Just do a Google search for "C++ too complex" and you get back a zillion hits and blog articles discussing the very topic.

http://www.werkema.com/tag/c/

When did C++ get so out-of-hand? Once upon a time, it was C, with classes. Neither more nor less complicated than that. It had C for getting stuff done and classes for keeping the stuff organized. Its current wranglers seem so intent on turning it from C-with-classes into a sophisticated exercise in type theory that the language no longer seems to actually do anything. In C, you can?t write more than a few lines of code without writing a line that at least does something ? it computes something, performs an operation, moves some data around, crunches some numbers. In C++, you can write ? quite literally ? thousands of lines of template classes that do absolutely nothing whatsoever but push types around without ever getting around to actually, y?know, solving a real-world problem.

Jamie Zawinski fought to keep C++ out of Netscape:

C++ is just an abomination. Everything is wrong with it in every way. So I really tried to avoid using that as much as I could and do everything in C at Netscape.
... When you?re programming C++ no one can ever agree on which ten percent of the language is safe to use. There?s going to be one guy who decides, ?I have to use templates.? And then you discover that there are no two compilers that implement templates the same way.

Confirming what I wrote about business only using parts of C++:

{Joshua BlochI think C++ was pushed well beyond its complexity threshold and yet there are a lot of people programming it. But what you do is you force people to subset it. So almost every shop that I know of that uses C++ says, ?Yes, we?re using C++ but we?re not doing multiple-implementation inheritance and we?re not using operator overloading.? There are just a bunch of features that you?re not going to use because the complexity of the resulting code is too high. And I don?t think it?s good when you have to start doing that. You lose this programmer portability where everyone can read everyone else?s code, which I think is such a good thing.

Over and over in the interview book Coders At Work you find the same refrain, even from Ken Thompson:

It certainly has its good points. But by and large I think it?s a bad language. It does a lot of things half well and it?s just a garbage heap of ideas that are mutually exclusive. Everybody I know, whether it?s personal or corporate, selects a subset and these subsets are different. So it?s not a good language to transport an algorithm?to say, ?I wrote it; here, take it.? It?s way too big, way too complex. And it?s obviously built by a committee.
Stroustrup campaigned for years and years and years, way beyond any sort of technical contributions he made to the language, to get it adopted and used. And he sort of ran all the standards committees with a whip and a chair. And he said ?no? to no one. He put every feature in that language that ever existed. It wasn?t cleanly designed?it was just the union of everything that came along. And I think it suffered drastically from that.

More at http://gigamonkeys.wordpress.com/200...s-c-plus-plus/

So, no, it's not some sort of crazy notion that C++ is complex. Heck, it has a separate template language! Between needing to be backwards compatible with C and being designed by a committee, you simply can't get a simple, clean language.

>python is designed from ground up for slowness

From what I've read on Guido Van Rossum's blog about the history of Python, "slowness" was never a design criterion. Readability is and changes that would make small improvements to speed at the expense of readability are routinely rejected. Given that a survey of top colleges has just shown that Python has pulled in front of Java as the most-used language for CS101 classes, it seems he did a good job of keeping his language contained.

[erendorn]

It's not "bullshit", it's the consensus opinion. Just do a Google search for "C++ too complex" and you get back a zillion hits and blog articles discussing the very topic.

http://www.werkema.com/tag/c/

Jamie Zawinski fought to keep C++ out of Netscape:

Complaining about template compiler support is sooo 1990.

Confirming what I wrote about business only using parts of C++:
Over and over in the interview book Coders At Work you find the same refrain, even from Ken Thompson:
More at http://gigamonkeys.wordpress.com/200...s-c-plus-plus/

Absolutely. But this is completely assumed, here are part of the "C++ philosophy" (wikipedia paraphrasing Stroustrup):

Programmers should be free to pick their own programming style, and that style should be fully supported by C++.
Allowing a useful feature is more important than preventing every possible misuse of C++.

C++ does not prevent misuse. That's a fact a bad and a good one. It's something some people hate, and some people love, no wonder you can find endless quotes of people complaining of it (and as many people praising it, although they tend not to speak up as much). There's nothing wrong about using only a subset of it. You simply use the subset most adapted to you problem at hand. The good thing is, if you have different problems in your code base, you still get to use a single language for the different styles.

[TheBlackCat]

of course it does

Assume that the string is UTF-8. What are the types of counts and ch?

[gamerk2]

C+ allows developers to get things done without holding their hands. C++ will let you do just about anything, but it won't hold your hand to protect you if you don't know what you're doing. It's really that simple.

[mmstick]

I did miss that. Although it only topples C for 2, and loses for 4.


Because I have enough work to do submitting patches for actual features to software I actually use on top of my normal day job to spend time submitting improved benchmarks for software I don't use and don't particularly care about the benchmarks for. Again, performance is not my primary concern. Getting my work done is. And the performance of the glue language I use is not a major limiting factor for that.



You can do that in python too, with the added benefit of not confusing function calls or numeric assignment with anonymous function assignment. Matlab made a similar mistake with confusing array indexing with function calls which led to no end of problems. I am disappointed to see Julia doing something similar. That sort of thing simply would not be allowed in python, since making sure the code is clear and unambiguous is a major concern.

Whatever the case, those are all at least as easy in python:

# gratio returns the value of the golden ratio.
gratio = lambda: (1 + 5**0.5) / 2
# Or you could do the sane thing and just assign a float:
gratio = (1 + 5**.5)/2

# fib prints the Nth fibonacci number(s), also works for arrays
fib = lambda n: (gratio**n - (1-gratio)**n) / 5**0.5

# squared squares the input, also works for arrays
squared = lambda x: x**2

# average returns the average, also works for arrays
average = lambda x: sum(x) / len(x)

# digits returns the number of digits in the number.
from math import trunc, log10
digits = lambda x: trunc(log10(x)) + 1

Now try doing this in Julia. Assume "readdata" is some arbitrary data-reading function. This will work on any platform with any directory separator:

import os
from os.path import join, splitext
# read all the data files in the directory and all subdirectories and store them in a dictionary (mapping) based on their path if their file extension is ".h5", if not skip them
res = {}
for path, _, filenames in os.walk('.'):
    for fname in filenames:
        if splitext(fname)[1] == ".h5":
            fullname = join(path, fname)
            res[fullname] = readdata(fullname)

# or an equivalent, but less clear, one-liner
res = {join(path, fname): readdata(join(path, fname)) for path, _, filenames in os.walk('.') for fname in filenames if splitext(fname)[1] == ".h5"}

# or an anonymous function for an arbitrary directory and extension:
res = lambda dirname, ext: {join(path, fname): readdata(join(path, fname)) for path, _, filenames in os.walk(dirname,) for fname in filenames if splitext(fname)[1] == ext}
res('.', '.h5')

Or what if you wanted to scrape data from tables on a web page? Or access data stored on an online resource like google docs? Or plot multiple interactive graphs where the dragging one plot or one marker also moved others? Or access multiple I/O bound tasks asynchronously? Despite your criticism of Python not being focused enough on numerical computing, these sorts of things are easy to do in Python specifically because it is explicitly designed and used as a general-purpose programming language. Things like advanced HTML handling, advanced network interfaces, advanced graphics toolkit interaction, and advanced asynchronous I/O handling were not created for numerical computing, and you don't see them on languages designed primarily for numerical computing, but people doing numerical computing benefit from work like this done for other purposes.

Considering you claim to have had the time to look at the benchmark and criticize their code, you have the time to write a simple patch to improve their code -- unless you are just blowing hot air. Julia has good support for processing text in files so that's not really a problem. Furthermore, considering Julia integrates with C and Python, you can use Python/C code inside Julia in case Julia's standard library is missing a feature you must have -- which is highly unlikely. What you seem to be unable to understand is that this entire thread is not about general purpose programming -- this is scientific computing. Julia is what you will use to write simulations. If you want to argue about general purpose languages, C/C++/Java/Python/etc. have nothing on Go and Rust.

[TheBlackCat]

Considering you claim to have had the time to look at the benchmark and criticize their code, you have the time to write a simple patch to improve their code -- unless you are just blowing hot air.

Taking a glance at the code is much quicker than cloning the git repo, not to mention installing Julia and all the other code for the benchmarks, compiling the code, running the benchmarks, writing a commit, writing a pull request, answering questions, etc. Have you ever actually submitted a patch to a project? I have submitted hundreds. It is not a few-second affair like looking at a few line function is even in a non-compiled language like Julia or Python. It is longer still when using a compiled language.

Julia has good support for processing text in files so that's not really a problem.

Did you even look at my description of what the code does? The problem isn't processing text files. Those weren't even text files, they were HDF5 files, which Julia can also handle but that isn't the point. The problem is recursively walking the filesystem hierarchy and handling only the files you want.

Furthermore, considering Julia integrates with C and Python, you can use Python/C code inside Julia in case Julia's standard library is missing a feature you must have -- which is highly unlikely.

Three problems with that:

1. I just pointed out an example of something Julia's standard library doesn't have.
2. This isn't about the standard library. Did you even look at my reasons for using Python over Julia? #1 was the much larger community and much larger set of available third-party libraries. I couldn't care less whether it is in the standard library or in a third-party library, all I care about is whether the tools are available to make my work easier.
3. I can call Julia from Python, too. That means in the 99.99% of my cases where code readability and library availability are more important than script speed, I can use Python, and write the rare slow loop in Julia (or C).

What you seem to be unable to understand is that this entire thread is not about general purpose programming -- this is scientific computing. Julia is what you will use to write simulations. If you want to argue about general purpose languages, C/C++/Java/Python/etc. have nothing on Go and Rust.

Now I know you didn't read my description of what the code does or why it does it. Hint: that was code for dealing with scientific data.

And what about my answer to your question? You ask how Julia code would be written in Python, then just ignore the answer? If you didn't care what the code would look like in Python, why did you ask?

I am done here. I thought we could have a reasonable discussion on the topic, but if you can't be bothered to actually read what I wrote, not to mention address answers to your own questions, then you obviously don't want to have a discussion, you just want to push Julia no matter what.

That is understandable, Julia will live or die based on the size and quality of its community, and that community is still extremely small compared to every competing language. Converting people is the only way Julia will survive. But you aren't going to do that by just sticking your head in the sand and ignoring the reasons people aren't already using it and just screaming "performance, performance, performance" over and over, not to mention pissing people off by not reading what they wrote and instead responding to some imaginary post from your own head.

[mmstick]

Taking a glance at the code is much quicker than cloning the git repo, not to mention installing Julia and all the other code for the benchmarks, compiling the code, running the benchmarks, writing a commit, writing a pull request, answering questions, etc. Have you ever actually submitted a patch to a project? I have submitted hundreds. It is not a few-second affair like looking at a few line function is even in a non-compiled language like Julia or Python. It is longer still when using a compiled language.


Did you even look at my description of what the code does? The problem isn't processing text files. Those weren't even text files, they were HDF5 files, which Julia can also handle but that isn't the point. The problem is recursively walking the filesystem hierarchy and handling only the files you want.


Three problems with that:

1. I just pointed out an example of something Julia's standard library doesn't have.
2. This isn't about the standard library. Did you even look at my reasons for using Python over Julia? #1 was the much larger community and much larger set of available third-party libraries. I couldn't care less whether it is in the standard library or in a third-party library, all I care about is whether the tools are available to make my work easier.
3. I can call Julia from Python, too. That means in the 99.99% of my cases where code readability and library availability are more important than script speed, I can use Python, and write the rare slow loop in Julia (or C).

Now I know you didn't read my description of what the code does or why it does it. Hint: that was code for dealing with scientific data.

And what about my answer to your question? You ask how Julia code would be written in Python, then just ignore the answer? If you didn't care what the code would look like in Python, why did you ask?

I am done here. I thought we could have a reasonable discussion on the topic, but if you can't be bothered to actually read what I wrote, not to mention address answers to your own questions, then you obviously don't want to have a discussion, you just want to push Julia no matter what.

That is understandable, Julia will live or die based on the size and quality of its community, and that community is still extremely small compared to every competing language. Converting people is the only way Julia will survive. But you aren't going to do that by just sticking your head in the sand and ignoring the reasons people aren't already using it and just screaming "performance, performance, performance" over and over, not to mention pissing people off by not reading what they wrote and instead responding to some imaginary post from your own head.

Taking a glance at their single file benchmark in github, click the edit button and then pressing submit after fixing it does not take more than a few minutes -- you're just trying to make stupid excuses. Julia isn't missing anything from it's standard library and it already has a large community -- I don't see how you can think something is missing when what you are looking for exists. Want to go through a list of files in a directory? Use the readdir function. There's wrappers for every library you could ever need for scientific computing, even for OpenCL. Who do you think created Julia? You can try to argue that Python has an irrelevantly-sized community, or that C has an irrelevantly-sized community, but the fact is that Julia was designed to interopt with Python and C and all libraries relevant to scientific computing already have Julia wrappers; therefore your argument is irrelevant.

[TheBlackCat]

Taking a glance at their single file benchmark in github, click the edit button and then pressing submit after fixing it does not take more than a few minutes -- you're just trying to make stupid excuses. Julia isn't missing anything from it's standard library and it already has a large community -- I don't see how you can think something is missing when what you are looking for exists. Want to go through a list of files in a directory? Use the readdir function. There's wrappers for every library you could ever need for scientific computing, even for OpenCL. Who do you think created Julia? You can try to argue that Python has an irrelevantly-sized community, or that C has an irrelevantly-sized community, but the fact is that Julia was designed to interopt with Python and C and all libraries relevant to scientific computing already have Julia wrappers; therefore your argument is irrelevant.

Thank you for again proving my point. Your stubborn refusal to actually read what I wrote, not to mention address it, is not helping convince me. Good day.

[brk0_0]

I did miss that. Although it only topples C for 2, and loses for 4.

Python loses for Julia and C for all of them.

Because I have enough work to do submitting patches for actual features to software I actually use on top of my normal day job to spend time submitting improved benchmarks for software I don't use and don't particularly care about the benchmarks for. Again, performance is not my primary concern. Getting my work done is. And the performance of the glue language I use is not a major limiting factor for that.

Writing the code takes less time than posting here. You should try. Also, for scientific computing and for any application that takes a huge amount of resources, performance matters. It's not your primary concern, but it is for many developers. The same is valid for glue code (that also benefits from neater Julia's syntax).

You can do that in python too, with the added benefit of not confusing function calls or numeric assignment with anonymous function assignment. Matlab made a similar mistake with confusing array indexing with function calls which led to no end of problems. I am disappointed to see Julia doing something similar. That sort of thing simply would not be allowed in python, since making sure the code is clear and unambiguous is a major concern.

You should confuse it, because on Julia they are exactly the same thing! That not even close to be ambiguous. Array indexing on MATLAB, in that aspect, is a completly different case. Next time, before posting things like this, check the Julia Documentation.

...Or plot multiple interactive graphs where the dragging one plot or one marker also moved others?

https://github.com/JuliaLang/Interact.jl

Take a look at the examples on the Notebooks.

...Or access multiple I/O bound tasks asynchronously?

https://github.com/JuliaLang/libuv

Again, read the docs.

Despite your criticism of Python not being focused enough on numerical computing, these sorts of things are easy to do in Python specifically because it is explicitly designed and used as a general-purpose programming language. Things like advanced HTML handling, advanced network interfaces, advanced graphics toolkit interaction, and advanced asynchronous I/O handling were not created for numerical computing, and you don't see them on languages designed primarily for numerical computing, but people doing numerical computing benefit from work like this done for other purposes.

Also, Julia is a general programming language. What led you to think it's not? Many of these features (Graphic Toolkits, Network Interfaces) are beign developed as we speak. The others can be implemented as well, and we need someone interested in developing that. Could you help us? Finally, at the same time Scientific Computing would benefit from all these features, all other areas benefit from performance, better interoperability, neater package organization (omg, that's neat), metaprogramming, etc.

Some nice bugs that will be squashed before v1.0:

Expressiveness

https://github.com/JuliaLang/julia/issues/4470

https://github.com/JuliaLang/julia/issues/550

Internal

https://github.com/JuliaLang/julia/issues/5155

https://github.com/JuliaLang/julia/pull/8791 (see Markdown.jl for usage example)

Performance

https://github.com/JuliaLang/julia/issues/3440

https://github.com/JuliaLang/julia/issues/261

https://github.com/JuliaLang/julia/issues/4373

https://github.com/JuliaLang/julia/pull/8745

The nice thing in Julia is that many apples are hanging low for us to catch.

[TheBlackCat]

Python loses for Julia and C for all of them.

I never claimed otherwise. I was specifically addressing mmstick's claims about C. I have said many times my primary reason for using python is not performance.

Also, for scientific computing and for any application that takes a huge amount of resources, performance matters. It's not your primary concern, but it is for many developers.

That is great for them. Again, I have said repeatedly that there different people have different needs and different workflows, and people should use the programming language that best suits their needs. If that is Julia, that is fine. If that is C, that is fine. It isn't for me.

You should confuse it, because on Julia they are exactly the same thing!

That is exactly the problem. But this is a philosophical argument about how languages should be structured, I don't think we are likely to convince each other on this one.

https://github.com/JuliaLang/Interact.jl

Take a look at the examples on the Notebooks.

From the Readme, it doesn't appear that does either of the things I listed.

https://github.com/JuliaLang/libuv

Again, read the docs.

Fair enough. However, I did search google for that and that didn't come up. All I could find was Julia's documentation talking about how asychonrous IO wasn't supported.

Also, Julia is a general programming language. What led you to think it's not?

I was responding to mmstick's argument that Julia is better than Python or C because it isn't a general-purpose programming language. If you have a problem with that characterization of the language, please take it up with mmstick.

But as for where I might get such an idea, look at the first part of the first sentence on the front page of Julia's website:

Julia is a high-level, high-performance dynamic programming language for technical computing

(emphasis added)

Many of these features (Graphic Toolkits, Network Interfaces) are beign developed as we speak. The others can be implemented as well, and we need someone interested in developing that.

Right, and that is reason #1 on my reasons why I use python. There are many more tools available, and those tools are much more mature.

Could you help us?

Sorry, as I said, I don't have the time. There are only so many hours in the day, I have a job, a family. The free time I have is taken either relaxing, or working on a language where the low-level tools I need are already available and I can focus on the rare niches specific to my job. I need to get work done now, so I need a language that has mature versions of the tools I need to make my work easier and more reliable. I also need tools that are compatible with what other people in my field are using (which is either Matlab or Python).

Perhaps if it was 10 years ago and I didn't have a wife, didn't have a baby, didn't have a job that has specific needs, I might have been able to help. But right now I need to focus my attention on work that will give me the most net benefit. And trying to build up the low-level tools for a language so that it might, one day, support the high-level needs I have now, does not give me much net benefit. This is especially when there is a language that already has those tools.

The nice thing in Julia is that many apples are hanging low for us to catch.

It may be nice for you, but for someone like me where I need to get my work done, it really gets in the way. I want a language where there is very little low-hanging fruit, where I don't need to submit many patches to get it to do what I need it to do.