A Proposal To Allow Python Scripting Within The GCC Compiler, Replacing AWK

[caligula]

The advantage of awk is that it's installed on virtually every Linux distro, has no shitty version incompatibility (python2 vs python3), and is lightweight.

Why would this be a problem? Of course you'd pick Python 2 these days.

[Weasel]

Well, from experience... I build complete distros for x86_64, i686, arm and aarch64. I have no problem compiling gcc on any of those platforms and that would not change if they require Python. As far as keeping my Python up to date, I change the version number in my build script and recompile. I have not had issues compiling Python since way back when it was young. It just works.

I was talking in general there, not specifically to this situation.

Someone mentioned version incompatibities between Python2/Python3. There is only one version to program for, Python3. Projects that are still dependant on Python2 are mostly not being maintained. Python2 development is ending at the end of the year.

And some projects like Wine still use C89 which "ended" in 1999 with C99, so your point is? Only that C99 is backwards compatible with it, unlike Python.

Imagine for a moment that they wrote the awk scripts in Python2 when it was the coolest new thing. According to you, since GCC is maintained, they would have to rewrite them for Python3.

Pure and simple, extra burden compared to a good language like AWK. You don't have to rewrite awk code as it's (mostly) backwards compatible (unless you use special GNU extensions or w/e, maybe).

With this precedent, who said there won't be a python 4? Imagine if they write so many build scripts on it and then they'll have to rewrite them, again and again. What a nightmare.

You can't blame the Python developers for making improvements/progress.

Unicode-by-default is not progress it's a nightmare for many situations especially on Linux. And yes you can totally blame them for that when they make people rewrite their stuff.

[Palu Macil]

I know a lot of Linux admins that have not learned awk. I believe they all respect it, but with all the relevant languages and tools to learn, learning one of the less used ones might seem inefficient. Personally I don't think I can maintain a high level of rapid efficiency in more than my five or six languages most used languages. If a project is written in a language that I have no other use for, I will probably spend my time contributing somewhere else where I can combine efficiencies.

The tooling around debugging Python and variety of quality IDEs is unparalleled, and the developer pool is deep. The disadvantages are certainly as you state them, but I don't think the impact is very large. I do think the benefit is large.

One more item where I might disagree with you is in regards to python being no different than any other language for text manipulation. I personally find it to be extraordinary comfortable for manipulating text. Perhaps that is a combination of a few things, but despite it being my 5th or 6th most used language, it is probably my favorite for text manipulation.

[kpedersen]

I have recently had the experience of building both GCC and Clang on Windows to build Emscripten/fastcomp.

I have to say that Clang was much easier due to its CMake build system. GCC is sticking with autotools (which isn't an issue on UNIX-like platforms) but is already extremely fiddly.

Lua would be my preferred choice. It is < 40 .c files and requires no other dependencies than C99.
And it wouldn't need maintaining. Just call it GCC-lua and leave it frozen at version 5.3. This constant keeping up with upstream versions is killing FOSS. Just look at Gtk+.

[Redfoxmoon]

Adding python as a hard dependency to GCC would make it an even worse mess to compile, *especially* on mingw-w64. Christ WTF?

[cochise]

IMHO, if some people likes AWK so much, they should step up an maintain the AWK code. If who is doing he job wants to change to another language it's fine, as they are doing the job. Is not that "the person should learn AWK", but "the person who do the job chooses the tools".
We need some compromises, as it is a collective work and the toll affect the final bundle, but this is what the developer is doing starting a discussion in the mailing list.

[kenjitamura]

Pure and simple, extra burden compared to a good language like AWK. You don't have to rewrite awk code as it's (mostly) backwards compatible (unless you use special GNU extensions or w/e, maybe).

With this precedent, who said there won't be a python 4? Imagine if they write so many build scripts on it and then they'll have to rewrite them, again and again. What a nightmare.

There WILL be a python 4 however a core developer had this to say about it:

My current expectation is that Python 4.0 will merely be "the release that comes after Python 3.9". That's it. No profound changes to the language, no major backwards compatibility breaks - going from Python 3.9 to 4.0 should be as uneventful as going from Python 3.3 to 3.4 (or from 2.6 to 2.7). I even expect the stable Application Binary Interface (as first defined in PEP 384) to be preserved across the boundary.

At the current rate of language feature releases (roughly every 18 months), that means we would likely see Python 4.0 some time in 2023, rather than seeing Python 3.10.

Update: After this post was originally written back in 2014, subsequent discussions on the core python-dev mailing list led to the conclusion that the release after 3.9 will probably just be 3.10. However, a 4.0 will presumably still happen some day, and the premise of this article is expected to hold for that release: it will be held to the same backwards compatibility obligations as a Python 3.X to 3.X+1 update.

I don't think they dare break backward compatibility again because it was so much of a nightmare and generated bad press they realize will never go away.

[hgoldfish]

GCC is a so heavy project, that we do not need to consider how heavy python is. And python 2.x is gone.

[Filiprino]

I don't have much experience with AWK and Python. But I know they are two languages with two different purposes.
Python is more of an object oriented programming language like Java, GNU R or JavaScript.
On the other hand AWK is a language oriented to text processing. It's simple, it has a reduced set of commands which are handy for string processing.

http://www.grymoire.com/Unix/Awk.html#uh-0

Why is AWK so important? It is an excellent filter and report writer. Many UNIX utilities generates rows and columns of information. AWK is an excellent tool for processing these rows and columns, and is easier to use AWK than most conventional programming languages. It can be considered to be a pseudo-C interpretor, as it understands the same arithmatic operators as C. AWK also has string manipulation functions, so it can search for particular strings and modify the output. AWK also has associative arrays, which are incredible useful, and is a feature most computing languages lack. Associative arrays can make a complex problem a trivial exercise.

AWK in GCC is used to parse files with constants, configuration flags and ISA definitions. Using Python for that I think it would add a dependency which doesn't pay off.

Other scripts use AWK but probably shouldn't be written in AWK. For example, this script pointed out in the mail list: https://gcc.gnu.org/git/?p=gcc.git;a...en.awk;hb=HEAD

There's a case for using an OO language there. But instead of Python, why don't you use Smalltalk? https://news.ycombinator.com/item?id=9023138
Smalltalk has a GNU implementation, you are still using GNU packages, something which I think is one of the purposes of GNU, having a GNU system (except X11 :P). Smalltalk influenced Python and many other languages. Was the first OO language to provide reflection.
And probably one great quality of Smalltalk is that it has been standardised. It is an ANSI standard.

But if one goes back to the basics, why don't use AWK and build data structures providing the needed functionality.
AWK even allows you to use TCP/IP: https://www.gnu.org/software/gawk/ma.../gawkinet.html

The copy&pasted code of the AWK script mentioned above could be replaced with a function and a single parameter. I think that the script was made "quick&dirty" and who knows if they used Yasnippet from Emacs or whatever :P
Maybe instead of replacing one language by another and adding an additional dependency the current state of the scripts can be improved (refactored) in order for them to be more maintainable.

Other questions arise, like Python runtime use of resources compared to AWK. The other day there was a post saying developers did not have enough memory in their server.

[Weasel]

Adding python as a hard dependency to GCC would make it an even worse mess to compile, *especially* on mingw-w64. Christ WTF?

Exactly my point. GCC is the only compiler I use even on Windows and yes I patch it also to enable plugins so I have to compile it myself.