PathScale Open-Sources The EKOPath 4 Compiler Suite

[ninez]

Check the path64-dev mailing list. I think the patch was like 1-5 lines with .org blah blah in the subject. Honestly don't spend time on this unless

1) You're ready to dig into the boot issues you'll encounter afterwards
2) You're ready to file good bug reports so we can work with you in getting them fixed

Please don't take this as discouraging, but it's a lot of work before we can support building the kernel in a high quality manor.

thanks for the information. this is helpful. i'll have a look. Yes, I kind of suspected there might be some boot issues - Intel's C compiler was the exact same way. but with some effort i was able to get it to work (after a few frustrating failed attempts...lol)...and seeking help through the Intel software developers portal / forums.... (this was 2-3 years ago)

I didn't take your comments as discouraging at all. you are just being honest and realistic, and i appreciate that.. it would be silly to have the expectation of having no issues at all, when substituting GCC with any other compiler for compiling the linux kernel...

When i get some free time, i will probably give it a shot. Maybe i will use a VM, and I can probably provide a decent bug report. but i will do some more investigating first. right now, im just happy to be able to use pathCC. I've already re-compiled a few libraries and applications with it, without too many hassles.

thanks codestrom,

cheerz

[xming]

For those who wants to see the Open64 commit log from pathscale
(http://svn.open64.net/log.php?repname=Open64&path=%2F&)

Rev Log message Author Age Path
66 Bootstrap the PathScale branch from the pro64 sources. marcel 2035d 06h /
71 Import of PathScale, version 1.0

The PathScale sources have been pre-molded in the following way:
1. The CVS keywords have been normalized for this repository and do
not generate any diffs. For filenames this means that they have
osprey1.0 and not kpro64 as the first level directory even though
the file lives under kpro64.
2. The CVS Log keyword in kpro64/arith/edition.c has been removed
and the delta caused by it due to Fred Chow's initial import has
been reverted.
3. The cygnus_20030612 directory has been renamed to cygnus. This
to reduce diffs with the Pro64 compiler, which had cygnus_20000828
and that got renamed to cygnus before importing. Future PathScale
releases have renamed this directory to cygnus as well.

Diff-reduction has been applied in the following way:
1. Source files that only contained copyright assertions but no other
(non-whitespace) changes have been reverted.

This commit does not remove files that are part of Pro64 but are not
part of PathScale 1.0. There are files we do not want to lose, so this
takes some additional effort. Also, no changes have been made yet to
account for the cygnus_20030612->cygnus name change. This will all be
resolved in follow-up commits. marcel 2034d 09h /
75 Post-import fixups of the PathScale bits to account for the renaming of
cygnus_20030612 to (just) cygnus. marcel 2034d 07h /
77 Remove files not part of PathScale 1.0. Files under the following
directories were kept:
kpro64/common/com/ia64
kpro64/init
kpro64/kgnu_common/libio
kpro64/kgnu_common/libstdc++
kpro64/targia64
kpro64/targcygnus_ia32_ia64
kpro64/targia32_ia64_nodebug

The libio and libstdc++ directories under the kgnu_common directory
are kept out of conservatism. I'm not sure at this time they should
be nuked and since they're not in the way, I can defer removing them
until I do know. Though, they really shouldn't be there... marcel 2034d 05h /
78 Branched off the /projects/pathscale branch after the 1.0 import.
Used for development based on the 1.0 sources. marcel 2030d 09h /
83 Import of PathScale version 1.3. marcel 2023d 07h /
84 Assert PathScale copyright for changes made in version 1.0 for which
no copyright was asserted then, but for which copyright is asserted
now. marcel 2023d 07h /
85 Bootstrap the PathScale 1.3 sandbox. marcel 2023d 06h /
86 Import of PathScale, version 2.2.1 marcel 2022d 09h /
87 The GNU linker testsuite was removed from PathScale 1.3. marcel 2022d 08h /
88 kgnu_common was removed from PathScale 2.2.1
libelf and libiberty were moved a directory level up.
We previously kept libstc++ and libio, so we lost those
with this commit. marcel 2022d 08h /
89 Remove files not part of PathScale 2.2.1. Some probably weren't
even part of PathScale 1.3, but I forgot too clean that up before
importing 2.2.1. marcel 2022d 08h /
90 Branch off the PathScale 2.2.1 import. marcel 2022d 08h /
135 Import PathScale, version 2.2.1 (take 2). marcel 2010d 05h /
136 Remove some Pro64 left-overs that we should not need, especially
since pathScale doesn't need it. marcel 2009d 12h /

[staalmannen]

thanks for the information. this is helpful. i'll have a look. Yes, I kind of suspected there might be some boot issues - Intel's C compiler was the exact same way. but with some effort i was able to get it to work (after a few frustrating failed attempts...lol)...and seeking help through the Intel software developers portal / forums.... (this was 2-3 years ago)

I didn't take your comments as discouraging at all. you are just being honest and realistic, and i appreciate that.. it would be silly to have the expectation of having no issues at all, when substituting GCC with any other compiler for compiling the linux kernel...

When i get some free time, i will probably give it a shot. Maybe i will use a VM, and I can probably provide a decent bug report. but i will do some more investigating first. right now, im just happy to be able to use pathCC. I've already re-compiled a few libraries and applications with it, without too many hassles.

thanks codestrom,

cheerz

I wonder if the work done on the Clang-compiled Linux kernel could be of benefit also to a Path64 - compiled kernel?

https://github.com/lll-project

[ninez]

I wonder if the work done on the Clang-compiled Linux kernel could be of benefit also to a Path64 - compiled kernel?

https://github.com/lll-project

if i remember correctly, for that project doesn't clang itself have to be patched in order to make it work???

Also, from my own experiences compiling the linux kernel with different compilers, usually the set of problems from compiler to compiler are very different...but i'm no expert - just some experience here and there...

i actually had that kernel installed for fun (easy install on Archlinux using AUR). but it wasn't as good as the kernel i usually use, so i removed it.

[staalmannen]

if i remember correctly, for that project doesn't clang itself have to be patched in order to make it work???

Also, from my own experiences compiling the linux kernel with different compilers, usually the set of problems from compiler to compiler are very different...but i'm no expert - just some experience here and there...

i actually had that kernel installed for fun (easy install on Archlinux using AUR). but it wasn't as good as the kernel i usually use, so i removed it.

Good to know I thought that one problem with compiling Linux with alternative compilers was that the code assumed GCC-specific behaviour. If that was the case, I thought it was logical that the set of problems should be about the same for all other compilers (enforcing true standard C rather than de-facto standard GCC-type C). On the other hand, if each other compiler solves the compiling Linux problem by "behaving like GCC", I guess different compilers might end up with different sub-sets of that behaviour implemented...

Yay for more diversity
Now Linux can be compiled with ICC (although useless for redistribution due to license terms), Clang and Path64. Reportedly, it was once possible to compile Linux with TCC too (I have not tried myself if this is still possible). I suppose it is just a manner of time before PCC can do the same (it compiles FreeBSD already).

[ninez]

Good to know I thought that one problem with compiling Linux with alternative compilers was that the code assumed GCC-specific behaviour. If that was the case, I thought it was logical that the set of problems should be about the same for all other compilers (enforcing true standard C rather than de-facto standard GCC-type C). On the other hand, if each other compiler solves the compiling Linux problem by "behaving like GCC", I guess different compilers might end up with different sub-sets of that behaviour implemented...

Yay for more diversity
Now Linux can be compiled with ICC (although useless for redistribution due to license terms), Clang and Path64. Reportedly, it was once possible to compile Linux with TCC too (I have not tried myself if this is still possible). I suppose it is just a manner of time before PCC can do the same (it compiles FreeBSD already).

Last time i checked LinuxDNA is pretty much a dead-project. (these were the guys patching the kernel for the intel compiler). I belong to the google group, and the last releqase i saw was 2.6.36. and there isn't much activity going on. (at all really). the one guy who was giving help to the odd person using x86 (32bit) - decided one day to start demanding that people pay him the value of the Intel XE compiler suite (like $500USD at the time) for his help or patch. (it was a lot of work for him to patch every kernel release, but still not worth me shelling out more money than the laptop i was using at the time was worth...lol). that was sort of the beginning of the end for me using Intel's compiler...lol. (with the licensing issues being a problem also)..

I think ekopath is better anyway...so far i have found it to be more to my liking and not a huge adjustment to use. (less than ICC anyway - for most apps/libs i have tried so far). But i haven't tried the kernel or gimp yet - so we will see what happens.

..about different compilers - as someone else pointed out (maybe codestrom?) compilers are inherently very complex creatures. they also can be very different internally - in how they actually work - so i tend to think that they each handle compatibility with GCC in different ways, that is likely not necessarily that transferable...this is my feeling anyway. but again, i am no expert

[codestr0m]

I wonder if the work done on the Clang-compiled Linux kernel could be of benefit also to a Path64 - compiled kernel?

https://github.com/lll-project

Coincidently - Bryce who afaik did most of the work for getting LLVM to build/boot the linux kernel also helped us. We had pretty good progress before (thank Gautam), but he did the last mile. Other than experience I don't think anything else between the projects could be shared.

[codestr0m]

Last time i checked LinuxDNA is pretty much a dead-project. (these were the guys patching the kernel for the intel compiler). I belong to the google group, and the last releqase i saw was 2.6.36. and there isn't much activity going on. (at all really). the one guy who was giving help to the odd person using x86 (32bit) - decided one day to start demanding that people pay him the value of the Intel XE compiler suite (like $500USD at the time) for his help or patch. (it was a lot of work for him to patch every kernel release, but still not worth me shelling out more money than the laptop i was using at the time was worth...lol). that was sort of the beginning of the end for me using Intel's compiler...lol. (with the licensing issues being a problem also)..

I think ekopath is better anyway...so far i have found it to be more to my liking and not a huge adjustment to use. (less than ICC anyway - for most apps/libs i have tried so far). But i haven't tried the kernel or gimp yet - so we will see what happens.

..about different compilers - as someone else pointed out (maybe codestrom?) compilers are inherently very complex creatures. they also can be very different internally - in how they actually work - so i tend to think that they each handle compatibility with GCC in different ways, that is likely not necessarily that transferable...this is my feeling anyway. but again, i am no expert

If you think of a compiler as a front-end and a backend then most of the source level compatibility happens at the front-end. In the case of GCC it has also adds a bunch of extensions for developers to use. The path64 front-end is gcc-4.2 which makes the parsing/semantic analysis easy, but the problem comes for handling those extensions on the backend. In some cases it's trivial and we just haven't gotten around to it. (Maybe they are rarely used or worse a stupid idea :P) For the most part we handle the most common ones and as a result it's easier for people to migrate code from GCC to Path64/EKOPath. The linux kernel poses a couple additional challenges

1) Heavy GNU extension use (We're missing some here)
2) Heavy inline asm
3) Debugging the bootup process (It's not easy to debug why the SMP built kernel doesn't boot)

Another example is glibc. In this case making our GCC front-end emit IR that is correct for the backend was a major pita. With ENZO we've made *huge* changes around this area and now a lot easier. (GCC/GNU code is gone entirely and why you may see ENZO building an small linux distribution from the ground up before EKOPath)

[leidola]

Am I doing something wrong?

Hi!

I'm evaluating the speed of the pathscale compiler. So I downloaded the nightly version from their homepage and ran some programs I compiled using this very compiler and compared the speed against GCC's version of the compiled code. In most of the cases GCC was faster. This was my own private source code, heavily relying on the Mersenne-Twister-Algorithm. I tested Matsumoto's version (dSFMT) and Boost's version.

I took the Boost implementation of Mersenne-Twister as reference, as I thought there might be some gcc-specific tweaks in dSFMT.

#include <sys/time.h>
#include <iostream>

#include <boost/random/linear_congruential.hpp>
#include <boost/random/uniform_int.hpp>
#include <boost/random/uniform_real.hpp>
#include <boost/random/variate_generator.hpp>
#include <boost/random/mersenne_twister.hpp>

// This is a typedef for a random number generator.
// Try boost::mt19937 or boost::ecuyer1988 instead of boost::minstd_rand
typedef boost::mt19937 base_generator_type;

int main() {
   long nIt=1L<<32;
   double x=0;

   base_generator_type generator(42u);

   boost::uniform_real<> uni_dist(0,1);
   boost::variate_generator<base_generator_type&, boost::uniform_real<> > uni(generator, uni_dist);

   for (long i=0; i < nIt; i++) {
	x +=  uni();
   }

   std::cout << "x=" << x << '\n';
}

On my CPU the pathscale compiled code is more than two times slower than GCC 4.3 when executing this code compiled with "-O3 -msse2". By the way, the latest Intel Compiler perfomed similar to GCC.

That's why I ran the phoronix suite on my machine, this time GCC 4.5 vs. yesterday's nightly version of the Pathscale compiler.

I used the linux-system suite, however not every benchmark compiled. The results are here:

http://openbenchmarking.org/result/1106167-GR-1106168GR02

This is what I did to run the benchmarks:

export CFLAGS="-O3 -msse2"
export CXXFLAGS="-O3 -msse2"
export CC=pathcc
export CXX=pathCC
export FFC=pathf90
export F90=pathf90
export F95=pathf95
export cc=pathcc
export cxx=pathCC
export ffc=pathf90
export f90=pathf90
export f95=pathf95

rm -Rf $HOME/.phoronix-test-suite/installed-tests/pts/
phoronix-test-suite batch-install pts/linux-system
phoronix-test-suite batch-run pts/linux-system

And then the same with GCC

export CFLAGS="-O3 -msse2"
export CXXFLAGS="-O3 -msse2"
export CC=gcc
export CXX=g++
unset FFC
unset F90
unset F95
export cc=gcc
export cxx=g++
unset ffc
unset f90
unset f95

rm -Rf $HOME/.phoronix-test-suite/installed-tests/pts/

phoronix-test-suite batch-install 1106168-GR-PATHSCALE46
phoronix-test-suite batch-run 1106168-GR-PATHSCALE46

Maybe I'm doing something wrong, I don't know. Comments are welcome.

[mlau]

@codestr0m:

do you also support other architectures, specifically MIPS (mips32) ?