Announcement

Collapse
No announcement yet.

What happened to Pathscale's open source path64 compiler suite?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • What happened to Pathscale's open source path64 compiler suite?

    Quite some time ago they deleted it from their github page at https://github.com/pathscale/path64-suite and/or https://github.com/path64/compiler.

    The website http://www.path64.org/ had its last update 6 years ago and doesn't say what happened.

    In fact with google I can find no news at all about the project being abandoned.

    I only found community (?) forks with not much activity either: https://github.com/Flameeyes/path64-suite/network

    Still, the nightly build of the closed source version of the ekopath compiler suite is constantly updated: http://www.pathscale.com/ekopath-compiler-suite
    (but the link isn't updated. You have to build the link yourself with the date like this:
    http://c591116.r16.cf2.rackcdn.com/e...-installer.run
    and just go a day back until you find the latest available file. Here's a python script that I use in my AUR package to return the latest download link: https://aur.archlinux.org/cgit/aur.g...?h=ekopath-bin)

    codestr0m Are you still here? What happened to path64?

    Edit: So I tried to compile mesa with ekopath-bin-20160624-1-x86_64.pkg.tar.gz.

    First, remove these two flags from mesa's configure:
    sed -i "s/-fno-trapping-math//g" configure
    sed -i "s/-Qunused-arguments//g" configure

    then, run configure without arguments and remove these two flags from these Makefiles:

    sed -i "s/-Wno-override-init//g" src/intel/vulkan/Makefile
    sed -i "s/-Wno-override-init//g" src/intel/isl/Makefile

    Also need to remove
    assert(expr);
    __builtin_assume(expr);
    from src/util/macros.h

    then, run make. It builds for a while, but then psclang segfaults:
    Code:
     CC       drivers/common/meta.lo
    psclang: error: unable to execute command: Segmentation fault (core dumped)
    psclang: error: clang frontend command failed due to signal (use -v to see invocation)
    clang version 3.7.0 (/root/cb_agent/cache/git/psclang-ng.git 1189e22d3c4a033b4f0296d8e1ab1f624602f66d) (/root/cb_agent/cache/git/psllvm.git 209fd5171479feced5594de1fa1c58a92beef735)
    Target: x86_64-unknown-linux-gnu
    Thread model: posix
    psclang: note: diagnostic msg: PLEASE submit a bug report to http://support.pathscale.com/ and include the crash backtrace, preprocessed source, and associated run script.
    psclang: error: unable to execute command: Segmentation fault (core dumped)
    psclang: note: diagnostic msg: Error generating preprocessed source(s).
    Eh, not really feeling like submitting a bug report to a proprietary only (?) compiler. They can figure it out themselves. Here, a backtrace. Without proper debugging information, because, you know, there is no source code to build the compiler with it:
    Code:
    Core was generated by `/usr/lib/6.0.844/x8664/psclang -cc1 -plugin whirl-print -triple x86_64-unknown-'.
    Program terminated with signal SIGSEGV, Segmentation fault.
    #0  0x0000000001469424 in Whirl::WhirlWnImpl::createConversionWn(Whirl::OPERATOR, unsigned char, Whirl::WN const*) ()
    (gdb) bt
    #0  0x0000000001469424 in Whirl::WhirlWnImpl::createConversionWn(Whirl::OPERATOR, unsigned char, Whirl::WN const*) ()
    #1  0x000000000137fed5 in Whirl::WhirlPrinterImpl::buildConvertingCast(Whirl::WN const*, clang::QualType, clang::QualType) ()
    #2  0x00000000013d8a5f in ?? ()
    #3  0x00000000013d8709 in ?? ()
    #4  0x00000000013d0ad6 in ?? ()
    #5  0x00000000013cf98e in Whirl::WhirlPrinterImpl::buildRValue(clang::Expr const*, Whirl::WnAcceptor*) ()
    #6  0x0000000001384ab1 in Whirl::WhirlPrinterImpl::buildCallArgWn(clang::QualType, clang::Expr const*, Whirl::WnAcceptor*) ()
    #7  0x0000000001384b6a in Whirl::WhirlPrinterImpl::buildCallArgList(clang::FunctionProtoType::ExtProtoInfo const&, clang::QualType const*, clang::QualType const*, clang::ConstExprIterator,
    clang::ConstExprIterator, std::__1::vector<Whirl::WN const*, std::__1::allocator<Whirl::WN const*> >*, Whirl::WnAcceptor*) ()
    #8  0x0000000001384ce3 in Whirl::WhirlPrinterImpl::buildCallArgList(clang::FunctionType const*, clang::ConstExprIterator, clang::ConstExprIterator, std::__1::vector<Whirl::WN const*, std::_
    _1::allocator<Whirl::WN const*> >*, Whirl::WnAcceptor*) ()
    #9  0x00000000013f2916 in ?? ()
    #10 0x00000000013cee16 in Whirl::WhirlPrinterImpl::buildCallExprRValue(clang::CallExpr const*, Whirl::WnAcceptor*, Whirl::WN const*) ()
    #11 0x00000000013d9d3a in ?? ()
    #12 0x00000000013d097a in ?? ()
    #13 0x00000000013cf98e in Whirl::WhirlPrinterImpl::buildRValue(clang::Expr const*, Whirl::WnAcceptor*) ()
    #14 0x00000000013f6c81 in Whirl::WhirlPrinterImpl::buildExprStmt(clang::Expr const*, Whirl::WnAcceptor*) ()
    #15 0x0000000001433f43 in ?? ()
    #16 0x00000000013f68ce in Whirl::WhirlPrinterImpl::buildStmtWNs(clang::Stmt const*, Whirl::WnAcceptor*) ()
    #17 0x0000000001434aa2 in ?? ()
    #18 0x00000000013f68ce in Whirl::WhirlPrinterImpl::buildStmtWNs(clang::Stmt const*, Whirl::WnAcceptor*) ()
    #19 0x00000000013f8197 in ?? ()
    #20 0x0000000001432712 in ?? ()
    #21 0x00000000013f68ce in Whirl::WhirlPrinterImpl::buildStmtWNs(clang::Stmt const*, Whirl::WnAcceptor*) ()
    #22 0x0000000001434aa2 in ?? ()
    #23 0x00000000013f68ce in Whirl::WhirlPrinterImpl::buildStmtWNs(clang::Stmt const*, Whirl::WnAcceptor*) ()
    #24 0x000000000142c96c in ?? ()
    #25 0x00000000013f68ce in Whirl::WhirlPrinterImpl::buildStmtWNs(clang::Stmt const*, Whirl::WnAcceptor*) ()
    #26 0x0000000001434aa2 in ?? ()
    ---Type <return> to continue, or q <return> to quit---
    #27 0x00000000013f68ce in Whirl::WhirlPrinterImpl::buildStmtWNs(clang::Stmt const*, Whirl::WnAcceptor*) ()
    #28 0x00000000013f699f in Whirl::WhirlPrinterImpl::buildFunctionBodyWNs(Whirl::DST_IDX, clang::Stmt const*, Whirl::WnAcceptor*) ()
    #29 0x0000000001378f56 in Whirl::WhirlPrinterImpl::PrintGlobalFunction(clang::GlobalDecl) ()
    #30 0x00000000013718a9 in Whirl::WhirlPrinterImpl::PrintFunction(clang::FunctionDecl const*) ()
    #31 0x000000000137b7cb in Whirl::WhirlPrinterImpl::PrintDecl(clang::Decl*) ()
    #32 0x0000000001391d58 in WhirlPrinterConsumer::HandleTranslationUnit(clang::ASTContext&) ()
    #33 0x00000000015bf6e1 in clang::ParseAST(clang::Sema&, bool, bool) ()
    #34 0x000000000128bfbd in clang::FrontendAction::Execute() ()
    #35 0x000000000123e35a in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) ()
    #36 0x000000000136cadc in clang::ExecuteCompilerInvocation(clang::CompilerInstance*) ()
    #37 0x00000000011408a2 in cc1_main(llvm::ArrayRef<char const*>, char const*, void*) ()
    #38 0x000000000113f751 in ?? ()
    #39 0x0000000001139404 in main ()
    Last edited by haagch; 24 June 2016, 07:46 AM.

  • #2
    Without going into a long diatribe of excuses and reasons it boils down to
    1) It confused our customers why they should pay for something if it was open source. Switching to a business model that worked for open source didn't seem possible. (I sincerely really did try) - So for anyone who wanted us to stay in business, well it was something we had to do..

    2) The time/cost/energy and resources to build an open source community around a compiler project is very large. "We", PathScale as a company, simply didn't have that. As a testament to this, when we closed things, almost nobody noticed or cared.
    -----------

    Side notes:
    Myself personally, I don't really feel guilty too much. The world has gcc and to an extent clang. In addition, there's a lot of bad information and habits taught in universities. I frequently see advisers and papers using source-to-source instead of requiring the end result to tie into something more robust. Basically, publish and forget...
    The learning curve is steep: codebase, algorithms, a ton of c++ and then sometimes expert level knowledge for some standard (c/c++/fortran). (Not to mention deep knowledge about the target hardware)

    Lastly and in direct response to the above -

    Our sweetspot is more on the HPC side of codes, I appreciate the guys effort to ask and even try to build something with our compiler. His attitude isn't all warm and fuzzy, but I don't blame him. I'll see what I can do to get this bug fixed and possibly make it easier to build this code in the future.

    The link on our site is static and there's a note that the URL needs to be manually updated. It's not perfect, but we find most people figure it out.

    I'll always do my best to be friendly with open source anywhere I can.. I personally open source stuff as often as possible. At the same time I try to respect the community and don't position PathScale as an open source company.

    Take a look at our github
    pathscale has 31 repositories available. Follow their code on GitHub.


    You'll be disappointed if you want to find our compiler, but..
    We have open sourced our c++ runtime
    We make public lots of changes which we are trying to get upstream (tradcpp/libunwind/openmp/etc)
    The code from papers and research, I push open so it's fully transparent where possible
    We pushed a cmake wrapper project that helps make it easier to build clang
    Small pieces that tie into the Gentoo project

    Indirectly, I advocate open standards and try to help facilitate open source as much as possible. The Microsoft C++AMP team has been great with helping make that an open standard as well as open sourcing a lot of side pieces like their internal testsuite.

    I hope this helps clarify what happened, where PathScale stands and where I personally stand.

    Comment


    • #3
      Thanks for the reply. Maybe you can post something like this to http://www.path64.org/ so there's a source on why it isn't available anymore other than a forum thread.

      I'll be fair: I understand your reasons. I was really looking forward to see distributions build some packages with it where it makes sense and a big community forming naturally around it very quickly, because... who wouldn't want to use an opensource compiler with advanced optimizations? But then it was open sourced and apart from 2 or 3 forum threads on phoronix almost nothing happened with it and people seemed generally disinterested... You made some comments in regards to the GPL situation that were probably a bit unprofessional, but I don't think that played a role. I just don't get why the community wasn't putting in more work to make path64 a thing.

      The thing that I was a bit salty about was that it was just deleted without much of an explanation, not because I'm hostile towards pathscale or ekopath, but because I really wanted to see the path64 suite succeed...

      I was going to ask whether you have a problem with people developing the still existing forks further, but then... they are several years behind now, so probably not that useful vs. today's clang and gcc anyway, except maybe as a reference.

      As for my edit:
      To be honest, I was just curious whether pathcc could actually compile mesa, because they're probably doing some funky low level optimization stuff in that codebase. Well, the flags that pathcc doesn't support don't seem so important and the __builtin_assume attribute doesn't seem very important either, just a bit annoying that you have to remove it. And then I was curious whether mesa built with pathcc would actually work. Last time I built mesa with clang, it was very crash prone...
      Other than satisfying my curiosity I had no real intention of doing anything with it.

      Well, if your compiler segfaults on some codebase, you probably want to fix it anyway, I'm just saying you don't need to waste time on this.

      Comment


      • #4
        If I had to guess a reason(s)
        The codebase was very old - You're looking at 1990's C code that was one day renamed .cxx as well as a quite complex lowering. In the beginning the build process was also so bad that it was beyond embarrassing. (Makefile hell like you wouldn't believe) Some of this has been cleaned up over the past 7 years, but for a passionate user to try to make the jump to a contributor would be exceptionally not easy. With clang you're dealing with mostly modern C++ code that has some level of reasonable design and lots of people to answer questions.

        In general I suspect people contribute to projects that they both love and find fun.. even if you loved using the compiler and even if we had the most friendly community in the world, you're still stuck with a rather unfun codebase and technical obstacle... c'est la vie..
        ------------
        builtins - I'm a fan of them over inline asm.. I lately tried to complain a bit with clang removing support for some. I even showed real world codes using them and nobody seemed to care. You may want to keep an eye on it if mesa breaks while trying to build with clang..

        I don't consider it a waste of time.. a compiler always has bugs, but it's just a matter of who and when it impacts them.

        Side side note: PathScale does actually give some people source access, but it's under NDA. This is so far from open source that it's not something I typically mention. The typical use case is researchers doing work on GPGPU or something else compiler related.

        I suspect the bottleneck in mesa performance isn't compiler related, but I could be mistaken - if it's doing something loop heavy or computationally complex. You're welcome to email me if you'd like to follow-up with this..

        cbergstrom ////at\\\\ pathscale.com

        cheers

        Comment


        • #5
          In pathcc's case it was pretty clear where the problem was because it complained about a missing symbols related to __builtin_assume when linking (I didn't save the error message), so I suppose it'll be easy to see too when clang breaks something. Thanks for the hint anyway.

          I think it's true, there shouldn't be bottlenecks in mesa that could be auto optimized away in a huge way. On the other hand, there has been a lot of new code to implement new features lately and there may or may not be some Make it Work, Make it Right, Make it Fast. stuck at step 2 in there.
          There have been some empirical results that suggest that a combination of autofdo, pgo, LTO, and O3 could - at least in some cases - have quite the impact: https://www.phoronix.com/scan.php?pa...r-opengl&num=2 but as far as I know, we still don't really know what exactly causes these different results.
          So measuring whether there is a difference is at least somewhat interesting. And if the result turns out to be that there is no noticeable difference than that's still an interesting result, because it (at least partly) confirms that there are no auto-optimizable bottlenecks in mesa - which would be good to know.

          Comment

          Working...
          X