Announcement

Collapse
No announcement yet.

The GCC Git Conversion Heats Up With Hopes Of Converting Over The Holidays

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • The GCC Git Conversion Heats Up With Hopes Of Converting Over The Holidays

    Phoronix: The GCC Git Conversion Heats Up With Hopes Of Converting Over The Holidays

    Decided back at the GNU Tools Cauldron was a timeline to aim converting from Subversion as their default revision control system to Git over the New Year's holiday. For that to happen, by the middle of December they wanted to decide what conversion system to use for bringing all their SVN commits to Git. As such, now it's heating up ahead of that decision...

    http://www.phoronix.com/scan.php?pag...ng-Up-Holidays

  • dreamer_
    replied
    Originally posted by alcalde View Post

    His original blog post announcing the rewrite to Go had several people in the comments section who had just that experiments and couldn't for the life of them understand what he was doing, including the need to load everything into memory at once. Not one could convince him that it was the approach he was using, not the language, that was the problem.
    I can't comment on this particular decision, as I don't know the details - but faced with a similar situation in the past I made a similar decision and it turned out to be correct.

    Except instead of writing my own tool from scratch I implemented additional functionality to BFG. BFG (written in Scala) loads everything into memory to speed up rewrite operation of whole DAG at once. It works several orders of magnitude faster than using git-filter-branch - when doing large migrations in the past I used it as one of several steps of converting repo to Git (it was used to rewrite binary artefacts from SVN to git-lfs, fix historical commit messages, and remove some broken commits causing problems from SVN history) - the speed of operation allowed me to run migration step on each commit to SVN (so I had new repo snapshot less than a minute after new commit landed in SVN) and allowed me to track progress in the original repo and find problematic areas of migration early (it would've been impossible if I worked with full SVN dumps offline).

    I imagine ESR might face similar problems I had and writing dedicated migration tool to deal with issues specific to GCC might be the correct approach.

    Leave a comment:


  • alcalde
    replied
    ESR: I didn't port to Go to fix bugs, I ported for better performance.
    The program only needs to be run once. It would not have taken the months to run that it took to convert.
    Python is a wonderful language for prototyping a tool like this, but it's too slow and memory-hungry for use at the GCC conversion's scale. Also doesn't parallelize worth a damn.
    In exchanges with him in the comment section of his blog post about the conversion, his displayed almost no (accurate) knowledge of writing parallelized Python code at all. I swear he said at the time that the code didn't employ parallelization either then.

    Leave a comment:


  • alcalde
    replied
    Originally posted by dreamer_ View Post
    Anyone claiming "he should just do this" or "just do that" probably never had a "pleasure" of converting huge SVN repo to Git.
    His original blog post announcing the rewrite to Go had several people in the comments section who had just that experiments and couldn't for the life of them understand what he was doing, including the need to load everything into memory at once. Not one could convince him that it was the approach he was using, not the language, that was the problem.

    Leave a comment:


  • alcalde
    replied
    Originally posted by cybertraveler View Post
    I don't know why some of you are so salty or angry about this.
    I'm salty because he decided to post an entire rant/essay disowning Python, which he blamed for his problems. Python programmers who looked at his code found it to be a nightmare, and translating it to Go just meant porting crappy code from one language to another. I swear ESR told me that the Python version (and the subsequent port) didn't employ any parallelization and his knowledge of Python debugging tools seemed scant. In the end, he achieves an order of magnitude less performance gain than he predicted, eating up months of time, showing it wasn't the language at all that was the problem, but the programmer. Similarly, his memory problems were caused by his insistence of trying to keep everything in memory at once, which others who had worked on similar projects tried and failed to convince him wasn't necessary. Meanwhile, those who didn't follow this closely or read his blog simply saw headlines about Python being a terrible language and some famous programmer abandoning it, tarnishing the language's reputation. That's not fair.

    Leave a comment:


  • anarki2
    replied
    Originally posted by cybertraveler View Post
    I don't know why some of you are so salty or angry about this.

    I just see a guy who wants to carefully and cleanly convert the data and has been working on doing this in the spare time he has available.

    Any cowboy with some basic coding knowledge could do a fast conversion using existing tools or even some homebrew software. However, the result would almost certainly be an imperfect conversion that would leave a trail of confusion and problems for years to come. But that cowboy would have "done his job" and received praise for doing it, so he probably wouldn't care. I've met those kind of coders; they are numerous. There are far fewer coders who are methodical, far-thinking and considerate.

    If I was a GCC dev I'd prefer that this guy takes another 2 years doing this task properly instead of it being done quickly with potential messy fallout.
    There's a difference between being pedantic and being an incompetent fool just stabbing in the dark without a clue.

    Leave a comment:


  • starshipeleven
    replied
    Originally posted by GI_Jack View Post
    If RAM was an issue, he could have gotten a decently fast SSD, perhaps even two in RAID-0, perhaps on any modern board with NVMe and just used them as 1TB+ or swap.
    That's... not what you do when you need a lot of RAM.

    Leave a comment:


  • GI_Jack
    replied
    ESR is a giant fucking tool and always has been. Some suckers just bought him a new rig, and that is fucking hillarious.

    If RAM was an issue, he could have gotten a decently fast SSD, perhaps even two in RAID-0, perhaps on any modern board with NVMe and just used them as 1TB+ or swap.

    Leave a comment:


  • stormcrow
    replied
    Originally posted by CommunityMember View Post
    And the gcc developers who prefer git are probably using it, with the git-svn integration, to let the upstream project continue to look like svn while having the ability to use tools that are a bit more modern and familiar and integrated. I have gone the git-svn route for a few past projects until the rest of the community agreed to do a (good enough) conversion. That said, those projects were not the size, complexity, or broken history of gcc, so I think we can understand that large projects without corporate commitment, resources, and decision making (such as what happened with Microsoft and their conversion to git for the Windows team) are not going to happen quickly.
    I didn't think about that when I wrote my reply, but your suggestion also makes sense. Probably plenty of contributors using existing compatibility tooling that more or less makes the conversion moot (to them).

    Leave a comment:


  • smitty3268
    replied
    Originally posted by Terrablit View Post
    If ESR wins, I'm afraid he'll slow down again without competition. Instead of getting the job done, he'll probably just find some random excuse to write a shitty clone of the JVM for "performance reasons". In a way, it's amazing. I've never seen anyone else with more than two years of experience NIH everything while being so bad at reinventing the wheel.
    The good news is it looks like they are committed to picking a solution next Monday, and then doing the conversion at the end of the year. If the reposurgeon conversion isn't fully working by next Monday, it will have presumably eliminated itself by then and Maxim's solution will win by default.

    Doesn't sound like they're willing to wait any longer.

    Leave a comment:

Working...
X