GCC's Conversion To Git Is Being Held Up By RAM, a.k.a. Crazy DDR4 Prices

[caligula]

Are you kidding? While this thing is processing, the repo will have to be read-only or offline -- i.e. nobody can commit to it while it's doing its thing. Committing in the middle of conversion will cause disasters obviously.

So if you think RAM is cheap then how about paying up their many man hours lost due to them waiting for the repo to become write-able again so they can commit their stuff? AFAIK, svn doesn't have local repositories like git.

I don't know how the legacy revision control systems work, but I'm pretty surprised if SVN potentially changes the state of the whole data set on each commit. I can understand that the 'head' of the revision tree is live and mutable, but isn't the whole point of revision control to maintain a immutable trace of the history? Can you rewrite the history in SVN? Does SVN restore a previous state by replaying the commits backwards, starting from the current state? I'd really expect that only the very latest tree heads may be mutable and changing, but the past history should be static, set in stone, and observable. Each state could be derived, starting from the initial commit, not the current one. Thus enabling read-only access even when the repository is being used. Am I missing something?

Other than that, in my book, a revision history is incremental and local. That is, changing a file state has effects only on a very limited subsection of the directory tree. I'd assume that the working set for tracking commits is actually very small and local, too. This would lead to good performance on paging memory systems. Requiring the whole repository state and history in fast RAM would imply that the access patterns were very non-uniform and all commits affected the whole state, which makes no sense. For example, adding a 100 byte file would mean you'd have to read 200 gigabytes of data from disk and store back 200 gigabytes? I highly doubt it.

It would be nice if some more knowledgeable could explain why this tool seems so crappy.

[Weasel]

I don't know how the legacy revision control systems work, but I'm pretty surprised if SVN potentially changes the state of the whole data set on each commit. I can understand that the 'head' of the revision tree is live and mutable, but isn't the whole point of revision control to maintain a immutable trace of the history? Can you rewrite the history in SVN? Does SVN restore a previous state by replaying the commits backwards, starting from the current state? I'd really expect that only the very latest tree heads may be mutable and changing, but the past history should be static, set in stone, and observable. Each state could be derived, starting from the initial commit, not the current one. Thus enabling read-only access even when the repository is being used. Am I missing something?

Other than that, in my book, a revision history is incremental and local. That is, changing a file state has effects only on a very limited subsection of the directory tree. I'd assume that the working set for tracking commits is actually very small and local, too. This would lead to good performance on paging memory systems. Requiring the whole repository state and history in fast RAM would imply that the access patterns were very non-uniform and all commits affected the whole state, which makes no sense. For example, adding a 100 byte file would mean you'd have to read 200 gigabytes of data from disk and store back 200 gigabytes? I highly doubt it.

It would be nice if some more knowledgeable could explain why this tool seems so crappy.

I have no idea, but clearly the tool has to somehow know all future points when it does its thing, otherwise why would it use so much RAM in the first place if it could simply "stream" it? (even if ESR is lazy, using the OS facilities like swap to swap out the "previous" blocks would work just fine in such streaming case)

[brrrrttttt]

It is not a straight conversion tool, hence the name reposurgeon, rather than repoconverter. You'd both be better served reading the original sources rather than typing up huge posts full of speculation here.

[vwtch]

It is rather astounding that ESR still does one-finger typing with his eyes 8 inches from the keyboard to see the letters on the keys.

[SystemCrasher]

and potentially shifting some code from Python to the Go programming language, but that could prove to be very tedious

That's how python software development happens: write shit, figure out it is shit for the real, the language is shit, the runtime is shit. And it shit, say, because it can't be fast and light, no matter what you try and what fanboys mumble in their lairs. Try to throw more and more hardware in hope it helps, fail that, get fed up with struggles and finally try some more sane language. Somehow this pattern is so damn typical around python...

[speculatrix]

He does not believe in swap.

memory is like sex - it's better when you don't fake it
:-)

[caligula]

That's how python software development happens: write shit, figure out it is shit for the real, the language is shit, the runtime is shit. And it shit, say, because it can't be fast and light, no matter what you try and what fanboys mumble in their lairs. Try to throw more and more hardware in hope it helps, fail that, get fed up with struggles and finally try some more sane language. Somehow this pattern is so damn typical around python...

They've even implemented Python to native compilers to "fix" the issues with Python. Still https://benchmarksgame-team.pages.de...thon3-gcc.html

[SystemCrasher]

You can't fully translate python into C or so without also demanding whole compiler being a part of "run time" - otherwise things like eval() wouldn't work. So it wouldn't be fully compatible. Dynamic types are also quite hostile to compiling it down to some optimized machine code. Atttempt to generate more or less optimized code at run time would imply severe lags and consumption of resources, comparable to what happens when running compiler. For compiled languages it happens just once, and at build time, where people care considerably less about resources and lags. But these do not generate code at run time usually. For that particular reason I guess. There're some attempts to get over it, but looking on how it ends like .NET on windows rebuilds about ~6Gb of assemblies, hogging computer resources about 2 hours after one "completes" .NET install... seems there is still some catch.

[stevea]

I pray that whatever caused this, "SVN -> git" computational nightmare is addressed I. The git design

[starshipeleven]

I pray that whatever caused this, "SVN -> git" computational nightmare is addressed I. The git design

AFAIK, Eric S Raymond is still a moron, so I'd say no.

On the flip side, most projects don't have him so they are fine.