The Leading Linux Desktop Platform Issues Of 2018

[Weasel]

Alright, I think you're a bot, you can't be so retarded as to copy-paste the exact same thing I proved wrong 3 times already.

And VirtualAlloc doesn't need a lock Lol. It's a syscall at its heart. The kernel does that by itself, no userspace locks are needed. (same reason you don't need locks to mmap on Linux)

Here's the deal: show me the memcpy that segfaults by copying from two runtimes without any buffer overflows (also show the code if you want), give me EXACT crash dump and stack trace, with assembly instructions (near the instruction pointer, RIP), or shut the fuck up you stupid clown. You make the claim, so YOU PROVE IT, because the burden of proof is on YOU.

And no, linking to an article that DOESN'T EVEN CONTAIN THE WORD YOU USE is not proof. You might as well link a random wikipedia article about some random animal and it would make more sense.

[oiaohm]

Here's the deal: show me the memcpy that segfaults by copying from two runtimes without any buffer overflows (also show the code if you want), give me EXACT crash dump and stack trace, with assembly instructions (near the instruction pointer, RIP), or shut the fuck up you stupid clown. You make the claim, so YOU PROVE IT, because the burden of proof is on YOU.

FuncC FuncD FuncF

Memory Access

Function call crosses CRT boundary. Accessing the allocated memory may not result into an error. However, interpreting the memory content may result into exceptions, data corruption, memory access error, or program crash.

http://siomsystems.com/mixing-visual-studio-versions/

Weasel why do I have to do wheel. This site is clear on the defects.

And VirtualAlloc doesn't need a lock Lol. It's a syscall at its heart. The kernel does that by itself, no userspace locks are needed. (same reason you don't need locks to mmap on Linux)

You are totally under estimating the evil. Where you hand out address to memory before you have in fact allocated it..
https://msdn.microsoft.com/en-us/lib...(v=vs.85).aspx
There is another way that some allocates do it as well MEM_RESERVE again this is where you have not allocated the pages really. So when something goes to use it you need to catch the signal process it and then allocate the memory for real.

There is more than 1 way to use VirtualAlloc and different versions of malloc have used them all.

May not result into an error is written there because all the problems are race conditions. I have not going to have a computer looping through test cases for possible days on end to trip a race that is already well documented.

shut the fuck up you stupid clown<< Really take this advice yourself Weasel. We would not have had this long fight if you were not keeping on claiming something that is documented as broken as working.

Yes using VirtualAlloc to allocate memory properly takes a large amount of time. This results in those optimising their malloc either allocating pointers to addresses before they are in fact requested by VirtualAlloc/mmap at addresses after fact or using MEM_RESERVE to mark out the pages but not in fact back them with memory. Both paths create exception that must be handled if the memory gets used before it really allocated to have real memory. When you use 2 CRT the handler argues leading to problems.

Please note MEM_RESERVE is a windows evil. Linux has it own evil where the memory is allocated but due to mbind its assigned to the wrong CPU if you are using a highly optimising allocate on Linux you need to trap these errors and handle them when they come up.

I m not going waste days running a test case to intentionally hit a race condition on a documented known problem.

[brrrrttttt]

How long can this go? >1 month so far .

[Weasel]

FuncC FuncD FuncF

Memory Access

Function call crosses CRT boundary. Accessing the allocated memory may not result into an error. However, interpreting the memory content may result into exceptions, data corruption, memory access error, or program crash.

http://siomsystems.com/mixing-visual-studio-versions/

Weasel why do I have to do wheel. This site is clear on the defects.

I asked you to show me the crash dump after a segfault. The fact that you did not is proof for me that you don't have any because you are FULL OF BULLSHIT.

I do not care about anything you say until you prove it by showing me THE EXACT SEGFAULT CRASH DUMP that happens on a memcpy that copies from two allocated buffers, without overflowing either. Understood? Until you do that, there's nothing else to discuss, you circus monkey.

You are totally under estimating the evil. Where you hand out address to memory before you have in fact allocated it..
https://msdn.microsoft.com/en-us/lib...(v=vs.85).aspx
There is another way that some allocates do it as well MEM_RESERVE again this is where you have not allocated the pages really. So when something goes to use it you need to catch the signal process it and then allocate the memory for real.

MEM_RESERVE doesn't allocate anything, neither commits, it just reserves address space. You can't be so stupid, surely? Nah, I'm wrong, you are even way more than that.

MEM_COMMIT commits memory. Memory only gets allocated the first time it's accessed after being committed (when pages get touched, they get allocated automatically by the OS).

This isn't even Windows specific, it's the same thing on Linux, since it's a CPU thing.

Maybe you should learn some basics first before opening your mouth.

[oiaohm]

I do not care about anything you say until you prove it by showing me THE EXACT SEGFAULT CRASH DUMP that happens on a memcpy that copies from two allocated buffers, without overflowing either. Understood? Until you do that, there's nothing else to discuss, you circus monkey.

The reality is the site that I have been quoting that author had done white paper that documents it. So I have no reason to reinvent the wheel. If you are not going to believe him why should I bother.

MEM_RESERVE doesn't allocate anything, neither commits, it just reserves address space. You can't be so stupid, surely? Nah, I'm wrong, you are even way more than that.

MEM_COMMIT commits memory. Memory only gets allocated the first time it's accessed after being committed (when pages get touched, they get allocated automatically by the OS).

Simpler question what one is faster. MEM_RESERVE right. You are not asking your self the right questions. Why would an allocation function choose MEM_RESERVE over MEM_COMMIT.

Maybe you should learn some basics first before opening your mouth.

Would pay if you understood allocate systems.

Lets run basic 101 of optimised allocate system.

1) Program starts the init straight up calls virtualalloc and allocates quite a large area of MEM_RESERVE.
2) program performs a malloc this returns a address in the MEM_RESERVE. This is going to error when it used.
3) When it used then catch the error and perform the MEM_COMMIT..

This at first seams worse. Here is a rough outline of what you see in real programs
void * buffer =malloc(10000);
if(something==1){
memcpy(buffer.somesource,10000)
/*then some more operations on buffer*/
}
free(buffer)

Notice something from point buffer is allocated to where it free it possible that nothing is done to buffer. This can be in some programs people use can be happening over 50% of the time. MEM_RESERVE is quite fast. MEM_COMMIT is slower.

There are historic benchmarks allocates huge number of buffers that they never use as well. Optimised allocation systems don't use the MEM_COMMIT when you think they would. You would think MEM_COMMIT would be done at the malloc when it fact in my example it would be done by the exception/signal handler the memcpy triggered. This is where optimising memory allocation can turn evil. Yes doing it this way works out faster.

[Weasel]

The reality is the site that I have been quoting that author had done white paper that documents it. So I have no reason to reinvent the wheel. If you are not going to believe him why should I bother.

I'm not going to "believe" anyone, because unlike you, I can read crash dumps, so I only believe FACTS. If you claim he has a crash dump, then link it?

He doesn't though -- not because he's incompetent, because he's not -- but because it has nothing to do with the techobabble bullshit you spew.

So again, until you can come to me with a crash dump, and show me the memcpy that caused it (knowing you, you'll just link a random crash dump of a totally unrelated thing!), without it being a buffer overflow (so it cannot go out of bounds compared to the amount allocated), I'll just be going to assume that you are full of shit that can't back up the diarrhea you spew.

I repeat: I don't care of anything else you say about this topic until you provide facts.

Simpler question what one is faster. MEM_RESERVE right. You are not asking your self the right questions. Why would an allocation function choose MEM_RESERVE over MEM_COMMIT.

They're about the same speed, mostly dominated by syscall overhead.

And why would it choose MEM_RESERVE? Maybe because they're two different things?!??? You can reserve terabytes of address space on x86-64... Petabytes even.

Try to commit that amount on Windows (since it has no overcommit unlike Linux) and see how far that gets you. For fuck's sake.

Would pay if you understood allocate systems.

Lets run basic 101 of optimised allocate system.

1) Program starts the init straight up calls virtualalloc and allocates quite a large area of MEM_RESERVE.
2) program performs a malloc this returns a address in the MEM_RESERVE. This is going to error when it used.
3) When it used then catch the error and perform the MEM_COMMIT..

It's not an "error", it's a page fault, and that's almost how the stack growth works. malloc does NOT return an address into MEM_RESERVE though.

malloc TOUCHES the memory already to allocate its structures (heap is a DATA STRUCTURE), so when it touches it gets transparently allocated (or not, it's up to the exception handler really).

There's no reason to do it this way though, it's just one of the possible implementations. You can always commit it all at once (even the stack, Wine does that for instance, it doesn't use MEM_RESERVE).

Committing doesn't allocate it, either. It's only when a committed page is accessed that it actually gets allocated.


Commit limit = all physical RAM + all swap space.

When something gets committed, no resources are used for it. None. The only thing it takes up is it eats up into this "commit limit", so you can end up out of it. You can't commit more than the commit limit, in other words. (On Linux you can, because of overcommit, but you can turn it off)

Allocation though (i.e. you touch the page the first time) is what actually consumes resources. That's when pages will have to be swapped out if unused to make room for the touched page, etc.

Reserve literally consumes nothing since it only reserves address space which is plentiful on 64-bit systems (and even on 32-bit apps if the app uses less than 1 GB of memory total).

[oiaohm]

They're about the same speed, mostly dominated by syscall overhead.

And why would it choose MEM_RESERVE? Maybe because they're two different things?!??? You can reserve terabytes of address space on x86-64... Petabytes even.
.

1) Program starts the init straight up calls virtualalloc and allocates quite a large area of MEM_RESERVE.
2) program performs a malloc this returns a address in the MEM_RESERVE. This is going to error when it used.
3) When it used then catch the error and perform the MEM_COMMIT..
Read again and work out how you are a moron.

Yes program starts it can reserve terabyte of stuff.

void * buffer =malloc(10000);
if(something==1){
memcpy(buffer.somesource,10000)
/*then some more operations on buffer*/
}
free(buffer)

Look at the example closer this time. So you have reserved a memory space. You have returned a pointer into that reserved space from malloc at this point you have not syscalled.
Error on memcpy then you syscall and do the MEM_COMMIT.
If you get to free and memory has not been changed from reserved to mem_commit again no syscall.

You can't commit more than the commit limit, in other words.

True you cannot commit more than commit limit but this method allows you to replicate Linux overcommit in a windows program using malloc and free. Why MEM_RESERVE is not MEM_COMMIT.

malloc TOUCHES the memory already to allocate its structures (heap is a DATA STRUCTURE), so when it touches it gets transparently allocated

This is not always true. Using MEM_RESERVE you still need to use a heap structure to record the allocated areas.

https://lwn.net/Articles/627632/

I am not the only one is is aware of the MEM_RESERVE. abuse.

(On Linux you can, because of overcommit, but you can turn it off)

You can overcommit on windows as well. Due to Linux having overcommit as a kernel feature you don't have runtimes implementing it on Linux using exception handling. Overcommit is implemented inside different visual studio runtimes by the MEM_RESERVE method. Kinda was more important early on.

[Weasel]

1) Program starts the init straight up calls virtualalloc and allocates quite a large area of MEM_RESERVE.
2) program performs a malloc this returns a address in the MEM_RESERVE. This is going to error when it used.
3) When it used then catch the error and perform the MEM_COMMIT..
Read again and work out how you are a moron.

Read again your bullshit?

YOUR REPEATED FUCKING CLAIMS ARE NOT PROOF YOU MORON. Shut the fuck up with copy-pasting your bullshit.

malloc does NOT return address in MEM_RESERVE. It returns address in COMMITTED memory, with MEM_COMMIT.

This is exactly why the PE format has a "Heap Reserve" and "Heap Commit" fields in the fucking format itself. When it needs to grow the heap, aka the "Heap Commit" is not enough, it uses VirtualAlloc with MEM_COMMIT first, then returns pointer to somewhere after placing there some heap structures.

void * buffer =malloc(10000);
if(something==1){
memcpy(buffer.somesource,10000)
/*then some more operations on buffer*/
}
free(buffer)

Look at the example closer this time. So you have reserved a memory space. You have returned a pointer into that reserved space from malloc at this point you have not syscalled.

No you have COMMITTED memory space. You CANNOT RESERVE memory with malloc. It allocates on the heap. Allocation happens transparently when a committed page is accessed the first time.

Since we're talking about Windows, why don't you use your task manager and post screenshots after allocating 1 GB with malloc?

Let's have a look at "commit limit", which btw it means what the name is, which is COMMIT limit not RESERVE limit. Reserving does not increase the commit limit, but malloc does. What does that tell you?

Error on memcpy then you syscall and do the MEM_COMMIT.
If you get to free and memory has not been changed from reserved to mem_commit again no syscall.

Dude, reserving itself is a syscall. Reserving address space can ONLY be done by the kernel. This applies to both Linux and Windows because it's a CPU thing.

Again, that's not an error, it's an exception, and you need guard page for that. Guess what function sets guard pages?

VirtualAlloc.

True you cannot commit more than commit limit but this method allows you to replicate Linux overcommit in a windows program using malloc and free. Why MEM_RESERVE is not MEM_COMMIT.

Bullshit. Prove it.

This is not always true. Using MEM_RESERVE you still need to use a heap structure to record the allocated areas.

https://lwn.net/Articles/627632/

I am not the only one is is aware of the MEM_RESERVE. abuse.

No but you and the idiot in that article seem to go along nicely, especially since the two comments tell him how wrong he is. But not like you read replies right? All you read is just bullshit and parrot it.

Overcommit means overCOMMIT. If you reserve, you DO NOT COMMIT.

No, you cannot overcommit on Windows. To overcommit, you have to use MEM_COMMIT.

It's really simple. "Commit" means something, "Reserve" means something else. You can't be so stupid.

[oiaohm]

No, you cannot overcommit on Windows. To overcommit, you have to use MEM_COMMIT.

No moron shut up and read.

https://docs.microsoft.com/en-us/windows/desktop/memory/reserving-and-committing-memory

This is the bare bones starting point to a optimise memory allocation under Windows.

Yes the optimised malloc will be returning a pointer into reserved memory not MEM_COMMIT.

The advantage of dynamic allocation is that it minimizes the total number of committed pages on the system. For very large allocations, pre-committing an entire allocation can cause the system to run out of committable pages, resulting in virtual memory allocation failures.

Love this lets not call it overcommit that sound too dangerous lets call it "dynamic allocation". Yes you have the same failure problem of linux kernel overcommit as Microsoft dynamic allocation where you attempt to allocate memory and you find out you don't have it.

malloc does NOT return address in MEM_RESERVE. It returns address in COMMITTED memory, with MEM_COMMIT.

This is that you have never configured your visual studio runtime for big data sets. Default malloc uses MEM_COMMIT but when its so called dynamic allocation mode malloc returns a point to MEM_RESERVE space and is depending on exception handler at this point visual studio runtines turn exceptionally nasty if you are using more than 1.

If you knew this topic correctly you should have said that Microsoft calls it "dynamic allocation".

This is exactly why the PE format has a "Heap Reserve" and "Heap Commit" fields in the fucking format itself. When it needs to grow the heap, aka the "Heap Commit" is not enough, it uses VirtualAlloc with MEM_COMMIT first, then returns pointer to somewhere after placing there some heap structures.

Again limited understanding on your part. Big data with "dynamic allocation" enabled MEM_RESERVE pointers are put into Heap and are converted to MEM_COMMIT by exception handler.

No but you and the idiot in that article seem to go along nicely, especially since the two comments tell him how wrong he is. But not like you read replies right? All you read is just bullshit and parrot it.

No you are the same as the two morons who said that reply was wrong. Neither of them quoted Microsoft documentation. Just like you moron you did not know where to look for it.