Linux Kernel Moving Ahead With Going From C89 To C11 Code

[coder]

I don't like to go overboard with it - I've seen some code that makes heavy use of the comma operator and nested conditional operators in order to force things into an expression form for SSA.

Wow, comma operator? Fortunately, I rarely see that used. I guess OpenCV found a debatably good use for it, by overloading & employing it in their matrix initialization syntax that even worked pre-C++11.

Single-use lambdas are one of the more common tricks for enabling SSA, at least in modern C++. I've gone there, but usually not. Heavy use of lambdas can add a lot of complexity to the code, since they tend to look like nested scopes but aren't.

I'm a fan of making variables "const"

Heh, I sometimes call that "const cancer". I have always used const where I thought it made sense, but I want to see it pull its syntactic weight. That means if there's some narrow scope where there aren't many variables and it's trivial to see something isn't modified, I won't bother. However, if there are some invariants above a nontrivial loop (or other control flow structure), I'll probably go ahead and make them const.

then you know the variable won't change, and the compiler helps spot your mistakes.

For non-trivial types, const semantics can get a little tricky and debatable. For instance, if a class has a pointer member to some object, do you make all the class' methods const which don't modify the pointer, itself? What if the class is defined as a proxy of the object to which it's pointing? The compiler will let you, but I'd argue a proxy class' const semantics should reflect whether member functions ultimately modify the the proxied object. I think that's fairly non-controversial, but I've had that debate with const enthusiasts, on more than one occasion. And there are yet other cases, where it hasn't been terribly obvious to me what the best answer is.

So, I've come to regard const member functions as something that pays biggest dividends on simple types. For sufficiently complex objects, if there's not an obvious and intuitive case for const-ness, I default to making methods non-const.

Meanwhile, some C programmers are certainly laughing at such consternation over these problems the C++ community has seemingly invented for ourselves. However, I'd argue these sorts of debates typically reveal deeper truths about code structures which they can only blunder through, in an indirect way, if only because their language doesn't give them a simple means of discussing these concepts in the abstract. It's a slight stretch, but I think linguistics tells us it's difficult to reason about concepts that a (spoken) language provides no easy way of expressing.

Many modern languages make "const" variables the default, and require an extra keyword for mutable variables - I think that's the right idea.

Agreed. If that's the language default (as I believe it is in R***), I'd not fight it.

What was the SSA language you used? At university I learned functional programming (with a language much like Haskell),

XSLT was that first plunge into the deep end. However, I use it so infrequently that I tend to forget most of what I (re)-learned about it, each time. Still, I find it a rewarding experience to puzzle through its often inside-out way of looking at problems.

while I don't do much functional programming now, it had a significant influence on my style.

For sure. Some of the best programming advice I got was to learn languages that give you a different way of looking at (some class of) problems. It makes me feel like people who've only ever used C-like languages don't know what they're missing. It's at least good to see some functional constructs gradually permeating other popular languages.

[coder]

The big problem I see with the theory that "malloc returns 0 on failure" or "new throws on failure" is that programmers handle it badly. It can often look good enough locally, but it rarely considers the big picture and any potential knock-on effects. And it is never tested. (The kind of programming tasks that justify full testing of all possible paths will not allow any use of dynamic memory.)

My take in this is that it's quite difficult -- and usually not worthwhile -- to write code that's robust to out-of-memory conditions. Even if the code encountering the failed allocation correctly cleans up and gracefully reports the problem, the caller is still typically left with no good options -- either retry the failed operation with little/no expectation it'll succeed upon retry, or simply defer the problem to the next higher level. And, because it's hard to correctly handle & test memory allocation failures, there's a good chance the program is left in a bad state. So, the best option is usually to fail fast & hard (at least making the problem easy to debug, if one is so inclined) and let whatever's running the program (i.e. user, systemd, shell script, etc.) decide how to handle it.

Or you are unlucky enough to get the memory but take so much of the stack that the program crashes later in the call tree, which makes it harder to identify the problem.

In fairness, this can also happen with heap-allocated memory. More often, a non-local heap error is due to one part of the program stepping on a different heap allocation used by another part. Fortunately, tools like valgrind can usually detect this -- with stack-based memory, you're basically on your own. Sanitizers and -fstack-protector are supposed to help with this, but I've not had success with the former and have no experience with the latter.

I am not at all convinced by the argument that heap allocation errors are easier to debug,

It's really a question of tools support. Again, just speaking from personal experience, I've found heap errors much easier to debug than most of the admittedly small sample of stack bugs I've had the misfortune of encountering.

[sinepgib]

About checking the result of malloc for NULL: pointless task on Linux, you essentially need to exhaust the address space (not memory, but address space) for it to ever fail. Linux does (at least by default) lazy overcommit of memory, which means it doesn't allocate, or check it can allocate, on calling to malloc, returning pretty much always a valid pointer and allocating on page fault. This means checking for NULL won't catch allocation failures, only segfaults will.

[coder]

About checking the result of malloc for NULL: pointless task on Linux, you essentially need to exhaust the address space (not memory, but address space) for it to ever fail. Linux does (at least by default) lazy overcommit of memory, which means it doesn't allocate, or check it can allocate, on calling to malloc,

For me, operator new throws std::bad_alloc at 2 GiB. It's a 64-bit executable (x86-64), which the file command confirms. I only checked powers of 2, but it did that twice in a row. Also, it slowed way down, as the allocations got bigger. That suggests there's a lot more going on than you say.

Now, when I retried it using plain old malloc(), it's indeed as you say. I could allocate as large as I dared (I think I went up to 2^54). It did seem to take roughly constant time. Weirdly, I still got a NULL result at 2^31 (repeatedly!), but then it continued to succeed beyond that point. I didn't then go back and adjust the operator new version to skip that case, as I don't really want to exhaust all my memory. Maybe I'll try it when booting from a USB stick or in rescue mode (i.e. no filesystems mounted).

The machine has 32 GiB of RAM and about as much swap. Most of it free, because I use it headless and don't have much running on it, right now.

[sinepgib]

For me, operator new throws std::bad_alloc at 2 GiB. It's a 64-bit executable (x86-64), which the file command confirms. I only checked powers of 2, but it did that twice in a row. Also, it slowed way down, as the allocations got bigger. That suggests there's a lot more going on than you say.

Now, when I retried it using plain old malloc(), it's indeed as you say. I could allocate as large as I dared (I think I went up to 2^54). It did seem to take roughly constant time. Weirdly, I still got a NULL result at 2^31 (repeatedly!), but then it continued to succeed beyond that point. I didn't then go back and adjust the operator new version to skip that case, as I don't really want to exhaust all my memory. Maybe I'll try it when booting from a USB stick or in rescue mode (i.e. no filesystems mounted).

The machine has 32 GiB of RAM and about as much swap. Most of it free, because I use it headless and don't have much running on it, right now.

Well, new does initialize memory by calling constructors IIRC, so that may trigger the page faults. How that gets caught by the runtime, I ignore, but maybe it installs a default signal handler or handles the fault itself. It may also be the case that it uses a different allocator that forces eager paging? For big allocs it's quite common for them to call mmap directly.

Note, tho, "exhausting the address space" was an over simplification. Fragmentation may cause a lack of contiguous addresses for the allocation, which is the actual limiting factor when using lazy overcommit.

[reba]

I wonder if anybody could tell me - do "typical" C programmers apply TDD (Test Driven Development) or alike (ATDD/BDD/DDD)?

[coder]

I wonder if anybody could tell me - do "typical" C programmers apply TDD (Test Driven Development) or alike (ATDD/BDD/DDD)?

If you explain what those mean and whether you're concerned with strictly C or more broadly C/C++ developers, a few of us might be able to relate our personal experiences and observations. And if that's not what you're looking for, then you're probably asking in the wrong place.

[reba]

If you explain what those mean and whether you're concerned with strictly C or more broadly C/C++ developers, a few of us might be able to relate our personal experiences and observations. And if that's not what you're looking for, then you're probably asking in the wrong place.

Oh, I see; C++ programmers, too, of course. Anyone using an "alike" and quite low-levelish language.

For explanations I'd like to redirect to Wikipedia, they probably do a better job than me with that:
- TDD (Test-driven development)
- ATDD (Acceptance test–driven development)
- BDD (Behavior-driven development)
- DDD (Domain-driven design)

[coder]

- TDD (Test-driven development)

Not really. Closer to ATDD.

- ATDD (Acceptance test–driven development)

When we had a QA team, both engineering and QA would start with product requirements. They would develop a test plan in parallel with engineering devising the functional spec + design documentation. Then, implementation would proceed in parallel, and functionality would get tested more or less as it was delivered. I can't really comment on their test plan, because (unlike my previous job), they never had us review it.

I know that's not really how acceptance testing is supposed to work, because they didn't exist in advance of the implementation and therefore engineering didn't run the them during development.

Since we lost our QA team, we haven't been doing much development of new tests. Certainly not to any rigorous degree.

- BDD (Behavior-driven development)

No, that's more formalized than what we do.

- DDD (Domain-driven design)

No. We used to document our designs before implementation, but that went away with the adoption of "Agile" and loss of staff.

[kpedersen]

I remember when I switched from C to C++. It takes a little getting used to, but I quickly grew to prefer the intermingled style.

I suppose you don't really get a choice with C++, you tend to not want to execute the constructor code of non-PODs until absolutely needed. Even order of variables matter because declaring them in many cases executes code (i.e Constructors) and some babysitting of RAII again needs the order.

In C having all variables at the top makes it easy to audit for i.e memory leaks, but I suppose C++ does help to solve that anyway.