Mastodawn

Using `make` to compile C programs (for non-C-programmers) https://jvns.ca/blog/2025/06/10/how-to-compile-a-c-program/

Using `make` to compile C programs (for non-C-programmers)

Using `make` to compile C programs (for non-C-programmers)

Julia Evans

Julia Evans Jun 11, 2025

related to this `make` post I've been very vaguely thinking of writing a "things that are useful to know about C even if you're never going to write a C program in your life" zine because I feel like I get a surprising amount of mileage out of understanding C at a very basic level even though I have never really written a C or C++ program and do not plan to

but I have to actually get the terminal zine published before I start thinking about whether I want to do that or not

Graydon Jun 11, 2025

@b0rk In John Barnes' odd fairytale _One For the Morning Glory_ the royal library has books titled _Highly Unpleasant Things It Is Sometimes Useful to Know_ and _Things It Is Not Good to Know at All_.

It seems like this organizing principle might be applied to the general subject of C programming.

Ángela Stella Matutina Jun 11, 2025

C is fine for some varieties of recreational programming, retrocomputing and so on. The unpleasantness and/or danger comes from maintaining C code in production.

Itamar Turner-Trauring Jun 11, 2025

@b0rk This knowledge really useful in a bunch of places, yes!

- Operating system APIs are all documented as C/C++ APIs
- All (or almost all?) books on low-level software performance assume you can read C

I'm thinking about the latter a lot at the moment since I'm trying to write a book that -doesn't- require that, but would be nice to have something I can point people at.

Julia Evans Jun 11, 2025

@itamarst it's difficult for me to tell how "hard" learning C at a basic level is. It always feels to me like the basic syntax is not that bad and that the "really hard" part is stuff like memory management and avoiding crashes which of course isn't important if you're not planning to actually write C programs

Julia Evans Jun 11, 2025

@itamarst you're making me wonder what the "minimal" set of C it's useful to understand is

for example I feel like understanding everything that's going on in this extrmely small program is a good start, but probably there are a couple of other things that are useful too?

Woke Leftist Trash Jun 11, 2025

@b0rk @itamarst not C specific but understanding calling conventions might also be useful since it’s what most FFI libraries will use to call C code.

I’ve seen folks struggle with pointers a lot but for non-C folks the one place that’s useful to them are “result” pointers: often a C function will take a pointer as an argument and it will initialize the data at that pointer instead of returning a value; this surprises users of higher level languages.

Also checking global, special variables to detect errors. Especially with system calls.

Sorry for the info dump. I’m a big fan of your work and I think this is a fantastic idea. 😇

Itamar Turner-Trauring Jun 11, 2025

The scoping of what to teach is a tricky problem yeah.

I think my success criteria (and there are other ones I'm sure) would be "are you able to read Kerrisk's The Linux Programming Interface book" which probably does a involve a bunch of additional syntax, just opening a random page (e.g. setting flags with |= operator).

The other question is... what can you assume the people you're writing for already know?

Someone with decent knowledge of Python you'd need to teach some syntax and surface semantics (straightforward, mostly), data types (straightforward, mostly), and pointers (complicated).

But "getting memory management right" is unnecessary as you said, and "here's how you make .h files for your library" is unnecessary, "what does static and friends mean, what are compilation units etc" is almost certainly unnecessary.

Julia Evans Jun 11, 2025

@itamarst oh man explaining pointers sounds like a huge weird project. recently I realized that the way pointers are used in C is a bit weird compared to in other languages (where instead of having a function return a value in a "normal" way, instead you'll just have a void* function which you pass a pointer to the struct that you want it to populate with the return value)

that's maybe just a consequence of C not having a garbage collector but it's still kind of weird

Denton Gentry Jun 11, 2025

@b0rk @itamarst

That is one of the things I would do without a second thought 30 years ago and now, when I am reminded of it, am appalled.

"You mean it just ASSUMES I'm passing in a chunk of memory big enough to hold whatever it is scribbling?"

Enno Rehling Jun 11, 2025

@b0rk @itamarst There are a few idiomatic solutions to object lifetime like this that you have to know, and will recognize in lots of libraries. Any discussion should start with stack vs. heap allocations, probably.

Enno Rehling Jun 11, 2025

@b0rk @itamarst those other languages you mention just take the shortcut of allocating everything on the heap, and don't have to worry about ownership and lifetime because of the GC.

Julia Evans Jun 11, 2025

@enno what reasons do you think there are to understand the stack vs the heap in C if you're never going to write a C program?

Enno Rehling Jun 11, 2025

@b0rk none. But someone mentioned pointers earlier in the thread. There's a lot that makes C difficult, and memory management is most of it.

Richard Barrell Jun 11, 2025

@b0rk @itamarst the first thing that comes to mind is the fact that a given chunk of memory can be accessed through pointers to different struct types so long as they have identical prefixes of fields.
Even though I feel like modern C library design sort of discourages it?

Weasel Jun 11, 2025

@b0rk @itamarst

That example requires understanding more than just the C programming language. It also requires understanding some system calls (section 2 of the manpages) and library functions (section 3).

The C language involves simple constructs like int, pointers, structures, if, else, while, return, etc.

Casual programmers learning C would need to understand a minimal subset of system or library calls, e.g. simple usage of printf(3) but not sigaction(2). Signals are quite complicated.

Viraptor Jun 11, 2025

@b0rk I wrote a massive slack thread at work explaining what include / library paths are, what pkg-config does, and how to deal with things that don't compile. (in context of Ruby extensions failing) Many people said it was useful to them even though we don't write any C normally.

Julia Evans Jun 11, 2025

@viraptor not all heroes wear capes

Nihilist Jun 11, 2025

@b0rk useful if you are a CPython user :)

Julia Evans Jun 11, 2025

@nihilistsf how so?

systemd-jaded.timer Jun 10, 2025

@b0rk CPPFLAGS is for the preprocessor, though I guess it's strictly correct to use instead of CFLAGS for -I in particular! I just think it might be confusing to refer to CPPFLAGS rather than the more common CFLAGS/CXXFLAGS.

I think one confusing binary toolchain thing that got me when I first learned is that I didn't understand that cc was both the c compiler *and* the linker depending on args, and that often tooling would be really confusing about what it actually is calling when it is "calling the linker" (-fuse-ld referring to an actual linker) *because* a lot of the time, including with rust, "calling the linker" means "calling the compiler driver in link mode".

source: https://docs.jade.fyi/gnu/make.html#index-CPPFLAGS

GNU make

GNU make

Julia Evans Jun 10, 2025

@leftpaddotpy ugh yeah this is kind of a weird example! the reason I ended up using CPPFLAGS instead of CXXFLAGS is that the makefile hardcodes CXXFLAGS to something else so that was the only way I could figure out to pass flags to the compiler without editing the makefile?

but maybe there's another way to do it that I'm missing

systemd-jaded.timer Jun 10, 2025

@b0rk The makefile is supposed to use some other assignment operator in that case, I think += possibly? It's supposed to be the case that you can add your own flags, but it's never quite ideal especially with crusty makefiles, so sometimes you have to do what you have to do.

In fact what it's ideally supposed to do is call pkg-config --cflags somedep and automatically resolve it from the .pc file shipped by the package (PKG_CONFIG_PATH to override those). Alas.

For this regrettable reason of bad build systems, nixpkgs chooses to replace the c compiler with its own little bad shell script, cc-wrapper, which is what processes those NIX_CFLAGS_COMPILE things and injects them into the actual compiler args. This is usually really confusing but at least if it fucks up you actually have a much nicer time than a normal compiler because you can NIX_DEBUG=4 (6? i forget what the just spammy enough one is) or so and get the compiler args dumped right out to stderr without putting up any fights.

Julia Evans Jun 10, 2025

@leftpaddotpy oh man I completely forgot about pkg-config which is probably for the best because i don't understand it at all

systemd-jaded.timer Jun 10, 2025

@b0rk it's *supposed* to be a pretty simple thing of "find the transitive dependency closure of the pkg-config package names you handed it and dump out the concatenation of the cflags/ldflags to use those packages as requested". idk i definitely should read the docs lmao!

@ariadne develops it and seems to be working on some really awesome looking modernization to hopefully reduce the amount of fiddling with cflags and overall crimes people put in their pkg-config files (aws-sdk-cpp for a long time was setting -std=c++17 in their requested cflags!! which is busted because you might want a newer standard yourself).

Ariadne Conill 🐰

@leftpaddotpy @b0rk minimum language version dependency is a complicated topic that is going to require some scoping, but yes, i am looking at more semantic ways of encoding this data

Zack Weinberg Jun 11, 2025

@b0rk @leftpaddotpy looking at the essay again, it seems like you consistently use environment variables to set makefile variables:

CFLAGS='...' make

But you can also set makefile variables as command line arguments:

make CFLAGS='...'

And this works kinda like CSS cascades. A variable value given on the command line overrides what was written in the makefile, but a value given in the environment only applies if there wasn't any value written in the makefile.

Julia Evans Jun 11, 2025

@zwol @leftpaddotpy is there an advantage to passing makefile variables as command line arguments? I noticed that you could do that but it felt kind of weird to me so I ignored it

Zack Weinberg Jun 11, 2025

@b0rk @leftpaddotpy just that if the make file already has a setting for CFLAGS or whatever, you can override that by putting your own setting on the command line but not by putting it in an environment variable. This is what I was trying to say in the first toot.

Julia Evans Jun 11, 2025

@zwol @leftpaddotpy ahh I get it, that's very cool! Will definitely add this to the post. Do you have a sense for which way people packaging software usually pass environment variables to `make`?

Andrew Jeffery Jun 11, 2025

@b0rk @zwol @leftpaddotpy sometimes the override interacts badly with packaging the software, so you can also override the override: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html

Override Directive (GNU make)

Override Directive (GNU make)

Andrew Jeffery Jun 11, 2025

@b0rk @zwol @leftpaddotpy for example: https://source.denx.de/u-boot/u-boot/-/commit/3b0825296aeba69c2cbfd3e179db2e9cbe5e70d7

Making sure you're not a bot!

Andrew Jeffery Jun 11, 2025

@b0rk @zwol @leftpaddotpy https://honk.bewilderbeest.net/u/zev/h/MC49Fd2RMmFyMhTBhG 😄

zev:

@[email protected] Shades of https://devblogs.microsoft.com/oldnewthing/20110310-00/?p=11253

Zack Weinberg Jun 11, 2025

@arj @b0rk @leftpaddotpy A very few times in my life I have actually had to debug such conflicts. It's not fun.

Ted Mielczarek Jun 11, 2025

@b0rk @zwol @leftpaddotpy if there's a configure script, the most well-supported way is to pass them as environment variables when running configure. It should persist them into the Makefile. As a bonus, configure should try running the compiler with them to make sure it actually works.

Zack Weinberg Jun 11, 2025

@tedmielczarek @b0rk @leftpaddotpy Note that configure scripts don't make as much distinction between "VAR=val ./configure" and "./configure VAR=val" as Make does. For Autoconf-generated configure scripts, if I'm reading the INSTALL file right, they're supposed to be equivalent.

pancomputans Jun 11, 2025

@zwol @tedmielczarek @b0rk @leftpaddotpy arguments get passed again during recheck (`./config.status --recheck`). That's not particularly relevant when just building an autoconf-using package, but quite valuable to persist build settings while developing the software in question.

pancomputans Jun 11, 2025

@zwol @tedmielczarek @b0rk @leftpaddotpy Variables that were only (temporarily) set in the environment on the other hand are not easily available when attempting to reconstruct what happened in a previous build.

Zack Weinberg Jun 11, 2025

@pancomputans @tedmielczarek @b0rk @leftpaddotpy ... I'll have to dig into the history to be sure but I think that might actually be a bug.

Zack Weinberg Jun 11, 2025

@b0rk @leftpaddotpy In my experience, almost always on the command line rather than in environment variables, because they want to make sure their settings are used. I haven't done much packaging work though.

Zack Weinberg Jun 11, 2025

@b0rk @leftpaddotpy this behavior is designed for people to set *defaults* for CPPFLAGS or whatever in their shell profile; the theory is that project specific settings (written into the make file) should take precedence over user defaults but then on an individual build basis you should be able to override both.

systemd-jaded.timer Jun 10, 2025

@b0rk For what it's worth, LD_LIBRARY_PATH problems are usually caused by projects misunderstanding the dynamic loader and screwing up something like omitting soname (-install_name on macOS) or otherwise having wrong rpath. Rust currently doesn't do this well with --crate-type=dylib, though that's admittedly rarely used.

I should write my own post about this stuff, how to use otool -L, install_name_tool, and similar to fix these rpath related load problems. Then maybe I can stop accidentally becoming the macOS domain expert 🙃.

Recently after several years of refusing to engage with autotools because of distaste for it (as an occasional begrudging C/C++ developer/packager) I finally spent 20 minutes reading the intro section to the autotools manual and found it really valuable in understanding how the project thinks about itself and *why* it's the way it is. I still don't like it but I understand better what it is trying to do. https://docs.jade.fyi/gnu/autoconf/autoconf.html#The-GNU-Build-System

Autoconf

Autoconf

Julia Evans Jun 10, 2025

@leftpaddotpy i would love to read that post! my understanding of how to do literally anything on a Mac is still extremely fragile

Zack Weinberg Jun 10, 2025

@b0rk I have a footnote: The program that actually does the work of linking is called `ld` and it identifies itself as such in error messages, but it's almost never run directly. Instead the makefile will feed all the .o files and linker options back into `gcc` or `clang`, which will pass them on to `ld` after adding a bunch more options. This is because the extra options are necessary for all but really unusual programs, but also mysterious even to most C programmers. (1/2)

Zack Weinberg Jun 10, 2025

@b0rk "Really unusual" here means things like the Linux kernel.

You can see what all the extra options are by picking a program that fits all in one .c file, like qf, and compiling it by hand with -v added to the options — for qf something like this should do it:

gcc -c -g -O -o qf.o qf.c
gcc -v -o qf qf.o -lncurses

this will spew out a lot of output, look for the "collect2" line (collect2 is *another* wrapper around ld but it doesn't do much of anything nowadays). (2/2)

Zack Weinberg Jun 10, 2025

@b0rk unrelatedly, if you ever do need to learn anything more about autotools I am happy to answer questions.

the esoteric programmer Jun 10, 2025

@b0rk awesome article! I struggled through these kinds of issues for so long, that I developed an intuition regarding how to solve some issues based on the compile errors I got lol. But yeah, this kind of article would have been very useful then :p. Also, you know what's worse than having the misfortune of compiling C programs? writing them, in particular contributing to programs which use autotools. The ./configure script is only one part of it, I forgot what I had to add where and in how many places just to convince autotools that yes,, this is a new file, and also for compiling it, one needs this library too, and I remember that the something else generated the ./configure script. But yeah, honestly, I don't think anyone should have to deal with building and contributing to anything autotools, that system is so oof that I imagine very few people actually know how to use it. You know, messing with that really tought me to appreciate rust more, or for mixed language projects, or c/c++ projects, to appreciate tools like cmake and meson. Btw, what do you think about meson? is it as bad as make in your opinion?

Julia Evans Jun 10, 2025

@esoteric_programmer I don't actually compile a lot of C programs so I don't think I've run into one that uses Meson yet

David Adam Jun 11, 2025

@esoteric_programmer @b0rk in about 2006 a friend of mine quipped that autotools is the second-worst build system, with the equal worst being every other build system. Handwritten configure scripts/Makefiles tended to fail in unusual ways. I don't think that's true any more, but there's still things that are easy in autoconf and hard in CMake, and Rust's feature detection is woeful in comparison.

Thibault Jun 10, 2025

@b0rk A small tidbit which has been *immensely* helpful when compiling makefile-based programs is the `--output-sync` option of make (see attached pic).

Compiler warnings are already hard to read, no need to mix them up with multiple `make` threads!

I think it'd be a good addition to the tips in section 4 of the article.

Julia Evans Jun 10, 2025

@thibault what do you set `--output-sync` to?

Thibault Jun 10, 2025

@b0rk Usually, I invoke make as `make -j10 --output-sync=target`. It allows me to see all of the compiler output of one failed "thread" at a time in the terminal.

Instead of interleaving warnings/errors for multiple in-progress builds, I can see them all at once once they've finished :)

Niccolò Maggioni Jun 11, 2025

@b0rk The tip about looking at how other packaging systems compile a given piece of code is hugely underrated: more than once I've peeked at Arch Linux's AUR just to double check some flags or testing steps.

Kevin Lyda Jun 11, 2025

@b0rk For fun, you can use make for scripting! Just put

#!/usr/bin/make -f

And then just write a normal Makefile syntax file. It's like an easy way to make a shell script with loads of subcommands that sometimes run.

No, it's not a good idea, but I've seen it done!

Mattias Wadman Jun 11, 2025

@lyda @b0rk want to see something cursed? you can change what shell make should use

blobcatgrimacing

https://github.com/lgersman/make-recipes-in-different-scripting-languages-demo

GitHub - lgersman/make-recipes-in-different-scripting-languages-demo: Did you kow that Make recipes can be written in almost any scripting language ? You don't need to know about shell scripting. Just use NodeJS, Typescript , Python or whatever you want !

Did you kow that Make recipes can be written in almost any scripting language ? You don't need to know about shell scripting. Just use NodeJS, Typescript , Python or whatever you want ! - lgers...

GitHub

Mattias Wadman Jun 11, 2025

@lyda @b0rk even more cursed you can use make with make as shell! https://github.com/lgersman/make-recipes-in-different-scripting-languages-demo/tree/main/make

make-recipes-in-different-scripting-languages-demo/make at main · lgersman/make-recipes-in-different-scripting-languages-demo

Did you kow that Make recipes can be written in almost any scripting language ? You don't need to know about shell scripting. Just use NodeJS, Typescript , Python or whatever you want ! - lgers...

GitHub

Kevin Lyda Jun 11, 2025

@wader @b0rk Wow. That's new to me, but I get it. I hate it, but I get it.

Awesome, I will now go terrify some people. Thanks!

Mattias Wadman Jun 11, 2025

@lyda @b0rk btw Dockerfile:s also support changing shell 🤔

Kevin Lyda Jun 11, 2025

@wader @b0rk noooooooooo!

Flesh & bone by the telephone Jun 11, 2025

@b0rk I am a C programmer, and know make rather well... But I struggle to explain it to people.

This is an excellent guide, and really nails a lot of the things that someone like me typically misses when trying to teach someone.

Thanks for writing it and thanks for sharing!