new achievement unlocked: use a syscall valgrind hasn't implemented
I think I need to call the linux kernel hotline, anyone know the number
god I hate reading C code. dude passes `&foo` to a function - is it reading from it? is it writing to it? who knows.
remember to remove binutils from your production systems so the exploits won't run, yeesh
infosec is at that rare skill intersection where the red team writes python code, but in C
copilot is now hallucinating more RAM

whoa, `println!("{:016x}")` writes every 0 with a single write, no buffering involved - presumably to avoid allocations.

maybe that only applies to stderr but... that can't be efficient 🤔

it's called a man page because it won't shut up about stuff you already know and doesn't actually have the answer to your question

okay I'm seriously starting to lose it - there's exactly two codepaths where userfaultfd_register can return ENOMEM:

* if mmget_not_zero is falsy (returns 0)
* if mas_find can't find the VMA

I can't tell why either of these would happen in my case. I really wish there was a linux kernel hotline.

https://elixir.bootlin.com/linux/v6.2.2/source/fs/userfaultfd.c#L1333

alriiiiiiiiight, all my crimes are working, thanks to @ljs for the assist! I'm super excited about this next project.

really the only question left is how many videos this is.

shorter more focused videos tend to work better, there's at least 3 separate concepts I haven't really covered anywhere that I need to introduce for this thing to make sense 🧐

@fasterthanlime The number of videos I generally need on any topic that doesn't involve cute animals or blowing things up is zero.
@fasterthanlime @ljs curious what the answer was, if it was something not too onerous to describe :)

@mokomull @fasterthanlime mmget_not_zero() due to fd referencing a now exec-cleared mm_struct.

I researched this in the least elegant way possible with the help of pr_err()'s

@ljs @mokomull and unfortunately the way around it involves setting up userfaultfd _after_ exec, which involves ptrace sadness or LD_PRELOAD trickery. it's fine though!

@fasterthanlime can also happen in split_vma() at https://elixir.bootlin.com/linux/v6.0/source/mm/mmap.c#L2740 if the map count exceeds max map count or if __split_vma() throws it too (see https://elixir.bootlin.com/linux/v6.0/source/mm/mmap.c#L2676 )

(might be more not dug thoroughly)

I am permanently stuck in v6.0 land due to book writing so forgive that but suspect code similar in 6.2 (just checked, is)

You under some terrible memory pressure somehow?

@ljs right. I don't think that's happening here.

I am doing a certain amount of crimes though - register only fails with ENOMEM _after_ exec.

The userfaultfd FD still exists after exec (it's not CLOEXEC), I just can't register anything to it - not from the "manager process" (tracer), not from the tracee.

I've checked everything I can think of and am losing hope at this point. Maybe just some unfortunate ptrace/userfaultfd interaction? Not sure.

@fasterthanlime is this easily repro? You can stuff it in a qemu setup and trace through.

Unfortunately userfaultfd is something I don't cover in the book but I might actually be using it soon...

If you exec you're clearing down all of the VMAs, and that fd somehow relates to exec'd (presumably forked) process's mmap this could be source of woes.

If you have a small repro I could try to play with it if I have a sec

@ljs is 400 lines small? I can add you to the github repo!
@fasterthanlime haha relatively, sure. https://github.com/lorenzo-stoakes is my github
lorenzo-stoakes - Overview

Coder with a particular interest in kernel, low-level and systems development. Writing a book on the linux kernel memory manager. - lorenzo-stoakes

GitHub
@fasterthanlime have you tried turning it off and on again? #TheITCrowd
@fasterthanlime
Fwiw I've used kgdb in the past with some success to solve the "why tf am I getting this error" quest ("some success" because I remember the setup being hell but idr why)

@fasterthanlime I frequently use ftrace to see the call trace through the kernel to figure out which error condition I'm using.

I even have little functions to turn on/off ftrace around a function I'm interested in

@fasterthanlime May I talk to you about our lord and saviour, GNU TeXinfo?
@fasterthanlime [just joking, I still do not know how to properly navigate the `info` command]

@schmittlauch imo (as the author of said page) the answer is https://docs.jade.fyi

(single page builds of all the info pages i could think of, which you can ctrl f)

texinfo is nice and has the only canonical docs but gnu hasn't posted adequate html builds of their stuff that aren't on a billion pages, but you may find their builds on Google sometimes.

@leftpaddotpy @schmittlauch this is an awesome site. Bookmarking it now!
@schmittlauch @fasterthanlime Nothing makes me madder fast than typing "man whatever" and being told to read the info file.
@resuna @schmittlauch @fasterthanlime How about software that fails to distribute any documentation and goes "lol go on our website" (where it's often been down or filled with dead links for years)?
@fasterthanlime I've said before that Linux is sexist because every time I need to figure something out I'm told to consult a man
@fasterthanlime OTOH, you can learn a lot of "oh cool, it can do that?" and "I didn't know about this pitfall" from skimming over a man page.
@fasterthanlime Linux is the best operating system because "wtf man" is a legitimate command.

@fasterthanlime i can't recall if you've been in today's lucky 10000 about this one yet, but here's a truly maddening fact: almost no GNU software has a complete man page.

gcc, gdb, and many others have the complete docs as info pages (which can be built to pdf or html, no need to use the mess that is info(1); such things can be acquired from https://www.gnu.org/manual/manual.html or https://docs.jade.fyi)

GNU Manuals Online - GNU Project - Free Software Foundation

@leftpaddotpy @fasterthanlime Yep, GNU tools are really horrible there. I long for the great SunOS man pages for which @alanc is currently still doing a lot of work. Outside of Solaris systems I mostly use the publicly available POSIX man pages which are *shorter* yet more to the point as they're concise and well-written:
https://pubs.opengroup.org/onlinepubs/9699919799/
Of course they may not document GNU extensions but it works for *most* of what I need just fine.
The Open Group Base Specifications Issue 7, 2018 edition

@ljrk @fasterthanlime @alanc i personally find man pages to be a seriously outmoded format: they predate "hyperlinks" and hooo boy do you ever need hyperlinks in docs. if you're just looking up function docs it's more ok, but man pages kind of suck. (also no fun to author)

my belief in hypertext led to making docs.jade.fyi

git docs and systemd docs both have canonical online versions that have working hyperlinks but are otherwise identical to their man pages.

@leftpaddotpy @ljrk @fasterthanlime
yeah - back in the 90's the Solaris man pages were converted to DocBook so that they could have hyperlinks, and a "Phase 2" project was promised to provide a man page viewer that could follow links. But Phase 2 never arrived, and the only links included were to other pages (which you can see in the HTML output), but not inter-page links that would be useful.

The man pages ended up as cumbersome reference manuals to search, but not good modern documentation.

@alanc @ljrk @fasterthanlime ah. what I've done to docbook stuff I've had to work with (notably the postgres manual) is to treat it like texinfo: build a single giant page and let the browser deal with it with hyperlinks and search. it mostly works but i want a sidebar. one day i might implement that.
@leftpaddotpy
Yeah, we did similar with Docbook at X.Org, but that was mainly for protocol & API specifications, not general purpose docs. Still, adding lots of internal links seemed to really help there.

@leftpaddotpy @fasterthanlime @alanc Absolutely, I prefer hyperlinked formats as well, the POSIX HTML man pages are exactly that though :-)

They only share the structure with the nroff variants. In background the nroff man pages and the HTML variants are generated from the same sources. Theoretically that could be implemented with the Linux man pages as well, you can even simply compile the roff files using groff to HTML but I think they're not interlinked.

@ljrk @alanc i get inter page linking by using the vim man viewer. intra page linking on the other hand is the real killer 😭

@ljrk You can convert man pages to hyperlinked HTML and/or PDF using [mandoc(1)](https://mandoc.bsd.lv/):

```
man -Thtml strcmp > strcmp.html
man -Tpdf strcmp > strcmp.pdf
```

Example of a generated HTML page: https://mandoc.bsd.lv/man/mandoc.1.html

However, IIRC, hyperlinks are only generated for mdoc(7) format, not man(7).

mandoc | UNIX manpage compiler

@fasterthanlime Actually, it's short for "manual". 🙂

@fasterthanlime Right, we can turn off the fediverse now. You’ve officially won it :)

👌🤣

@fasterthanlime You can tune a filesystem, but you can't tune a fish.
@katzmandu @fasterthanlime but you can tuna fish, and in a can.
@fasterthanlime manuals used to be good
@fasterthanlime nah, it's called a man page because it actually gives you a direct answer to the question you asked.
@fasterthanlime I heard that's a problem with betas, the alphas are much better
@fasterthanlime First of all, ok fine yeah, I get the joke. Haha. But also I learned perl largely offline at a time when I didn't have continuous internet access thanks to the manual pages. Manual pages are pretty great, or at least were pretty great.
@fasterthanlime It should be a "ual" page like... "you'll" figure it out.

@fasterthanlime you’ll be delighted* to learn** that bro pages are a thing http://bropages.org/

* probably not actually delighted
** unless you knew about these already

bro: just get to the point!

@T045T I did know about bro pages! I prefer https://tldr.sh/ myself
tldr pages

Simplified and community-driven man pages

@fasterthanlime nice! The „bro“ name always felt a bit cringey to me, especially the „bro thanks“ and „bro …no“ upvote/downvote mechanism…
@T045T @fasterthanlime many bro thanks died to bring us this information
@fasterthanlime I miss man pages, particularly now that I'm having to depend on rustdocs.

@fasterthanlime `man splain`

>diagnostics, splain - produce verbose warning diagnostics

yeah, that tracks.

GitHub - tldr-pages/tldr: 📚 Collaborative cheatsheets for console commands

📚 Collaborative cheatsheets for console commands. Contribute to tldr-pages/tldr development by creating an account on GitHub.

GitHub