Mastodawn

Any ideas why perf_event_open() would often return 0 CPU instructions retired when I pin a thread to a particular core?

I guess sometimes the kernel can't put enough counters in the limited number provided by the hardware, and then it either samples (giving reduced data) or just... returns 0?

And kinda seems like whatever is using the counters is some other process, since I still have intermittent issues when I use a lock to limit this to one counter at a time in my code.

And you can actually see the time enabled vs time running and yeah this is a problem.

Show thread

Per Vognsen Mar 23

@itamarst IIRC, since Sandy Bridge (i.e. a long time ago) you can have at least 4 hardware performance counters active at a time per core. I know it's 6 on newer Zens. But yeah, if you try to enable more counters than that, the software driver (perf in this case) has to time-multiplex the hardware counters and rely on statistical sampling to estimate the count. Not sure if that's the actual reason for what you're seeing, though.

Show thread

Itamar Turner-Trauring 6d ago

@pervognsen I eventually discovered that setting _both_ CPU core restriction _and_ process id restriction made schedaffinity no longer break the results from perf_event_open. So now I'm fiddling with APIs, then docs, then a release: https://codeberg.org/itamarst/bigo

(I guess also figure out if I want to fight a name squatter or pick another crate name.)

UPDATE: Nope, still broken, I was being fooled by my fallback logic which switches to time-based measures.

bigo

Codeberg.org

Show thread

Alexander Monakov 6d ago

@itamarst @pervognsen I'd suggest to use .observe_self instead of .observe_pid (monitoring another task needs elevated privileges, I'm surprised it worked at all for you; .observe_self, and passing pid=0 to perf_event_open is specifically there to support the self-monitoring use-case which needs less privs)

Show thread

Itamar Turner-Trauring 5d ago

@amonakov @pervognsen I want to observe a single thread, not the whole process (there are other threads I want to observe). Also it's not clear to me you actually _ought_ to need more privileges given it's a thread within the same process. The man page might be wrong in this edge case? Would have to dig into Linux kernel source code to check though.

Show thread

Paul Khuong

@itamarst @amonakov @pervognsen you can specify the TID. IIRC, the problem with specifying the core is you can get access to counters that don't clearly map to a process/uid.

Show thread

Itamar Turner-Trauring 5d ago

@pkhuong I'm only doing retired CPU instructions. As mentioned the reason I need to specify CPU core is that otherwise schedaffinity limiting to that core results in getting results of 0 instead of the real numbers... At some point I should verify if that happens with a different high level wrapper I guess.

UPDATE: Nope, still broken when using both, I was being fooled by my fallback logic which switches to time-based measures.