01 / The problem
A slow service call hid an inefficient poller.
Waybar requests one JSON status update on a fixed schedule. rstat's original Waybar module took roughly 800 milliseconds per update because it launched a shell script that invoked command-line tools including awk, bc and powerprofilesctl.
We ask: Why is it slow? And we don’t stop until we have a real, measurable answer:
- Service round trip ~800 ms with the D-Bus request in the update path
powerprofilesctlcrossed process and D-Bus boundaries to retrieve the active power profile, even though Linux exposed the same value through a sysfs file.- Process churn Shell + at least 3 child processes / update
- Each external utility required
fork(),exec(), dynamic loading and process teardown. The shell implementation paid those costs again on every update. - Text interfaces ~15 ms remained after the D-Bus call was removed
- Linux's process filesystem, procfs, converted kernel counters into text. The command-line utilities then parsed that text and passed more text through the shell.
- No in-process history 0 counters, buffers or file descriptors retained
- A short-lived script couldn't retain previous counters, file descriptors or buffers in memory between Waybar refreshes.
02 / The progression
Remove the dominant delay, then measure what remained.
We first replaced the one-shot shell module with a long-running Rust collector that retained counters, buffers and previous samples between Waybar updates. The Rust collector removed nearly every external utility, but powerprofilesctl still contacted the system power-profile daemon over D-Bus inside every update.
Reading the active power profile directly from /sys/firmware/acpi/platform_profile removed that round trip. The Rust procfs collector then collapsed three process-directory walks into one, reused an 8 KB read buffer, took resident-set size (RSS) from /proc/[pid]/stat instead of opening statm, and identified kernel threads from the same record so it could skip their I/O file in the default view. Together, those changes cut each update from roughly 800 to 15 milliseconds.
The remaining cost came from the design. Every refresh still enumerated /proc, opened files for each live process and asked the kernel to turn counters into text. Local tuning could reduce that walk, but it couldn't remove it.
03 / The architecture
Stop asking the kernel to describe what it already knows.
The optimized Rust procfs collector still scaled with the number of live processes. It opened stat for every process and io for each ordinary process. Across roughly 300 processes, those files can require about 1,800 open, read and close system calls per refresh, before directory enumeration and global metrics. We replaced that refresh-time reconstruction with event-time accounting. A small extended Berkeley Packet Filter (eBPF) program, checked by the Linux kernel before loading, now records CPU residency and snapshots the kernel's RSS and task-I/O counters when scheduler events already identify the relevant task.
Four scheduler tracepoints maintain task state as processes fork, run, exit and are freed. Every record is keyed by its process identifier (PID) and kernel start timestamp, preventing a reused PID from joining two unrelated process lifetimes. A retired record remains in the shared eBPF statistics map until the Rust daemon consumes its final counters, so short-lived work remains visible. One procfs scan at startup seeds blocked threads and zombie process leaders that already existed when the eBPF program attached; that scan doesn't recur during refreshes.
sched_process_fork
sched_switch
sched_process_exit
sched_process_free
Four tracepoints create, update and retire task state.
CPU ← every switch
RSS / I/O ← gated
At most once per task every 10 ms; exit forces the final snapshot.
stats[PID, start time]
Final counters survive task exit until userspace consumes them.
batch map lookup
reused buffers → Waybar JSON
04 / The hot path
Once the architecture was right, small costs became visible.
The eBPF design removed the recurring procfs walk. We then bounded every output list and reused the Rust daemon's allocated storage after it reached its steady working set.
- Batch the statistics map
- Linux's
BPF_MAP_LOOKUP_BATCHoperation transfers many task records into preallocated key and value arrays in one system call. The iterative fallback requires separate key-advance and lookup calls for each record. - Gate the snapshots
- CPU residency is charged on every switch. RSS and I/O are sampled at most once per task every 10 milliseconds, with a forced final snapshot when the task exits.
- Preallocate the working set
- The statistics, scheduler-start and PID-to-start-time hash maps have fixed capacities and are created before their tracepoints attach. Userspace reserves its batch arrays and process-aggregation tables once, then clears them without releasing their storage.
- Reuse contiguous snapshots
- Two vectors retain capacity and swap roles between samples. Sorting records by PID and start time gives cache-friendly traversal and binary-search lookups against the previous sample.
- Minimise system reads
sysinfo()returns all three load averages in one system call. The daemon discovers and opens its sysfs temperature, frequency, graphics-processor and throttling files once;pread()fills fixed byte buffers, and numeric values are parsed directly from those bytes.- Write the fixed schema directly
write!appends the known Waybar schema to reusable tooltip and JSON strings. A small emitter escapes task names in place without a general serializer or temporary formatted strings.- Bound the result sets
- CPU, memory and I/O leaders are selected into fixed five-entry arrays, while the list of tasks blocked in Linux's uninterruptible D state is capped at ten. Only those small arrays are sorted for output.