$RodHat_
Console Tips

taskset pins a process to specific CPUs. numactl keeps its memory local too.

Published by

taskset pins a process to specific CPUs. numactl keeps its memory local too.
Photo: AI-generated — no human photographer / RodHat AI Cover

perf stat showed your process at 0.6 IPC with an elevated cache-miss rate. The code isn’t wrong. The working set isn’t enormous. The hardware is fast. And the process is getting shuffled between cores every few hundred milliseconds by a scheduler that has no idea this particular thread cares deeply about L3 cache locality.

This is a real category of problem, not a contrived benchmark scenario. High-throughput network daemons, in-process caches, latency-sensitive services on multi-socket servers — all of them can suffer from the scheduler doing its job correctly and still ruining yours.

taskset is the fix. numactl is its smarter sibling. Neither of them is elegant. Both work.

Know your topology first

Before pinning anything, know what you’re pinning to:

lscpu -e

Output on a dual-socket, 8-core-per-socket box:

CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
  0    0      0    0 0:0:0:0          yes
  1    0      0    1 1:1:1:0          yes
  2    0      0    2 2:2:2:0          yes
  3    0      0    3 3:3:3:0          yes
  4    0      0    4 4:4:4:0          yes
  5    0      0    5 5:5:5:0          yes
  6    0      0    6 6:6:6:0          yes
  7    0      0    7 7:7:7:0          yes
  8    1      1    8 8:8:8:1          yes
...
 15    1      1   15 15:15:15:1       yes
 16    0      0    0 0:0:0:0          yes  ← HT sibling of CPU 0

CPUs 0–7 are physical cores on socket 0, NUMA node 0. CPUs 8–15 are on socket 1, NUMA node 1. CPUs 16–31 (if hyperthreading is on) are the logical siblings sharing physical cores with CPUs 0–15.

This matters because pinning to CPUs 0 and 16 means both logical threads compete for the same physical core. Pinning to CPUs 0 and 1 means two independent physical cores with shared L3. Pinning to CPU 0 and CPU 8 means two different sockets — memory accesses that cross the NUMA boundary.

numactl --hardware gives a broader picture:

numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23
node 0 size: 64370 MB
node 0 free: 51203 MB
node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31
node 1 size: 64403 MB
node 1 free: 49811 MB
node distances:
node   0   1
  0:  10  21
  1:  21  10

Distance 10 is local. Distance 21 is remote — a cross-node memory fetch costs roughly 2x a local one on most NUMA systems. If your process allocates on node 1 and runs on node 0, you are paying that tax on every cache miss.

taskset: pin at launch or to a running process

Launch a process pinned to CPU 0:

taskset -c 0 ./myapp

Pin to a range:

taskset -c 0-3 ./myapp

Pin to a non-contiguous set:

taskset -c 0,2,4,6 ./myapp

Read the current affinity of a running process:

taskset -cp $pid
pid 4821's current affinity list: 0-31

Set the affinity of a running process — no restart required:

taskset -cp 0-7 $pid

This takes effect immediately. The process migrates to a CPU in the new set at the next scheduler tick. Affinity is inherited by child processes and threads, so pin the parent and all workers pin with it.

The -c flag uses the comma/range list format. Without -c, taskset takes and emits a hex bitmask (0xff for CPUs 0–7). The list format is more legible for anything beyond a handful of CPUs.

numactl: CPU binding plus memory binding

taskset controls which CPUs a process can run on. It does nothing about where memory is allocated. A process pinned to CPUs 0–7 on node 0 can still allocate from node 1 if node 0 is under memory pressure. numactl pins both:

numactl --cpunodebind=0 --membind=0 ./myapp

All threads run on CPUs in NUMA node 0. All memory allocations come from node 0. Local access from start to finish, assuming the process fits in node 0’s memory.

If you want to specify CPUs directly rather than by NUMA node:

numactl --physcpubind=0-7 --membind=0 ./myapp

For a running process, numactl does not take a -p flag the way taskset does — it only applies at launch. To move a running process’s memory policy, you need set_mempolicy(2) from inside the process, or restart with numactl. Taskset’s -p flag is the runtime knob; numactl is the launch-time knob.

After the fact, check where a process’s memory actually landed:

numastat -p $pid
Per-node process memory usage (in MBs) for PID 4821 (myapp)
                           Node 0          Node 1           Total
                  --------------- --------------- ---------------
Huge                         0.00            0.00            0.00
Heap                       412.31           87.44          499.75
Stack                        0.02            0.00            0.02
Private                    201.14          312.49          513.63

That heap and private split tells the story. 312MB of private memory is on node 1 while the process runs on node 0. Every access to those pages crosses the NUMA interconnect. numactl --membind=0 at relaunch fixes it.

IRQ affinity: the part you’ll forget

Pin a network daemon to CPUs 0–3 and still see inter-core interrupts in perf stat? The NIC’s interrupts are probably being handled by a different CPU, then the daemon is woken up there, then migrated back. You pinned the process but not the interrupt.

Find which IRQ belongs to your NIC:

cat /proc/interrupts | grep eth0
 42:      0    142839      0      0  PCI-MSI 524288-edge  eth0-TxRx-0
 43:      0         0 219441      0  PCI-MSI 524289-edge  eth0-TxRx-1

IRQ 42’s affinity mask — a hex bitmask of which CPUs can handle it:

cat /proc/irq/42/smp_affinity
ffffffff

All CPUs. Set it to CPU 0 only (0x1):

echo 1 > /proc/irq/42/smp_affinity

Or use the list format:

echo 0 > /proc/irq/42/smp_affinity_list

IRQ 43 to CPU 1:

echo 1 > /proc/irq/43/smp_affinity_list

Now the NIC’s receive interrupts are handled on the same CPUs where your daemon runs. Packet arrives, interrupt fires on CPU 0, daemon wakes up on CPU 0, data is already in L1. One hop, not three.

irqbalance runs on most Linux systems and will undo this — it’s a daemon that periodically redistributes IRQ affinities for load balancing. Disable it or add your IRQs to its exclusion list if you’re doing manual IRQ pinning:

systemctl stop irqbalance   # if you have systemd (condolences)

On FreeBSD, interrupt affinity is handled through pcpu and the dev.*.N.%parent sysctl hierarchy — different model, same problem.

When NOT to pin

Pinning is a scalpel that becomes a club if you wave it at everything.

A multithreaded application with more threads than pinned CPUs will have its threads fighting over the pinned set. You traded scheduler noise for lock contention and throughput collapse. Pin to at least as many CPUs as the thread pool is wide.

A process that’s CPU-bound for two minutes then spends ten minutes waiting on I/O should probably not be pinned — the pinned CPUs sit idle while the process blocks, and the scheduler can’t use them for anything else without violating your affinity mask.

Short-lived processes — anything that forks and exits in under a second — cost more to pin than you’ll save. The overhead of taskset itself dwarfs the cache-warmup benefit.

The correct order: profile first with perf stat, confirm the bottleneck is cache-miss rate or NUMA cross-node traffic (via numastat), then pin. Pinning without evidence is chasing shadows and adding complexity you’ll forget about in six months.

Taskset in a container

Most runtimes respect the task’s CPU affinity mask. Docker respects it; Kubernetes does too, via the kubelet’s CPU manager policy. In a container, taskset inside the container sets affinity within the cpuset that cgroup already constrains it to — you cannot escape the cgroup cpuset, but you can subset it.

For NUMA-aware container placement, Kubernetes’ topologyManager with single-numa-node policy is the right lever if you’re operating at that scale. For a single box, numactl at the container’s --cpuset-cpus + a host-side numactl wrapper is more straightforward.

The theory behind why cache topology matters — cache lines, TLB shootdowns, NUMA memory latency and how MOESI protocol coherency interacts with memory placement — is in the architecture chapters of The Linux Programming Interface. Kerrisk’s treatment of NUMA in the virtual memory chapters is the most grounded explanation I’ve read that doesn’t require owning the hardware to follow.