$RodHat_/search
Console Tips

SCHED_DEADLINE: ask the kernel for a CPU budget and get told no

Published by · Updated

This article may contain Amazon affiliate links. I may earn a commission if you buy through them. Disclosure.

Editorial card: SCHED_DEADLINE: ask the kernel for a CPU budget and get told no
Editorial card: RodHat
On this page · 4 sections
  1. The three numbers
  2. The kernel can refuse
  3. Three ways it bites
  4. Where it fits

Every real-time scheduling story I know ends the same way. Somebody sets SCHED_FIFO priority 99 on their audio thread or their trading loop, it works great, and then a second thing also set to 99 shows up and the box wedges. FIFO is a priority system. It has no idea what you actually need, and it never says no.

SCHED_DEADLINE does say no. It has been in Linux since 3.14, and almost nobody uses it, which is a shame, because it is the only policy where you state the requirement instead of a rank.

The three numbers

You give the kernel a reservation in nanoseconds:

  • runtime: CPU time the task needs per period
  • deadline: by when, relative to period start, that runtime must be delivered
  • period: how often the whole thing repeats

The constraint is runtime <= deadline <= period. The scheduler runs tasks earliest-deadline-first. No glibc wrapper, so from C you call syscall(SYS_sched_setattr, ...) with a struct sched_attr. From a shell, chrt does it:

# 5ms of CPU every 10ms
chrt -d --sched-runtime 5000000 --sched-deadline 10000000 \
     --sched-period 10000000 0 ./rt-worker

That trailing 0 is the priority argument, which must be zero for this policy. It needs CAP_SYS_NICE; plain users get EPERM.

The kernel can refuse

This is the whole point. When you ask for a reservation, admission control checks whether the total deadline bandwidth (sum of runtime/period across tasks) fits within what the RT throttling knobs allow, by default 950000 of every 1000000 µs (/proc/sys/kernel/sched_rt_runtime_us and sched_rt_period_us), scaled across CPUs. If it does not fit, you get EBUSY at the call, not a missed deadline at 3am.

chrt -d --sched-runtime 9000000 --sched-deadline 10000000 \
     --sched-period 10000000 0 ./a &   # fits
chrt -d --sched-runtime 9000000 --sched-deadline 10000000 \
     --sched-period 10000000 0 ./b     # may fail: Device or resource busy

Whether the second one fails depends on your CPU count, so check nproc before you panic. Admission is a bandwidth check, not a proof your workload actually meets its deadline. That part is still on you.

Three ways it bites

Runtime is a hard budget. Burn through it before the period ends and the task is throttled until the next period. Size runtime from your measured worst case, not the average. You can ask for SCHED_FLAG_DL_OVERRUN to get SIGXCPU when it happens, which beats guessing.

No fork by default. fork(2) from a deadline thread fails with EAGAIN unless you set SCHED_FLAG_RESET_ON_FORK, in which case the child falls back to SCHED_OTHER. Wrapping a shell script in chrt -d and wondering why nothing spawns is a classic.

Cgroup RT limits. If you are running under a cgroup hierarchy with the cpu controller, check how your setup treats realtime bandwidth before assuming the reservation is available to your service. I have seen this bite container setups; read the cgroups v2 tip for how the controller tree is laid out.

Where it fits

Not general servers. Use it for periodic work with a real deadline: media pipelines, control loops, a sampler that has to tick on time. If your problem is cache thrash from migration, that is CPU pinning, a different lever, and the two combine fine. For I/O fairness rather than CPU, see ionice.

chrt -p <pid> shows the current policy. SCHED_FLAG_RECLAIM lets a deadline task soak up bandwidth other deadline tasks reserved but did not use, if you would rather be greedy than tidy.

BSD side: FreeBSD has rtprio(2) and POSIX FIFO/RR, but nothing with admission control. This one is Linux’s, and it is genuinely good. I hate admitting it.

Sources

  1. sched_setattr(2) Linux man page (man7.org)
  2. sched(7) Linux man page (man7.org)
  3. chrt(1) Linux man page (man7.org)