PERCPU(9)

PERCPU(9) | Kernel Developer's Manual | PERCPU(9)

# NAME

`percpu`, `percpu_alloc`, `percpu_create`, `percpu_free`, `percpu_getref`, `percpu_putref`, `percpu_foreach`, `percpu_foreach_xcall` — per-CPU storage allocator

# SYNOPSIS

`#include <sys/percpu.h>`

typedef void (*percpu_callback_t)(void *, void *, struct cpu_info *);

percpu_t *
`percpu_alloc`(size_t size);

percpu_t *
`percpu_create`(size_t size, percpu_callback_t ctor, percpu_callback_t dtor, void *arg);

void
`percpu_free`(percpu_t *pc, size_t size);

void *
`percpu_getref`(percpu_t *pc);

void
`percpu_putref`(percpu_t *pc);

void
`percpu_foreach`(percpu_t *pc, percpu_callback_t cb, void *arg);

void
`percpu_foreach_xcall`(percpu_t *pc, u_int xcflags, percpu_callback_t cb, void *arg);

# DESCRIPTION

The machine-independent `percpu` interface provides per-CPU, CPU-local memory reservations to kernel subsystems. `percpu_alloc`(size); reserves on each CPU an independent memory region of size bytes that is local to that CPU, returning a handle (percpu_t) to those regions. A thread may subsequently ask for a pointer, p, to the region held by the percpu_t on the thread's current CPU. Until the thread relinquishes the pointer, or voluntarily sleeps, the thread may read or write the region at p without causing interprocessor memory synchronization.

# FUNCTIONS

`percpu_alloc`(size)
  Call this in thread context to allocate size bytes of local storage on each CPU. The storage is initialized with zeroes. Treat this as an expensive operation. `percpu_alloc`() returns a handle for the per-CPU storage.
`percpu_create`(size, ctor, dtor, arg)
  Like `percpu_alloc`(), but before any access to the storage on any CPU, arrange to call `(*ctor)`(p, arg, ci), where p is the pointer to that CPU's storage and ci is the struct cpu_info * for that CPU. This may happen synchronously in `percpu_create`() or when a CPU is attached later. Further, arrange that `percpu_free`() will do the same with `(*dtor)`(p, arg, ci).

  ctor and dtor MAY sleep, e.g. to allocate memory or to wait for users to drain before deallocating memory. Do not rely on any particular order of iteration over the CPUs.

`percpu_free`(pc, size)
  Call this in thread context to return to the system the per-CPU storage held by pc. size should match the size passed to `percpu_alloc`() or `percpu_create`(). When `percpu_free`() returns, pc is undefined. Treat this as an expensive operation.
`percpu_getref`(pc)
  Disable preemption and return a pointer to the storage held by pc on the local CPU. Use `percpu_getref`() in either thread or interrupt context. Follow each `percpu_getref`() call with a matching call to `percpu_putref`().

  Caller MUST NOT sleep after `percpu_getref`(), not even on an adaptive lock, before `percpu_putref`().

`percpu_putref`(pc)
  Indicate that the thread is finished with the pointer returned by the matching call to `percpu_getref`(). Re-enables preemption.
`percpu_foreach`(pc, cb, arg)
  For each CPU, with ci being the corresponding struct cpu_info * and p the CPU-local storage held by pc, run `(*cb)`(p, arg, ci);. The call to cb runs in the current thread; use `percpu_foreach_xcall`() to execute the call to cb on the remote CPUs.

  Must be used in thread context. cb MUST NOT sleep except on adaptive locks, and should be fast. Do not rely on any particular order of iteration over the CPUs.

`percpu_foreach_xcall`(pc, xcflags, cb, arg)
  Like `percpu_foreach`(), except the call to cb executes on the remote CPUs. The xcflags argument specifies how the cross calls are invoked; see xc_broadcast(9). The same caveats and restrictions that apply to `percpu_foreach`() also apply to `percpu_foreach_xcall`().

# CODE REFERENCES

The `percpu` interface is implemented within the file sys/kern/subr_percpu.c.

# SEE ALSO

atomic_ops(3), kmem(9), pcq(9), pool_cache(9), xcall(9)

# HISTORY

The `percpu` interface first appeared in NetBSD 6.0.

# AUTHORS

YAMAMOTO Takashi <yamt@NetBSD.org>

February 7, 2020 | NetBSD 11.0