In the past years, a new interface for futex had been under development, in the hope to solve the issues found in the current syscall. One of the remaining things to be fixed is to add NUMA-awareness to it. Currently, all kernelspace memory resources are stored in a single memory node, adding latency for every futex operation that happens outside of this node. The goal of this session is to present an interface for NUMA operation for futex2, collect feedback and discuss ideas around futex scalability. Here's the proposed interface: https://lore.kernel.org/lkml/36a8f60a-69b2-4586-434e-29820a64cd88@igalia.com/ (c) Linux Plumbers Conference 2022 Dublin, Ireland Sept. 12-14, 2022 https://lpc.events/event/16/page/169-lpc-2022-overview

1. A NUMA interface for
futex2
André Almeida
LPC 2022
1

2. futex2
Ongoing effort to solve futex issues
futex_waitv() merged
2

3. NUMA bottleneck
For each FUTEX_WAIT operation, there's a entry in a
kernelside hash table
And there's a single hash table, in one node
This creates a cost for the other nodes to access the
table
3

4. futex2 design
A single syscall per operation (no multiplex)
Designed to solve a bunch of issues at once (waitv,
variable size, NUMA)
4

5. futex2 design
futex_wait(void *uaddr, unsigned int val, unsigned i
struct timespec *timo)
futex_wake(void *uaddr, unsigned long nr_wake,
unsigned int flags)
5

6. futex2 design
futex_waitv(struct futex_waitv *waiters,
unsigned int nr_futexes,
unsigned int flags,
struct timespec *timeout,
clockid_t clockid)
struct futex_waitv {
__u64 val;
__u64 uaddr;
__u32 flags;
__u32 __reserved;
};
6

7. futex2 design
futex_requeue(struct futex_requeue *rq1,
struct futex_requeue *rq2,
unsigned int nr_wake,
unsigned int nr_requeue,
u64 cmpval, unsigned int flags)
struct futex_requeue {
void *uaddr;
unsigned int flags;
};
7

8. FUTEX_NUMA_FLAG
struct futexX_numa {
__uX value;
__sX hint;
};
struct futex32_numa f = {.value = 0, hint = -1};
8

9. FUTEX_NUMA_FLAG
struct futexX_numa {
__uX value;
__sX hint;
};
value is the futex value
hint can be [0, MAX_NUMA_NODES) to specify a
node or -1 for the current node
9

10. FUTEX_NUMA_FLAG
struct futexX_numa {
__uX value;
__sX hint;
};
struct futex32_numa f = {.value = 0, hint = -1};
futex_wait(&f, 0, FUTEX_NUMA | FUTEX_32, NULL);
// getting the lock
f.value = 1;
10

11. FUTEX_NUMA_FLAG
struct futex32_numa f = {.value = 0, hint = 2};
futex_wait(&f, 0, FUTEX_NUMA, NULL); // T1, N3
futex_wait(&f, 0, FUTEX_NUMA, NULL); // T2, N0
futex_wait(&f, 0, FUTEX_NUMA, NULL); // T3, N2
futex_wake(&f, 2, FUTEX_NUMA); // T4, N1
11

12. Thanks!
12

13. 13