OSDN Git Service

io_uring: optimise putting task struct
authorPavel Begunkov <asml.silence@gmail.com>
Mon, 9 Aug 2021 12:04:20 +0000 (13:04 +0100)
committerJens Axboe <axboe@kernel.dk>
Mon, 23 Aug 2021 19:08:06 +0000 (13:08 -0600)
commite9dbe221f5d1c974c853da94eee456803239cab5
tree7de722898f1f55dacfed8d7fc938763ab931f7b5
parentaf066f31eb3dac2a11516315d47a286a7b3b07df
io_uring: optimise putting task struct

We cache all the reference to task + tctx, so if io_put_task() is
called by the corresponding task itself, we can save on atomics and
return the refs right back into the cache.

It's beneficial for all inline completions, and also iopolling, when
polling and submissions are done by the same task, including
SQPOLL|IOPOLL.

Note: io_uring_cancel_generic() can return refs to the cache as well,
so those should be flushed in the loop for tctx_inflight() to work
right.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/6fe9646b3cb70e46aca1f58426776e368c8926b3.1628471125.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c