user/sven/linux.git/io_uring, branch v6.6.26

io_uring/kbuf: hold io_buffer_list reference over mmap

2024-04-10T14:36:03Z

commit 561e4f9451d65fc2f7eef564e0064373e3019793 upstream. If we look up the kbuf, ensure that it doesn't get unregistered until after we're done with it. Since we're inside mmap, we cannot safely use the io_uring lock. Rely on the fact that we can lookup the buffer list under RCU now and grab a reference to it, preventing it from being unregistered until we're done with it. The lookup returns the io_buffer_list directly with it referenced. Cc: stable@vger.kernel.org # v6.4+ Fixes: 5cf4f52e6d8a ("io_uring: free io_buffer_list entries via RCU") Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: use private workqueue for exit work

2024-04-10T14:36:03Z

commit 73eaa2b583493b680c6f426531d6736c39643bfb upstream. Rather than use the system unbound event workqueue, use an io_uring specific one. This avoids dependencies with the tty, which also uses the system_unbound_wq, and issues flushes of said workqueue from inside its poll handling. Cc: stable@vger.kernel.org Reported-by: Rasmus Karlsson Tested-by: Rasmus Karlsson Tested-by: Iskren Chernev Link: https://github.com/axboe/liburing/issues/1113 Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring/kbuf: protect io_buffer_list teardown with a reference

2024-04-10T14:36:02Z

commit 6b69c4ab4f685327d9e10caf0d84217ba23a8c4b upstream. No functional changes in this patch, just in preparation for being able to keep the buffer list alive outside of the ctx->uring_lock. Cc: stable@vger.kernel.org # v6.4+ Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring/kbuf: get rid of bl->is_ready

2024-04-10T14:36:02Z

commit 3b80cff5a4d117c53d38ce805823084eaeffbde6 upstream. Now that xarray is being exclusively used for the buffer_list lookup, this check is no longer needed. Get rid of it and the is_ready member. Cc: stable@vger.kernel.org # v6.4+ Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring/kbuf: get rid of lower BGID lists

2024-04-10T14:36:02Z

commit 09ab7eff38202159271534d2f5ad45526168f2a5 upstream. Just rely on the xarray for any kind of bgid. This simplifies things, and it really doesn't bring us much, if anything. Cc: stable@vger.kernel.org # v6.4+ Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: clean rings on NO_MMAP alloc fail

2024-04-03T13:28:31Z

[ Upstream commit cef59d1ea7170ec753182302645a0191c8aa3382 ] We make a few cancellation judgements based on ctx->rings, so let's zero it afer deallocation for IORING_SETUP_NO_MMAP just like it's done with the mmap case. Likely, it's not a real problem, but zeroing is safer and better tested. Cc: stable@vger.kernel.org Fixes: 03d89a2de25bbc ("io_uring: support for user allocated memory for rings/sqes") Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/9ff6cdf91429b8a51699c210e1f6af6ea3f8bdcf.1710255382.git.asml.silence@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin

io_uring: fix mshot io-wq checks

2024-04-03T13:28:29Z

[ Upstream commit 3a96378e22cc46c7c49b5911f6c8631527a133a9 ] When checking for concurrent CQE posting, we're not only interested in requests running from the poll handler but also strayed requests ended up in normal io-wq execution. We're disallowing multishots in general from io-wq, not only when they came in a certain way. Cc: stable@vger.kernel.org Fixes: 17add5cea2bba ("io_uring: force multishot CQEs into task context") Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/d8c5b36a39258036f93301cd60d3cd295e40653d.1709905727.git.asml.silence@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin

io_uring/net: correctly handle multishot recvmsg retry setup

2024-04-03T13:28:29Z

[ Upstream commit deaef31bc1ec7966698a427da8c161930830e1cf ] If we loop for multishot receive on the initial attempt, and then abort later on to wait for more, we miss a case where we should be copying the io_async_msghdr from the stack to stable storage. This leads to the next retry potentially failing, if the application had the msghdr on the stack. Cc: stable@vger.kernel.org Fixes: 9bb66906f23e ("io_uring: support multishot in recvmsg") Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin

io_uring: fix poll_remove stalled req completion

2024-03-26T22:20:09Z

[ Upstream commit 5e3afe580a9f5ca173a6bd55ffe10948796ef7e5 ] Taking the ctx lock is not enough to use the deferred request completion infrastructure, it'll get queued into the list but no one would expect it there, so it will sit there until next io_submit_flush_completions(). It's hard to care about the cancellation path, so complete it via tw. Fixes: ef7dfac51d8ed ("io_uring/poll: serialize poll linked timer start with poll removal") Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/c446740bc16858f8a2a8dcdce899812f21d15f23.1710514702.git.asml.silence@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin

io_uring: Fix release of pinned pages when __io_uaddr_map fails

2024-03-26T22:20:08Z

[ Upstream commit 67d1189d1095d471ed7fa426c7e384a7140a5dd7 ] Looking at the error path of __io_uaddr_map, if we fail after pinning the pages for any reasons, ret will be set to -EINVAL and the error handler won't properly release the pinned pages. I didn't manage to trigger it without forcing a failure, but it can happen in real life when memory is heavily fragmented. Signed-off-by: Gabriel Krisman Bertazi Fixes: 223ef4743164 ("io_uring: don't allow IORING_SETUP_NO_MMAP rings on highmem pages") Link: https://lore.kernel.org/r/20240313213912.1920-1-krisman@suse.de Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin