user/sven/linux.git/io_uring/io_uring.c, branch v6.1.149

io_uring: account drain memory to cgroup

2025-06-27T10:07:26Z

commit f979c20547e72568e3c793bc92c7522bc3166246 upstream. Account drain allocations against memcg. It's not a big problem as each such allocation is paired with a request, which is accounted, but it's nicer to follow the limits more closely. Cc: stable@vger.kernel.org # 6.1 Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/f8dfdbd755c41fd9c75d12b858af07dfba5bbb68.1746788718.git.asml.silence@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

Revert "io_uring: ensure deferred completions are posted for multishot"

2025-06-27T10:07:24Z

This reverts commit b82c386898f7b00cb49abe3fbd622017aaa61230 which is commit 687b2bae0efff9b25e071737d6af5004e6e35af5 upstream. Jens writes: There's some missing dependencies that makes this not work right, I'll bring it back in a series instead. Link: https://lore.kernel.org/r/313f2335-626f-4eea-8502-d5c3773db35a@kernel.dk Reported-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman Signed-off-by: Greg Kroah-Hartman

io_uring: fix overflow resched cqe reordering

2025-06-04T12:40:20Z

[ Upstream commit a7d755ed9ce9738af3db602eb29d32774a180bc7 ] Leaving the CQ critical section in the middle of a overflow flushing can cause cqe reordering since the cache cq pointers are reset and any new cqe emitters that might get called in between are not going to be forced into io_cqe_cache_refill(). Fixes: eac2ca2d682f9 ("io_uring: check if we need to reschedule during overflow flush") Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/90ba817f1a458f091f355f407de1c911d2b93bbf.1747483784.git.asml.silence@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin

io_uring: ensure deferred completions are posted for multishot

2025-05-18T06:21:25Z

Commit 687b2bae0efff9b25e071737d6af5004e6e35af5 upstream. Multishot normally uses io_req_post_cqe() to post completions, but when stopping it, it may finish up with a deferred completion. This is fine, except if another multishot event triggers before the deferred completions get flushed. If this occurs, then CQEs may get reordered in the CQ ring, and cause confusion on the application side. When multishot posting via io_req_post_cqe(), flush any pending deferred completions first, if any. Cc: stable@vger.kernel.org # 6.1+ Reported-by: Norman Maurer Reported-by: Christian Mazakas Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: always arm linked timeouts prior to issue

2025-05-18T06:21:25Z

Commit b53e523261bf058ea4a518b482222e7a277b186b upstream. There are a few spots where linked timeouts are armed, and not all of them adhere to the pre-arm, attempt issue, post-arm pattern. This can be problematic if the linked request returns that it will trigger a callback later, and does so before the linked timeout is fully armed. Consolidate all the linked timeout handling into __io_issue_sqe(), rather than have it spread throughout the various issue entry points. Cc: stable@vger.kernel.org Link: https://github.com/axboe/liburing/issues/1390 Reported-by: Chase Hiltz Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: fix corner case forgetting to vunmap

2025-03-28T20:58:53Z

Commit 43eef70e7e2ac74e7767731dd806720c7fb5e010 upstream. io_pages_unmap() is a bit tricky in trying to figure whether the pages were previously vmap'ed or not. In particular If there is juts one page it belives there is no need to vunmap. Paired io_pages_map(), however, could've failed io_mem_alloc_compound() and attempted to io_mem_alloc_single(), which does vmap, and that leads to unpaired vmap. The solution is to fail if io_mem_alloc_compound() can't allocate a single page. That's the easiest way to deal with it, and those two functions are getting removed soon, so no need to overcomplicate it. Cc: stable@vger.kernel.org Fixes: 3ab1db3c6039e ("io_uring: get rid of remap_pfn_range() for mapping rings/sqes") Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/477e75a3907a2fe83249e49c0a92cd480b2c60e0.1732569842.git.asml.silence@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: don't attempt to mmap larger than what the user asks for

2025-03-28T20:58:53Z

Commit 06fe9b1df1086b42718d632aa57e8f7cd1a66a21 upstream. If IORING_FEAT_SINGLE_MMAP is ignored, as can happen if an application uses an ancient liburing or does setup manually, then 3 mmap's are required to map the ring into userspace. The kernel will still have collapsed the mappings, however userspace may ask for mapping them individually. If so, then we should not use the full number of ring pages, as it may exceed the partial mapping. Doing so will yield an -EFAULT from vm_insert_pages(), as we pass in more pages than what the application asked for. Cap the number of pages to match what the application asked for, for the particular mapping operation. Reported-by: Lucas Mülling Link: https://github.com/axboe/liburing/issues/1157 Fixes: 3ab1db3c6039 ("io_uring: get rid of remap_pfn_range() for mapping rings/sqes") Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: get rid of remap_pfn_range() for mapping rings/sqes

2025-03-28T20:58:53Z

Commit 3ab1db3c6039e02a9deb9d5091d28d559917a645 upstream. Rather than use remap_pfn_range() for this and manually free later, switch to using vm_insert_pages() and have it Just Work. If possible, allocate a single compound page that covers the range that is needed. If that works, then we can just use page_address() on that page. If we fail to get a compound page, allocate single pages and use vmap() to map them into the kernel virtual address space. This just covers the rings/sqes, the other remaining user of the mmap remap_pfn_range() user will be converted separately. Once that is done, we can kill the old alloc/free code. Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: add ring freeing helper

2025-03-28T20:58:53Z

Commit 9c189eee73af1825ea9c895fafad469de5f82641 upstream. We do rings and sqes separately, move them into a helper that does both the freeing and clearing of the memory. Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring: return error pointer from io_mem_alloc()

2025-03-28T20:58:53Z

Commit e27cef86a0edd4ef7f8b4670f508a03b509cbbb2 upstream. In preparation for having more than one time of ring allocator, make the existing one return valid/error-pointer rather than just NULL. Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman