| Age | Commit message (Collapse) | Author |
|---|
|
AIO was adding a new context's max requests to the global total before
testing if that resulting total was over the global limit. This let
innocent tasks get their new limit tested along with a racing guilty task
that was crossing the limit. This serializes the _nr accounting with a
spinlock It also switches to using unsigned long for the global totals.
Individual contexts are still limited to an unsigned int's worth of
requests by the syscall interface.
The problem and fix were verified with a simple program that spun creating
and destroying a context while holding on to another long lived context.
Before the patch a task creating a tiny context could get a spurious EAGAIN
if it raced with a task creating a very large context that overran the
limit.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
Cc: Benjamin LaHaise <bcrl@kvack.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
lock_kiocb() was introduced to serialize retrying and cancellation. In the
process of doing so it tried to sleep waiting for KIF_LOCKED while holding
the ctx_lock spinlock. Recent fixes have ensured that multiple concurrent
retries won't be attempted for a given iocb. Cancel has other problems and
has no significant in-tree users that have been complaining about it. So
for the immediate future we'll revert sleeping with the lock held and will
address proper cancellation and retry serialization in the future.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
Acked-by: Benjamin LaHaise <bcrl@kvack.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
Only one of the run or kick path is supposed to put an iocb on the run
list. If both of them do it than one of them can end up referencing a
freed iocb. The kick path could delete the task_list item from the wait
queue before getting the ctx_lock and putting the iocb on the run list.
The run path was testing the task_list item outside the lock so that it
could catch ki_retry methods that return -EIOCBRETRY *without* putting the
iocb on a wait queue and promising to call kick_iocb. This unlocked check
could then race with the kick path to cause both to try and put the iocb on
the run list.
The patch stops the run path from testing task_list by requring that any
ki_retry that returns -EIOCBRETRY *must* guarantee that kick_iocb() will be
called in the future. aio_p{read,write}, the only in-tree -EIOCBRETRY
users, are updated.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
Signed-off-by: Benjamin LaHaise <bcrl@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
->ki_buf is always a userland pointer.
|
|
From: Daniel McNeil <daniel@osdl.org>
From: Chris Mason <mason@suse.com>
AIO: retry infrastructure fixes and enhancements
Reorganises, comments and fixes the AIO retry logic. Fixes
and enhancements include:
- Split iocb setup and execution in io_submit
(also fixes io_submit error reporting)
- Use aio workqueue instead of keventd for retries
- Default high level retry methods
- Subtle use_mm/unuse_mm fix
- Code commenting
- Fix aio process hang on EINVAL (Daniel McNeil)
- Hold the context lock across unuse_mm
- Acquire task_lock in use_mm()
- Allow fops to override the retry method with their own
- Elevated ref count for AIO retries (Daniel McNeil)
- set_fs needed when calling use_mm
- Flush workqueue on __put_ioctx (Chris Mason)
- Fix io_cancel to work with retries (Chris Mason)
- Read-immediate option for socket/pipe retry support
Note on default high-level retry methods support
================================================
High-level retry methods allows an AIO request to be executed as a series of
non-blocking iterations, where each iteration retries the remaining part of
the request from where the last iteration left off, by reissuing the
corresponding AIO fop routine with modified arguments representing the
remaining I/O. The retries are "kicked" via the AIO waitqueue callback
aio_wake_function() which replaces the default wait queue entry used for
blocking waits.
The high level retry infrastructure is responsible for running the
iterations in the mm context (address space) of the caller, and ensures that
only one retry instance is active at a given time, thus relieving the fops
themselves from having to deal with potential races of that sort.
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
sizeof(struct kiocb) is dangerously large for a structure commonly
allocated on-stack. This patch converts the 24*sizeof(long) field,
->private, to a void pointer for use by file_operations entrypoints.
A ->dtor() method is added to the kiocb in order to support the release
of dynamically allocated structures referred to by ->private.
The sole in-tree users of ->private are async network read/write,
which are not, in fact, async, and so need not handle preallocated
->private as they would need to if ->ki_retry were ever used. The sole
truly async operations are direct IO pread()/pwrite() which do not
now use ->ki_retry(). All they would need to do in that case is to
check for ->private already being allocated for async kiocbs.
This rips 88B off the stack on 32-bit in the common case.
Signed-off-by: William Lee Irwin III <wli@holomorphy.com>
Signed-off-by: David S. Miller <davem@redhat.com>
|
|
This documents how this member is used in two
ways, 1) as a pointer to a user ioctx and 2)
as a pointer to a task struct to wake up.
|
|
|
|
Every pointer in <syscalls.h> had better be a user
pointer. Also add some others that a quick sanity check
picked up on.
|
|
From: Janet Morgan <janetmor@us.ibm.com>
It looks like aio_nr and aio_max_nr were intended to be sysctl parameters.
|
|
- comment fixes
- duplicated assignments
- Remove a prototype which is in aio.h already
- Some __user annotation
- use the existing symbolic names, not magic numbers.
|
|
Fix some warnings in the ppc64 build.
Also declare a couple of AIO functions in aio.h rather than aio.c They are
needed for 32-bit emulation support.
|
|
The new aio emulation in x86-64 needs a few aio symbols exported.
Export them.
|
|
Patch from: Suparna Bhattacharya <suparna@in.ibm.com>
Just an obvious fix.
The kiocbClearX macros were doing a set_bit !
They should be calling clear_bit.
Ran into this now that I'm actually using kiocbClearKicked.
|
|
|
|
the mm
context of the submitter to allow the use of copy_*_user.
|
|
|
|
support for file descriptors with only async ops in vfs_{read,write}
|
|
This adds support for synchronous iocbs and converts generic_file_read
to use a sync iocb to call into generic_file_aio_read.
The tests I've run with lmbench on a piii-866 showed no difference in
file re-read speed when forced to use a completion path via aio_complete
and an -EIOCBQUEUED return from generic_file_aio_read -- people with
slower machines might want to test this to see if we can tune it any
better. Also, a bug fix to correct a missing call into the aio code
from the fork code is present. This patch sets things up for making
generic_file_aio_read actually asynchronous.
|
|
|
