| Age | Commit message (Collapse) | Author |
|
Use hlist_unhashed() rather than accessing inside data structure.
Signed-off-by: Akinobu Mita <mita@miraclelinux.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
For iterating over list of given type continuing from existing point.
Signed-off-by: Arnaldo Carvalho de Melo <acme@mandriva.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
For iterate over list of given type from existing point safe against removal of
list entry.
Signed-off-by: Arnaldo Carvalho de Melo <acme@mandriva.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
I noticed that list.h init functions were evaluating macro arguments
multiple times and thought it might be nice to protect the unsuspecting
caller. Converting the macros to inline functions seems to reduce code
size, too. A i386 defconfig build with gcc 3.3.3 from fc4:
text data bss dec hex filename
3573148 565664 188828 4327640 4208d8 vmlinux.before
3572177 565664 188828 4326669 42050d vmlinux
add/remove: 0/0 grow/shrink: 11/144 up/down: 88/-1016 (-928)
There was no difference in checkstack output.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
Add list_for_each_entry_safe_reverse() to linux/list.h
This is needed by unmerged cachefs and be an as-yet-unreviewed
device_shutdown() fix.
Signed-off-by: David Howells <dhowells@redhat.com>
Cc: Patrick Mochel <mochel@digitalimplant.org>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
Add list_replace_rcu: replace old entry by new one.
Signed-off-by: Paul E. McKenney <paulmck@us.ibm.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
hlist_for_each_entry_rcu
Remove the hlist_for_each_rcu() API, which is used only in one place, and
is trivially converted to hlist_for_each_entry_rcu(), making the code
shorter and more readable. Any out-of-tree uses may be similarly
converted.
Signed-off-by: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
It seems that all the list_*_rcu primitives are missing a memory barrier
on the very first dereference. For example,
#define list_for_each_rcu(pos, head) \
for (pos = (head)->next; prefetch(pos->next), pos != (head); \
pos = rcu_dereference(pos->next))
It will go something like:
pos = (head)->next
prefetch(pos->next)
pos != (head)
do stuff
We're missing a barrier here.
pos = rcu_dereference(pos->next)
fetch pos->next
barrier given by rcu_dereference(pos->next)
store pos
Without the missing barrier, the pos->next value may turn out to be stale.
In fact, if "do stuff" were also dereferencing pos and relying on
list_for_each_rcu to provide the barrier then it may also break.
So here is a patch to make sure that we have a barrier for the first
element in the list.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
* RCU versions of hlist_***_rcu
* fib_alias partial rcu port just whats needed now.
Signed-off-by: Robert Olsson <Robert.Olsson@data.slu.se>
Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
This also changes the list_for_each_entry_safe_continue behaviour to match its
kerneldoc comment, that is, to start after the pos passed.
Also adds several helper functions from previously open coded fragments, making
the code more clear.
Signed-off-by: Arnaldo Carvalho de Melo <acme@mandriva.com>
|
|
Used in the dccp CCID3 code, that is going to be submitted RSN.
Signed-off-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
2.6.12-rc6-mm1 has a few remaining synchronize_kernel()s, some (but not
all) in comments. This patch changes these synchronize_kernel() calls (and
comments) to synchronize_rcu() or synchronize_sched() as follows:
- arch/x86_64/kernel/mce.c mce_read(): change to synchronize_sched() to
handle races with machine-check exceptions (synchronize_rcu() would not cut
it given RCU implementations intended for hardcore realtime use.
- drivers/input/serio/i8042.c i8042_stop(): change to synchronize_sched() to
handle races with i8042_interrupt() interrupt handler. Again,
synchronize_rcu() would not cut it given RCU implementations intended for
hardcore realtime use.
- include/*/kdebug.h comments: change to synchronize_sched() to handle races
with NMIs. As before, synchronize_rcu() would not cut it...
- include/linux/list.h comment: change to synchronize_rcu(), since this
comment is for list_del_rcu().
- security/keys/key.c unregister_key_type(): change to synchronize_rcu(),
since this is interacting with RCU read side.
- security/keys/process_keys.c install_session_keyring(): change to
synchronize_rcu(), since this is interacting with RCU read side.
Signed-off-by: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
The hlist_for_each_entry_rcu() comment block refers to a nonexistent
hlist_add_rcu() API, needs to change to hlist_add_head_rcu().
Signed-off-by: Paul E. McKenney <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
This patch changes list_for_each_xxx iterators
from:
for (pos = (head)->next, prefetch(pos->next);
pos != (head);
pos = pos->next, prefetch(pos->next))
to:
for (pos = (head)->next;
prefetch(pos->next), pos != (head);
pos = pos->next)
Reduces my vmlinux .text size by 4401 bytes.
Signed-off-by: Oleg Nesterov <oleg@tv-sign.ru>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
This patch uses the rcu_assign_pointer() API to eliminate a number of explicit
memory barriers from code using RCU. This has been tested successfully on
i386 and ppc64.
Signed-off-by: <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
This attached patch adds list_replace_rcu() to include/linux/list.h for
atomic updating operations according to RCU-model.
void list_replace_rcu(struct list_head *old, struct list_head *new)
The 'old' element is detached from the linked list, and the 'new' element
is inserted to the same point of the linked list concurrently.
This patch is necessary for the performance improvement of SELinux.
See, http://lkml.org/lkml/2004/8/16/54
(Subject: RCU issue with SELinux)
http://lkml.org/lkml/2004/8/30/63
(Subject: [PATCH]SELinux performance improvement by RCU)
Signed-off-by: KaiGai, Kohei <kaigai@ak.jp.nec.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
Use abstracted RCU API to dereference RCU protected data. Hides barrier
details. Patch from Paul McKenney.
This patch introduced an rcu_dereference() macro that replaces most uses of
smp_read_barrier_depends(). The new macro has the advantage of explicitly
documenting which pointers are protected by RCU -- in contrast, it is
sometimes difficult to figure out which pointer is being protected by a given
smp_read_barrier_depends() call.
Signed-off-by: Paul McKenney <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
into nuts.davemloft.net:/disk1/BK/net-2.6
|
|
Convert the bridge forwarding database over to using RCU.
This avoids a read_lock and atomic_inc/dec in the fast path
of output.
Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
Signed-off-by: David S. Miller <davem@redhat.com>
|
|
Make code for hlist_for_each_safe use better code (same as
hlist_for_each_entry_safe). Get rid of comment about prefetch, because
that was fixed a while ago. Only current use of this is in the bridge
code, that I maintain.
Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
|
- missing ; between default: and } in sun4setup.c
- cast of pointer to unsigned long long instead of unsigned long in
x86_64 signal.c
- missed annotations for ioctl structure in sparc64 openpromio.h
(should've been in the same patch as the rest of drivers/sbus/*
annotations)
- 0->NULL in list.h and pmdisk.c
|
|
From: "Pedro Emanuel M. D. Pinto" <pepinto@student.dei.uc.pt>
This currently-unused function is incorrectly implemented. Fix.
|
|
- s/__inline__/inline/
- Remove lots of extraneous andi-was-here trailing whitespace
|
|
From: "Paul E. McKenney" <paulmck@us.ibm.com>
The attached patch improves the documentation of the _rcu list primitives.
|
|
correctly handle the "restart from this device/driver" case, and
caused oopses with ieee1394.
This just uses "list_for_each_entry_continue()" instead.
Add helper macro to make usage of "list_for_each_entry_continue()"
a bit more readable.
|
|
From: Ingo Molnar <mingo@elte.hu>
I'd also suggest the following patch below, to clarify the use of
unsynchronized list_empty(). list_empty_careful() can only be safe in the
very specific case of "one-shot" list entries which might be removed by
another CPU. (but nothing else can happen to them and this is their only
final state.) list_empty_careful() is otherwise completely unsynchronized
on both the compiler and CPU level and is not 'SMP safe' in any way.
|
|
corruption on SMP because of another CPU still accessing a waitqueue
even after it was de-allocated.
Use a careful version of the list emptiness check to make sure we
don't de-allocate the stack frame before the waitqueue is all done.
|
|
From: "Perez-Gonzalez, Inaky" <inaky.perez-gonzalez@intel.com>
|
|
From: Mitchell Blank Jr <mitch@sfgoth.com>
Make some more of the hlist functions accept constant arguments.
|
|
From: Michael Still <mikal@stillhq.com>
The patch squelches build errors in the kernel-doc make targets by adding
documentation to arguements previously not documented, and updating the
argument names where they have changed.
|
|
|
|
|
|
as we delete the entry, we can only poison the back pointer, not the
traversal pointer (rcu traversal only ever walks forward).
Make __d_drop() take this into account.
|
|
into home.transmeta.com:/home/torvalds/v2.5/linux
|
|
|
|
list pointers to give us a nice oops if somebody is doing something
bad.
Also introduce hlist_del_rcu_init() - same as hlist_del_init().
|
|
|
|
This changeset:
1. Implements hlist_add_after
2. uses prefetch in hlist_for_each, using a trick that ends up being
equivalent to having the prefetch instruction in the first block
of the hlist_for_each for block, the compiler optimizes the second
"test" away, as its result is constant
3. implements hlist_for_each_entry and hlist_for_each_entry safe,
using a struct hlist_node as iterator to avoid the extra branches a
similar implementation to list_for_each_entry would have if used
a typed iterator, but while avoiding having to have the explicit
hlist_entry as in hlist_for_each.
4. Converts the hlist_for_each users that had explicit prefetches, i.e.
removed the explicit prefetch
5. fix a harmless list_entry use in a hlist_for_each in inode.c
|
|
|
|
|
|
|
|
|
|
From: Dipankar Sarma <dipankar@in.ibm.com>
This patch makes the list macros use smp-only version of the barriers,
no need to hurt UP performance.
|
|
|
|
This patches add an include to stddef.h into include/linux/list.h.
It uses the NULL define.
|
|
list.h must now include stddef since it uses NULL.
|
|
- Inode and dcache Hash table only needs half the memory/cache because
of using hlists.
- Simplify dcache-rcu code. With NULL end markers in the hlists
is_bucket is not needed anymore. Also the list walking code
generates better code on x86 now because it doesn't need to dedicate
a register for the list head.
- Reorganize struct dentry to be more cache friendly. All the state
accessed for the hash walk is in one chunk now together with the
inline name (all at the end)
- Add prefetching for all the list walks. Old hash lookup code didn't
use it.
- Some other minor cleanup.
|
|
Now that the devicemapper hit the tree there's no more reason
to keep the uncompiling LVM1 code around and it's various hacks
to other files around, this patch removes it.
|
|
This adds a set of list macros that make handling of list protected
by RCU simpler. The interfaces added are -
list_add_rcu
list_add_tail_rcu
- Adds an element by taking care of memory barrier (wmb()).
list_del_rcu
- Deletes an element but doesn't re-initialize the pointers in
the element for supporting RCU based traversal.
list_for_each_rcu
__list_for_each_rcu
- Traversal of RCU protected list - takes care of memory barriers
transparently.
|