user/sven/linux.git/lib/flex_array.c, branch leds/master

reciprocal_divide: update/correction of the algorithm

2014-01-22T07:17:20Z

Jakub Zawadzki noticed that some divisions by reciprocal_divide() were not correct [1][2], which he could also show with BPF code after divisions are transformed into reciprocal_value() for runtime invariance which can be passed to reciprocal_divide() later on; reverse in BPF dump ended up with a different, off-by-one K in some situations. This has been fixed by Eric Dumazet in commit aee636c4809fa5 ("bpf: do not use reciprocal divide"). This follow-up patch improves reciprocal_value() and reciprocal_divide() to work in all cases by using Granlund and Montgomery method, so that also future use is safe and without any non-obvious side-effects. Known problems with the old implementation were that division by 1 always returned 0 and some off-by-ones when the dividend and divisor where very large. This seemed to not be problematic with its current users, as far as we can tell. Eric Dumazet checked for the slab usage, we cannot surely say so in the case of flex_array. Still, in order to fix that, we propose an extension from the original implementation from commit 6a2d7a955d8d resp. [3][4], by using the algorithm proposed in "Division by Invariant Integers Using Multiplication" [5], Torbjörn Granlund and Peter L. Montgomery, that is, pseudocode for q = n/d where q, n, d is in u32 universe: 1) Initialization: int l = ceil(log_2 d) uword m' = floor((1<<32)*((1<>32 q = (t+((n-t)>>sh_1))>>sh_2 The assembler implementation from Agner Fog [6] also helped a lot while implementing. We have tested the implementation on x86_64, ppc64, i686, s390x; on x86_64/haswell we're still half the latency compared to normal divide. Joint work with Daniel Borkmann. [1] http://www.wireshark.org/~darkjames/reciprocal-buggy.c [2] http://www.wireshark.org/~darkjames/set-and-dump-filter-k-bug.c [3] https://gmplib.org/~tege/division-paper.pdf [4] http://homepage.cs.uiowa.edu/~jones/bcd/divide.html [5] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.1.2556 [6] http://www.agner.org/optimize/asmlib.zip Reported-by: Jakub Zawadzki Cc: Eric Dumazet Cc: Austin S Hemmelgarn Cc: linux-kernel@vger.kernel.org Cc: Jesse Gross Cc: Jamal Hadi Salim Cc: Stephen Hemminger Cc: Matt Mackall Cc: Pekka Enberg Cc: Christoph Lameter Cc: Andy Gospodarek Cc: Veaceslav Falico Cc: Jay Vosburgh Cc: Jakub Zawadzki Signed-off-by: Daniel Borkmann Signed-off-by: Hannes Frederic Sowa Signed-off-by: David S. Miller

lib: reduce the use of module.h wherever possible

2012-03-07T20:04:04Z

For files only using THIS_MODULE and/or EXPORT_SYMBOL, map them onto including export.h -- or if the file isn't even using those, then just delete the include. Fix up any implicit include dependencies that were being masked by module.h along the way. Signed-off-by: Paul Gortmaker

flex_array: avoid divisions when accessing elements

2011-05-27T00:12:33Z

On most architectures division is an expensive operation and accessing an element currently requires four of them. This performance penalty effectively precludes flex arrays from being used on any kind of fast path. However, two of these divisions can be handled at creation time and the others can be replaced by a reciprocal divide, completely avoiding real divisions on access. [eparis@redhat.com: rebase on top of changes to support 0 len elements] [eparis@redhat.com: initialize part_nr when array fits entirely in base] Signed-off-by: Jesse Gross Signed-off-by: Eric Paris Cc: Dave Hansen Cc: David Rientjes Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

flex_array: allow 0 length elements

2011-04-28T19:56:07Z

flex_arrays are supposed to be a replacement for: kmalloc(num_elements * sizeof(element)) If kmalloc is given 0 num_elements or a 0 size element it will happily return ZERO_SIZE_PTR. Which looks like a valid allocation, but which will explode if something actually try to use it. The current flex_array code will return an equivalent result if num_elements is 0, but will fail to work if sizeof(element) is 0. This patch allows allocation to work even for 0 size elements. It will cause flex_arrays to explode though if they are used. Imitating the kmalloc behavior. Based-on-patch-by: Steffen Klassert Signed-off-by: Eric Paris Acked-by: Dave Hansen

flex_arrays: allow zero length flex arrays

2011-04-28T19:56:07Z

Just like kmalloc will allow one to allocate a 0 length segment of memory flex arrays should do the same thing. It should bomb if you try to use something, but it should at least allow the allocation. This is needed because when SELinux switched to using flex_arrays in 2.6.38 the inability to allocate a 0 length array resulted in SELinux policy load returning -ENOSPC when previously it worked. Based-on-patch-by: Steffen Klassert Signed-off-by: Eric Paris Tested-by: Chris Richards Cc: stable@kernel.org [2.6.38+]

flex_array: flex_array_prealloc takes a number of elements, not an end

2011-04-28T19:56:06Z

Change flex_array_prealloc to take the number of elements for which space should be allocated instead of the last (inclusive) element. Users and documentation are updated accordingly. flex_arrays got introduced before they had users. When folks started using it, they ended up needing a different API than was coded up originally. This swaps over to the API that folks apparently need. Based-on-patch-by: Steffen Klassert Signed-off-by: Eric Paris Tested-by: Chris Richards Acked-by: Dave Hansen Cc: stable@kernel.org [2.6.38+]

flex_array: export symbols to modules

2011-01-13T16:03:11Z

Alex said: I want to use flex_array to store a sparse array of ATM cell re-assembly buffers for my ATM over Ethernet driver. Using the per-vcc user_back structure causes problems when stacked with things like br2684. Add EXPORT_SYMBOL() for all publically accessible flex array functions and move to obj-y so that modules may use this library. Signed-off-by: David Rientjes Cc: Dave Hansen Cc: Paul Mundt Reported-by: Alex Bennee Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

flex_array: add helpers to get and put to make pointers easy to use

2010-08-10T03:45:09Z

Getting and putting arrays of pointers with flex arrays is a PITA. You have to remember to pass &ptr to the _put and you have to do weird and wacky casting to get the ptr back from the _get. Add two functions flex_array_get_ptr() and flex_array_put_ptr() to handle all of the magic. [akpm@linux-foundation.org: simplification suggested by Joe] Signed-off-by: Eric Paris Cc: David Rientjes Cc: Dave Hansen Cc: Joe Perches Cc: James Morris Cc: Joe Perches Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

flex_array: fix the panic when calling flex_array_alloc() without __GFP_ZERO

2010-04-24T18:31:24Z

memset() is called with the wrong address and the kernel panics. Signed-off-by: Changli Gao Cc: Patrick McHardy Acked-by: David Rientjes Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

flex_array: add missing kerneldoc annotations

2009-09-22T14:17:47Z

Add kerneldoc annotations for function formals of type struct flex_array and gfp_t which are currently lacking. Signed-off-by: David Rientjes Cc: Dave Hansen Cc: Randy Dunlap Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds