user/sven/linux.git/crypto/Makefile, branch v3.18.75

crypto: improve gcc optimization flags for serpent and wp512

2017-04-18T05:55:50Z

commit 7d6e9105026788c497f0ab32fa16c82f4ab5ff61 upstream. An ancient gcc bug (first reported in 2003) has apparently resurfaced on MIPS, where kernelci.org reports an overly large stack frame in the whirlpool hash algorithm: crypto/wp512.c:987:1: warning: the frame size of 1112 bytes is larger than 1024 bytes [-Wframe-larger-than=] With some testing in different configurations, I'm seeing large variations in stack frames size up to 1500 bytes for what should have around 300 bytes at most. I also checked the reference implementation, which is essentially the same code but also comes with some test and benchmarking infrastructure. It seems that recent compiler versions on at least arm, arm64 and powerpc have a partial fix for this problem, but enabling "-fsched-pressure", but even with that fix they suffer from the issue to a certain degree. Some testing on arm64 shows that the time needed to hash a given amount of data is roughly proportional to the stack frame size here, which makes sense given that the wp512 implementation is doing lots of loads for table lookups, and the problem with the overly large stack is a result of doing a lot more loads and stores for spilled registers (as seen from inspecting the object code). Disabling -fschedule-insns consistently fixes the problem for wp512, in my collection of cross-compilers, the results are consistently better or identical when comparing the stack sizes in this function, though some architectures (notable x86) have schedule-insns disabled by default. The four columns are: default: -O2 press: -O2 -fsched-pressure nopress: -O2 -fschedule-insns -fno-sched-pressure nosched: -O2 -no-schedule-insns (disables sched-pressure) default press nopress nosched alpha-linux-gcc-4.9.3 1136 848 1136 176 am33_2.0-linux-gcc-4.9.3 2100 2076 2100 2104 arm-linux-gnueabi-gcc-4.9.3 848 848 1048 352 cris-linux-gcc-4.9.3 272 272 272 272 frv-linux-gcc-4.9.3 1128 1000 1128 280 hppa64-linux-gcc-4.9.3 1128 336 1128 184 hppa-linux-gcc-4.9.3 644 308 644 276 i386-linux-gcc-4.9.3 352 352 352 352 m32r-linux-gcc-4.9.3 720 656 720 268 microblaze-linux-gcc-4.9.3 1108 604 1108 256 mips64-linux-gcc-4.9.3 1328 592 1328 208 mips-linux-gcc-4.9.3 1096 624 1096 240 powerpc64-linux-gcc-4.9.3 1088 432 1088 160 powerpc-linux-gcc-4.9.3 1080 584 1080 224 s390-linux-gcc-4.9.3 456 456 624 360 sh3-linux-gcc-4.9.3 292 292 292 292 sparc64-linux-gcc-4.9.3 992 240 992 208 sparc-linux-gcc-4.9.3 680 592 680 312 x86_64-linux-gcc-4.9.3 224 240 272 224 xtensa-linux-gcc-4.9.3 1152 704 1152 304 aarch64-linux-gcc-7.0.0 224 224 1104 208 arm-linux-gnueabi-gcc-7.0.1 824 824 1048 352 mips-linux-gcc-7.0.0 1120 648 1120 272 x86_64-linux-gcc-7.0.1 240 240 304 240 arm-linux-gnueabi-gcc-4.4.7 840 392 arm-linux-gnueabi-gcc-4.5.4 784 728 784 320 arm-linux-gnueabi-gcc-4.6.4 736 728 736 304 arm-linux-gnueabi-gcc-4.7.4 944 784 944 352 arm-linux-gnueabi-gcc-4.8.5 464 464 760 352 arm-linux-gnueabi-gcc-4.9.3 848 848 1048 352 arm-linux-gnueabi-gcc-5.3.1 824 824 1064 336 arm-linux-gnueabi-gcc-6.1.1 808 808 1056 344 arm-linux-gnueabi-gcc-7.0.1 824 824 1048 352 Trying the same test for serpent-generic, the picture is a bit different, and while -fno-schedule-insns is generally better here than the default, -fsched-pressure wins overall, so I picked that instead. default press nopress nosched alpha-linux-gcc-4.9.3 1392 864 1392 960 am33_2.0-linux-gcc-4.9.3 536 524 536 528 arm-linux-gnueabi-gcc-4.9.3 552 552 776 536 cris-linux-gcc-4.9.3 528 528 528 528 frv-linux-gcc-4.9.3 536 400 536 504 hppa64-linux-gcc-4.9.3 524 208 524 480 hppa-linux-gcc-4.9.3 768 472 768 508 i386-linux-gcc-4.9.3 564 564 564 564 m32r-linux-gcc-4.9.3 712 576 712 532 microblaze-linux-gcc-4.9.3 724 392 724 512 mips64-linux-gcc-4.9.3 720 384 720 496 mips-linux-gcc-4.9.3 728 384 728 496 powerpc64-linux-gcc-4.9.3 704 304 704 480 powerpc-linux-gcc-4.9.3 704 296 704 480 s390-linux-gcc-4.9.3 560 560 592 536 sh3-linux-gcc-4.9.3 540 540 540 540 sparc64-linux-gcc-4.9.3 544 352 544 496 sparc-linux-gcc-4.9.3 544 344 544 496 x86_64-linux-gcc-4.9.3 528 536 576 528 xtensa-linux-gcc-4.9.3 752 544 752 544 aarch64-linux-gcc-7.0.0 432 432 656 480 arm-linux-gnueabi-gcc-7.0.1 616 616 808 536 mips-linux-gcc-7.0.0 720 464 720 488 x86_64-linux-gcc-7.0.1 536 528 600 536 arm-linux-gnueabi-gcc-4.4.7 592 440 arm-linux-gnueabi-gcc-4.5.4 776 448 776 544 arm-linux-gnueabi-gcc-4.6.4 776 448 776 544 arm-linux-gnueabi-gcc-4.7.4 768 448 768 544 arm-linux-gnueabi-gcc-4.8.5 488 488 776 544 arm-linux-gnueabi-gcc-4.9.3 552 552 776 536 arm-linux-gnueabi-gcc-5.3.1 552 552 776 536 arm-linux-gnueabi-gcc-6.1.1 560 560 776 536 arm-linux-gnueabi-gcc-7.0.1 616 616 808 536 I did not do any runtime tests with serpent, so it is possible that stack frame size does not directly correlate with runtime performance here and it actually makes things worse, but it's more likely to help here, and the reduced stack frame size is probably enough reason to apply the patch, especially given that the crypto code is often used in deep call chains. Link: https://kernelci.org/build/id/58797d7559b5149efdf6c3a9/logs/ Link: http://www.larc.usp.br/~pbarreto/WhirlpoolPage.html Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11488 Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79149 Cc: Ralf Baechle Signed-off-by: Arnd Bergmann Signed-off-by: Herbert Xu Signed-off-by: Greg Kroah-Hartman

crypto: sha-mb - multibuffer crypto infrastructure

2014-08-25T12:32:25Z

This patch introduces the multi-buffer crypto daemon which is responsible for submitting crypto jobs in a work queue to the responsible multi-buffer crypto algorithm. The idea of the multi-buffer algorihtm is to put data streams from multiple jobs in a wide (AVX2) register and then take advantage of SIMD instructions to do crypto computation on several buffers simultaneously. The multi-buffer crypto daemon is also responsbile for flushing the remaining buffers to complete the computation if no new buffers arrive for a while. Signed-off-by: Tim Chen Signed-off-by: Herbert Xu

crypto: drbg - Use Kconfig to ensure at least one RNG option is set

2014-07-04T14:15:08Z

This patch removes the build-time test that ensures at least one RNG is set. Instead we will simply not build drbg if no options are set through Kconfig. This also fixes a typo in the name of the Kconfig option CRYTPO_DRBG (should be CRYPTO_DRBG). Signed-off-by: Herbert Xu

crypto: drbg - compile the DRBG code

2014-06-20T13:26:10Z

Signed-off-by: Stephan Mueller Signed-off-by: Herbert Xu

CRC32C: Add soft module dependency to load other accelerated crc32c modules

2014-02-25T11:45:04Z

We added the soft module dependency of crc32c module alias to generic crc32c module so other hardware accelerated crc32c modules could get loaded and used before the generic version. We also renamed the crypto/crc32c.c containing the generic crc32c crypto computation to crypto/crc32c_generic.c according to convention. Signed-off-by: Tim Chen Signed-off-by: Herbert Xu

crypto: more robust crypto_memneq

2013-12-05T13:28:41Z

Disabling compiler optimizations can be fragile, since a new optimization could be added to -O0 or -Os that breaks the assumptions the code is making. Instead of disabling compiler optimizations, use a dummy inline assembly (based on RELOC_HIDE) to block the problematic kinds of optimization, while still allowing other optimizations to be applied to the code. The dummy inline assembly is added after every OR, and has the accumulator variable as its input and output. The compiler is forced to assume that the dummy inline assembly could both depend on the accumulator variable and change the accumulator variable, so it is forced to compute the value correctly before the inline assembly, and cannot assume anything about its value after the inline assembly. This change should be enough to make crypto_memneq work correctly (with data-independent timing) even if it is inlined at its call sites. That can be done later in a followup patch. Compile-tested on x86_64. Signed-off-by: Cesar Eduardo Barros Acked-by: Daniel Borkmann Signed-off-by: Herbert Xu

Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6

2013-11-24T00:18:25Z

Pull crypto update from Herbert Xu: - Made x86 ablk_helper generic for ARM - Phase out chainiv in favour of eseqiv (affects IPsec) - Fixed aes-cbc IV corruption on s390 - Added constant-time crypto_memneq which replaces memcmp - Fixed aes-ctr in omap-aes - Added OMAP3 ROM RNG support - Add PRNG support for MSM SoC's - Add and use Job Ring API in caam - Misc fixes [ NOTE! This pull request was sent within the merge window, but Herbert has some questionable email sending setup that makes him public enemy #1 as far as gmail is concerned. So most of his emails seem to be trapped by gmail as spam, resulting in me not seeing them. - Linus ] * git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (49 commits) crypto: s390 - Fix aes-cbc IV corruption crypto: omap-aes - Fix CTR mode counter length crypto: omap-sham - Add missing modalias padata: make the sequence counter an atomic_t crypto: caam - Modify the interface layers to use JR API's crypto: caam - Add API's to allocate/free Job Rings crypto: caam - Add Platform driver for Job Ring hwrng: msm - Add PRNG support for MSM SoC's ARM: DT: msm: Add Qualcomm's PRNG driver binding document crypto: skcipher - Use eseqiv even on UP machines crypto: talitos - Simplify key parsing crypto: picoxcell - Simplify and harden key parsing crypto: ixp4xx - Simplify and harden key parsing crypto: authencesn - Simplify key parsing crypto: authenc - Export key parsing helper function crypto: mv_cesa: remove deprecated IRQF_DISABLED hwrng: OMAP3 ROM Random Number Generator support crypto: sha256_ssse3 - also test for BMI2 crypto: mv_cesa - Remove redundant of_match_ptr crypto: sahara - Remove redundant of_match_ptr ...

crypto: provide single place for hash algo information

2013-10-25T21:14:03Z

This patch provides a single place for information about hash algorithms, such as hash sizes and kernel driver names, which will be used by IMA and the public key code. Changelog: - Fix sparse and checkpatch warnings - Move hash algo enums to uapi for userspace signing functions. Signed-off-by: Dmitry Kasatkin Signed-off-by: Mimi Zohar Acked-by: Herbert Xu

crypto: crypto_memneq - add equality testing of memory regions w/o timing leaks

2013-10-07T06:17:06Z

When comparing MAC hashes, AEAD authentication tags, or other hash values in the context of authentication or integrity checking, it is important not to leak timing information to a potential attacker, i.e. when communication happens over a network. Bytewise memory comparisons (such as memcmp) are usually optimized so that they return a nonzero value as soon as a mismatch is found. E.g, on x86_64/i5 for 512 bytes this can be ~50 cyc for a full mismatch and up to ~850 cyc for a full match (cold). This early-return behavior can leak timing information as a side channel, allowing an attacker to iteratively guess the correct result. This patch adds a new method crypto_memneq ("memory not equal to each other") to the crypto API that compares memory areas of the same length in roughly "constant time" (cache misses could change the timing, but since they don't reveal information about the content of the strings being compared, they are effectively benign). Iow, best and worst case behaviour take the same amount of time to complete (in contrast to memcmp). Note that crypto_memneq (unlike memcmp) can only be used to test for equality or inequality, NOT for lexicographical order. This, however, is not an issue for its use-cases within the crypto API. We tried to locate all of the places in the crypto API where memcmp was being used for authentication or integrity checking, and convert them over to crypto_memneq. crypto_memneq is declared noinline, placed in its own source file, and compiled with optimizations that might increase code size disabled ("Os") because a smart compiler (or LTO) might notice that the return value is always compared against zero/nonzero, and might then reintroduce the same early-return optimization that we are trying to avoid. Using #pragma or __attribute__ optimization annotations of the code for disabling optimization was avoided as it seems to be considered broken or unmaintained for long time in GCC [1]. Therefore, we work around that by specifying the compile flag for memneq.o directly in the Makefile. We found that this seems to be most appropriate. As we use ("Os"), this patch also provides a loop-free "fast-path" for frequently used 16 byte digests. Similarly to kernel library string functions, leave an option for future even further optimized architecture specific assembler implementations. This was a joint work of James Yonan and Daniel Borkmann. Also thanks for feedback from Florian Weimer on this and earlier proposals [2]. [1] http://gcc.gnu.org/ml/gcc/2012-07/msg00211.html [2] https://lkml.org/lkml/2013/2/10/131 Signed-off-by: James Yonan Signed-off-by: Daniel Borkmann Cc: Florian Weimer Signed-off-by: Herbert Xu

crypto: create generic version of ablk_helper

2013-09-23T20:02:24Z

Create a generic version of ablk_helper so it can be reused by other architectures. Acked-by: Jussi Kivilinna Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu