This is the start of the stable review cycle for the 4.9.320 release.
There are 264 patches in this series, all will be posted as a response
to this one. If anyone has any issues with these being applied, please
let me know.
Responses should be made by Sat, 25 Jun 2022 16:43:11 +0000.
Anything received after that time might be too late.
The whole patch series can be found in one patch at:
https://www.kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.9.320-rc1.gz
or in the git tree and branch at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-4.9.y
and the diffstat can be found below.
thanks,
greg k-h
-------------
Pseudo-Shortlog of commits:
Greg Kroah-Hartman <[email protected]>
Linux 4.9.320-rc1
Willy Tarreau <[email protected]>
tcp: drop the hash_32() part from the index calculation
Willy Tarreau <[email protected]>
tcp: increase source port perturb table to 2^16
Willy Tarreau <[email protected]>
tcp: dynamically allocate the perturb table used by source ports
Willy Tarreau <[email protected]>
tcp: add small random increments to the source port
Willy Tarreau <[email protected]>
tcp: use different parts of the port_offset for index and offset
Willy Tarreau <[email protected]>
secure_seq: use the 64 bits of the siphash for port offset calculation
Eric Dumazet <[email protected]>
tcp: add some entropy in __inet_hash_connect()
Eric Dumazet <[email protected]>
tcp: change source port randomizarion at connect() time
Miklos Szeredi <[email protected]>
fuse: fix pipe buffer lifetime for direct_io
Linus Torvalds <[email protected]>
Reinstate some of "swiotlb: rework "fix info leak with DMA_FROM_DEVICE""
Halil Pasic <[email protected]>
swiotlb: fix info leak with DMA_FROM_DEVICE
Colin Ian King <[email protected]>
xprtrdma: fix incorrect header size calculations
Christian Borntraeger <[email protected]>
s390/mm: use non-quiescing sske for KVM switch to keyed guest
James Chapman <[email protected]>
l2tp: fix race in pppol2tp_release with session object destroy
James Chapman <[email protected]>
l2tp: don't use inet_shutdown on ppp session destroy
Zhang Yi <[email protected]>
ext4: add reserved GDT blocks check
Ding Xiang <[email protected]>
ext4: make variable "count" signed
Baokun Li <[email protected]>
ext4: fix bug_on ext4_mb_use_inode_pa
Ilpo Järvinen <[email protected]>
serial: 8250: Store to lsr_save_flags after lsr read
Miaoqian Lin <[email protected]>
usb: gadget: lpc32xx_udc: Fix refcount leak in lpc32xx_udc_probe
Robert Eckelmann <[email protected]>
USB: serial: io_ti: add Agilent E5805A support
Slark Xiao <[email protected]>
USB: serial: option: add support for Cinterion MV31 with new baseline
Ian Abbott <[email protected]>
comedi: vmk80xx: fix expression for tx buffer size
zijun_hu <[email protected]>
irqchip/gic-v3: Iterate over possible CPUs by for_each_possible_cpu()
Miaoqian Lin <[email protected]>
irqchip/gic/realview: Fix refcount leak in realview_gic_of_init
Miaoqian Lin <[email protected]>
misc: atmel-ssc: Fix IRQ check in ssc_probe
Trond Myklebust <[email protected]>
pNFS: Don't keep retrying if the server replied NFS4ERR_LAYOUTUNAVAILABLE
Jason A. Donenfeld <[email protected]>
random: credit cpu and bootloader seeds by default
Chen Lin <[email protected]>
net: ethernet: mtk_eth_soc: fix misuse of mem alloc interface netdev[napi]_alloc_frag
Wang Yufen <[email protected]>
ipv6: Fix signed integer overflow in l2tp_ip6_sendmsg
Xiaohui Zhang <[email protected]>
nfc: nfcmrvl: Fix memory leak in nfcmrvl_play_deferred
chengkaitao <[email protected]>
virtio-mmio: fix missing put_device() when vm_cmdline_parent registration failed
James Smart <[email protected]>
scsi: lpfc: Fix port stuck in bypassed state after LIP in PT2PT topology
Wentao Wang <[email protected]>
scsi: vmw_pvscsi: Expand vcpuHint to 16 bits
Adam Ford <[email protected]>
ASoC: wm8962: Fix suspend while playing music
Sergey Shtylyov <[email protected]>
ata: libata-core: fix NULL pointer deref in ata_host_alloc_pinfo()
Charles Keepax <[email protected]>
ASoC: cs42l56: Correct typo in minimum level for SX volume controls
Charles Keepax <[email protected]>
ASoC: cs42l52: Correct TLV for Bypass Volume
Charles Keepax <[email protected]>
ASoC: cs53l30: Correct number of volume levels on SX controls
Charles Keepax <[email protected]>
ASoC: cs42l52: Fix TLV scales for mixer controls
Jason A. Donenfeld <[email protected]>
random: account for arch randomness in bits
Jason A. Donenfeld <[email protected]>
random: mark bootloader randomness code as __init
Jason A. Donenfeld <[email protected]>
random: avoid checking crng_ready() twice in random_init()
Nicolai Stange <[email protected]>
crypto: drbg - make reseeding from get_random_bytes() synchronous
Stephan Müller <[email protected]>
crypto: drbg - always try to free Jitter RNG instance
Nicolai Stange <[email protected]>
crypto: drbg - move dynamic ->reseed_threshold adjustments to __drbg_seed()
Nicolai Stange <[email protected]>
crypto: drbg - track whether DRBG was seeded with !rng_is_initialized()
Nicolai Stange <[email protected]>
crypto: drbg - prepare for more fine-grained tracking of seeding state
Stephan Müller <[email protected]>
crypto: drbg - always seeded with SP800-90B compliant noise source
Stephan Mueller <[email protected]>
crypto: drbg - add FIPS 140-2 CTRNG for noise source
Jason A. Donenfeld <[email protected]>
Revert "random: use static branch for crng_ready()"
Jason A. Donenfeld <[email protected]>
random: check for signals after page of pool writes
Jens Axboe <[email protected]>
random: wire up fops->splice_{read,write}_iter()
Jens Axboe <[email protected]>
random: convert to using fops->write_iter()
Jason A. Donenfeld <[email protected]>
random: move randomize_page() into mm where it belongs
Jason A. Donenfeld <[email protected]>
random: move initialization functions out of hot pages
Jason A. Donenfeld <[email protected]>
random: use proper return types on get_random_{int,long}_wait()
Jason A. Donenfeld <[email protected]>
random: remove extern from functions in header
Jason A. Donenfeld <[email protected]>
random: use static branch for crng_ready()
Jason A. Donenfeld <[email protected]>
random: credit architectural init the exact amount
Jason A. Donenfeld <[email protected]>
random: handle latent entropy and command line from random_init()
Jason A. Donenfeld <[email protected]>
random: use proper jiffies comparison macro
Jason A. Donenfeld <[email protected]>
random: remove ratelimiting for in-kernel unseeded randomness
Jason A. Donenfeld <[email protected]>
random: avoid initializing twice in credit race
Jason A. Donenfeld <[email protected]>
random: use symbolic constants for crng_init states
Jason A. Donenfeld <[email protected]>
siphash: use one source of truth for siphash permutations
Jason A. Donenfeld <[email protected]>
random: help compiler out with fast_mix() by using simpler arguments
Jason A. Donenfeld <[email protected]>
random: do not use input pool from hard IRQs
Jason A. Donenfeld <[email protected]>
random: order timer entropy functions below interrupt functions
Jason A. Donenfeld <[email protected]>
random: do not pretend to handle premature next security model
Jason A. Donenfeld <[email protected]>
random: do not use batches when !crng_ready()
Jason A. Donenfeld <[email protected]>
random: insist on random_get_entropy() existing in order to simplify
Yury Norov <[email protected]>
uapi: rename ext2_swab() to swab() and share globally in swab.h
Jason A. Donenfeld <[email protected]>
xtensa: use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
sparc: use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
um: use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
x86/tsc: Use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
nios2: use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
arm: use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
mips: use fallback for random_get_entropy() instead of just c0 random
Jason A. Donenfeld <[email protected]>
m68k: use fallback for random_get_entropy() instead of zero
Jason A. Donenfeld <[email protected]>
timekeeping: Add raw clock fallback for random_get_entropy()
Jason A. Donenfeld <[email protected]>
powerpc: define get_cycles macro for arch-override
Jason A. Donenfeld <[email protected]>
alpha: define get_cycles macro for arch-override
Jason A. Donenfeld <[email protected]>
parisc: define get_cycles macro for arch-override
Jason A. Donenfeld <[email protected]>
s390: define get_cycles macro for arch-override
Jason A. Donenfeld <[email protected]>
ia64: define get_cycles macro for arch-override
Jason A. Donenfeld <[email protected]>
init: call time_init() before rand_initialize()
Jason A. Donenfeld <[email protected]>
random: fix sysctl documentation nits
Jason A. Donenfeld <[email protected]>
random: document crng_fast_key_erasure() destination possibility
Jason A. Donenfeld <[email protected]>
random: make random_get_entropy() return an unsigned long
Jason A. Donenfeld <[email protected]>
random: check for signals every PAGE_SIZE chunk of /dev/[u]random
Jann Horn <[email protected]>
random: check for signal_pending() outside of need_resched() check
Jason A. Donenfeld <[email protected]>
random: do not allow user to keep crng key around on stack
Jan Varho <[email protected]>
random: do not split fast init input in add_hwgenerator_randomness()
Jason A. Donenfeld <[email protected]>
random: mix build-time latent entropy into pool at init
Jason A. Donenfeld <[email protected]>
random: re-add removed comment about get_random_{u32,u64} reseeding
Jason A. Donenfeld <[email protected]>
random: treat bootloader trust toggle the same way as cpu trust toggle
Jason A. Donenfeld <[email protected]>
random: skip fast_init if hwrng provides large chunk of entropy
Jason A. Donenfeld <[email protected]>
random: check for signal and try earlier when generating entropy
Jason A. Donenfeld <[email protected]>
random: reseed more often immediately after booting
Jason A. Donenfeld <[email protected]>
random: make consistent usage of crng_ready()
Jason A. Donenfeld <[email protected]>
random: use SipHash as interrupt entropy accumulator
Jason A. Donenfeld <[email protected]>
random: replace custom notifier chain with standard one
Jason A. Donenfeld <[email protected]>
random: don't let 644 read-only sysctls be written to
Jason A. Donenfeld <[email protected]>
random: give sysctl_random_min_urandom_seed a more sensible value
Jason A. Donenfeld <[email protected]>
random: do crng pre-init loading in worker rather than irq
Jason A. Donenfeld <[email protected]>
random: unify cycles_t and jiffies usage and types
Jason A. Donenfeld <[email protected]>
random: cleanup UUID handling
Jason A. Donenfeld <[email protected]>
random: only wake up writers after zap if threshold was passed
Jason A. Donenfeld <[email protected]>
random: round-robin registers as ulong, not u32
Jason A. Donenfeld <[email protected]>
random: clear fast pool, crng, and batches in cpuhp bring up
Jason A. Donenfeld <[email protected]>
random: pull add_hwgenerator_randomness() declaration into random.h
Harald Freudenberger <[email protected]>
hwrng: remember rng chosen by user
Harald Freudenberger <[email protected]>
hwrng: use rng source with best quality
Corentin LABBE <[email protected]>
hwrng: core - remove unused PFX macro
Corentin LABBE <[email protected]>
hwrng: core - Move hwrng miscdev minor number to include/linux/miscdevice.h
Corentin LABBE <[email protected]>
hwrng: core - Rewrite the header
Corentin LABBE <[email protected]>
hwrng: core - rewrite better comparison to NULL
Corentin LABBE <[email protected]>
hwrng: core - do not use multiple blank lines
Jason A. Donenfeld <[email protected]>
random: check for crng_init == 0 in add_device_randomness()
Jason A. Donenfeld <[email protected]>
random: unify early init crng load accounting
Jason A. Donenfeld <[email protected]>
random: do not take pool spinlock at boot
Jason A. Donenfeld <[email protected]>
random: defer fast pool mixing to worker
Tejun Heo <[email protected]>
workqueue: make workqueue available early during boot
Jason A. Donenfeld <[email protected]>
random: rewrite header introductory comment
Jason A. Donenfeld <[email protected]>
random: group sysctl functions
Jason A. Donenfeld <[email protected]>
random: group userspace read/write functions
Jason A. Donenfeld <[email protected]>
random: group entropy collection functions
Jason A. Donenfeld <[email protected]>
random: group entropy extraction functions
Jason A. Donenfeld <[email protected]>
random: group initialization wait functions
Jason A. Donenfeld <[email protected]>
random: remove whitespace and reorder includes
Jason A. Donenfeld <[email protected]>
random: remove useless header comment
Jason A. Donenfeld <[email protected]>
random: introduce drain_entropy() helper to declutter crng_reseed()
Jason A. Donenfeld <[email protected]>
random: deobfuscate irq u32/u64 contributions
Jason A. Donenfeld <[email protected]>
random: add proper SPDX header
Jason A. Donenfeld <[email protected]>
random: remove unused tracepoints
Jason A. Donenfeld <[email protected]>
random: remove ifdef'd out interrupt bench
Jason A. Donenfeld <[email protected]>
random: tie batched entropy generation to base_crng generation
Jason A. Donenfeld <[email protected]>
random: zero buffer after reading entropy from userspace
Jason A. Donenfeld <[email protected]>
random: remove outdated INT_MAX >> 6 check in urandom_read()
Jason A. Donenfeld <[email protected]>
random: use hash function for crng_slow_load()
Jason A. Donenfeld <[email protected]>
random: absorb fast pool into input pool after fast load
Jason A. Donenfeld <[email protected]>
random: do not xor RDRAND when writing into /dev/random
Jason A. Donenfeld <[email protected]>
random: ensure early RDSEED goes through mixer on init
Jason A. Donenfeld <[email protected]>
random: inline leaves of rand_initialize()
Jason A. Donenfeld <[email protected]>
random: use RDSEED instead of RDRAND in entropy extraction
Dominik Brodowski <[email protected]>
random: fix locking in crng_fast_load()
Jason A. Donenfeld <[email protected]>
random: remove batched entropy locking
Eric Biggers <[email protected]>
random: remove use_input_pool parameter from crng_reseed()
Jason A. Donenfeld <[email protected]>
random: make credit_entropy_bits() always safe
Jason A. Donenfeld <[email protected]>
random: always wake up entropy writers after extraction
Jason A. Donenfeld <[email protected]>
random: use linear min-entropy accumulation crediting
Jason A. Donenfeld <[email protected]>
random: simplify entropy debiting
Jason A. Donenfeld <[email protected]>
random: use computational hash for entropy extraction
Dominik Brodowski <[email protected]>
random: only call crng_finalize_init() for primary_crng
Dominik Brodowski <[email protected]>
random: access primary_pool directly rather than through pointer
Dominik Brodowski <[email protected]>
random: continually use hwgenerator randomness
Jason A. Donenfeld <[email protected]>
random: simplify arithmetic function flow in account()
Jason A. Donenfeld <[email protected]>
random: access input_pool_data directly rather than through pointer
Jason A. Donenfeld <[email protected]>
random: cleanup fractional entropy shift constants
Jason A. Donenfeld <[email protected]>
random: prepend remaining pool constants with POOL_
Jason A. Donenfeld <[email protected]>
random: de-duplicate INPUT_POOL constants
Jason A. Donenfeld <[email protected]>
random: remove unused OUTPUT_POOL constants
Jason A. Donenfeld <[email protected]>
random: rather than entropy_store abstraction, use global
Linus Torvalds <[email protected]>
random: try to actively add entropy rather than passively wait for it
Jason A. Donenfeld <[email protected]>
random: remove unused extract_entropy() reserved argument
Jason A. Donenfeld <[email protected]>
random: remove incomplete last_data logic
Jason A. Donenfeld <[email protected]>
random: cleanup integer types
Eric Biggers <[email protected]>
crypto: chacha20 - Fix chacha20_block() keystream alignment (again)
Jason A. Donenfeld <[email protected]>
random: cleanup poolinfo abstraction
Schspa Shi <[email protected]>
random: fix typo in comments
Jann Horn <[email protected]>
random: don't reset crng_init_cnt on urandom_read()
Jason A. Donenfeld <[email protected]>
random: avoid superfluous call to RDRAND in CRNG extraction
Dominik Brodowski <[email protected]>
random: early initialization of ChaCha constants
Eric Biggers <[email protected]>
random: initialize ChaCha20 constants with correct endianness
Jason A. Donenfeld <[email protected]>
random: use IS_ENABLED(CONFIG_NUMA) instead of ifdefs
Dominik Brodowski <[email protected]>
random: harmonize "crng init done" messages
Jason A. Donenfeld <[email protected]>
random: mix bootloader randomness into pool
Jason A. Donenfeld <[email protected]>
random: do not re-init if crng_reseed completes before primary init
Jason A. Donenfeld <[email protected]>
random: do not sign extend bytes for rotation when mixing
Jason A. Donenfeld <[email protected]>
random: use BLAKE2s instead of SHA1 in extraction
Sebastian Andrzej Siewior <[email protected]>
random: remove unused irq_flags argument from add_interrupt_randomness()
Mark Brown <[email protected]>
random: document add_hwgenerator_randomness() with other input functions
Eric Biggers <[email protected]>
crypto: blake2s - adjust include guard naming
Eric Biggers <[email protected]>
crypto: blake2s - include <linux/bug.h> instead of <asm/bug.h>
Jason A. Donenfeld <[email protected]>
MAINTAINERS: co-maintain random.c
Eric Biggers <[email protected]>
random: remove dead code left over from blocking pool
Ard Biesheuvel <[email protected]>
random: avoid arch_get_random_seed_long() when collecting IRQ randomness
Mark Rutland <[email protected]>
random: add arch_get_random_*long_early()
Richard Henderson <[email protected]>
powerpc: Use bool in archrandom.h
Richard Henderson <[email protected]>
linux/random.h: Mark CONFIG_ARCH_RANDOM functions __must_check
Richard Henderson <[email protected]>
linux/random.h: Use false with bool
Richard Henderson <[email protected]>
linux/random.h: Remove arch_has_random, arch_has_random_seed
Richard Henderson <[email protected]>
powerpc: Remove arch_has_random, arch_has_random_seed
Richard Henderson <[email protected]>
x86: Remove arch_has_random, arch_has_random_seed
Mark Rutland <[email protected]>
random: avoid warnings for !CONFIG_NUMA builds
Mark Rutland <[email protected]>
random: split primary/secondary crng init paths
Yangtao Li <[email protected]>
random: remove some dead code of poolinfo
Yangtao Li <[email protected]>
random: fix typo in add_timer_randomness()
Yangtao Li <[email protected]>
random: Add and use pr_fmt()
Yangtao Li <[email protected]>
random: convert to ENTROPY_BITS for better code readability
Yangtao Li <[email protected]>
random: remove unnecessary unlikely()
Andy Lutomirski <[email protected]>
random: remove kernel.random.read_wakeup_threshold
Andy Lutomirski <[email protected]>
random: delete code to pull data into pools
Andy Lutomirski <[email protected]>
random: remove the blocking pool
Dominik Brodowski <[email protected]>
random: fix crash on multiple early calls to add_bootloader_randomness()
Andy Lutomirski <[email protected]>
random: make /dev/random be almost like /dev/urandom
Andy Lutomirski <[email protected]>
random: ignore GRND_RANDOM in getentropy(2)
Andy Lutomirski <[email protected]>
random: add GRND_INSECURE to return best-effort non-cryptographic bytes
Andy Lutomirski <[email protected]>
random: Add a urandom_read_nowait() for random APIs that don't warn
Andy Lutomirski <[email protected]>
random: Don't wake crng_init_wait when crng_init == 1
Jason A. Donenfeld <[email protected]>
lib/crypto: sha1: re-roll loops to reduce code size
Jason A. Donenfeld <[email protected]>
lib/crypto: blake2s: move hmac construction into wireguard
Jason A. Donenfeld <[email protected]>
crypto: blake2s - generic C library implementation and selftest
Andy Shevchenko <[email protected]>
crypto: Deduplicate le32_to_cpu_array() and cpu_to_le32_array()
Herbert Xu <[email protected]>
Revert "hwrng: core - Freeze khwrng thread during suspend"
Borislav Petkov <[email protected]>
char/random: Add a newline at the end of the file
Stephen Boyd <[email protected]>
random: Use wait_event_freezable() in add_hwgenerator_randomness()
Hsin-Yi Wang <[email protected]>
fdt: add support for rng-seed
Stephen Boyd <[email protected]>
random: Support freezable kthreads in add_hwgenerator_randomness()
Theodore Ts'o <[email protected]>
random: fix soft lockup when trying to read from an uninitialized blocking pool
Vasily Gorbik <[email protected]>
latent_entropy: avoid build error when plugin cflags are not set
George Spelvin <[email protected]>
random: document get_random_int() family
Kees Cook <[email protected]>
random: move rand_initialize() earlier
Theodore Ts'o <[email protected]>
random: only read from /dev/random after its pool has received 128 bits
Rasmus Villemoes <[email protected]>
drivers/char/random.c: make primary_crng static
Rasmus Villemoes <[email protected]>
drivers/char/random.c: remove unused stuct poolinfo::poolbits
Rasmus Villemoes <[email protected]>
drivers/char/random.c: constify poolinfo_table
Kees Cook <[email protected]>
random: make CPU trust a boot parameter
Jason A. Donenfeld <[email protected]>
random: Make crng state queryable
Ingo Molnar <[email protected]>
random: remove preempt disabled region
Theodore Ts'o <[email protected]>
random: add a config option to trust the CPU's hwrng
Tobin C. Harding <[email protected]>
random: Return nbytes filled from hw RNG
Tobin C. Harding <[email protected]>
random: Fix whitespace pre random-bytes work
Rasmus Villemoes <[email protected]>
drivers/char/random.c: remove unused dont_count_entropy
Andi Kleen <[email protected]>
random: optimize add_interrupt_randomness
Jason A. Donenfeld <[email protected]>
random: always fill buffer in get_random_bytes_wait
Eric Biggers <[email protected]>
crypto: chacha20 - Fix keystream alignment for chacha20_block()
Eric Biggers <[email protected]>
random: fix data race on crng_node_pool
Jason A. Donenfeld <[email protected]>
random: always use batched entropy for get_random_u{32,64}
Greg Kroah-Hartman <[email protected]>
Revert "char/random: silence a lockdep splat with printk()"
Sergey Senozhatsky <[email protected]>
char/random: silence a lockdep splat with printk()
Sebastian Andrzej Siewior <[email protected]>
random: add a spinlock_t to struct batched_entropy
Theodore Ts'o <[email protected]>
random: rate limit unseeded randomness warnings
Theodore Ts'o <[email protected]>
random: fix possible sleeping allocation from irq context
Theodore Ts'o <[email protected]>
random: set up the NUMA crng instances after the CRNG is fully initialized
Theodore Ts'o <[email protected]>
random: use a different mixing algorithm for add_device_randomness()
Helge Deller <[email protected]>
random: fix warning message on ia64 and parisc
Sebastian Andrzej Siewior <[email protected]>
random: reorder READ_ONCE() in get_random_uXX
Theodore Ts'o <[email protected]>
random: suppress spammy warnings about unseeded randomness
Kees Cook <[email protected]>
random: do not ignore early device randomness
Jason A. Donenfeld <[email protected]>
random: warn when kernel uses unseeded randomness
Jason A. Donenfeld <[email protected]>
random: add get_random_{bytes,u32,u64,int,long,once}_wait family
Jason A. Donenfeld <[email protected]>
random: add wait_for_random_bytes() API
Jason A. Donenfeld <[email protected]>
random: silence compiler warnings and fix race
Jason A. Donenfeld <[email protected]>
random: invalidate batched entropy after crng init
Fabio Estevam <[email protected]>
random: move random_min_urandom_seed into CONFIG_SYSCTL ifdef block
Jason A. Donenfeld <[email protected]>
random: convert get_random_int/long into get_random_u32/u64
Stephan Müller <[email protected]>
random: fix comment for unused random_min_urandom_seed
Stephan Müller <[email protected]>
random: remove variable limit
Stephan Müller <[email protected]>
random: remove stale urandom_init_wait
Stephan Mueller <[email protected]>
random: remove stale maybe_reseed_primary_crng
Al Viro <[email protected]>
9p: missing chunk of "fs/9p: Don't update file type when updating file attributes"
-------------
Diffstat:
Documentation/kernel-parameters.txt | 12 +
Documentation/sysctl/kernel.txt | 35 +-
MAINTAINERS | 1 +
Makefile | 4 +-
arch/alpha/include/asm/timex.h | 1 +
arch/arm/include/asm/timex.h | 1 +
arch/ia64/include/asm/timex.h | 1 +
arch/m68k/include/asm/timex.h | 2 +-
arch/mips/include/asm/timex.h | 17 +-
arch/nios2/include/asm/timex.h | 3 +
arch/parisc/include/asm/timex.h | 3 +-
arch/powerpc/include/asm/archrandom.h | 27 +-
arch/powerpc/include/asm/timex.h | 1 +
arch/s390/include/asm/timex.h | 1 +
arch/s390/mm/pgtable.c | 2 +-
arch/sparc/include/asm/timex_32.h | 4 +-
arch/um/include/asm/timex.h | 9 +-
arch/x86/include/asm/archrandom.h | 12 +-
arch/x86/include/asm/timex.h | 9 +
arch/x86/include/asm/tsc.h | 7 +-
arch/xtensa/include/asm/timex.h | 6 +-
crypto/chacha20_generic.c | 3 +-
crypto/drbg.c | 220 +-
crypto/md4.c | 17 -
crypto/md5.c | 17 -
drivers/ata/libata-core.c | 4 +-
drivers/char/Kconfig | 38 +-
drivers/char/hw_random/core.c | 93 +-
drivers/char/random.c | 2891 ++++++++++-----------------
drivers/hv/vmbus_drv.c | 2 +-
drivers/irqchip/irq-gic-realview.c | 1 +
drivers/irqchip/irq-gic-v3.c | 8 +-
drivers/misc/atmel-ssc.c | 4 +-
drivers/net/ethernet/mediatek/mtk_eth_soc.c | 21 +-
drivers/nfc/nfcmrvl/usb.c | 16 +-
drivers/of/fdt.c | 14 +-
drivers/scsi/lpfc/lpfc_nportdisc.c | 3 +-
drivers/scsi/vmw_pvscsi.h | 4 +-
drivers/staging/comedi/drivers/vmk80xx.c | 2 +-
drivers/tty/serial/8250/8250_port.c | 2 +
drivers/usb/gadget/udc/lpc32xx_udc.c | 1 +
drivers/usb/serial/io_ti.c | 2 +
drivers/usb/serial/io_usbvend.h | 1 +
drivers/usb/serial/option.c | 6 +
drivers/virtio/virtio_mmio.c | 1 +
fs/9p/vfs_inode_dotl.c | 10 +-
fs/ext4/mballoc.c | 9 +
fs/ext4/namei.c | 3 +-
fs/ext4/resize.c | 10 +
fs/fuse/dev.c | 12 +-
fs/fuse/file.c | 1 +
fs/fuse/fuse_i.h | 2 +
fs/nfs/pnfs.c | 6 +
include/crypto/blake2s.h | 102 +
include/crypto/chacha20.h | 17 +-
include/crypto/drbg.h | 18 +-
include/crypto/internal/blake2s.h | 19 +
include/linux/byteorder/generic.h | 17 +
include/linux/cpuhotplug.h | 2 +
include/linux/hw_random.h | 2 -
include/linux/miscdevice.h | 1 +
include/linux/mm.h | 2 +
include/linux/net.h | 2 +
include/linux/once.h | 2 +
include/linux/prandom.h | 23 +-
include/linux/random.h | 133 +-
include/linux/siphash.h | 28 +
include/linux/swab.h | 1 +
include/linux/timex.h | 10 +-
include/linux/uuid.h | 1 +
include/linux/workqueue.h | 7 +-
include/net/inet_hashtables.h | 2 +-
include/net/secure_seq.h | 4 +-
include/trace/events/random.h | 315 ---
include/uapi/linux/random.h | 4 +-
include/uapi/linux/swab.h | 10 +
init/main.c | 22 +
kernel/cpu.c | 11 +
kernel/irq/handle.c | 2 +-
kernel/time/timekeeping.c | 16 +
kernel/workqueue.c | 76 +-
lib/Kconfig.debug | 27 +
lib/Makefile | 2 +
lib/chacha20.c | 6 +-
lib/crypto/Makefile | 7 +
lib/crypto/blake2s-generic.c | 111 +
lib/crypto/blake2s-selftest.c | 591 ++++++
lib/crypto/blake2s.c | 78 +
lib/find_bit.c | 16 +-
lib/random32.c | 15 +-
lib/sha1.c | 95 +-
lib/siphash.c | 32 +-
lib/swiotlb.c | 11 +-
mm/util.c | 33 +
net/core/secure_seq.c | 4 +-
net/ipv4/inet_hashtables.c | 46 +-
net/ipv6/inet6_hashtables.c | 4 +-
net/l2tp/l2tp_ip6.c | 5 +-
net/l2tp/l2tp_ppp.c | 60 +-
net/sunrpc/xprtrdma/rpc_rdma.c | 4 +-
sound/soc/codecs/cs42l52.c | 8 +-
sound/soc/codecs/cs42l56.c | 4 +-
sound/soc/codecs/cs53l30.c | 16 +-
sound/soc/codecs/wm8962.c | 1 +
104 files changed, 2966 insertions(+), 2641 deletions(-)
From: "Stephan M?ller" <[email protected]>
commit 43d8a72cd985ca5279a9eb84d61fcbb3ee3d3774 upstream.
The variable limit was used to identify the nonblocking pool's unlimited
random number generation. As the nonblocking pool is a thing of the
past, remove the limit variable and any conditions around it (i.e.
preserve the branches for limit == 1).
Signed-off-by: Stephan Mueller <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 32 ++++++++------------------------
1 file changed, 8 insertions(+), 24 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -478,7 +478,6 @@ struct entropy_store {
int entropy_count;
int entropy_total;
unsigned int initialized:1;
- unsigned int limit:1;
unsigned int last_data_init:1;
__u8 last_data[EXTRACT_SIZE];
};
@@ -496,7 +495,6 @@ static __u32 blocking_pool_data[OUTPUT_P
static struct entropy_store input_pool = {
.poolinfo = &poolinfo_table[0],
.name = "input",
- .limit = 1,
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
.pool = input_pool_data
};
@@ -504,7 +502,6 @@ static struct entropy_store input_pool =
static struct entropy_store blocking_pool = {
.poolinfo = &poolinfo_table[1],
.name = "blocking",
- .limit = 1,
.pull = &input_pool,
.lock = __SPIN_LOCK_UNLOCKED(blocking_pool.lock),
.pool = blocking_pool_data,
@@ -1280,15 +1277,6 @@ static void xfer_secondary_pool(struct e
r->entropy_count > r->poolinfo->poolfracbits)
return;
- if (r->limit == 0 && random_min_urandom_seed) {
- unsigned long now = jiffies;
-
- if (time_before(now,
- r->last_pulled + random_min_urandom_seed * HZ))
- return;
- r->last_pulled = now;
- }
-
_xfer_secondary_pool(r, nbytes);
}
@@ -1296,8 +1284,6 @@ static void _xfer_secondary_pool(struct
{
__u32 tmp[OUTPUT_POOL_WORDS];
- /* For /dev/random's pool, always leave two wakeups' worth */
- int rsvd_bytes = r->limit ? 0 : random_read_wakeup_bits / 4;
int bytes = nbytes;
/* pull at least as much as a wakeup */
@@ -1308,7 +1294,7 @@ static void _xfer_secondary_pool(struct
trace_xfer_secondary_pool(r->name, bytes * 8, nbytes * 8,
ENTROPY_BITS(r), ENTROPY_BITS(r->pull));
bytes = extract_entropy(r->pull, tmp, bytes,
- random_read_wakeup_bits / 8, rsvd_bytes);
+ random_read_wakeup_bits / 8, 0);
mix_pool_bytes(r, tmp, bytes);
credit_entropy_bits(r, bytes*8);
}
@@ -1336,7 +1322,7 @@ static void push_to_pool(struct work_str
static size_t account(struct entropy_store *r, size_t nbytes, int min,
int reserved)
{
- int entropy_count, orig;
+ int entropy_count, orig, have_bytes;
size_t ibytes, nfrac;
BUG_ON(r->entropy_count > r->poolinfo->poolfracbits);
@@ -1345,14 +1331,12 @@ static size_t account(struct entropy_sto
retry:
entropy_count = orig = ACCESS_ONCE(r->entropy_count);
ibytes = nbytes;
- /* If limited, never pull more than available */
- if (r->limit) {
- int have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
-
- if ((have_bytes -= reserved) < 0)
- have_bytes = 0;
- ibytes = min_t(size_t, ibytes, have_bytes);
- }
+ /* never pull more than available */
+ have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
+
+ if ((have_bytes -= reserved) < 0)
+ have_bytes = 0;
+ ibytes = min_t(size_t, ibytes, have_bytes);
if (ibytes < min)
ibytes = 0;
From: Kees Cook <[email protected]>
commit 9b25436662d5fb4c66eb527ead53cab15f596ee0 upstream.
Instead of forcing a distro or other system builder to choose
at build time whether the CPU is trusted for CRNG seeding via
CONFIG_RANDOM_TRUST_CPU, provide a boot-time parameter for end users to
control the choice. The CONFIG will set the default state instead.
Signed-off-by: Kees Cook <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/kernel-parameters.txt | 6 ++++++
drivers/char/Kconfig | 4 ++--
drivers/char/random.c | 11 ++++++++---
3 files changed, 16 insertions(+), 5 deletions(-)
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -3577,6 +3577,12 @@ bytes respectively. Such letter suffixes
ramdisk_size= [RAM] Sizes of RAM disks in kilobytes
See Documentation/blockdev/ramdisk.txt.
+ random.trust_cpu={on,off}
+ [KNL] Enable or disable trusting the use of the
+ CPU's random number generator (if available) to
+ fully seed the kernel's CRNG. Default is controlled
+ by CONFIG_RANDOM_TRUST_CPU.
+
rcu_nocbs= [KNL]
The argument is a cpu list, as described above.
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -607,5 +607,5 @@ config RANDOM_TRUST_CPU
that CPU manufacturer (perhaps with the insistence or mandate
of a Nation State's intelligence or law enforcement agencies)
has not installed a hidden back door to compromise the CPU's
- random number generation facilities.
-
+ random number generation facilities. This can also be configured
+ at boot with "random.trust_cpu=on/off".
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -780,6 +780,13 @@ static struct crng_state **crng_node_poo
static void invalidate_batched_entropy(void);
+static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
+static int __init parse_trust_cpu(char *arg)
+{
+ return kstrtobool(arg, &trust_cpu);
+}
+early_param("random.trust_cpu", parse_trust_cpu);
+
static void crng_initialize(struct crng_state *crng)
{
int i;
@@ -800,12 +807,10 @@ static void crng_initialize(struct crng_
}
crng->state[i] ^= rv;
}
-#ifdef CONFIG_RANDOM_TRUST_CPU
- if (arch_init) {
+ if (trust_cpu && arch_init) {
crng_init = 2;
pr_notice("random: crng done (trusting CPU's manufacturer)\n");
}
-#endif
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
From: Andi Kleen <[email protected]>
commit e8e8a2e47db6bb85bb0cb21e77b5c6aaedf864b4 upstream.
add_interrupt_randomess always wakes up
code blocking on /dev/random. This wake up is done
unconditionally. Unfortunately this means all interrupts
take the wait queue spinlock, which can be rather expensive
on large systems processing lots of interrupts.
We saw 1% cpu time spinning on this on a large macro workload
running on a large system.
I believe it's a recent regression (?)
Always check if there is a waiter on the wait queue
before waking up. This check can be done without
taking a spinlock.
1.06% 10460 [kernel.vmlinux] [k] native_queued_spin_lock_slowpath
|
---native_queued_spin_lock_slowpath
|
--0.57%--_raw_spin_lock_irqsave
|
--0.56%--__wake_up_common_lock
credit_entropy_bits
add_interrupt_randomness
handle_irq_event_percpu
handle_irq_event
handle_edge_irq
handle_irq
do_IRQ
common_interrupt
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -722,7 +722,8 @@ retry:
}
/* should we wake readers? */
- if (entropy_bits >= random_read_wakeup_bits) {
+ if (entropy_bits >= random_read_wakeup_bits &&
+ wq_has_sleeper(&random_read_wait)) {
wake_up_interruptible(&random_read_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
}
From: Herbert Xu <[email protected]>
commit 08e97aec700aeff54c4847f170e566cbd7e14e81 upstream.
This reverts commit 03a3bb7ae631 ("hwrng: core - Freeze khwrng
thread during suspend"), ff296293b353 ("random: Support freezable
kthreads in add_hwgenerator_randomness()") and 59b569480dc8 ("random:
Use wait_event_freezable() in add_hwgenerator_randomness()").
These patches introduced regressions and we need more time to
get them ready for mainline.
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -327,7 +327,6 @@
#include <linux/percpu.h>
#include <linux/cryptohash.h>
#include <linux/fips.h>
-#include <linux/freezer.h>
#include <linux/ptrace.h>
#include <linux/kmemcheck.h>
#include <linux/workqueue.h>
@@ -2484,8 +2483,7 @@ void add_hwgenerator_randomness(const ch
* We'll be woken up again once below random_write_wakeup_thresh,
* or when the calling thread is about to terminate.
*/
- wait_event_freezable(random_write_wait,
- kthread_should_stop() ||
+ wait_event_interruptible(random_write_wait, kthread_should_stop() ||
ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
mix_pool_bytes(poolp, buffer, count);
credit_entropy_bits(poolp, entropy);
From: Sebastian Andrzej Siewior <[email protected]>
[ Upstream commit b7d5dc21072cda7124d13eae2aefb7343ef94197 ]
The per-CPU variable batched_entropy_uXX is protected by get_cpu_var().
This is just a preempt_disable() which ensures that the variable is only
from the local CPU. It does not protect against users on the same CPU
from another context. It is possible that a preemptible context reads
slot 0 and then an interrupt occurs and the same value is read again.
The above scenario is confirmed by lockdep if we add a spinlock:
| ================================
| WARNING: inconsistent lock state
| 5.1.0-rc3+ #42 Not tainted
| --------------------------------
| inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
| ksoftirqd/9/56 [HC0[0]:SC1[1]:HE0:SE0] takes:
| (____ptrval____) (batched_entropy_u32.lock){+.?.}, at: get_random_u32+0x3e/0xe0
| {SOFTIRQ-ON-W} state was registered at:
| _raw_spin_lock+0x2a/0x40
| get_random_u32+0x3e/0xe0
| new_slab+0x15c/0x7b0
| ___slab_alloc+0x492/0x620
| __slab_alloc.isra.73+0x53/0xa0
| kmem_cache_alloc_node+0xaf/0x2a0
| copy_process.part.41+0x1e1/0x2370
| _do_fork+0xdb/0x6d0
| kernel_thread+0x20/0x30
| kthreadd+0x1ba/0x220
| ret_from_fork+0x3a/0x50
…
| other info that might help us debug this:
| Possible unsafe locking scenario:
|
| CPU0
| ----
| lock(batched_entropy_u32.lock);
| <Interrupt>
| lock(batched_entropy_u32.lock);
|
| *** DEADLOCK ***
|
| stack backtrace:
| Call Trace:
…
| kmem_cache_alloc_trace+0x20e/0x270
| ipmi_alloc_recv_msg+0x16/0x40
…
| __do_softirq+0xec/0x48d
| run_ksoftirqd+0x37/0x60
| smpboot_thread_fn+0x191/0x290
| kthread+0xfe/0x130
| ret_from_fork+0x3a/0x50
Add a spinlock_t to the batched_entropy data structure and acquire the
lock while accessing it. Acquire the lock with disabled interrupts
because this function may be used from interrupt context.
Remove the batched_entropy_reset_lock lock. Now that we have a lock for
the data scructure, we can access it from a remote CPU.
Signed-off-by: Sebastian Andrzej Siewior <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 52 +++++++++++++++++++++++++-------------------------
1 file changed, 27 insertions(+), 25 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2228,8 +2228,8 @@ struct batched_entropy {
u32 entropy_u32[CHACHA20_BLOCK_SIZE / sizeof(u32)];
};
unsigned int position;
+ spinlock_t batch_lock;
};
-static rwlock_t batched_entropy_reset_lock = __RW_LOCK_UNLOCKED(batched_entropy_reset_lock);
/*
* Get a random word for internal kernel use only. The quality of the random
@@ -2239,12 +2239,14 @@ static rwlock_t batched_entropy_reset_lo
* wait_for_random_bytes() should be called and return 0 at least once
* at any point prior.
*/
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64);
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
+ .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
+};
+
u64 get_random_u64(void)
{
u64 ret;
- bool use_lock;
- unsigned long flags = 0;
+ unsigned long flags;
struct batched_entropy *batch;
static void *previous;
@@ -2259,28 +2261,25 @@ u64 get_random_u64(void)
warn_unseeded_randomness(&previous);
- use_lock = READ_ONCE(crng_init) < 2;
- batch = &get_cpu_var(batched_entropy_u64);
- if (use_lock)
- read_lock_irqsave(&batched_entropy_reset_lock, flags);
+ batch = raw_cpu_ptr(&batched_entropy_u64);
+ spin_lock_irqsave(&batch->batch_lock, flags);
if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) {
extract_crng((u8 *)batch->entropy_u64);
batch->position = 0;
}
ret = batch->entropy_u64[batch->position++];
- if (use_lock)
- read_unlock_irqrestore(&batched_entropy_reset_lock, flags);
- put_cpu_var(batched_entropy_u64);
+ spin_unlock_irqrestore(&batch->batch_lock, flags);
return ret;
}
EXPORT_SYMBOL(get_random_u64);
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32);
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
+ .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock),
+};
u32 get_random_u32(void)
{
u32 ret;
- bool use_lock;
- unsigned long flags = 0;
+ unsigned long flags;
struct batched_entropy *batch;
static void *previous;
@@ -2289,18 +2288,14 @@ u32 get_random_u32(void)
warn_unseeded_randomness(&previous);
- use_lock = READ_ONCE(crng_init) < 2;
- batch = &get_cpu_var(batched_entropy_u32);
- if (use_lock)
- read_lock_irqsave(&batched_entropy_reset_lock, flags);
+ batch = raw_cpu_ptr(&batched_entropy_u32);
+ spin_lock_irqsave(&batch->batch_lock, flags);
if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) {
extract_crng((u8 *)batch->entropy_u32);
batch->position = 0;
}
ret = batch->entropy_u32[batch->position++];
- if (use_lock)
- read_unlock_irqrestore(&batched_entropy_reset_lock, flags);
- put_cpu_var(batched_entropy_u32);
+ spin_unlock_irqrestore(&batch->batch_lock, flags);
return ret;
}
EXPORT_SYMBOL(get_random_u32);
@@ -2314,12 +2309,19 @@ static void invalidate_batched_entropy(v
int cpu;
unsigned long flags;
- write_lock_irqsave(&batched_entropy_reset_lock, flags);
for_each_possible_cpu (cpu) {
- per_cpu_ptr(&batched_entropy_u32, cpu)->position = 0;
- per_cpu_ptr(&batched_entropy_u64, cpu)->position = 0;
+ struct batched_entropy *batched_entropy;
+
+ batched_entropy = per_cpu_ptr(&batched_entropy_u32, cpu);
+ spin_lock_irqsave(&batched_entropy->batch_lock, flags);
+ batched_entropy->position = 0;
+ spin_unlock(&batched_entropy->batch_lock);
+
+ batched_entropy = per_cpu_ptr(&batched_entropy_u64, cpu);
+ spin_lock(&batched_entropy->batch_lock);
+ batched_entropy->position = 0;
+ spin_unlock_irqrestore(&batched_entropy->batch_lock, flags);
}
- write_unlock_irqrestore(&batched_entropy_reset_lock, flags);
}
/**
From: Theodore Ts'o <[email protected]>
commit 39a8883a2b989d1d21bd8dd99f5557f0c5e89694 upstream.
This gives the user building their own kernel (or a Linux
distribution) the option of deciding whether or not to trust the CPU's
hardware random number generator (e.g., RDRAND for x86 CPU's) as being
correctly implemented and not having a back door introduced (perhaps
courtesy of a Nation State's law enforcement or intelligence
agencies).
This will prevent getrandom(2) from blocking, if there is a
willingness to trust the CPU manufacturer.
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/Kconfig | 14 ++++++++++++++
drivers/char/random.c | 11 ++++++++++-
2 files changed, 24 insertions(+), 1 deletion(-)
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -595,3 +595,17 @@ source "drivers/char/xillybus/Kconfig"
endmenu
+config RANDOM_TRUST_CPU
+ bool "Trust the CPU manufacturer to initialize Linux's CRNG"
+ depends on X86 || S390 || PPC
+ default n
+ help
+ Assume that CPU manufacturer (e.g., Intel or AMD for RDSEED or
+ RDRAND, IBM for the S390 and Power PC architectures) is trustworthy
+ for the purposes of initializing Linux's CRNG. Since this is not
+ something that can be independently audited, this amounts to trusting
+ that CPU manufacturer (perhaps with the insistence or mandate
+ of a Nation State's intelligence or law enforcement agencies)
+ has not installed a hidden back door to compromise the CPU's
+ random number generation facilities.
+
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -783,6 +783,7 @@ static void invalidate_batched_entropy(v
static void crng_initialize(struct crng_state *crng)
{
int i;
+ int arch_init = 1;
unsigned long rv;
memcpy(&crng->state[0], "expand 32-byte k", 16);
@@ -793,10 +794,18 @@ static void crng_initialize(struct crng_
_get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
for (i = 4; i < 16; i++) {
if (!arch_get_random_seed_long(&rv) &&
- !arch_get_random_long(&rv))
+ !arch_get_random_long(&rv)) {
rv = random_get_entropy();
+ arch_init = 0;
+ }
crng->state[i] ^= rv;
}
+#ifdef CONFIG_RANDOM_TRUST_CPU
+ if (arch_init) {
+ crng_init = 2;
+ pr_notice("random: crng done (trusting CPU's manufacturer)\n");
+ }
+#endif
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
From: "Jason A. Donenfeld" <[email protected]>
commit c440408cf6901eeb2c09563397e24a9097907078 upstream.
Many times, when a user wants a random number, he wants a random number
of a guaranteed size. So, thinking of get_random_int and get_random_long
in terms of get_random_u32 and get_random_u64 makes it much easier to
achieve this. It also makes the code simpler.
On 32-bit platforms, get_random_int and get_random_long are both aliased
to get_random_u32. On 64-bit platforms, int->u32 and long->u64.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 63 ++++++++++++++++++++++++++-----------------------
include/linux/random.h | 17 +++++++++++--
2 files changed, 49 insertions(+), 31 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2089,57 +2089,62 @@ struct ctl_table random_table[] = {
struct batched_entropy {
union {
- unsigned long entropy_long[CHACHA20_BLOCK_SIZE / sizeof(unsigned long)];
- unsigned int entropy_int[CHACHA20_BLOCK_SIZE / sizeof(unsigned int)];
+ u64 entropy_u64[CHACHA20_BLOCK_SIZE / sizeof(u64)];
+ u32 entropy_u32[CHACHA20_BLOCK_SIZE / sizeof(u32)];
};
unsigned int position;
};
/*
* Get a random word for internal kernel use only. The quality of the random
- * number is good as /dev/urandom, but there is no backtrack protection, with
- * the goal of being quite fast and not depleting entropy.
+ * number is either as good as RDRAND or as good as /dev/urandom, with the
+ * goal of being quite fast and not depleting entropy.
*/
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_long);
-unsigned long get_random_long(void)
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64);
+u64 get_random_u64(void)
{
- unsigned long ret;
+ u64 ret;
struct batched_entropy *batch;
- batch = &get_cpu_var(batched_entropy_long);
- if (batch->position % ARRAY_SIZE(batch->entropy_long) == 0) {
- extract_crng((u8 *)batch->entropy_long);
+#if BITS_PER_LONG == 64
+ if (arch_get_random_long((unsigned long *)&ret))
+ return ret;
+#else
+ if (arch_get_random_long((unsigned long *)&ret) &&
+ arch_get_random_long((unsigned long *)&ret + 1))
+ return ret;
+#endif
+
+ batch = &get_cpu_var(batched_entropy_u64);
+ if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) {
+ extract_crng((u8 *)batch->entropy_u64);
batch->position = 0;
}
- ret = batch->entropy_long[batch->position++];
- put_cpu_var(batched_entropy_long);
+ ret = batch->entropy_u64[batch->position++];
+ put_cpu_var(batched_entropy_u64);
return ret;
}
-EXPORT_SYMBOL(get_random_long);
+EXPORT_SYMBOL(get_random_u64);
-#if BITS_PER_LONG == 32
-unsigned int get_random_int(void)
-{
- return get_random_long();
-}
-#else
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_int);
-unsigned int get_random_int(void)
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32);
+u32 get_random_u32(void)
{
- unsigned int ret;
+ u32 ret;
struct batched_entropy *batch;
- batch = &get_cpu_var(batched_entropy_int);
- if (batch->position % ARRAY_SIZE(batch->entropy_int) == 0) {
- extract_crng((u8 *)batch->entropy_int);
+ if (arch_get_random_int(&ret))
+ return ret;
+
+ batch = &get_cpu_var(batched_entropy_u32);
+ if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) {
+ extract_crng((u8 *)batch->entropy_u32);
batch->position = 0;
}
- ret = batch->entropy_int[batch->position++];
- put_cpu_var(batched_entropy_int);
+ ret = batch->entropy_u32[batch->position++];
+ put_cpu_var(batched_entropy_u32);
return ret;
}
-#endif
-EXPORT_SYMBOL(get_random_int);
+EXPORT_SYMBOL(get_random_u32);
/**
* randomize_page - Generate a random, page aligned address
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -42,8 +42,21 @@ extern void get_random_bytes_arch(void *
extern const struct file_operations random_fops, urandom_fops;
#endif
-unsigned int get_random_int(void);
-unsigned long get_random_long(void);
+u32 get_random_u32(void);
+u64 get_random_u64(void);
+static inline unsigned int get_random_int(void)
+{
+ return get_random_u32();
+}
+static inline unsigned long get_random_long(void)
+{
+#if BITS_PER_LONG == 64
+ return get_random_u64();
+#else
+ return get_random_u32();
+#endif
+}
+
unsigned long randomize_page(unsigned long start, unsigned long range);
/*
From: "Stephan M?ller" <[email protected]>
commit 5d0e5ea343a0f70351428476bcf8715e0731f26a upstream.
The variable random_min_urandom_seed is not needed any more as it
defined the reseeding behavior of the nonblocking pool. Though it is not
needed any more, it is left in the code for user space interface
compatibility.
Signed-off-by: Stephan Mueller <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -314,9 +314,7 @@ static int random_read_wakeup_bits = 64;
static int random_write_wakeup_bits = 28 * OUTPUT_POOL_WORDS;
/*
- * The minimum number of seconds between urandom pool reseeding. We
- * do this to limit the amount of entropy that can be drained from the
- * input pool even if there are heavy demands on /dev/urandom.
+ * Variable is currently unused by left for user space compatibility.
*/
static int random_min_urandom_seed = 60;
From: Theodore Ts'o <[email protected]>
commit dc12baacb95f205948f64dc936a47d89ee110117 upstream.
add_device_randomness() use of crng_fast_load() was highly
problematic. Some callers of add_device_randomness() can pass in a
large amount of static information. This would immediately promote
the crng_init state from 0 to 1, without really doing much to
initialize the primary_crng's internal state with something even
vaguely unpredictable.
Since we don't have the speed constraints of add_interrupt_randomness(),
we can do a better job mixing in the what unpredictability a device
driver or architecture maintainer might see fit to give us, and do it
in a way which does not bump the crng_init_cnt variable.
Also, since add_device_randomness() doesn't bump any entropy
accounting in crng_init state 0, mix the device randomness into the
input_pool entropy pool as well. This is related to CVE-2018-1108.
Reported-by: Jann Horn <[email protected]>
Fixes: ee7998c50c26 ("random: do not ignore early device randomness")
Cc: [email protected] # 4.13+
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 51 insertions(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -799,6 +799,10 @@ static void crng_initialize(struct crng_
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
+/*
+ * crng_fast_load() can be called by code in the interrupt service
+ * path. So we can't afford to dilly-dally.
+ */
static int crng_fast_load(const char *cp, size_t len)
{
unsigned long flags;
@@ -876,6 +880,51 @@ static struct crng_state *select_crng(vo
}
#endif
+/*
+ * crng_slow_load() is called by add_device_randomness, which has two
+ * attributes. (1) We can't trust the buffer passed to it is
+ * guaranteed to be unpredictable (so it might not have any entropy at
+ * all), and (2) it doesn't have the performance constraints of
+ * crng_fast_load().
+ *
+ * So we do something more comprehensive which is guaranteed to touch
+ * all of the primary_crng's state, and which uses a LFSR with a
+ * period of 255 as part of the mixing algorithm. Finally, we do
+ * *not* advance crng_init_cnt since buffer we may get may be something
+ * like a fixed DMI table (for example), which might very well be
+ * unique to the machine, but is otherwise unvarying.
+ */
+static int crng_slow_load(const char *cp, size_t len)
+{
+ unsigned long flags;
+ static unsigned char lfsr = 1;
+ unsigned char tmp;
+ unsigned i, max = CHACHA20_KEY_SIZE;
+ const char * src_buf = cp;
+ char * dest_buf = (char *) &primary_crng.state[4];
+
+ if (!spin_trylock_irqsave(&primary_crng.lock, flags))
+ return 0;
+ if (crng_init != 0) {
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
+ return 0;
+ }
+ if (len > max)
+ max = len;
+
+ for (i = 0; i < max ; i++) {
+ tmp = lfsr;
+ lfsr >>= 1;
+ if (tmp & 1)
+ lfsr ^= 0xE1;
+ tmp = dest_buf[i % CHACHA20_KEY_SIZE];
+ dest_buf[i % CHACHA20_KEY_SIZE] ^= src_buf[i % len] ^ lfsr;
+ lfsr += (tmp << 3) | (tmp >> 5);
+ }
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
+ return 1;
+}
+
static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
{
unsigned long flags;
@@ -1046,10 +1095,8 @@ void add_device_randomness(const void *b
unsigned long time = random_get_entropy() ^ jiffies;
unsigned long flags;
- if (!crng_ready()) {
- crng_fast_load(buf, size);
- return;
- }
+ if (!crng_ready() && size)
+ crng_slow_load(buf, size);
trace_add_device_randomness(size, _RET_IP_);
spin_lock_irqsave(&input_pool.lock, flags);
From: Theodore Ts'o <[email protected]>
commit 6c1e851c4edc13a43adb3ea4044e3fc8f43ccf7d upstream.
We can do a sleeping allocation from an irq context when CONFIG_NUMA
is enabled. Fix this by initializing the NUMA crng instances in a
workqueue.
Reported-by: Tetsuo Handa <[email protected]>
Reported-by: [email protected]
Fixes: 8ef35c866f8862df ("random: set up the NUMA crng instances...")
Cc: [email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -800,7 +800,7 @@ static void crng_initialize(struct crng_
}
#ifdef CONFIG_NUMA
-static void numa_crng_init(void)
+static void do_numa_crng_init(struct work_struct *work)
{
int i;
struct crng_state *crng;
@@ -821,6 +821,13 @@ static void numa_crng_init(void)
kfree(pool);
}
}
+
+static DECLARE_WORK(numa_crng_init_work, do_numa_crng_init);
+
+static void numa_crng_init(void)
+{
+ schedule_work(&numa_crng_init_work);
+}
#else
static void numa_crng_init(void) {}
#endif
From: Eric Biggers <[email protected]>
commit 5d73d1e320c3fd94ea15ba5f79301da9a8bcc7de upstream.
extract_crng() and crng_backtrack_protect() load crng_node_pool with a
plain load, which causes undefined behavior if do_numa_crng_init()
modifies it concurrently.
Fix this by using READ_ONCE(). Note: as per the previous discussion
https://lore.kernel.org/lkml/[email protected]/T/#u,
READ_ONCE() is believed to be sufficient here, and it was requested that
it be used here instead of smp_load_acquire().
Also change do_numa_crng_init() to set crng_node_pool using
cmpxchg_release() instead of mb() + cmpxchg(), as the former is
sufficient here but is more lightweight.
Fixes: 1e7f583af67b ("random: make /dev/urandom scalable for silly userspace programs")
Cc: [email protected]
Signed-off-by: Eric Biggers <[email protected]>
Acked-by: Paul E. McKenney <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -814,8 +814,8 @@ static void do_numa_crng_init(struct wor
crng_initialize(crng);
pool[i] = crng;
}
- mb();
- if (cmpxchg(&crng_node_pool, NULL, pool)) {
+ /* pairs with READ_ONCE() in select_crng() */
+ if (cmpxchg_release(&crng_node_pool, NULL, pool) != NULL) {
for_each_node(i)
kfree(pool[i]);
kfree(pool);
@@ -828,8 +828,26 @@ static void numa_crng_init(void)
{
schedule_work(&numa_crng_init_work);
}
+
+static struct crng_state *select_crng(void)
+{
+ struct crng_state **pool;
+ int nid = numa_node_id();
+
+ /* pairs with cmpxchg_release() in do_numa_crng_init() */
+ pool = READ_ONCE(crng_node_pool);
+ if (pool && pool[nid])
+ return pool[nid];
+
+ return &primary_crng;
+}
#else
static void numa_crng_init(void) {}
+
+static struct crng_state *select_crng(void)
+{
+ return &primary_crng;
+}
#endif
/*
From: "Jason A. Donenfeld" <[email protected]>
commit a254a0e4093fce8c832414a83940736067eed515 upstream.
Now that have_bytes is never modified, we can simplify this function.
First, we move the check for negative entropy_count to be first. That
ensures that subsequent reads of this will be non-negative. Then,
have_bytes and ibytes can be folded into their one use site in the
min_t() function.
Suggested-by: Dominik Brodowski <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 17 ++++++-----------
1 file changed, 6 insertions(+), 11 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1344,7 +1344,7 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
*/
static size_t account(size_t nbytes, int min)
{
- int entropy_count, orig, have_bytes;
+ int entropy_count, orig;
size_t ibytes, nfrac;
BUG_ON(input_pool.entropy_count > POOL_FRACBITS);
@@ -1352,20 +1352,15 @@ static size_t account(size_t nbytes, int
/* Can we pull enough? */
retry:
entropy_count = orig = READ_ONCE(input_pool.entropy_count);
- ibytes = nbytes;
- /* never pull more than available */
- have_bytes = entropy_count >> (POOL_ENTROPY_SHIFT + 3);
-
- if (have_bytes < 0)
- have_bytes = 0;
- ibytes = min_t(size_t, ibytes, have_bytes);
- if (ibytes < min)
- ibytes = 0;
-
if (WARN_ON(entropy_count < 0)) {
pr_warn("negative entropy count: count %d\n", entropy_count);
entropy_count = 0;
}
+
+ /* never pull more than available */
+ ibytes = min_t(size_t, nbytes, entropy_count >> (POOL_ENTROPY_SHIFT + 3));
+ if (ibytes < min)
+ ibytes = 0;
nfrac = ibytes << (POOL_ENTROPY_SHIFT + 3);
if ((size_t)entropy_count > nfrac)
entropy_count -= nfrac;
From: Theodore Ts'o <[email protected]>
commit 4e00b339e264802851aff8e73cde7d24b57b18ce upstream.
On systems without sufficient boot randomness, no point spamming dmesg.
Signed-off-by: Theodore Ts'o <[email protected]>
Cc: [email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1637,8 +1637,9 @@ static void _warn_unseeded_randomness(co
#ifndef CONFIG_WARN_ALL_UNSEEDED_RANDOM
print_once = true;
#endif
- pr_notice("random: %s called from %pS with crng_init=%d\n",
- func_name, caller, crng_init);
+ if (__ratelimit(&unseeded_warning))
+ pr_notice("random: %s called from %pS with crng_init=%d\n",
+ func_name, caller, crng_init);
}
/*
From: Dominik Brodowski <[email protected]>
commit 161212c7fd1d9069b232785c75492e50941e2ea8 upstream.
We print out "crng init done" for !TRUST_CPU, so we should also print
out the same for TRUST_CPU.
Signed-off-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -833,7 +833,7 @@ static void __init crng_initialize_prima
invalidate_batched_entropy();
numa_crng_init();
crng_init = 2;
- pr_notice("crng done (trusting CPU's manufacturer)\n");
+ pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 57826feeedb63b091f807ba8325d736775d39afd upstream.
If we're trusting bootloader randomness, crng_fast_load() is called by
add_hwgenerator_randomness(), which sets us to crng_init==1. However,
usually it is only called once for an initial 64-byte push, so bootloader
entropy will not mix any bytes into the input pool. So it's conceivable
that crng_init==1 when crng_initialize_primary() is called later, but
then the input pool is empty. When that happens, the crng state key will
be overwritten with extracted output from the empty input pool. That's
bad.
In contrast, if we're not trusting bootloader randomness, we call
crng_slow_load() *and* we call mix_pool_bytes(), so that later
crng_initialize_primary() isn't drawing on nothing.
In order to prevent crng_initialize_primary() from extracting an empty
pool, have the trusted bootloader case mirror that of the untrusted
bootloader case, mixing the input into the pool.
[[email protected]: rewrite commit message]
Signed-off-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2285,8 +2285,12 @@ void add_hwgenerator_randomness(const ch
struct entropy_store *poolp = &input_pool;
if (unlikely(crng_init == 0)) {
- crng_fast_load(buffer, count);
- return;
+ size_t ret = crng_fast_load(buffer, count);
+ mix_pool_bytes(poolp, buffer, ret);
+ count -= ret;
+ buffer += ret;
+ if (!count || crng_init == 0)
+ return;
}
/* Suspend writing if we're above the trickle threshold.
From: Rasmus Villemoes <[email protected]>
commit 3bd0b5bf7dc3ea02070fcbcd682ecf628269e8ef upstream.
This field is never used, might as well remove it.
Signed-off-by: Rasmus Villemoes <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -296,7 +296,7 @@
* To allow fractional bits to be tracked, the entropy_count field is
* denominated in units of 1/8th bits.
*
- * 2*(ENTROPY_SHIFT + log2(poolbits)) must <= 31, or the multiply in
+ * 2*(ENTROPY_SHIFT + poolbitshift) must <= 31, or the multiply in
* credit_entropy_bits() needs to be 64 bits wide.
*/
#define ENTROPY_SHIFT 3
@@ -361,8 +361,8 @@ static int random_write_wakeup_bits = 28
* irreducible, which we have made here.
*/
static const struct poolinfo {
- int poolbitshift, poolwords, poolbytes, poolbits, poolfracbits;
-#define S(x) ilog2(x)+5, (x), (x)*4, (x)*32, (x) << (ENTROPY_SHIFT+5)
+ int poolbitshift, poolwords, poolbytes, poolfracbits;
+#define S(x) ilog2(x)+5, (x), (x)*4, (x) << (ENTROPY_SHIFT+5)
int tap1, tap2, tap3, tap4, tap5;
} poolinfo_table[] = {
/* was: x^128 + x^103 + x^76 + x^51 +x^25 + x + 1 */
From: "Jason A. Donenfeld" <[email protected]>
commit 69efea712f5b0489e67d07565aad5c94e09a3e52 upstream.
It turns out that RDRAND is pretty slow. Comparing these two
constructions:
for (i = 0; i < CHACHA_BLOCK_SIZE; i += sizeof(ret))
arch_get_random_long(&ret);
and
long buf[CHACHA_BLOCK_SIZE / sizeof(long)];
extract_crng((u8 *)buf);
it amortizes out to 352 cycles per long for the top one and 107 cycles
per long for the bottom one, on Coffee Lake Refresh, Intel Core i9-9880H.
And importantly, the top one has the drawback of not benefiting from the
real rng, whereas the bottom one has all the nice benefits of using our
own chacha rng. As get_random_u{32,64} gets used in more places (perhaps
beyond what it was originally intended for when it was introduced as
get_random_{int,long} back in the md5 monstrosity era), it seems like it
might be a good thing to strengthen its posture a tiny bit. Doing this
should only be stronger and not any weaker because that pool is already
initialized with a bunch of rdrand data (when available). This way, we
get the benefits of the hardware rng as well as our own rng.
Another benefit of this is that we no longer hit pitfalls of the recent
stream of AMD bugs in RDRAND. One often used code pattern for various
things is:
do {
val = get_random_u32();
} while (hash_table_contains_key(val));
That recent AMD bug rendered that pattern useless, whereas we're really
very certain that chacha20 output will give pretty distributed numbers,
no matter what.
So, this simplification seems better both from a security perspective
and from a performance perspective.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Reviewed-by: Greg Kroah-Hartman <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 20 ++++----------------
1 file changed, 4 insertions(+), 16 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2233,11 +2233,11 @@ struct batched_entropy {
/*
* Get a random word for internal kernel use only. The quality of the random
- * number is either as good as RDRAND or as good as /dev/urandom, with the
- * goal of being quite fast and not depleting entropy. In order to ensure
+ * number is good as /dev/urandom, but there is no backtrack protection, with
+ * the goal of being quite fast and not depleting entropy. In order to ensure
* that the randomness provided by this function is okay, the function
- * wait_for_random_bytes() should be called and return 0 at least once
- * at any point prior.
+ * wait_for_random_bytes() should be called and return 0 at least once at any
+ * point prior.
*/
static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
.batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
@@ -2250,15 +2250,6 @@ u64 get_random_u64(void)
struct batched_entropy *batch;
static void *previous;
-#if BITS_PER_LONG == 64
- if (arch_get_random_long((unsigned long *)&ret))
- return ret;
-#else
- if (arch_get_random_long((unsigned long *)&ret) &&
- arch_get_random_long((unsigned long *)&ret + 1))
- return ret;
-#endif
-
warn_unseeded_randomness(&previous);
batch = raw_cpu_ptr(&batched_entropy_u64);
@@ -2283,9 +2274,6 @@ u32 get_random_u32(void)
struct batched_entropy *batch;
static void *previous;
- if (arch_get_random_int(&ret))
- return ret;
-
warn_unseeded_randomness(&previous);
batch = raw_cpu_ptr(&batched_entropy_u32);
From: "Jason A. Donenfeld" <[email protected]>
commit c570449094844527577c5c914140222cb1893e3f upstream.
30e37ec516ae ("random: account for entropy loss due to overwrites")
assumed that adding new entropy to the LFSR pool probabilistically
cancelled out old entropy there, so entropy was credited asymptotically,
approximating Shannon entropy of independent sources (rather than a
stronger min-entropy notion) using 1/8th fractional bits and replacing
a constant 2-2/√???? term (~0.786938) with 3/4 (0.75) to slightly
underestimate it. This wasn't superb, but it was perhaps better than
nothing, so that's what was done. Which entropy specifically was being
cancelled out and how much precisely each time is hard to tell, though
as I showed with the attack code in my previous commit, a motivated
adversary with sufficient information can actually cancel out
everything.
Since we're no longer using an LFSR for entropy accumulation, this
probabilistic cancellation is no longer relevant. Rather, we're now
using a computational hash function as the accumulator and we've
switched to working in the random oracle model, from which we can now
revisit the question of min-entropy accumulation, which is done in
detail in <https://eprint.iacr.org/2019/198>.
Consider a long input bit string that is built by concatenating various
smaller independent input bit strings. Each one of these inputs has a
designated min-entropy, which is what we're passing to
credit_entropy_bits(h). When we pass the concatenation of these to a
random oracle, it means that an adversary trying to receive back the
same reply as us would need to become certain about each part of the
concatenated bit string we passed in, which means becoming certain about
all of those h values. That means we can estimate the accumulation by
simply adding up the h values in calls to credit_entropy_bits(h);
there's no probabilistic cancellation at play like there was said to be
for the LFSR. Incidentally, this is also what other entropy accumulators
based on computational hash functions do as well.
So this commit replaces credit_entropy_bits(h) with essentially `total =
min(POOL_BITS, total + h)`, done with a cmpxchg loop as before.
What if we're wrong and the above is nonsense? It's not, but let's
assume we don't want the actual _behavior_ of the code to change much.
Currently that behavior is not extracting from the input pool until it
has 128 bits of entropy in it. With the old algorithm, we'd hit that
magic 128 number after roughly 256 calls to credit_entropy_bits(1). So,
we can retain more or less the old behavior by waiting to extract from
the input pool until it hits 256 bits of entropy using the new code. For
people concerned about this change, it means that there's not that much
practical behavioral change. And for folks actually trying to model
the behavior rigorously, it means that we have an even higher margin
against attacks.
Cc: Theodore Ts'o <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Jean-Philippe Aumasson <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 114 ++++++++------------------------------------------
1 file changed, 20 insertions(+), 94 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -287,17 +287,9 @@
/* #define ADD_INTERRUPT_BENCH */
-enum poolinfo {
+enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
- POOL_BITSHIFT = ilog2(POOL_BITS),
- POOL_MIN_BITS = POOL_BITS / 2,
-
- /* To allow fractional bits to be tracked, the entropy_count field is
- * denominated in units of 1/8th bits. */
- POOL_ENTROPY_SHIFT = 3,
-#define POOL_ENTROPY_BITS() (input_pool.entropy_count >> POOL_ENTROPY_SHIFT)
- POOL_FRACBITS = POOL_BITS << POOL_ENTROPY_SHIFT,
- POOL_MIN_FRACBITS = POOL_MIN_BITS << POOL_ENTROPY_SHIFT
+ POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
};
/*
@@ -310,7 +302,7 @@ static struct fasync_struct *fasync;
* should wake up processes which are selecting or polling on write
* access to /dev/random.
*/
-static int random_write_wakeup_bits = POOL_BITS * 3 / 4;
+static int random_write_wakeup_bits = POOL_MIN_BITS;
static DEFINE_SPINLOCK(random_ready_list_lock);
static LIST_HEAD(random_ready_list);
@@ -470,66 +462,18 @@ static void process_random_ready_list(vo
static void credit_entropy_bits(int nbits)
{
int entropy_count, orig;
- int nfrac = nbits << POOL_ENTROPY_SHIFT;
-
- /* Ensure that the multiplication can avoid being 64 bits wide. */
- BUILD_BUG_ON(2 * (POOL_ENTROPY_SHIFT + POOL_BITSHIFT) > 31);
if (!nbits)
return;
-retry:
- entropy_count = orig = READ_ONCE(input_pool.entropy_count);
- if (nfrac < 0) {
- /* Debit */
- entropy_count += nfrac;
- } else {
- /*
- * Credit: we have to account for the possibility of
- * overwriting already present entropy. Even in the
- * ideal case of pure Shannon entropy, new contributions
- * approach the full value asymptotically:
- *
- * entropy <- entropy + (pool_size - entropy) *
- * (1 - exp(-add_entropy/pool_size))
- *
- * For add_entropy <= pool_size/2 then
- * (1 - exp(-add_entropy/pool_size)) >=
- * (add_entropy/pool_size)*0.7869...
- * so we can approximate the exponential with
- * 3/4*add_entropy/pool_size and still be on the
- * safe side by adding at most pool_size/2 at a time.
- *
- * The use of pool_size-2 in the while statement is to
- * prevent rounding artifacts from making the loop
- * arbitrarily long; this limits the loop to log2(pool_size)*2
- * turns no matter how large nbits is.
- */
- int pnfrac = nfrac;
- const int s = POOL_BITSHIFT + POOL_ENTROPY_SHIFT + 2;
- /* The +2 corresponds to the /4 in the denominator */
-
- do {
- unsigned int anfrac = min(pnfrac, POOL_FRACBITS / 2);
- unsigned int add =
- ((POOL_FRACBITS - entropy_count) * anfrac * 3) >> s;
-
- entropy_count += add;
- pnfrac -= anfrac;
- } while (unlikely(entropy_count < POOL_FRACBITS - 2 && pnfrac));
- }
-
- if (WARN_ON(entropy_count < 0)) {
- pr_warn("negative entropy/overflow: count %d\n", entropy_count);
- entropy_count = 0;
- } else if (entropy_count > POOL_FRACBITS)
- entropy_count = POOL_FRACBITS;
- if (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig)
- goto retry;
+ do {
+ orig = READ_ONCE(input_pool.entropy_count);
+ entropy_count = min(POOL_BITS, orig + nbits);
+ } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
- trace_credit_entropy_bits(nbits, entropy_count >> POOL_ENTROPY_SHIFT, _RET_IP_);
+ trace_credit_entropy_bits(nbits, entropy_count, _RET_IP_);
- if (crng_init < 2 && entropy_count >= POOL_MIN_FRACBITS)
+ if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
crng_reseed(&primary_crng, true);
}
@@ -842,7 +786,7 @@ static void crng_reseed(struct crng_stat
int entropy_count;
do {
entropy_count = READ_ONCE(input_pool.entropy_count);
- if (entropy_count < POOL_MIN_FRACBITS)
+ if (entropy_count < POOL_MIN_BITS)
return;
} while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
extract_entropy(buf.key, sizeof(buf.key));
@@ -1065,7 +1009,7 @@ void add_input_randomness(unsigned int t
last_value = value;
add_timer_randomness(&input_timer_state,
(type << 4) ^ code ^ (code >> 4) ^ value);
- trace_add_input_randomness(POOL_ENTROPY_BITS());
+ trace_add_input_randomness(input_pool.entropy_count);
}
EXPORT_SYMBOL_GPL(add_input_randomness);
@@ -1163,7 +1107,7 @@ void add_disk_randomness(struct gendisk
return;
/* first major is 1, so we get >= 0x200 here */
add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
- trace_add_disk_randomness(disk_devt(disk), POOL_ENTROPY_BITS());
+ trace_add_disk_randomness(disk_devt(disk), input_pool.entropy_count);
}
EXPORT_SYMBOL_GPL(add_disk_randomness);
#endif
@@ -1188,7 +1132,7 @@ static void extract_entropy(void *buf, s
} block;
size_t i;
- trace_extract_entropy(nbytes, POOL_ENTROPY_BITS());
+ trace_extract_entropy(nbytes, input_pool.entropy_count);
for (i = 0; i < ARRAY_SIZE(block.rdrand); ++i) {
if (!arch_get_random_long(&block.rdrand[i]))
@@ -1537,9 +1481,9 @@ static ssize_t urandom_read_nowarn(struc
{
int ret;
- nbytes = min_t(size_t, nbytes, INT_MAX >> (POOL_ENTROPY_SHIFT + 3));
+ nbytes = min_t(size_t, nbytes, INT_MAX >> 6);
ret = extract_crng_user(buf, nbytes);
- trace_urandom_read(8 * nbytes, 0, POOL_ENTROPY_BITS());
+ trace_urandom_read(8 * nbytes, 0, input_pool.entropy_count);
return ret;
}
@@ -1578,7 +1522,7 @@ static unsigned int random_poll(struct f
mask = 0;
if (crng_ready())
mask |= POLLIN | POLLRDNORM;
- if (POOL_ENTROPY_BITS() < random_write_wakeup_bits)
+ if (input_pool.entropy_count < random_write_wakeup_bits)
mask |= POLLOUT | POLLWRNORM;
return mask;
}
@@ -1633,8 +1577,7 @@ static long random_ioctl(struct file *f,
switch (cmd) {
case RNDGETENTCNT:
/* inherently racy, no point locking */
- ent_count = POOL_ENTROPY_BITS();
- if (put_user(ent_count, p))
+ if (put_user(input_pool.entropy_count, p))
return -EFAULT;
return 0;
case RNDADDTOENTCNT:
@@ -1780,23 +1723,6 @@ static int proc_do_uuid(struct ctl_table
return proc_dostring(&fake_table, write, buffer, lenp, ppos);
}
-/*
- * Return entropy available scaled to integral bits
- */
-static int proc_do_entropy(struct ctl_table *table, int write,
- void __user *buffer, size_t *lenp, loff_t *ppos)
-{
- struct ctl_table fake_table;
- int entropy_count;
-
- entropy_count = *(int *)table->data >> POOL_ENTROPY_SHIFT;
-
- fake_table.data = &entropy_count;
- fake_table.maxlen = sizeof(entropy_count);
-
- return proc_dointvec(&fake_table, write, buffer, lenp, ppos);
-}
-
static int sysctl_poolsize = POOL_BITS;
extern struct ctl_table random_table[];
struct ctl_table random_table[] = {
@@ -1809,10 +1735,10 @@ struct ctl_table random_table[] = {
},
{
.procname = "entropy_avail",
+ .data = &input_pool.entropy_count,
.maxlen = sizeof(int),
.mode = 0444,
- .proc_handler = proc_do_entropy,
- .data = &input_pool.entropy_count,
+ .proc_handler = proc_dointvec,
},
{
.procname = "write_wakeup_threshold",
@@ -2008,7 +1934,7 @@ void add_hwgenerator_randomness(const ch
*/
wait_event_interruptible_timeout(random_write_wait,
!system_wq || kthread_should_stop() ||
- POOL_ENTROPY_BITS() <= random_write_wakeup_bits,
+ input_pool.entropy_count <= random_write_wakeup_bits,
CRNG_RESEED_INTERVAL);
mix_pool_bytes(buffer, count);
credit_entropy_bits(entropy);
From: "Jason A. Donenfeld" <[email protected]>
commit 489c7fc44b5740d377e8cfdbf0851036e493af00 upstream.
Now that POOL_BITS == POOL_MIN_BITS, we must unconditionally wake up
entropy writers after every extraction. Therefore there's no point of
write_wakeup_threshold, so we can move it to the dustbin of unused
compatibility sysctls. While we're at it, we can fix a small comparison
where we were waking up after <= min rather than < min.
Cc: Theodore Ts'o <[email protected]>
Suggested-by: Eric Biggers <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/sysctl/kernel.txt | 44 ++++++++++++++++++++++++++++++++++++++--
drivers/char/random.c | 36 ++++++++++++--------------------
2 files changed, 56 insertions(+), 24 deletions(-)
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -777,9 +777,49 @@ The kernel command line parameter printk
a one-time setting until next reboot: once set, it cannot be changed by
this sysctl interface anymore.
-==============================================================
+pty
+===
-randomize_va_space:
+See Documentation/filesystems/devpts.rst.
+
+
+random
+======
+
+This is a directory, with the following entries:
+
+* ``boot_id``: a UUID generated the first time this is retrieved, and
+ unvarying after that;
+
+* ``entropy_avail``: the pool's entropy count, in bits;
+
+* ``poolsize``: the entropy pool size, in bits;
+
+* ``urandom_min_reseed_secs``: obsolete (used to determine the minimum
+ number of seconds between urandom pool reseeding). This file is
+ writable for compatibility purposes, but writing to it has no effect
+ on any RNG behavior.
+
+* ``uuid``: a UUID generated every time this is retrieved (this can
+ thus be used to generate UUIDs at will);
+
+* ``write_wakeup_threshold``: when the entropy count drops below this
+ (as a number of bits), processes waiting to write to ``/dev/random``
+ are woken up. This file is writable for compatibility purposes, but
+ writing to it has no effect on any RNG behavior.
+
+If ``drivers/char/random.c`` is built with ``ADD_INTERRUPT_BENCH``
+defined, these additional entries are present:
+
+* ``add_interrupt_avg_cycles``: the average number of cycles between
+ interrupts used to feed the pool;
+
+* ``add_interrupt_avg_deviation``: the standard deviation seen on the
+ number of cycles between interrupts used to feed the pool.
+
+
+randomize_va_space
+==================
This option can be used to select the type of process address
space randomization that is used in the system, for architectures
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -297,12 +297,6 @@ enum {
*/
static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
static struct fasync_struct *fasync;
-/*
- * If the entropy count falls under this number of bits, then we
- * should wake up processes which are selecting or polling on write
- * access to /dev/random.
- */
-static int random_write_wakeup_bits = POOL_MIN_BITS;
static DEFINE_SPINLOCK(random_ready_list_lock);
static LIST_HEAD(random_ready_list);
@@ -790,10 +784,8 @@ static void crng_reseed(struct crng_stat
return;
} while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
extract_entropy(buf.key, sizeof(buf.key));
- if (random_write_wakeup_bits) {
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
- }
+ wake_up_interruptible(&random_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
} else {
_extract_crng(&primary_crng, buf.block);
_crng_backtrack_protect(&primary_crng, buf.block,
@@ -1522,7 +1514,7 @@ static unsigned int random_poll(struct f
mask = 0;
if (crng_ready())
mask |= POLLIN | POLLRDNORM;
- if (input_pool.entropy_count < random_write_wakeup_bits)
+ if (input_pool.entropy_count < POOL_MIN_BITS)
mask |= POLLOUT | POLLWRNORM;
return mask;
}
@@ -1607,7 +1599,10 @@ static long random_ioctl(struct file *f,
*/
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- input_pool.entropy_count = 0;
+ if (xchg(&input_pool.entropy_count, 0)) {
+ wake_up_interruptible(&random_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
+ }
return 0;
case RNDRESEEDCRNG:
if (!capable(CAP_SYS_ADMIN))
@@ -1682,9 +1677,9 @@ SYSCALL_DEFINE3(getrandom, char __user *
#include <linux/sysctl.h>
-static int min_write_thresh;
-static int max_write_thresh = POOL_BITS;
static int random_min_urandom_seed = 60;
+static int random_write_wakeup_bits = POOL_MIN_BITS;
+static int sysctl_poolsize = POOL_BITS;
static char sysctl_bootid[16];
/*
@@ -1723,7 +1718,6 @@ static int proc_do_uuid(struct ctl_table
return proc_dostring(&fake_table, write, buffer, lenp, ppos);
}
-static int sysctl_poolsize = POOL_BITS;
extern struct ctl_table random_table[];
struct ctl_table random_table[] = {
{
@@ -1745,9 +1739,7 @@ struct ctl_table random_table[] = {
.data = &random_write_wakeup_bits,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &min_write_thresh,
- .extra2 = &max_write_thresh,
+ .proc_handler = proc_dointvec,
},
{
.procname = "urandom_min_reseed_secs",
@@ -1928,13 +1920,13 @@ void add_hwgenerator_randomness(const ch
}
/* Throttle writing if we're above the trickle threshold.
- * We'll be woken up again once below random_write_wakeup_thresh,
- * when the calling thread is about to terminate, or once
- * CRNG_RESEED_INTERVAL has lapsed.
+ * We'll be woken up again once below POOL_MIN_BITS, when
+ * the calling thread is about to terminate, or once
+ * CRNG_RESEED_INTERVAL has elapsed.
*/
wait_event_interruptible_timeout(random_write_wait,
!system_wq || kthread_should_stop() ||
- input_pool.entropy_count <= random_write_wakeup_bits,
+ input_pool.entropy_count < POOL_MIN_BITS,
CRNG_RESEED_INTERVAL);
mix_pool_bytes(buffer, count);
credit_entropy_bits(entropy);
From: "Jason A. Donenfeld" <[email protected]>
commit 7b87324112df2e1f9b395217361626362dcfb9fb upstream.
Rather than an awkward combination of ifdefs and __maybe_unused, we can
ensure more source gets parsed, regardless of the configuration, by
using IS_ENABLED for the CONFIG_NUMA conditional code. This makes things
cleaner and easier to follow.
I've confirmed that on !CONFIG_NUMA, we don't wind up with excess code
by accident; the generated object file is the same.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 32 ++++++++++++--------------------
1 file changed, 12 insertions(+), 20 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -762,7 +762,6 @@ static int credit_entropy_bits_safe(stru
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
-#ifdef CONFIG_NUMA
/*
* Hack to deal with crazy userspace progams when they are all trying
* to access /dev/urandom in parallel. The programs are almost
@@ -770,7 +769,6 @@ static DECLARE_WAIT_QUEUE_HEAD(crng_init
* their brain damage.
*/
static struct crng_state **crng_node_pool __read_mostly;
-#endif
static void invalidate_batched_entropy(void);
@@ -817,7 +815,7 @@ static bool __init crng_init_try_arch_ea
return arch_init;
}
-static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
+static void crng_initialize_secondary(struct crng_state *crng)
{
memcpy(&crng->state[0], "expand 32-byte k", 16);
_get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
@@ -868,7 +866,6 @@ static void crng_finalize_init(struct cr
}
}
-#ifdef CONFIG_NUMA
static void do_numa_crng_init(struct work_struct *work)
{
int i;
@@ -895,29 +892,24 @@ static DECLARE_WORK(numa_crng_init_work,
static void numa_crng_init(void)
{
- schedule_work(&numa_crng_init_work);
+ if (IS_ENABLED(CONFIG_NUMA))
+ schedule_work(&numa_crng_init_work);
}
static struct crng_state *select_crng(void)
{
- struct crng_state **pool;
- int nid = numa_node_id();
-
- /* pairs with cmpxchg_release() in do_numa_crng_init() */
- pool = READ_ONCE(crng_node_pool);
- if (pool && pool[nid])
- return pool[nid];
-
- return &primary_crng;
-}
-#else
-static void numa_crng_init(void) {}
+ if (IS_ENABLED(CONFIG_NUMA)) {
+ struct crng_state **pool;
+ int nid = numa_node_id();
+
+ /* pairs with cmpxchg_release() in do_numa_crng_init() */
+ pool = READ_ONCE(crng_node_pool);
+ if (pool && pool[nid])
+ return pool[nid];
+ }
-static struct crng_state *select_crng(void)
-{
return &primary_crng;
}
-#endif
/*
* crng_fast_load() can be called by code in the interrupt service
From: Dominik Brodowski <[email protected]>
commit ebf7606388732ecf2821ca21087e9446cb4a5b57 upstream.
Both crng_initialize_primary() and crng_init_try_arch_early() are
only called for the primary_pool. Accessing it directly instead of
through a function parameter simplifies the code.
Signed-off-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -762,7 +762,7 @@ static bool crng_init_try_arch(struct cr
return arch_init;
}
-static bool __init crng_init_try_arch_early(struct crng_state *crng)
+static bool __init crng_init_try_arch_early(void)
{
int i;
bool arch_init = true;
@@ -774,7 +774,7 @@ static bool __init crng_init_try_arch_ea
rv = random_get_entropy();
arch_init = false;
}
- crng->state[i] ^= rv;
+ primary_crng.state[i] ^= rv;
}
return arch_init;
@@ -788,16 +788,16 @@ static void crng_initialize_secondary(st
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
-static void __init crng_initialize_primary(struct crng_state *crng)
+static void __init crng_initialize_primary(void)
{
- _extract_entropy(&crng->state[4], sizeof(u32) * 12);
- if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
+ _extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
+ if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
crng_init = 2;
pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
- crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
+ primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
static void crng_finalize_init(struct crng_state *crng)
@@ -1749,7 +1749,7 @@ int __init rand_initialize(void)
init_std_data();
if (crng_need_final_init)
crng_finalize_init(&primary_crng);
- crng_initialize_primary(&primary_crng);
+ crng_initialize_primary();
crng_global_init_time = jiffies;
if (ratelimit_disable) {
urandom_warning.interval = 0;
From: "Jason A. Donenfeld" <[email protected]>
commit 92c653cf14400946f376a29b828d6af7e01f38dd upstream.
This pulls all of the entropy collection-focused functions into the
fourth labeled section.
No functional changes.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 370 +++++++++++++++++++++++++++-----------------------
1 file changed, 206 insertions(+), 164 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1035,60 +1035,112 @@ static bool drain_entropy(void *buf, siz
return true;
}
-struct fast_pool {
- union {
- u32 pool32[4];
- u64 pool64[2];
- };
- unsigned long last;
- u16 reg_idx;
- u8 count;
-};
+
+/**********************************************************************
+ *
+ * Entropy collection routines.
+ *
+ * The following exported functions are used for pushing entropy into
+ * the above entropy accumulation routines:
+ *
+ * void add_device_randomness(const void *buf, size_t size);
+ * void add_input_randomness(unsigned int type, unsigned int code,
+ * unsigned int value);
+ * void add_disk_randomness(struct gendisk *disk);
+ * void add_hwgenerator_randomness(const void *buffer, size_t count,
+ * size_t entropy);
+ * void add_bootloader_randomness(const void *buf, size_t size);
+ * void add_interrupt_randomness(int irq);
+ *
+ * add_device_randomness() adds data to the input pool that
+ * is likely to differ between two devices (or possibly even per boot).
+ * This would be things like MAC addresses or serial numbers, or the
+ * read-out of the RTC. This does *not* credit any actual entropy to
+ * the pool, but it initializes the pool to different values for devices
+ * that might otherwise be identical and have very little entropy
+ * available to them (particularly common in the embedded world).
+ *
+ * add_input_randomness() uses the input layer interrupt timing, as well
+ * as the event type information from the hardware.
+ *
+ * add_disk_randomness() uses what amounts to the seek time of block
+ * layer request events, on a per-disk_devt basis, as input to the
+ * entropy pool. Note that high-speed solid state drives with very low
+ * seek times do not make for good sources of entropy, as their seek
+ * times are usually fairly consistent.
+ *
+ * The above two routines try to estimate how many bits of entropy
+ * to credit. They do this by keeping track of the first and second
+ * order deltas of the event timings.
+ *
+ * add_hwgenerator_randomness() is for true hardware RNGs, and will credit
+ * entropy as specified by the caller. If the entropy pool is full it will
+ * block until more entropy is needed.
+ *
+ * add_bootloader_randomness() is the same as add_hwgenerator_randomness() or
+ * add_device_randomness(), depending on whether or not the configuration
+ * option CONFIG_RANDOM_TRUST_BOOTLOADER is set.
+ *
+ * add_interrupt_randomness() uses the interrupt timing as random
+ * inputs to the entropy pool. Using the cycle counters and the irq source
+ * as inputs, it feeds the input pool roughly once a second or after 64
+ * interrupts, crediting 1 bit of entropy for whichever comes first.
+ *
+ **********************************************************************/
+
+static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
+static int __init parse_trust_cpu(char *arg)
+{
+ return kstrtobool(arg, &trust_cpu);
+}
+early_param("random.trust_cpu", parse_trust_cpu);
/*
- * This is a fast mixing routine used by the interrupt randomness
- * collector. It's hardcoded for an 128 bit pool and assumes that any
- * locks that might be needed are taken by the caller.
+ * The first collection of entropy occurs at system boot while interrupts
+ * are still turned off. Here we push in RDSEED, a timestamp, and utsname().
+ * Depending on the above configuration knob, RDSEED may be considered
+ * sufficient for initialization. Note that much earlier setup may already
+ * have pushed entropy into the input pool by the time we get here.
*/
-static void fast_mix(u32 pool[4])
+int __init rand_initialize(void)
{
- u32 a = pool[0], b = pool[1];
- u32 c = pool[2], d = pool[3];
-
- a += b; c += d;
- b = rol32(b, 6); d = rol32(d, 27);
- d ^= a; b ^= c;
+ size_t i;
+ ktime_t now = ktime_get_real();
+ bool arch_init = true;
+ unsigned long rv;
- a += b; c += d;
- b = rol32(b, 16); d = rol32(d, 14);
- d ^= a; b ^= c;
+ for (i = 0; i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
+ if (!arch_get_random_seed_long_early(&rv) &&
+ !arch_get_random_long_early(&rv)) {
+ rv = random_get_entropy();
+ arch_init = false;
+ }
+ mix_pool_bytes(&rv, sizeof(rv));
+ }
+ mix_pool_bytes(&now, sizeof(now));
+ mix_pool_bytes(utsname(), sizeof(*(utsname())));
- a += b; c += d;
- b = rol32(b, 6); d = rol32(d, 27);
- d ^= a; b ^= c;
+ extract_entropy(base_crng.key, sizeof(base_crng.key));
+ ++base_crng.generation;
- a += b; c += d;
- b = rol32(b, 16); d = rol32(d, 14);
- d ^= a; b ^= c;
+ if (arch_init && trust_cpu && crng_init < 2) {
+ crng_init = 2;
+ pr_notice("crng init done (trusting CPU's manufacturer)\n");
+ }
- pool[0] = a; pool[1] = b;
- pool[2] = c; pool[3] = d;
+ if (ratelimit_disable) {
+ urandom_warning.interval = 0;
+ unseeded_warning.interval = 0;
+ }
+ return 0;
}
-/*********************************************************************
- *
- * Entropy input management
- *
- *********************************************************************/
-
/* There is one of these per entropy source */
struct timer_rand_state {
cycles_t last_time;
long last_delta, last_delta2;
};
-#define INIT_TIMER_RAND_STATE { INITIAL_JIFFIES, };
-
/*
* Add device- or boot-specific data to the input pool to help
* initialize it.
@@ -1112,8 +1164,6 @@ void add_device_randomness(const void *b
}
EXPORT_SYMBOL(add_device_randomness);
-static struct timer_rand_state input_timer_state = INIT_TIMER_RAND_STATE;
-
/*
* This function adds entropy to the entropy "pool" by using timing
* delays. It uses the timer_rand_state structure to make an estimate
@@ -1175,8 +1225,9 @@ void add_input_randomness(unsigned int t
unsigned int value)
{
static unsigned char last_value;
+ static struct timer_rand_state input_timer_state = { INITIAL_JIFFIES };
- /* ignore autorepeat and the like */
+ /* Ignore autorepeat and the like. */
if (value == last_value)
return;
@@ -1186,6 +1237,119 @@ void add_input_randomness(unsigned int t
}
EXPORT_SYMBOL_GPL(add_input_randomness);
+#ifdef CONFIG_BLOCK
+void add_disk_randomness(struct gendisk *disk)
+{
+ if (!disk || !disk->random)
+ return;
+ /* First major is 1, so we get >= 0x200 here. */
+ add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
+}
+EXPORT_SYMBOL_GPL(add_disk_randomness);
+
+void rand_initialize_disk(struct gendisk *disk)
+{
+ struct timer_rand_state *state;
+
+ /*
+ * If kzalloc returns null, we just won't use that entropy
+ * source.
+ */
+ state = kzalloc(sizeof(struct timer_rand_state), GFP_KERNEL);
+ if (state) {
+ state->last_time = INITIAL_JIFFIES;
+ disk->random = state;
+ }
+}
+#endif
+
+/*
+ * Interface for in-kernel drivers of true hardware RNGs.
+ * Those devices may produce endless random bits and will be throttled
+ * when our pool is full.
+ */
+void add_hwgenerator_randomness(const void *buffer, size_t count,
+ size_t entropy)
+{
+ if (unlikely(crng_init == 0)) {
+ size_t ret = crng_fast_load(buffer, count);
+ mix_pool_bytes(buffer, ret);
+ count -= ret;
+ buffer += ret;
+ if (!count || crng_init == 0)
+ return;
+ }
+
+ /*
+ * Throttle writing if we're above the trickle threshold.
+ * We'll be woken up again once below POOL_MIN_BITS, when
+ * the calling thread is about to terminate, or once
+ * CRNG_RESEED_INTERVAL has elapsed.
+ */
+ wait_event_interruptible_timeout(random_write_wait,
+ !system_wq || kthread_should_stop() ||
+ input_pool.entropy_count < POOL_MIN_BITS,
+ CRNG_RESEED_INTERVAL);
+ mix_pool_bytes(buffer, count);
+ credit_entropy_bits(entropy);
+}
+EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
+
+/*
+ * Handle random seed passed by bootloader.
+ * If the seed is trustworthy, it would be regarded as hardware RNGs. Otherwise
+ * it would be regarded as device data.
+ * The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
+ */
+void add_bootloader_randomness(const void *buf, size_t size)
+{
+ if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
+ add_hwgenerator_randomness(buf, size, size * 8);
+ else
+ add_device_randomness(buf, size);
+}
+EXPORT_SYMBOL_GPL(add_bootloader_randomness);
+
+struct fast_pool {
+ union {
+ u32 pool32[4];
+ u64 pool64[2];
+ };
+ unsigned long last;
+ u16 reg_idx;
+ u8 count;
+};
+
+/*
+ * This is a fast mixing routine used by the interrupt randomness
+ * collector. It's hardcoded for an 128 bit pool and assumes that any
+ * locks that might be needed are taken by the caller.
+ */
+static void fast_mix(u32 pool[4])
+{
+ u32 a = pool[0], b = pool[1];
+ u32 c = pool[2], d = pool[3];
+
+ a += b; c += d;
+ b = rol32(b, 6); d = rol32(d, 27);
+ d ^= a; b ^= c;
+
+ a += b; c += d;
+ b = rol32(b, 16); d = rol32(d, 14);
+ d ^= a; b ^= c;
+
+ a += b; c += d;
+ b = rol32(b, 6); d = rol32(d, 27);
+ d ^= a; b ^= c;
+
+ a += b; c += d;
+ b = rol32(b, 16); d = rol32(d, 14);
+ d ^= a; b ^= c;
+
+ pool[0] = a; pool[1] = b;
+ pool[2] = c; pool[3] = d;
+}
+
static DEFINE_PER_CPU(struct fast_pool, irq_randomness);
static u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
@@ -1255,22 +1419,11 @@ void add_interrupt_randomness(int irq)
fast_pool->count = 0;
- /* award one bit for the contents of the fast pool */
+ /* Award one bit for the contents of the fast pool. */
credit_entropy_bits(1);
}
EXPORT_SYMBOL_GPL(add_interrupt_randomness);
-#ifdef CONFIG_BLOCK
-void add_disk_randomness(struct gendisk *disk)
-{
- if (!disk || !disk->random)
- return;
- /* first major is 1, so we get >= 0x200 here */
- add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
-}
-EXPORT_SYMBOL_GPL(add_disk_randomness);
-#endif
-
/*
* Each time the timer fires, we expect that we got an unpredictable
* jump in the cycle counter. Even if the timer is running on another
@@ -1320,73 +1473,6 @@ static void try_to_generate_entropy(void
mix_pool_bytes(&stack.now, sizeof(stack.now));
}
-static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
-static int __init parse_trust_cpu(char *arg)
-{
- return kstrtobool(arg, &trust_cpu);
-}
-early_param("random.trust_cpu", parse_trust_cpu);
-
-/*
- * Note that setup_arch() may call add_device_randomness()
- * long before we get here. This allows seeding of the pools
- * with some platform dependent data very early in the boot
- * process. But it limits our options here. We must use
- * statically allocated structures that already have all
- * initializations complete at compile time. We should also
- * take care not to overwrite the precious per platform data
- * we were given.
- */
-int __init rand_initialize(void)
-{
- size_t i;
- ktime_t now = ktime_get_real();
- bool arch_init = true;
- unsigned long rv;
-
- for (i = 0; i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
- if (!arch_get_random_seed_long_early(&rv) &&
- !arch_get_random_long_early(&rv)) {
- rv = random_get_entropy();
- arch_init = false;
- }
- mix_pool_bytes(&rv, sizeof(rv));
- }
- mix_pool_bytes(&now, sizeof(now));
- mix_pool_bytes(utsname(), sizeof(*(utsname())));
-
- extract_entropy(base_crng.key, sizeof(base_crng.key));
- ++base_crng.generation;
-
- if (arch_init && trust_cpu && crng_init < 2) {
- crng_init = 2;
- pr_notice("crng init done (trusting CPU's manufacturer)\n");
- }
-
- if (ratelimit_disable) {
- urandom_warning.interval = 0;
- unseeded_warning.interval = 0;
- }
- return 0;
-}
-
-#ifdef CONFIG_BLOCK
-void rand_initialize_disk(struct gendisk *disk)
-{
- struct timer_rand_state *state;
-
- /*
- * If kzalloc returns null, we just won't use that entropy
- * source.
- */
- state = kzalloc(sizeof(struct timer_rand_state), GFP_KERNEL);
- if (state) {
- state->last_time = INITIAL_JIFFIES;
- disk->random = state;
- }
-}
-#endif
-
static ssize_t urandom_read(struct file *file, char __user *buf, size_t nbytes,
loff_t *ppos)
{
@@ -1669,47 +1755,3 @@ struct ctl_table random_table[] = {
{ }
};
#endif /* CONFIG_SYSCTL */
-
-/* Interface for in-kernel drivers of true hardware RNGs.
- * Those devices may produce endless random bits and will be throttled
- * when our pool is full.
- */
-void add_hwgenerator_randomness(const void *buffer, size_t count,
- size_t entropy)
-{
- if (unlikely(crng_init == 0)) {
- size_t ret = crng_fast_load(buffer, count);
- mix_pool_bytes(buffer, ret);
- count -= ret;
- buffer += ret;
- if (!count || crng_init == 0)
- return;
- }
-
- /* Throttle writing if we're above the trickle threshold.
- * We'll be woken up again once below POOL_MIN_BITS, when
- * the calling thread is about to terminate, or once
- * CRNG_RESEED_INTERVAL has elapsed.
- */
- wait_event_interruptible_timeout(random_write_wait,
- !system_wq || kthread_should_stop() ||
- input_pool.entropy_count < POOL_MIN_BITS,
- CRNG_RESEED_INTERVAL);
- mix_pool_bytes(buffer, count);
- credit_entropy_bits(entropy);
-}
-EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
-
-/* Handle random seed passed by bootloader.
- * If the seed is trustworthy, it would be regarded as hardware RNGs. Otherwise
- * it would be regarded as device data.
- * The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
- */
-void add_bootloader_randomness(const void *buf, size_t size)
-{
- if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
- add_hwgenerator_randomness(buf, size, size * 8);
- else
- add_device_randomness(buf, size);
-}
-EXPORT_SYMBOL_GPL(add_bootloader_randomness);
From: "Jason A. Donenfeld" <[email protected]>
commit 91ec0fe138f107232cb36bc6112211db37cb5306 upstream.
Now that we're only using one polynomial, we can cleanup its
representation into constants, instead of passing around pointers
dynamically to select different polynomials. This improves the codegen
and makes the code a bit more straightforward.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 67 ++++++++++++++++++++++----------------------------
1 file changed, 30 insertions(+), 37 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -432,14 +432,20 @@ static int random_write_wakeup_bits = 28
* polynomial which improves the resulting TGFSR polynomial to be
* irreducible, which we have made here.
*/
-static const struct poolinfo {
- int poolbitshift, poolwords, poolbytes, poolfracbits;
-#define S(x) ilog2(x)+5, (x), (x)*4, (x) << (ENTROPY_SHIFT+5)
- int tap1, tap2, tap3, tap4, tap5;
-} poolinfo_table[] = {
- /* was: x^128 + x^103 + x^76 + x^51 +x^25 + x + 1 */
+enum poolinfo {
+ POOL_WORDS = 128,
+ POOL_WORDMASK = POOL_WORDS - 1,
+ POOL_BYTES = POOL_WORDS * sizeof(u32),
+ POOL_BITS = POOL_BYTES * 8,
+ POOL_BITSHIFT = ilog2(POOL_WORDS) + 5,
+ POOL_FRACBITS = POOL_WORDS << (ENTROPY_SHIFT + 5),
+
/* x^128 + x^104 + x^76 + x^51 +x^25 + x + 1 */
- { S(128), 104, 76, 51, 25, 1 },
+ POOL_TAP1 = 104,
+ POOL_TAP2 = 76,
+ POOL_TAP3 = 51,
+ POOL_TAP4 = 25,
+ POOL_TAP5 = 1
};
/*
@@ -506,7 +512,6 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
struct entropy_store;
struct entropy_store {
/* read-only data: */
- const struct poolinfo *poolinfo;
__u32 *pool;
const char *name;
@@ -528,7 +533,6 @@ static void crng_reseed(struct crng_stat
static __u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
static struct entropy_store input_pool = {
- .poolinfo = &poolinfo_table[0],
.name = "input",
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
.pool = input_pool_data
@@ -551,33 +555,26 @@ static __u32 const twist_table[8] = {
static void _mix_pool_bytes(struct entropy_store *r, const void *in,
int nbytes)
{
- unsigned long i, tap1, tap2, tap3, tap4, tap5;
+ unsigned long i;
int input_rotate;
- int wordmask = r->poolinfo->poolwords - 1;
const unsigned char *bytes = in;
__u32 w;
- tap1 = r->poolinfo->tap1;
- tap2 = r->poolinfo->tap2;
- tap3 = r->poolinfo->tap3;
- tap4 = r->poolinfo->tap4;
- tap5 = r->poolinfo->tap5;
-
input_rotate = r->input_rotate;
i = r->add_ptr;
/* mix one byte at a time to simplify size handling and churn faster */
while (nbytes--) {
w = rol32(*bytes++, input_rotate);
- i = (i - 1) & wordmask;
+ i = (i - 1) & POOL_WORDMASK;
/* XOR in the various taps */
w ^= r->pool[i];
- w ^= r->pool[(i + tap1) & wordmask];
- w ^= r->pool[(i + tap2) & wordmask];
- w ^= r->pool[(i + tap3) & wordmask];
- w ^= r->pool[(i + tap4) & wordmask];
- w ^= r->pool[(i + tap5) & wordmask];
+ w ^= r->pool[(i + POOL_TAP1) & POOL_WORDMASK];
+ w ^= r->pool[(i + POOL_TAP2) & POOL_WORDMASK];
+ w ^= r->pool[(i + POOL_TAP3) & POOL_WORDMASK];
+ w ^= r->pool[(i + POOL_TAP4) & POOL_WORDMASK];
+ w ^= r->pool[(i + POOL_TAP5) & POOL_WORDMASK];
/* Mix the result back in with a twist */
r->pool[i] = (w >> 3) ^ twist_table[w & 7];
@@ -675,7 +672,6 @@ static void process_random_ready_list(vo
static void credit_entropy_bits(struct entropy_store *r, int nbits)
{
int entropy_count, orig;
- const int pool_size = r->poolinfo->poolfracbits;
int nfrac = nbits << ENTROPY_SHIFT;
if (!nbits)
@@ -709,25 +705,25 @@ retry:
* turns no matter how large nbits is.
*/
int pnfrac = nfrac;
- const int s = r->poolinfo->poolbitshift + ENTROPY_SHIFT + 2;
+ const int s = POOL_BITSHIFT + ENTROPY_SHIFT + 2;
/* The +2 corresponds to the /4 in the denominator */
do {
- unsigned int anfrac = min(pnfrac, pool_size/2);
+ unsigned int anfrac = min(pnfrac, POOL_FRACBITS/2);
unsigned int add =
- ((pool_size - entropy_count)*anfrac*3) >> s;
+ ((POOL_FRACBITS - entropy_count)*anfrac*3) >> s;
entropy_count += add;
pnfrac -= anfrac;
- } while (unlikely(entropy_count < pool_size-2 && pnfrac));
+ } while (unlikely(entropy_count < POOL_FRACBITS-2 && pnfrac));
}
if (WARN_ON(entropy_count < 0)) {
pr_warn("negative entropy/overflow: pool %s count %d\n",
r->name, entropy_count);
entropy_count = 0;
- } else if (entropy_count > pool_size)
- entropy_count = pool_size;
+ } else if (entropy_count > POOL_FRACBITS)
+ entropy_count = POOL_FRACBITS;
if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
goto retry;
@@ -744,13 +740,11 @@ retry:
static int credit_entropy_bits_safe(struct entropy_store *r, int nbits)
{
- const int nbits_max = r->poolinfo->poolwords * 32;
-
if (nbits < 0)
return -EINVAL;
/* Cap the value to avoid overflows */
- nbits = min(nbits, nbits_max);
+ nbits = min(nbits, POOL_BITS);
credit_entropy_bits(r, nbits);
return 0;
@@ -1394,7 +1388,7 @@ static size_t account(struct entropy_sto
int entropy_count, orig, have_bytes;
size_t ibytes, nfrac;
- BUG_ON(r->entropy_count > r->poolinfo->poolfracbits);
+ BUG_ON(r->entropy_count > POOL_FRACBITS);
/* Can we pull enough? */
retry:
@@ -1460,8 +1454,7 @@ static void extract_buf(struct entropy_s
/* Generate a hash across the pool */
spin_lock_irqsave(&r->lock, flags);
- blake2s_update(&state, (const u8 *)r->pool,
- r->poolinfo->poolwords * sizeof(*r->pool));
+ blake2s_update(&state, (const u8 *)r->pool, POOL_BYTES);
blake2s_final(&state, hash); /* final zeros out state */
/*
@@ -1756,7 +1749,7 @@ static void __init init_std_data(struct
unsigned long rv;
mix_pool_bytes(r, &now, sizeof(now));
- for (i = r->poolinfo->poolbytes; i > 0; i -= sizeof(rv)) {
+ for (i = POOL_BYTES; i > 0; i -= sizeof(rv)) {
if (!arch_get_random_seed_long(&rv) &&
!arch_get_random_long(&rv))
rv = random_get_entropy();
From: "Jason A. Donenfeld" <[email protected]>
commit 95e6060c20a7f5db60163274c5222a725ac118f9 upstream.
With tools like kbench9000 giving more finegrained responses, and this
basically never having been used ever since it was initially added,
let's just get rid of this. There *is* still work to be done on the
interrupt handler, but this really isn't the way it's being developed.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/sysctl/kernel.txt | 9 ---------
drivers/char/random.c | 40 ----------------------------------------
2 files changed, 49 deletions(-)
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -808,15 +808,6 @@ This is a directory, with the following
are woken up. This file is writable for compatibility purposes, but
writing to it has no effect on any RNG behavior.
-If ``drivers/char/random.c`` is built with ``ADD_INTERRUPT_BENCH``
-defined, these additional entries are present:
-
-* ``add_interrupt_avg_cycles``: the average number of cycles between
- interrupts used to feed the pool;
-
-* ``add_interrupt_avg_deviation``: the standard deviation seen on the
- number of cycles between interrupts used to feed the pool.
-
randomize_va_space
==================
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -241,8 +241,6 @@
#define CREATE_TRACE_POINTS
#include <trace/events/random.h>
-/* #define ADD_INTERRUPT_BENCH */
-
enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
@@ -858,27 +856,6 @@ EXPORT_SYMBOL_GPL(add_input_randomness);
static DEFINE_PER_CPU(struct fast_pool, irq_randomness);
-#ifdef ADD_INTERRUPT_BENCH
-static unsigned long avg_cycles, avg_deviation;
-
-#define AVG_SHIFT 8 /* Exponential average factor k=1/256 */
-#define FIXED_1_2 (1 << (AVG_SHIFT - 1))
-
-static void add_interrupt_bench(cycles_t start)
-{
- long delta = random_get_entropy() - start;
-
- /* Use a weighted moving average */
- delta = delta - ((avg_cycles + FIXED_1_2) >> AVG_SHIFT);
- avg_cycles += delta;
- /* And average deviation */
- delta = abs(delta) - ((avg_deviation + FIXED_1_2) >> AVG_SHIFT);
- avg_deviation += delta;
-}
-#else
-#define add_interrupt_bench(x)
-#endif
-
static u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
{
u32 *ptr = (u32 *)regs;
@@ -915,7 +892,6 @@ void add_interrupt_randomness(int irq)
(sizeof(ip) > 4) ? ip >> 32 : get_reg(fast_pool, regs);
fast_mix(fast_pool);
- add_interrupt_bench(cycles);
if (unlikely(crng_init == 0)) {
if (fast_pool->count >= 64 &&
@@ -1623,22 +1599,6 @@ struct ctl_table random_table[] = {
.mode = 0444,
.proc_handler = proc_do_uuid,
},
-#ifdef ADD_INTERRUPT_BENCH
- {
- .procname = "add_interrupt_avg_cycles",
- .data = &avg_cycles,
- .maxlen = sizeof(avg_cycles),
- .mode = 0444,
- .proc_handler = proc_doulongvec_minmax,
- },
- {
- .procname = "add_interrupt_avg_deviation",
- .data = &avg_deviation,
- .maxlen = sizeof(avg_deviation),
- .mode = 0444,
- .proc_handler = proc_doulongvec_minmax,
- },
-#endif
{ }
};
#endif /* CONFIG_SYSCTL */
From: "Jason A. Donenfeld" <[email protected]>
commit 66e4c2b9541503d721e936cc3898c9f25f4591ff upstream.
Since we have a hash function that's really fast, and the goal of
crng_slow_load() is reportedly to "touch all of the crng's state", we
can just hash the old state together with the new state and call it a
day. This way we dont need to reason about another LFSR or worry about
various attacks there. This code is only ever used at early boot and
then never again.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 163 +++++++++++++++++-------------------------
include/linux/hw_random.h | 2
include/linux/random.h | 10 +-
include/trace/events/random.h | 79 +++++++++-----------
4 files changed, 113 insertions(+), 141 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -69,7 +69,7 @@
*
* The primary kernel interfaces are:
*
- * void get_random_bytes(void *buf, int nbytes);
+ * void get_random_bytes(void *buf, size_t nbytes);
* u32 get_random_u32()
* u64 get_random_u64()
* unsigned int get_random_int()
@@ -97,14 +97,14 @@
* The current exported interfaces for gathering environmental noise
* from the devices are:
*
- * void add_device_randomness(const void *buf, unsigned int size);
+ * void add_device_randomness(const void *buf, size_t size);
* void add_input_randomness(unsigned int type, unsigned int code,
* unsigned int value);
* void add_interrupt_randomness(int irq);
* void add_disk_randomness(struct gendisk *disk);
- * void add_hwgenerator_randomness(const char *buffer, size_t count,
+ * void add_hwgenerator_randomness(const void *buffer, size_t count,
* size_t entropy);
- * void add_bootloader_randomness(const void *buf, unsigned int size);
+ * void add_bootloader_randomness(const void *buf, size_t size);
*
* add_device_randomness() is for adding data to the random pool that
* is likely to differ between two devices (or possibly even per boot).
@@ -269,7 +269,7 @@ static int crng_init = 0;
#define crng_ready() (likely(crng_init > 1))
static int crng_init_cnt = 0;
static void process_random_ready_list(void);
-static void _get_random_bytes(void *buf, int nbytes);
+static void _get_random_bytes(void *buf, size_t nbytes);
static struct ratelimit_state unseeded_warning =
RATELIMIT_STATE_INIT("warn_unseeded_randomness", HZ, 3);
@@ -291,7 +291,7 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
static struct {
struct blake2s_state hash;
spinlock_t lock;
- int entropy_count;
+ unsigned int entropy_count;
} input_pool = {
.hash.h = { BLAKE2S_IV0 ^ (0x01010000 | BLAKE2S_HASH_SIZE),
BLAKE2S_IV1, BLAKE2S_IV2, BLAKE2S_IV3, BLAKE2S_IV4,
@@ -309,18 +309,12 @@ static void crng_reseed(void);
* update the entropy estimate. The caller should call
* credit_entropy_bits if this is appropriate.
*/
-static void _mix_pool_bytes(const void *in, int nbytes)
+static void _mix_pool_bytes(const void *in, size_t nbytes)
{
blake2s_update(&input_pool.hash, in, nbytes);
}
-static void __mix_pool_bytes(const void *in, int nbytes)
-{
- trace_mix_pool_bytes_nolock(nbytes, _RET_IP_);
- _mix_pool_bytes(in, nbytes);
-}
-
-static void mix_pool_bytes(const void *in, int nbytes)
+static void mix_pool_bytes(const void *in, size_t nbytes)
{
unsigned long flags;
@@ -384,18 +378,18 @@ static void process_random_ready_list(vo
spin_unlock_irqrestore(&random_ready_list_lock, flags);
}
-static void credit_entropy_bits(int nbits)
+static void credit_entropy_bits(size_t nbits)
{
- int entropy_count, orig;
+ unsigned int entropy_count, orig, add;
- if (nbits <= 0)
+ if (!nbits)
return;
- nbits = min(nbits, POOL_BITS);
+ add = min_t(size_t, nbits, POOL_BITS);
do {
orig = READ_ONCE(input_pool.entropy_count);
- entropy_count = min(POOL_BITS, orig + nbits);
+ entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
} while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
trace_credit_entropy_bits(nbits, entropy_count, _RET_IP_);
@@ -442,10 +436,10 @@ static void invalidate_batched_entropy(v
* path. So we can't afford to dilly-dally. Returns the number of
* bytes processed from cp.
*/
-static size_t crng_fast_load(const u8 *cp, size_t len)
+static size_t crng_fast_load(const void *cp, size_t len)
{
unsigned long flags;
- u8 *p;
+ const u8 *src = (const u8 *)cp;
size_t ret = 0;
if (!spin_trylock_irqsave(&base_crng.lock, flags))
@@ -454,10 +448,9 @@ static size_t crng_fast_load(const u8 *c
spin_unlock_irqrestore(&base_crng.lock, flags);
return 0;
}
- p = base_crng.key;
while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
- p[crng_init_cnt % sizeof(base_crng.key)] ^= *cp;
- cp++; crng_init_cnt++; len--; ret++;
+ base_crng.key[crng_init_cnt % sizeof(base_crng.key)] ^= *src;
+ src++; crng_init_cnt++; len--; ret++;
}
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
invalidate_batched_entropy();
@@ -527,42 +520,30 @@ static struct crng_state *select_crng(vo
* all), and (2) it doesn't have the performance constraints of
* crng_fast_load().
*
- * So we do something more comprehensive which is guaranteed to touch
- * all of the primary_crng's state, and which uses a LFSR with a
- * period of 255 as part of the mixing algorithm. Finally, we do
- * *not* advance crng_init_cnt since buffer we may get may be something
- * like a fixed DMI table (for example), which might very well be
- * unique to the machine, but is otherwise unvarying.
+ * So, we simply hash the contents in with the current key. Finally,
+ * we do *not* advance crng_init_cnt since buffer we may get may be
+ * something like a fixed DMI table (for example), which might very
+ * well be unique to the machine, but is otherwise unvarying.
*/
-static int crng_slow_load(const u8 *cp, size_t len)
+static void crng_slow_load(const void *cp, size_t len)
{
unsigned long flags;
- static u8 lfsr = 1;
- u8 tmp;
- unsigned int i, max = sizeof(base_crng.key);
- const u8 *src_buf = cp;
- u8 *dest_buf = base_crng.key;
+ struct blake2s_state hash;
+
+ blake2s_init(&hash, sizeof(base_crng.key));
if (!spin_trylock_irqsave(&base_crng.lock, flags))
- return 0;
+ return;
if (crng_init != 0) {
spin_unlock_irqrestore(&base_crng.lock, flags);
- return 0;
+ return;
}
- if (len > max)
- max = len;
- for (i = 0; i < max; i++) {
- tmp = lfsr;
- lfsr >>= 1;
- if (tmp & 1)
- lfsr ^= 0xE1;
- tmp = dest_buf[i % sizeof(base_crng.key)];
- dest_buf[i % sizeof(base_crng.key)] ^= src_buf[i % len] ^ lfsr;
- lfsr += (tmp << 3) | (tmp >> 5);
- }
+ blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
+ blake2s_update(&hash, cp, len);
+ blake2s_final(&hash, base_crng.key);
+
spin_unlock_irqrestore(&base_crng.lock, flags);
- return 1;
}
static void crng_reseed(void)
@@ -718,14 +699,15 @@ static void crng_make_state(u32 chacha_s
static ssize_t get_random_bytes_user(void __user *buf, size_t nbytes)
{
bool large_request = nbytes > 256;
- ssize_t ret = 0, len;
+ ssize_t ret = 0;
+ size_t len;
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
u8 output[CHACHA20_BLOCK_SIZE];
if (!nbytes)
return 0;
- len = min_t(ssize_t, 32, nbytes);
+ len = min_t(size_t, 32, nbytes);
crng_make_state(chacha_state, output, len);
if (copy_to_user(buf, output, len))
@@ -745,7 +727,7 @@ static ssize_t get_random_bytes_user(voi
if (unlikely(chacha_state[12] == 0))
++chacha_state[13];
- len = min_t(ssize_t, nbytes, CHACHA20_BLOCK_SIZE);
+ len = min_t(size_t, nbytes, CHACHA20_BLOCK_SIZE);
if (copy_to_user(buf, output, len)) {
ret = -EFAULT;
break;
@@ -783,7 +765,7 @@ struct timer_rand_state {
* the entropy pool having similar initial state across largely
* identical devices.
*/
-void add_device_randomness(const void *buf, unsigned int size)
+void add_device_randomness(const void *buf, size_t size)
{
unsigned long time = random_get_entropy() ^ jiffies;
unsigned long flags;
@@ -811,7 +793,7 @@ static struct timer_rand_state input_tim
* keyboard scan codes, and 256 upwards for interrupts.
*
*/
-static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
+static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
{
struct {
long jiffies;
@@ -855,7 +837,7 @@ static void add_timer_randomness(struct
* Round down by 1 bit on general principles,
* and limit entropy estimate to 12 bits.
*/
- credit_entropy_bits(min_t(int, fls(delta >> 1), 11));
+ credit_entropy_bits(min_t(unsigned int, fls(delta >> 1), 11));
}
void add_input_randomness(unsigned int type, unsigned int code,
@@ -936,8 +918,8 @@ void add_interrupt_randomness(int irq)
add_interrupt_bench(cycles);
if (unlikely(crng_init == 0)) {
- if ((fast_pool->count >= 64) &&
- crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)) > 0) {
+ if (fast_pool->count >= 64 &&
+ crng_fast_load(fast_pool->pool, sizeof(fast_pool->pool)) > 0) {
fast_pool->count = 0;
fast_pool->last = now;
if (spin_trylock(&input_pool.lock)) {
@@ -955,7 +937,7 @@ void add_interrupt_randomness(int irq)
return;
fast_pool->last = now;
- __mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool));
+ _mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool));
spin_unlock(&input_pool.lock);
fast_pool->count = 0;
@@ -1064,18 +1046,18 @@ static void _warn_unseeded_randomness(co
* wait_for_random_bytes() should be called and return 0 at least once
* at any point prior.
*/
-static void _get_random_bytes(void *buf, int nbytes)
+static void _get_random_bytes(void *buf, size_t nbytes)
{
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
u8 tmp[CHACHA20_BLOCK_SIZE];
- ssize_t len;
+ size_t len;
trace_get_random_bytes(nbytes, _RET_IP_);
if (!nbytes)
return;
- len = min_t(ssize_t, 32, nbytes);
+ len = min_t(size_t, 32, nbytes);
crng_make_state(chacha_state, buf, len);
nbytes -= len;
buf += len;
@@ -1098,7 +1080,7 @@ static void _get_random_bytes(void *buf,
memzero_explicit(chacha_state, sizeof(chacha_state));
}
-void get_random_bytes(void *buf, int nbytes)
+void get_random_bytes(void *buf, size_t nbytes)
{
static void *previous;
@@ -1259,25 +1241,19 @@ EXPORT_SYMBOL(del_random_ready_callback)
/*
* This function will use the architecture-specific hardware random
- * number generator if it is available. The arch-specific hw RNG will
- * almost certainly be faster than what we can do in software, but it
- * is impossible to verify that it is implemented securely (as
- * opposed, to, say, the AES encryption of a sequence number using a
- * key known by the NSA). So it's useful if we need the speed, but
- * only if we're willing to trust the hardware manufacturer not to
- * have put in a back door.
- *
- * Return number of bytes filled in.
+ * number generator if it is available. It is not recommended for
+ * use. Use get_random_bytes() instead. It returns the number of
+ * bytes filled in.
*/
-int __must_check get_random_bytes_arch(void *buf, int nbytes)
+size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes)
{
- int left = nbytes;
+ size_t left = nbytes;
u8 *p = buf;
trace_get_random_bytes_arch(left, _RET_IP_);
while (left) {
unsigned long v;
- int chunk = min_t(int, left, sizeof(unsigned long));
+ size_t chunk = min_t(size_t, left, sizeof(unsigned long));
if (!arch_get_random_long(&v))
break;
@@ -1310,12 +1286,12 @@ early_param("random.trust_cpu", parse_tr
*/
int __init rand_initialize(void)
{
- int i;
+ size_t i;
ktime_t now = ktime_get_real();
bool arch_init = true;
unsigned long rv;
- for (i = BLAKE2S_BLOCK_SIZE; i > 0; i -= sizeof(rv)) {
+ for (i = 0; i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
if (!arch_get_random_seed_long_early(&rv) &&
!arch_get_random_long_early(&rv)) {
rv = random_get_entropy();
@@ -1364,7 +1340,7 @@ static ssize_t urandom_read_nowarn(struc
nbytes = min_t(size_t, nbytes, INT_MAX >> 6);
ret = get_random_bytes_user(buf, nbytes);
- trace_urandom_read(8 * nbytes, 0, input_pool.entropy_count);
+ trace_urandom_read(nbytes, input_pool.entropy_count);
return ret;
}
@@ -1408,19 +1384,18 @@ static unsigned int random_poll(struct f
return mask;
}
-static int write_pool(const char __user *buffer, size_t count)
+static int write_pool(const char __user *ubuf, size_t count)
{
- size_t bytes;
- u8 buf[BLAKE2S_BLOCK_SIZE];
- const char __user *p = buffer;
-
- while (count > 0) {
- bytes = min(count, sizeof(buf));
- if (copy_from_user(buf, p, bytes))
+ size_t len;
+ u8 block[BLAKE2S_BLOCK_SIZE];
+
+ while (count) {
+ len = min(count, sizeof(block));
+ if (copy_from_user(block, ubuf, len))
return -EFAULT;
- count -= bytes;
- p += bytes;
- mix_pool_bytes(buf, bytes);
+ count -= len;
+ ubuf += len;
+ mix_pool_bytes(block, len);
cond_resched();
}
@@ -1430,7 +1405,7 @@ static int write_pool(const char __user
static ssize_t random_write(struct file *file, const char __user *buffer,
size_t count, loff_t *ppos)
{
- size_t ret;
+ int ret;
ret = write_pool(buffer, count);
if (ret)
@@ -1524,8 +1499,6 @@ const struct file_operations urandom_fop
SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count, unsigned int,
flags)
{
- int ret;
-
if (flags & ~(GRND_NONBLOCK | GRND_RANDOM | GRND_INSECURE))
return -EINVAL;
@@ -1540,6 +1513,8 @@ SYSCALL_DEFINE3(getrandom, char __user *
count = INT_MAX;
if (!(flags & GRND_INSECURE) && !crng_ready()) {
+ int ret;
+
if (flags & GRND_NONBLOCK)
return -EAGAIN;
ret = wait_for_random_bytes();
@@ -1798,7 +1773,7 @@ unsigned long randomize_page(unsigned lo
* Those devices may produce endless random bits and will be throttled
* when our pool is full.
*/
-void add_hwgenerator_randomness(const char *buffer, size_t count,
+void add_hwgenerator_randomness(const void *buffer, size_t count,
size_t entropy)
{
if (unlikely(crng_init == 0)) {
@@ -1829,7 +1804,7 @@ EXPORT_SYMBOL_GPL(add_hwgenerator_random
* it would be regarded as device data.
* The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
*/
-void add_bootloader_randomness(const void *buf, unsigned int size)
+void add_bootloader_randomness(const void *buf, size_t size)
{
if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
add_hwgenerator_randomness(buf, size, size * 8);
--- a/include/linux/hw_random.h
+++ b/include/linux/hw_random.h
@@ -61,6 +61,6 @@ extern int devm_hwrng_register(struct de
extern void hwrng_unregister(struct hwrng *rng);
extern void devm_hwrng_unregister(struct device *dve, struct hwrng *rng);
/** Feed random bits into the pool. */
-extern void add_hwgenerator_randomness(const char *buffer, size_t count, size_t entropy);
+extern void add_hwgenerator_randomness(const void *buffer, size_t count, size_t entropy);
#endif /* LINUX_HWRANDOM_H_ */
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -19,8 +19,8 @@ struct random_ready_callback {
struct module *owner;
};
-extern void add_device_randomness(const void *, unsigned int);
-extern void add_bootloader_randomness(const void *, unsigned int);
+extern void add_device_randomness(const void *, size_t);
+extern void add_bootloader_randomness(const void *, size_t);
#if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__)
static inline void add_latent_entropy(void)
@@ -36,13 +36,13 @@ extern void add_input_randomness(unsigne
unsigned int value) __latent_entropy;
extern void add_interrupt_randomness(int irq) __latent_entropy;
-extern void get_random_bytes(void *buf, int nbytes);
+extern void get_random_bytes(void *buf, size_t nbytes);
extern int wait_for_random_bytes(void);
extern int __init rand_initialize(void);
extern bool rng_is_initialized(void);
extern int add_random_ready_callback(struct random_ready_callback *rdy);
extern void del_random_ready_callback(struct random_ready_callback *rdy);
-extern int __must_check get_random_bytes_arch(void *buf, int nbytes);
+extern size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes);
#ifndef MODULE
extern const struct file_operations random_fops, urandom_fops;
@@ -65,7 +65,7 @@ static inline unsigned long get_random_l
/* Calls wait_for_random_bytes() and then calls get_random_bytes(buf, nbytes).
* Returns the result of the call to wait_for_random_bytes. */
-static inline int get_random_bytes_wait(void *buf, int nbytes)
+static inline int get_random_bytes_wait(void *buf, size_t nbytes)
{
int ret = wait_for_random_bytes();
get_random_bytes(buf, nbytes);
--- a/include/trace/events/random.h
+++ b/include/trace/events/random.h
@@ -8,13 +8,13 @@
#include <linux/tracepoint.h>
TRACE_EVENT(add_device_randomness,
- TP_PROTO(int bytes, unsigned long IP),
+ TP_PROTO(size_t bytes, unsigned long IP),
TP_ARGS(bytes, IP),
TP_STRUCT__entry(
- __field( int, bytes )
- __field(unsigned long, IP )
+ __field(size_t, bytes )
+ __field(unsigned long, IP )
),
TP_fast_assign(
@@ -22,18 +22,18 @@ TRACE_EVENT(add_device_randomness,
__entry->IP = IP;
),
- TP_printk("bytes %d caller %pS",
+ TP_printk("bytes %zu caller %pS",
__entry->bytes, (void *)__entry->IP)
);
DECLARE_EVENT_CLASS(random__mix_pool_bytes,
- TP_PROTO(int bytes, unsigned long IP),
+ TP_PROTO(size_t bytes, unsigned long IP),
TP_ARGS(bytes, IP),
TP_STRUCT__entry(
- __field( int, bytes )
- __field(unsigned long, IP )
+ __field(size_t, bytes )
+ __field(unsigned long, IP )
),
TP_fast_assign(
@@ -41,12 +41,12 @@ DECLARE_EVENT_CLASS(random__mix_pool_byt
__entry->IP = IP;
),
- TP_printk("input pool: bytes %d caller %pS",
+ TP_printk("input pool: bytes %zu caller %pS",
__entry->bytes, (void *)__entry->IP)
);
DEFINE_EVENT(random__mix_pool_bytes, mix_pool_bytes,
- TP_PROTO(int bytes, unsigned long IP),
+ TP_PROTO(size_t bytes, unsigned long IP),
TP_ARGS(bytes, IP)
);
@@ -58,13 +58,13 @@ DEFINE_EVENT(random__mix_pool_bytes, mix
);
TRACE_EVENT(credit_entropy_bits,
- TP_PROTO(int bits, int entropy_count, unsigned long IP),
+ TP_PROTO(size_t bits, size_t entropy_count, unsigned long IP),
TP_ARGS(bits, entropy_count, IP),
TP_STRUCT__entry(
- __field( int, bits )
- __field( int, entropy_count )
+ __field(size_t, bits )
+ __field(size_t, entropy_count )
__field(unsigned long, IP )
),
@@ -74,34 +74,34 @@ TRACE_EVENT(credit_entropy_bits,
__entry->IP = IP;
),
- TP_printk("input pool: bits %d entropy_count %d caller %pS",
+ TP_printk("input pool: bits %zu entropy_count %zu caller %pS",
__entry->bits, __entry->entropy_count, (void *)__entry->IP)
);
TRACE_EVENT(add_input_randomness,
- TP_PROTO(int input_bits),
+ TP_PROTO(size_t input_bits),
TP_ARGS(input_bits),
TP_STRUCT__entry(
- __field( int, input_bits )
+ __field(size_t, input_bits )
),
TP_fast_assign(
__entry->input_bits = input_bits;
),
- TP_printk("input_pool_bits %d", __entry->input_bits)
+ TP_printk("input_pool_bits %zu", __entry->input_bits)
);
TRACE_EVENT(add_disk_randomness,
- TP_PROTO(dev_t dev, int input_bits),
+ TP_PROTO(dev_t dev, size_t input_bits),
TP_ARGS(dev, input_bits),
TP_STRUCT__entry(
- __field( dev_t, dev )
- __field( int, input_bits )
+ __field(dev_t, dev )
+ __field(size_t, input_bits )
),
TP_fast_assign(
@@ -109,17 +109,17 @@ TRACE_EVENT(add_disk_randomness,
__entry->input_bits = input_bits;
),
- TP_printk("dev %d,%d input_pool_bits %d", MAJOR(__entry->dev),
+ TP_printk("dev %d,%d input_pool_bits %zu", MAJOR(__entry->dev),
MINOR(__entry->dev), __entry->input_bits)
);
DECLARE_EVENT_CLASS(random__get_random_bytes,
- TP_PROTO(int nbytes, unsigned long IP),
+ TP_PROTO(size_t nbytes, unsigned long IP),
TP_ARGS(nbytes, IP),
TP_STRUCT__entry(
- __field( int, nbytes )
+ __field(size_t, nbytes )
__field(unsigned long, IP )
),
@@ -128,29 +128,29 @@ DECLARE_EVENT_CLASS(random__get_random_b
__entry->IP = IP;
),
- TP_printk("nbytes %d caller %pS", __entry->nbytes, (void *)__entry->IP)
+ TP_printk("nbytes %zu caller %pS", __entry->nbytes, (void *)__entry->IP)
);
DEFINE_EVENT(random__get_random_bytes, get_random_bytes,
- TP_PROTO(int nbytes, unsigned long IP),
+ TP_PROTO(size_t nbytes, unsigned long IP),
TP_ARGS(nbytes, IP)
);
DEFINE_EVENT(random__get_random_bytes, get_random_bytes_arch,
- TP_PROTO(int nbytes, unsigned long IP),
+ TP_PROTO(size_t nbytes, unsigned long IP),
TP_ARGS(nbytes, IP)
);
DECLARE_EVENT_CLASS(random__extract_entropy,
- TP_PROTO(int nbytes, int entropy_count),
+ TP_PROTO(size_t nbytes, size_t entropy_count),
TP_ARGS(nbytes, entropy_count),
TP_STRUCT__entry(
- __field( int, nbytes )
- __field( int, entropy_count )
+ __field( size_t, nbytes )
+ __field( size_t, entropy_count )
),
TP_fast_assign(
@@ -158,37 +158,34 @@ DECLARE_EVENT_CLASS(random__extract_entr
__entry->entropy_count = entropy_count;
),
- TP_printk("input pool: nbytes %d entropy_count %d",
+ TP_printk("input pool: nbytes %zu entropy_count %zu",
__entry->nbytes, __entry->entropy_count)
);
DEFINE_EVENT(random__extract_entropy, extract_entropy,
- TP_PROTO(int nbytes, int entropy_count),
+ TP_PROTO(size_t nbytes, size_t entropy_count),
TP_ARGS(nbytes, entropy_count)
);
TRACE_EVENT(urandom_read,
- TP_PROTO(int got_bits, int pool_left, int input_left),
+ TP_PROTO(size_t nbytes, size_t entropy_count),
- TP_ARGS(got_bits, pool_left, input_left),
+ TP_ARGS(nbytes, entropy_count),
TP_STRUCT__entry(
- __field( int, got_bits )
- __field( int, pool_left )
- __field( int, input_left )
+ __field( size_t, nbytes )
+ __field( size_t, entropy_count )
),
TP_fast_assign(
- __entry->got_bits = got_bits;
- __entry->pool_left = pool_left;
- __entry->input_left = input_left;
+ __entry->nbytes = nbytes;
+ __entry->entropy_count = entropy_count;
),
- TP_printk("got_bits %d nonblocking_pool_entropy_left %d "
- "input_entropy_left %d", __entry->got_bits,
- __entry->pool_left, __entry->input_left)
+ TP_printk("reading: nbytes %zu entropy_count %zu",
+ __entry->nbytes, __entry->entropy_count)
);
#endif /* _TRACE_RANDOM_H */
From: "Jason A. Donenfeld" <[email protected]>
commit 9c07f57869e90140080cfc282cc628d123e27704 upstream.
Our pool is 256 bits, and we only ever use all of it or don't use it at
all, which is decided by whether or not it has at least 128 bits in it.
So we can drastically simplify the accounting and cmpxchg loop to do
exactly this. While we're at it, we move the minimum bit size into a
constant so it can be shared between the two places where it matters.
The reason we want any of this is for the case in which an attacker has
compromised the current state, and then bruteforces small amounts of
entropy added to it. By demanding a particular minimum amount of entropy
be present before reseeding, we make that bruteforcing difficult.
Note that this rationale no longer includes anything about /dev/random
blocking at the right moment, since /dev/random no longer blocks (except
for at ~boot), but rather uses the crng. In a former life, /dev/random
was different and therefore required a more nuanced account(), but this
is no longer.
Behaviorally, nothing changes here. This is just a simplification of
the code.
Cc: Theodore Ts'o <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 91 +++++++++---------------------------------
include/trace/events/random.h | 30 ++-----------
2 files changed, 27 insertions(+), 94 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -290,12 +290,14 @@
enum poolinfo {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
POOL_BITSHIFT = ilog2(POOL_BITS),
+ POOL_MIN_BITS = POOL_BITS / 2,
/* To allow fractional bits to be tracked, the entropy_count field is
* denominated in units of 1/8th bits. */
POOL_ENTROPY_SHIFT = 3,
#define POOL_ENTROPY_BITS() (input_pool.entropy_count >> POOL_ENTROPY_SHIFT)
- POOL_FRACBITS = POOL_BITS << POOL_ENTROPY_SHIFT
+ POOL_FRACBITS = POOL_BITS << POOL_ENTROPY_SHIFT,
+ POOL_MIN_FRACBITS = POOL_MIN_BITS << POOL_ENTROPY_SHIFT
};
/*
@@ -376,8 +378,7 @@ static struct {
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
};
-static bool extract_entropy(void *buf, size_t nbytes, int min);
-static void _extract_entropy(void *buf, size_t nbytes);
+static void extract_entropy(void *buf, size_t nbytes);
static void crng_reseed(struct crng_state *crng, bool use_input_pool);
@@ -468,7 +469,7 @@ static void process_random_ready_list(vo
*/
static void credit_entropy_bits(int nbits)
{
- int entropy_count, entropy_bits, orig;
+ int entropy_count, orig;
int nfrac = nbits << POOL_ENTROPY_SHIFT;
/* Ensure that the multiplication can avoid being 64 bits wide. */
@@ -528,8 +529,7 @@ retry:
trace_credit_entropy_bits(nbits, entropy_count >> POOL_ENTROPY_SHIFT, _RET_IP_);
- entropy_bits = entropy_count >> POOL_ENTROPY_SHIFT;
- if (crng_init < 2 && entropy_bits >= 128)
+ if (crng_init < 2 && entropy_count >= POOL_MIN_FRACBITS)
crng_reseed(&primary_crng, true);
}
@@ -618,7 +618,7 @@ static void crng_initialize_secondary(st
static void __init crng_initialize_primary(void)
{
- _extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
+ extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
@@ -839,8 +839,17 @@ static void crng_reseed(struct crng_stat
} buf;
if (use_input_pool) {
- if (!extract_entropy(&buf, 32, 16))
- return;
+ int entropy_count;
+ do {
+ entropy_count = READ_ONCE(input_pool.entropy_count);
+ if (entropy_count < POOL_MIN_FRACBITS)
+ return;
+ } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
+ extract_entropy(buf.key, sizeof(buf.key));
+ if (random_write_wakeup_bits) {
+ wake_up_interruptible(&random_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
+ }
} else {
_extract_crng(&primary_crng, buf.block);
_crng_backtrack_protect(&primary_crng, buf.block,
@@ -1166,51 +1175,10 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
*********************************************************************/
/*
- * This function decides how many bytes to actually take from the
- * given pool, and also debits the entropy count accordingly.
- */
-static size_t account(size_t nbytes, int min)
-{
- int entropy_count, orig;
- size_t ibytes, nfrac;
-
- BUG_ON(input_pool.entropy_count > POOL_FRACBITS);
-
- /* Can we pull enough? */
-retry:
- entropy_count = orig = READ_ONCE(input_pool.entropy_count);
- if (WARN_ON(entropy_count < 0)) {
- pr_warn("negative entropy count: count %d\n", entropy_count);
- entropy_count = 0;
- }
-
- /* never pull more than available */
- ibytes = min_t(size_t, nbytes, entropy_count >> (POOL_ENTROPY_SHIFT + 3));
- if (ibytes < min)
- ibytes = 0;
- nfrac = ibytes << (POOL_ENTROPY_SHIFT + 3);
- if ((size_t)entropy_count > nfrac)
- entropy_count -= nfrac;
- else
- entropy_count = 0;
-
- if (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig)
- goto retry;
-
- trace_debit_entropy(8 * ibytes);
- if (ibytes && POOL_ENTROPY_BITS() < random_write_wakeup_bits) {
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
- }
-
- return ibytes;
-}
-
-/*
* This is an HKDF-like construction for using the hashed collected entropy
* as a PRF key, that's then expanded block-by-block.
*/
-static void _extract_entropy(void *buf, size_t nbytes)
+static void extract_entropy(void *buf, size_t nbytes)
{
unsigned long flags;
u8 seed[BLAKE2S_HASH_SIZE], next_key[BLAKE2S_HASH_SIZE];
@@ -1220,6 +1188,8 @@ static void _extract_entropy(void *buf,
} block;
size_t i;
+ trace_extract_entropy(nbytes, POOL_ENTROPY_BITS());
+
for (i = 0; i < ARRAY_SIZE(block.rdrand); ++i) {
if (!arch_get_random_long(&block.rdrand[i]))
block.rdrand[i] = random_get_entropy();
@@ -1251,25 +1221,6 @@ static void _extract_entropy(void *buf,
memzero_explicit(&block, sizeof(block));
}
-/*
- * This function extracts randomness from the "entropy pool", and
- * returns it in a buffer.
- *
- * The min parameter specifies the minimum amount we can pull before
- * failing to avoid races that defeat catastrophic reseeding. If we
- * have less than min entropy available, we return false and buf is
- * not filled.
- */
-static bool extract_entropy(void *buf, size_t nbytes, int min)
-{
- trace_extract_entropy(nbytes, POOL_ENTROPY_BITS(), _RET_IP_);
- if (account(nbytes, min)) {
- _extract_entropy(buf, nbytes);
- return true;
- }
- return false;
-}
-
#define warn_unseeded_randomness(previous) \
_warn_unseeded_randomness(__func__, (void *)_RET_IP_, (previous))
--- a/include/trace/events/random.h
+++ b/include/trace/events/random.h
@@ -78,22 +78,6 @@ TRACE_EVENT(credit_entropy_bits,
__entry->bits, __entry->entropy_count, (void *)__entry->IP)
);
-TRACE_EVENT(debit_entropy,
- TP_PROTO(int debit_bits),
-
- TP_ARGS( debit_bits),
-
- TP_STRUCT__entry(
- __field( int, debit_bits )
- ),
-
- TP_fast_assign(
- __entry->debit_bits = debit_bits;
- ),
-
- TP_printk("input pool: debit_bits %d", __entry->debit_bits)
-);
-
TRACE_EVENT(add_input_randomness,
TP_PROTO(int input_bits),
@@ -160,31 +144,29 @@ DEFINE_EVENT(random__get_random_bytes, g
);
DECLARE_EVENT_CLASS(random__extract_entropy,
- TP_PROTO(int nbytes, int entropy_count, unsigned long IP),
+ TP_PROTO(int nbytes, int entropy_count),
- TP_ARGS(nbytes, entropy_count, IP),
+ TP_ARGS(nbytes, entropy_count),
TP_STRUCT__entry(
__field( int, nbytes )
__field( int, entropy_count )
- __field(unsigned long, IP )
),
TP_fast_assign(
__entry->nbytes = nbytes;
__entry->entropy_count = entropy_count;
- __entry->IP = IP;
),
- TP_printk("input pool: nbytes %d entropy_count %d caller %pS",
- __entry->nbytes, __entry->entropy_count, (void *)__entry->IP)
+ TP_printk("input pool: nbytes %d entropy_count %d",
+ __entry->nbytes, __entry->entropy_count)
);
DEFINE_EVENT(random__extract_entropy, extract_entropy,
- TP_PROTO(int nbytes, int entropy_count, unsigned long IP),
+ TP_PROTO(int nbytes, int entropy_count),
- TP_ARGS(nbytes, entropy_count, IP)
+ TP_ARGS(nbytes, entropy_count)
);
TRACE_EVENT(urandom_read,
From: "Tobin C. Harding" <[email protected]>
commit 753d433b586d1d43c487e3d660f5778c7c8d58ea upstream.
Currently the function get_random_bytes_arch() has return value 'void'.
If the hw RNG fails we currently fall back to using get_random_bytes().
This defeats the purpose of requesting random material from the hw RNG
in the first place.
There are currently no intree users of get_random_bytes_arch().
Only get random bytes from the hw RNG, make function return the number
of bytes retrieved from the hw RNG.
Acked-by: Theodore Ts'o <[email protected]>
Reviewed-by: Steven Rostedt (VMware) <[email protected]>
Signed-off-by: Tobin C. Harding <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 16 +++++++++-------
include/linux/random.h | 2 +-
2 files changed, 10 insertions(+), 8 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1783,26 +1783,28 @@ EXPORT_SYMBOL(del_random_ready_callback)
* key known by the NSA). So it's useful if we need the speed, but
* only if we're willing to trust the hardware manufacturer not to
* have put in a back door.
+ *
+ * Return number of bytes filled in.
*/
-void get_random_bytes_arch(void *buf, int nbytes)
+int __must_check get_random_bytes_arch(void *buf, int nbytes)
{
+ int left = nbytes;
char *p = buf;
- trace_get_random_bytes_arch(nbytes, _RET_IP_);
- while (nbytes) {
+ trace_get_random_bytes_arch(left, _RET_IP_);
+ while (left) {
unsigned long v;
- int chunk = min(nbytes, (int)sizeof(unsigned long));
+ int chunk = min_t(int, left, sizeof(unsigned long));
if (!arch_get_random_long(&v))
break;
memcpy(p, &v, chunk);
p += chunk;
- nbytes -= chunk;
+ left -= chunk;
}
- if (nbytes)
- get_random_bytes(p, nbytes);
+ return nbytes - left;
}
EXPORT_SYMBOL(get_random_bytes_arch);
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -37,7 +37,7 @@ extern void get_random_bytes(void *buf,
extern int wait_for_random_bytes(void);
extern int add_random_ready_callback(struct random_ready_callback *rdy);
extern void del_random_ready_callback(struct random_ready_callback *rdy);
-extern void get_random_bytes_arch(void *buf, int nbytes);
+extern int __must_check get_random_bytes_arch(void *buf, int nbytes);
#ifndef MODULE
extern const struct file_operations random_fops, urandom_fops;
From: Linus Torvalds <[email protected]>
commit 50ee7529ec4500c88f8664560770a7a1b65db72b upstream.
For 5.3 we had to revert a nice ext4 IO pattern improvement, because it
caused a bootup regression due to lack of entropy at bootup together
with arguably broken user space that was asking for secure random
numbers when it really didn't need to.
See commit 72dbcf721566 (Revert "ext4: make __ext4_get_inode_loc plug").
This aims to solve the issue by actively generating entropy noise using
the CPU cycle counter when waiting for the random number generator to
initialize. This only works when you have a high-frequency time stamp
counter available, but that's the case on all modern x86 CPU's, and on
most other modern CPU's too.
What we do is to generate jitter entropy from the CPU cycle counter
under a somewhat complex load: calling the scheduler while also
guaranteeing a certain amount of timing noise by also triggering a
timer.
I'm sure we can tweak this, and that people will want to look at other
alternatives, but there's been a number of papers written on jitter
entropy, and this should really be fairly conservative by crediting one
bit of entropy for every timer-induced jump in the cycle counter. Not
because the timer itself would be all that unpredictable, but because
the interaction between the timer and the loop is going to be.
Even if (and perhaps particularly if) the timer actually happens on
another CPU, the cacheline interaction between the loop that reads the
cycle counter and the timer itself firing is going to add perturbations
to the cycle counter values that get mixed into the entropy pool.
As Thomas pointed out, with a modern out-of-order CPU, even quite simple
loops show a fair amount of hard-to-predict timing variability even in
the absense of external interrupts. But this tries to take that further
by actually having a fairly complex interaction.
This is not going to solve the entropy issue for architectures that have
no CPU cycle counter, but it's not clear how (and if) that is solvable,
and the hardware in question is largely starting to be irrelevant. And
by doing this we can at least avoid some of the even more contentious
approaches (like making the entropy waiting time out in order to avoid
the possibly unbounded waiting).
Cc: Ahmed Darwish <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Cc: Nicholas Mc Guire <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Kees Cook <[email protected]>
Cc: Willy Tarreau <[email protected]>
Cc: Alexander E. Patrakov <[email protected]>
Cc: Lennart Poettering <[email protected]>
Cc: Noah Meyerhans <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 61 insertions(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1573,6 +1573,56 @@ void get_random_bytes(void *buf, int nby
}
EXPORT_SYMBOL(get_random_bytes);
+
+/*
+ * Each time the timer fires, we expect that we got an unpredictable
+ * jump in the cycle counter. Even if the timer is running on another
+ * CPU, the timer activity will be touching the stack of the CPU that is
+ * generating entropy..
+ *
+ * Note that we don't re-arm the timer in the timer itself - we are
+ * happy to be scheduled away, since that just makes the load more
+ * complex, but we do not want the timer to keep ticking unless the
+ * entropy loop is running.
+ *
+ * So the re-arming always happens in the entropy loop itself.
+ */
+static void entropy_timer(unsigned long data)
+{
+ credit_entropy_bits(&input_pool, 1);
+}
+
+/*
+ * If we have an actual cycle counter, see if we can
+ * generate enough entropy with timing noise
+ */
+static void try_to_generate_entropy(void)
+{
+ struct {
+ unsigned long now;
+ struct timer_list timer;
+ } stack;
+
+ stack.now = random_get_entropy();
+
+ /* Slow counter - or none. Don't even bother */
+ if (stack.now == random_get_entropy())
+ return;
+
+ __setup_timer_on_stack(&stack.timer, entropy_timer, 0, 0);
+ while (!crng_ready()) {
+ if (!timer_pending(&stack.timer))
+ mod_timer(&stack.timer, jiffies+1);
+ mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+ schedule();
+ stack.now = random_get_entropy();
+ }
+
+ del_timer_sync(&stack.timer);
+ destroy_timer_on_stack(&stack.timer);
+ mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+}
+
/*
* Wait for the urandom pool to be seeded and thus guaranteed to supply
* cryptographically secure random numbers. This applies to: the /dev/urandom
@@ -1587,7 +1637,17 @@ int wait_for_random_bytes(void)
{
if (likely(crng_ready()))
return 0;
- return wait_event_interruptible(crng_init_wait, crng_ready());
+
+ do {
+ int ret;
+ ret = wait_event_interruptible_timeout(crng_init_wait, crng_ready(), HZ);
+ if (ret)
+ return ret > 0 ? 0 : ret;
+
+ try_to_generate_entropy();
+ } while (!crng_ready());
+
+ return 0;
}
EXPORT_SYMBOL(wait_for_random_bytes);
From: "Jason A. Donenfeld" <[email protected]>
commit 90ed1e67e896cc8040a523f8428fc02f9b164394 upstream.
Originally, the RNG used several pools, so having things abstracted out
over a generic entropy_store object made sense. These days, there's only
one input pool, and then an uneven mix of usage via the abstraction and
usage via &input_pool. Rather than this uneasy mixture, just get rid of
the abstraction entirely and have things always use the global. This
simplifies the code and makes reading it a bit easier.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 219 ++++++++++++++++++------------------------
include/trace/events/random.h | 56 ++++------
2 files changed, 117 insertions(+), 158 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -376,7 +376,7 @@
* credit_entropy_bits() needs to be 64 bits wide.
*/
#define ENTROPY_SHIFT 3
-#define ENTROPY_BITS(r) ((r)->entropy_count >> ENTROPY_SHIFT)
+#define ENTROPY_BITS() (input_pool.entropy_count >> ENTROPY_SHIFT)
/*
* If the entropy count falls under this number of bits, then we
@@ -506,33 +506,27 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
*
**********************************************************************/
-struct entropy_store;
-struct entropy_store {
+static u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
+
+static struct {
/* read-only data: */
u32 *pool;
- const char *name;
/* read-write data: */
spinlock_t lock;
u16 add_ptr;
u16 input_rotate;
int entropy_count;
-};
-
-static ssize_t extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes, int min);
-static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes);
-
-static void crng_reseed(struct crng_state *crng, struct entropy_store *r);
-static u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
-
-static struct entropy_store input_pool = {
- .name = "input",
+} input_pool = {
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
.pool = input_pool_data
};
+static ssize_t extract_entropy(void *buf, size_t nbytes, int min);
+static ssize_t _extract_entropy(void *buf, size_t nbytes);
+
+static void crng_reseed(struct crng_state *crng, bool use_input_pool);
+
static u32 const twist_table[8] = {
0x00000000, 0x3b6e20c8, 0x76dc4190, 0x4db26158,
0xedb88320, 0xd6d6a3e8, 0x9b64c2b0, 0xa00ae278 };
@@ -547,16 +541,15 @@ static u32 const twist_table[8] = {
* it's cheap to do so and helps slightly in the expected case where
* the entropy is concentrated in the low-order bits.
*/
-static void _mix_pool_bytes(struct entropy_store *r, const void *in,
- int nbytes)
+static void _mix_pool_bytes(const void *in, int nbytes)
{
unsigned long i;
int input_rotate;
const u8 *bytes = in;
u32 w;
- input_rotate = r->input_rotate;
- i = r->add_ptr;
+ input_rotate = input_pool.input_rotate;
+ i = input_pool.add_ptr;
/* mix one byte at a time to simplify size handling and churn faster */
while (nbytes--) {
@@ -564,15 +557,15 @@ static void _mix_pool_bytes(struct entro
i = (i - 1) & POOL_WORDMASK;
/* XOR in the various taps */
- w ^= r->pool[i];
- w ^= r->pool[(i + POOL_TAP1) & POOL_WORDMASK];
- w ^= r->pool[(i + POOL_TAP2) & POOL_WORDMASK];
- w ^= r->pool[(i + POOL_TAP3) & POOL_WORDMASK];
- w ^= r->pool[(i + POOL_TAP4) & POOL_WORDMASK];
- w ^= r->pool[(i + POOL_TAP5) & POOL_WORDMASK];
+ w ^= input_pool.pool[i];
+ w ^= input_pool.pool[(i + POOL_TAP1) & POOL_WORDMASK];
+ w ^= input_pool.pool[(i + POOL_TAP2) & POOL_WORDMASK];
+ w ^= input_pool.pool[(i + POOL_TAP3) & POOL_WORDMASK];
+ w ^= input_pool.pool[(i + POOL_TAP4) & POOL_WORDMASK];
+ w ^= input_pool.pool[(i + POOL_TAP5) & POOL_WORDMASK];
/* Mix the result back in with a twist */
- r->pool[i] = (w >> 3) ^ twist_table[w & 7];
+ input_pool.pool[i] = (w >> 3) ^ twist_table[w & 7];
/*
* Normally, we add 7 bits of rotation to the pool.
@@ -583,26 +576,24 @@ static void _mix_pool_bytes(struct entro
input_rotate = (input_rotate + (i ? 7 : 14)) & 31;
}
- r->input_rotate = input_rotate;
- r->add_ptr = i;
+ input_pool.input_rotate = input_rotate;
+ input_pool.add_ptr = i;
}
-static void __mix_pool_bytes(struct entropy_store *r, const void *in,
- int nbytes)
+static void __mix_pool_bytes(const void *in, int nbytes)
{
- trace_mix_pool_bytes_nolock(r->name, nbytes, _RET_IP_);
- _mix_pool_bytes(r, in, nbytes);
+ trace_mix_pool_bytes_nolock(nbytes, _RET_IP_);
+ _mix_pool_bytes(in, nbytes);
}
-static void mix_pool_bytes(struct entropy_store *r, const void *in,
- int nbytes)
+static void mix_pool_bytes(const void *in, int nbytes)
{
unsigned long flags;
- trace_mix_pool_bytes(r->name, nbytes, _RET_IP_);
- spin_lock_irqsave(&r->lock, flags);
- _mix_pool_bytes(r, in, nbytes);
- spin_unlock_irqrestore(&r->lock, flags);
+ trace_mix_pool_bytes(nbytes, _RET_IP_);
+ spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(in, nbytes);
+ spin_unlock_irqrestore(&input_pool.lock, flags);
}
struct fast_pool {
@@ -664,16 +655,16 @@ static void process_random_ready_list(vo
* Use credit_entropy_bits_safe() if the value comes from userspace
* or otherwise should be checked for extreme values.
*/
-static void credit_entropy_bits(struct entropy_store *r, int nbits)
+static void credit_entropy_bits(int nbits)
{
- int entropy_count, orig;
+ int entropy_count, entropy_bits, orig;
int nfrac = nbits << ENTROPY_SHIFT;
if (!nbits)
return;
retry:
- entropy_count = orig = ACCESS_ONCE(r->entropy_count);
+ entropy_count = orig = READ_ONCE(input_pool.entropy_count);
if (nfrac < 0) {
/* Debit */
entropy_count += nfrac;
@@ -714,26 +705,21 @@ retry:
}
if (WARN_ON(entropy_count < 0)) {
- pr_warn("negative entropy/overflow: pool %s count %d\n",
- r->name, entropy_count);
+ pr_warn("negative entropy/overflow: count %d\n", entropy_count);
entropy_count = 0;
} else if (entropy_count > POOL_FRACBITS)
entropy_count = POOL_FRACBITS;
- if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
+ if (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig)
goto retry;
- trace_credit_entropy_bits(r->name, nbits,
- entropy_count >> ENTROPY_SHIFT, _RET_IP_);
+ trace_credit_entropy_bits(nbits, entropy_count >> ENTROPY_SHIFT, _RET_IP_);
- if (r == &input_pool) {
- int entropy_bits = entropy_count >> ENTROPY_SHIFT;
-
- if (crng_init < 2 && entropy_bits >= 128)
- crng_reseed(&primary_crng, r);
- }
+ entropy_bits = entropy_count >> ENTROPY_SHIFT;
+ if (crng_init < 2 && entropy_bits >= 128)
+ crng_reseed(&primary_crng, true);
}
-static int credit_entropy_bits_safe(struct entropy_store *r, int nbits)
+static int credit_entropy_bits_safe(int nbits)
{
if (nbits < 0)
return -EINVAL;
@@ -741,7 +727,7 @@ static int credit_entropy_bits_safe(stru
/* Cap the value to avoid overflows */
nbits = min(nbits, POOL_BITS);
- credit_entropy_bits(r, nbits);
+ credit_entropy_bits(nbits);
return 0;
}
@@ -818,7 +804,7 @@ static void crng_initialize_secondary(st
static void __init crng_initialize_primary(struct crng_state *crng)
{
- _extract_entropy(&input_pool, &crng->state[4], sizeof(u32) * 12);
+ _extract_entropy(&crng->state[4], sizeof(u32) * 12);
if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
@@ -1030,7 +1016,7 @@ static int crng_slow_load(const u8 *cp,
return 1;
}
-static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
+static void crng_reseed(struct crng_state *crng, bool use_input_pool)
{
unsigned long flags;
int i, num;
@@ -1039,8 +1025,8 @@ static void crng_reseed(struct crng_stat
u32 key[8];
} buf;
- if (r) {
- num = extract_entropy(r, &buf, 32, 16);
+ if (use_input_pool) {
+ num = extract_entropy(&buf, 32, 16);
if (num == 0)
return;
} else {
@@ -1071,8 +1057,7 @@ static void _extract_crng(struct crng_st
init_time = READ_ONCE(crng->init_time);
if (time_after(READ_ONCE(crng_global_init_time), init_time) ||
time_after(jiffies, init_time + CRNG_RESEED_INTERVAL))
- crng_reseed(crng, crng == &primary_crng ?
- &input_pool : NULL);
+ crng_reseed(crng, crng == &primary_crng);
}
spin_lock_irqsave(&crng->lock, flags);
chacha20_block(&crng->state[0], out);
@@ -1183,8 +1168,8 @@ void add_device_randomness(const void *b
trace_add_device_randomness(size, _RET_IP_);
spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(&input_pool, buf, size);
- _mix_pool_bytes(&input_pool, &time, sizeof(time));
+ _mix_pool_bytes(buf, size);
+ _mix_pool_bytes(&time, sizeof(time));
spin_unlock_irqrestore(&input_pool.lock, flags);
}
EXPORT_SYMBOL(add_device_randomness);
@@ -1203,7 +1188,6 @@ static struct timer_rand_state input_tim
*/
static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
{
- struct entropy_store *r;
struct {
long jiffies;
unsigned int cycles;
@@ -1214,8 +1198,7 @@ static void add_timer_randomness(struct
sample.jiffies = jiffies;
sample.cycles = random_get_entropy();
sample.num = num;
- r = &input_pool;
- mix_pool_bytes(r, &sample, sizeof(sample));
+ mix_pool_bytes(&sample, sizeof(sample));
/*
* Calculate number of bits of randomness we probably added.
@@ -1247,7 +1230,7 @@ static void add_timer_randomness(struct
* Round down by 1 bit on general principles,
* and limit entropy estimate to 12 bits.
*/
- credit_entropy_bits(r, min_t(int, fls(delta>>1), 11));
+ credit_entropy_bits(min_t(int, fls(delta>>1), 11));
}
void add_input_randomness(unsigned int type, unsigned int code,
@@ -1262,7 +1245,7 @@ void add_input_randomness(unsigned int t
last_value = value;
add_timer_randomness(&input_timer_state,
(type << 4) ^ code ^ (code >> 4) ^ value);
- trace_add_input_randomness(ENTROPY_BITS(&input_pool));
+ trace_add_input_randomness(ENTROPY_BITS());
}
EXPORT_SYMBOL_GPL(add_input_randomness);
@@ -1306,7 +1289,6 @@ static u32 get_reg(struct fast_pool *f,
void add_interrupt_randomness(int irq)
{
- struct entropy_store *r;
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
struct pt_regs *regs = get_irq_regs();
unsigned long now = jiffies;
@@ -1341,18 +1323,17 @@ void add_interrupt_randomness(int irq)
!time_after(now, fast_pool->last + HZ))
return;
- r = &input_pool;
- if (!spin_trylock(&r->lock))
+ if (!spin_trylock(&input_pool.lock))
return;
fast_pool->last = now;
- __mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
- spin_unlock(&r->lock);
+ __mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool));
+ spin_unlock(&input_pool.lock);
fast_pool->count = 0;
/* award one bit for the contents of the fast pool */
- credit_entropy_bits(r, 1);
+ credit_entropy_bits(1);
}
EXPORT_SYMBOL_GPL(add_interrupt_randomness);
@@ -1363,7 +1344,7 @@ void add_disk_randomness(struct gendisk
return;
/* first major is 1, so we get >= 0x200 here */
add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
- trace_add_disk_randomness(disk_devt(disk), ENTROPY_BITS(&input_pool));
+ trace_add_disk_randomness(disk_devt(disk), ENTROPY_BITS());
}
EXPORT_SYMBOL_GPL(add_disk_randomness);
#endif
@@ -1378,16 +1359,16 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
* This function decides how many bytes to actually take from the
* given pool, and also debits the entropy count accordingly.
*/
-static size_t account(struct entropy_store *r, size_t nbytes, int min)
+static size_t account(size_t nbytes, int min)
{
int entropy_count, orig, have_bytes;
size_t ibytes, nfrac;
- BUG_ON(r->entropy_count > POOL_FRACBITS);
+ BUG_ON(input_pool.entropy_count > POOL_FRACBITS);
/* Can we pull enough? */
retry:
- entropy_count = orig = ACCESS_ONCE(r->entropy_count);
+ entropy_count = orig = READ_ONCE(input_pool.entropy_count);
ibytes = nbytes;
/* never pull more than available */
have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
@@ -1399,8 +1380,7 @@ retry:
ibytes = 0;
if (WARN_ON(entropy_count < 0)) {
- pr_warn("negative entropy count: pool %s count %d\n",
- r->name, entropy_count);
+ pr_warn("negative entropy count: count %d\n", entropy_count);
entropy_count = 0;
}
nfrac = ibytes << (ENTROPY_SHIFT + 3);
@@ -1409,11 +1389,11 @@ retry:
else
entropy_count = 0;
- if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
+ if (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig)
goto retry;
- trace_debit_entropy(r->name, 8 * ibytes);
- if (ibytes && ENTROPY_BITS(r) < random_write_wakeup_bits) {
+ trace_debit_entropy(8 * ibytes);
+ if (ibytes && ENTROPY_BITS() < random_write_wakeup_bits) {
wake_up_interruptible(&random_write_wait);
kill_fasync(&fasync, SIGIO, POLL_OUT);
}
@@ -1426,7 +1406,7 @@ retry:
*
* Note: we assume that .poolwords is a multiple of 16 words.
*/
-static void extract_buf(struct entropy_store *r, u8 *out)
+static void extract_buf(u8 *out)
{
struct blake2s_state state __aligned(__alignof__(unsigned long));
u8 hash[BLAKE2S_HASH_SIZE];
@@ -1448,8 +1428,8 @@ static void extract_buf(struct entropy_s
}
/* Generate a hash across the pool */
- spin_lock_irqsave(&r->lock, flags);
- blake2s_update(&state, (const u8 *)r->pool, POOL_BYTES);
+ spin_lock_irqsave(&input_pool.lock, flags);
+ blake2s_update(&state, (const u8 *)input_pool.pool, POOL_BYTES);
blake2s_final(&state, hash); /* final zeros out state */
/*
@@ -1461,8 +1441,8 @@ static void extract_buf(struct entropy_s
* brute-forcing the feedback as hard as brute-forcing the
* hash.
*/
- __mix_pool_bytes(r, hash, sizeof(hash));
- spin_unlock_irqrestore(&r->lock, flags);
+ __mix_pool_bytes(hash, sizeof(hash));
+ spin_unlock_irqrestore(&input_pool.lock, flags);
/* Note that EXTRACT_SIZE is half of hash size here, because above
* we've dumped the full length back into mixer. By reducing the
@@ -1472,14 +1452,13 @@ static void extract_buf(struct entropy_s
memzero_explicit(hash, sizeof(hash));
}
-static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes)
+static ssize_t _extract_entropy(void *buf, size_t nbytes)
{
ssize_t ret = 0, i;
u8 tmp[EXTRACT_SIZE];
while (nbytes) {
- extract_buf(r, tmp);
+ extract_buf(tmp);
i = min_t(int, nbytes, EXTRACT_SIZE);
memcpy(buf, tmp, i);
nbytes -= i;
@@ -1500,12 +1479,11 @@ static ssize_t _extract_entropy(struct e
* The min parameter specifies the minimum amount we can pull before
* failing to avoid races that defeat catastrophic reseeding.
*/
-static ssize_t extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes, int min)
+static ssize_t extract_entropy(void *buf, size_t nbytes, int min)
{
- trace_extract_entropy(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
- nbytes = account(r, nbytes, min);
- return _extract_entropy(r, buf, nbytes);
+ trace_extract_entropy(nbytes, ENTROPY_BITS(), _RET_IP_);
+ nbytes = account(nbytes, min);
+ return _extract_entropy(buf, nbytes);
}
#define warn_unseeded_randomness(previous) \
@@ -1589,7 +1567,7 @@ EXPORT_SYMBOL(get_random_bytes);
*/
static void entropy_timer(unsigned long data)
{
- credit_entropy_bits(&input_pool, 1);
+ credit_entropy_bits(1);
}
/*
@@ -1613,14 +1591,14 @@ static void try_to_generate_entropy(void
while (!crng_ready()) {
if (!timer_pending(&stack.timer))
mod_timer(&stack.timer, jiffies+1);
- mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+ mix_pool_bytes(&stack.now, sizeof(stack.now));
schedule();
stack.now = random_get_entropy();
}
del_timer_sync(&stack.timer);
destroy_timer_on_stack(&stack.timer);
- mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+ mix_pool_bytes(&stack.now, sizeof(stack.now));
}
/*
@@ -1761,26 +1739,24 @@ EXPORT_SYMBOL(get_random_bytes_arch);
/*
* init_std_data - initialize pool with system data
*
- * @r: pool to initialize
- *
* This function clears the pool's entropy count and mixes some system
* data into the pool to prepare it for use. The pool is not cleared
* as that can only decrease the entropy in the pool.
*/
-static void __init init_std_data(struct entropy_store *r)
+static void __init init_std_data(void)
{
int i;
ktime_t now = ktime_get_real();
unsigned long rv;
- mix_pool_bytes(r, &now, sizeof(now));
+ mix_pool_bytes(&now, sizeof(now));
for (i = POOL_BYTES; i > 0; i -= sizeof(rv)) {
if (!arch_get_random_seed_long(&rv) &&
!arch_get_random_long(&rv))
rv = random_get_entropy();
- mix_pool_bytes(r, &rv, sizeof(rv));
+ mix_pool_bytes(&rv, sizeof(rv));
}
- mix_pool_bytes(r, utsname(), sizeof(*(utsname())));
+ mix_pool_bytes(utsname(), sizeof(*(utsname())));
}
/*
@@ -1795,7 +1771,7 @@ static void __init init_std_data(struct
*/
int __init rand_initialize(void)
{
- init_std_data(&input_pool);
+ init_std_data();
if (crng_need_final_init)
crng_finalize_init(&primary_crng);
crng_initialize_primary(&primary_crng);
@@ -1832,7 +1808,7 @@ urandom_read_nowarn(struct file *file, c
nbytes = min_t(size_t, nbytes, INT_MAX >> (ENTROPY_SHIFT + 3));
ret = extract_crng_user(buf, nbytes);
- trace_urandom_read(8 * nbytes, 0, ENTROPY_BITS(&input_pool));
+ trace_urandom_read(8 * nbytes, 0, ENTROPY_BITS());
return ret;
}
@@ -1872,13 +1848,13 @@ random_poll(struct file *file, poll_tabl
mask = 0;
if (crng_ready())
mask |= POLLIN | POLLRDNORM;
- if (ENTROPY_BITS(&input_pool) < random_write_wakeup_bits)
+ if (ENTROPY_BITS() < random_write_wakeup_bits)
mask |= POLLOUT | POLLWRNORM;
return mask;
}
static int
-write_pool(struct entropy_store *r, const char __user *buffer, size_t count)
+write_pool(const char __user *buffer, size_t count)
{
size_t bytes;
u32 t, buf[16];
@@ -1900,7 +1876,7 @@ write_pool(struct entropy_store *r, cons
count -= bytes;
p += bytes;
- mix_pool_bytes(r, buf, bytes);
+ mix_pool_bytes(buf, bytes);
cond_resched();
}
@@ -1912,7 +1888,7 @@ static ssize_t random_write(struct file
{
size_t ret;
- ret = write_pool(&input_pool, buffer, count);
+ ret = write_pool(buffer, count);
if (ret)
return ret;
@@ -1928,7 +1904,7 @@ static long random_ioctl(struct file *f,
switch (cmd) {
case RNDGETENTCNT:
/* inherently racy, no point locking */
- ent_count = ENTROPY_BITS(&input_pool);
+ ent_count = ENTROPY_BITS();
if (put_user(ent_count, p))
return -EFAULT;
return 0;
@@ -1937,7 +1913,7 @@ static long random_ioctl(struct file *f,
return -EPERM;
if (get_user(ent_count, p))
return -EFAULT;
- return credit_entropy_bits_safe(&input_pool, ent_count);
+ return credit_entropy_bits_safe(ent_count);
case RNDADDENTROPY:
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -1947,11 +1923,10 @@ static long random_ioctl(struct file *f,
return -EINVAL;
if (get_user(size, p++))
return -EFAULT;
- retval = write_pool(&input_pool, (const char __user *)p,
- size);
+ retval = write_pool((const char __user *)p, size);
if (retval < 0)
return retval;
- return credit_entropy_bits_safe(&input_pool, ent_count);
+ return credit_entropy_bits_safe(ent_count);
case RNDZAPENTCNT:
case RNDCLEARPOOL:
/*
@@ -1967,7 +1942,7 @@ static long random_ioctl(struct file *f,
return -EPERM;
if (crng_init < 2)
return -ENODATA;
- crng_reseed(&primary_crng, &input_pool);
+ crng_reseed(&primary_crng, true);
WRITE_ONCE(crng_global_init_time, jiffies - 1);
return 0;
default:
@@ -2289,11 +2264,9 @@ randomize_page(unsigned long start, unsi
void add_hwgenerator_randomness(const char *buffer, size_t count,
size_t entropy)
{
- struct entropy_store *poolp = &input_pool;
-
if (unlikely(crng_init == 0)) {
size_t ret = crng_fast_load(buffer, count);
- mix_pool_bytes(poolp, buffer, ret);
+ mix_pool_bytes(buffer, ret);
count -= ret;
buffer += ret;
if (!count || crng_init == 0)
@@ -2306,9 +2279,9 @@ void add_hwgenerator_randomness(const ch
*/
wait_event_interruptible(random_write_wait,
!system_wq || kthread_should_stop() ||
- ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
- mix_pool_bytes(poolp, buffer, count);
- credit_entropy_bits(poolp, entropy);
+ ENTROPY_BITS() <= random_write_wakeup_bits);
+ mix_pool_bytes(buffer, count);
+ credit_entropy_bits(entropy);
}
EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
--- a/include/trace/events/random.h
+++ b/include/trace/events/random.h
@@ -27,80 +27,71 @@ TRACE_EVENT(add_device_randomness,
);
DECLARE_EVENT_CLASS(random__mix_pool_bytes,
- TP_PROTO(const char *pool_name, int bytes, unsigned long IP),
+ TP_PROTO(int bytes, unsigned long IP),
- TP_ARGS(pool_name, bytes, IP),
+ TP_ARGS(bytes, IP),
TP_STRUCT__entry(
- __field( const char *, pool_name )
__field( int, bytes )
__field(unsigned long, IP )
),
TP_fast_assign(
- __entry->pool_name = pool_name;
__entry->bytes = bytes;
__entry->IP = IP;
),
- TP_printk("%s pool: bytes %d caller %pS",
- __entry->pool_name, __entry->bytes, (void *)__entry->IP)
+ TP_printk("input pool: bytes %d caller %pS",
+ __entry->bytes, (void *)__entry->IP)
);
DEFINE_EVENT(random__mix_pool_bytes, mix_pool_bytes,
- TP_PROTO(const char *pool_name, int bytes, unsigned long IP),
+ TP_PROTO(int bytes, unsigned long IP),
- TP_ARGS(pool_name, bytes, IP)
+ TP_ARGS(bytes, IP)
);
DEFINE_EVENT(random__mix_pool_bytes, mix_pool_bytes_nolock,
- TP_PROTO(const char *pool_name, int bytes, unsigned long IP),
+ TP_PROTO(int bytes, unsigned long IP),
- TP_ARGS(pool_name, bytes, IP)
+ TP_ARGS(bytes, IP)
);
TRACE_EVENT(credit_entropy_bits,
- TP_PROTO(const char *pool_name, int bits, int entropy_count,
- unsigned long IP),
+ TP_PROTO(int bits, int entropy_count, unsigned long IP),
- TP_ARGS(pool_name, bits, entropy_count, IP),
+ TP_ARGS(bits, entropy_count, IP),
TP_STRUCT__entry(
- __field( const char *, pool_name )
__field( int, bits )
__field( int, entropy_count )
__field(unsigned long, IP )
),
TP_fast_assign(
- __entry->pool_name = pool_name;
__entry->bits = bits;
__entry->entropy_count = entropy_count;
__entry->IP = IP;
),
- TP_printk("%s pool: bits %d entropy_count %d caller %pS",
- __entry->pool_name, __entry->bits,
- __entry->entropy_count, (void *)__entry->IP)
+ TP_printk("input pool: bits %d entropy_count %d caller %pS",
+ __entry->bits, __entry->entropy_count, (void *)__entry->IP)
);
TRACE_EVENT(debit_entropy,
- TP_PROTO(const char *pool_name, int debit_bits),
+ TP_PROTO(int debit_bits),
- TP_ARGS(pool_name, debit_bits),
+ TP_ARGS( debit_bits),
TP_STRUCT__entry(
- __field( const char *, pool_name )
__field( int, debit_bits )
),
TP_fast_assign(
- __entry->pool_name = pool_name;
__entry->debit_bits = debit_bits;
),
- TP_printk("%s: debit_bits %d", __entry->pool_name,
- __entry->debit_bits)
+ TP_printk("input pool: debit_bits %d", __entry->debit_bits)
);
TRACE_EVENT(add_input_randomness,
@@ -169,36 +160,31 @@ DEFINE_EVENT(random__get_random_bytes, g
);
DECLARE_EVENT_CLASS(random__extract_entropy,
- TP_PROTO(const char *pool_name, int nbytes, int entropy_count,
- unsigned long IP),
+ TP_PROTO(int nbytes, int entropy_count, unsigned long IP),
- TP_ARGS(pool_name, nbytes, entropy_count, IP),
+ TP_ARGS(nbytes, entropy_count, IP),
TP_STRUCT__entry(
- __field( const char *, pool_name )
__field( int, nbytes )
__field( int, entropy_count )
__field(unsigned long, IP )
),
TP_fast_assign(
- __entry->pool_name = pool_name;
__entry->nbytes = nbytes;
__entry->entropy_count = entropy_count;
__entry->IP = IP;
),
- TP_printk("%s pool: nbytes %d entropy_count %d caller %pS",
- __entry->pool_name, __entry->nbytes, __entry->entropy_count,
- (void *)__entry->IP)
+ TP_printk("input pool: nbytes %d entropy_count %d caller %pS",
+ __entry->nbytes, __entry->entropy_count, (void *)__entry->IP)
);
DEFINE_EVENT(random__extract_entropy, extract_entropy,
- TP_PROTO(const char *pool_name, int nbytes, int entropy_count,
- unsigned long IP),
+ TP_PROTO(int nbytes, int entropy_count, unsigned long IP),
- TP_ARGS(pool_name, nbytes, entropy_count, IP)
+ TP_ARGS(nbytes, entropy_count, IP)
);
TRACE_EVENT(urandom_read,
From: Dominik Brodowski <[email protected]>
commit 9d5505f1eebeca778074a0260ed077fd85f8792c upstream.
crng_finalize_init() returns instantly if it is called for another pool
than primary_crng. The test whether crng_finalize_init() is still required
can be moved to the relevant caller in crng_reseed(), and
crng_need_final_init can be reset to false if crng_finalize_init() is
called with workqueues ready. Then, no previous callsite will call
crng_finalize_init() unless it is needed, and we can get rid of the
superfluous function parameter.
Signed-off-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -800,10 +800,8 @@ static void __init crng_initialize_prima
primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
-static void crng_finalize_init(struct crng_state *crng)
+static void crng_finalize_init(void)
{
- if (crng != &primary_crng || crng_init >= 2)
- return;
if (!system_wq) {
/* We can't call numa_crng_init until we have workqueues,
* so mark this for processing later. */
@@ -814,6 +812,7 @@ static void crng_finalize_init(struct cr
invalidate_batched_entropy();
numa_crng_init();
crng_init = 2;
+ crng_need_final_init = false;
process_random_ready_list();
wake_up_interruptible(&crng_init_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
@@ -1031,7 +1030,8 @@ static void crng_reseed(struct crng_stat
memzero_explicit(&buf, sizeof(buf));
WRITE_ONCE(crng->init_time, jiffies);
spin_unlock_irqrestore(&crng->lock, flags);
- crng_finalize_init(crng);
+ if (crng == &primary_crng && crng_init < 2)
+ crng_finalize_init();
}
static void _extract_crng(struct crng_state *crng, u8 out[CHACHA20_BLOCK_SIZE])
@@ -1748,7 +1748,7 @@ int __init rand_initialize(void)
{
init_std_data();
if (crng_need_final_init)
- crng_finalize_init(&primary_crng);
+ crng_finalize_init();
crng_initialize_primary();
crng_global_init_time = jiffies;
if (ratelimit_disable) {
From: "Jason A. Donenfeld" <[email protected]>
commit 0d9488ffbf2faddebc6bac055bfa6c93b94056a3 upstream.
By using `char` instead of `unsigned char`, certain platforms will sign
extend the byte when `w = rol32(*bytes++, input_rotate)` is called,
meaning that bit 7 is overrepresented when mixing. This isn't a real
problem (unless the mixer itself is already broken) since it's still
invertible, but it's not quite correct either. Fix this by using an
explicit unsigned type.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -550,7 +550,7 @@ static void _mix_pool_bytes(struct entro
unsigned long i, tap1, tap2, tap3, tap4, tap5;
int input_rotate;
int wordmask = r->poolinfo->poolwords - 1;
- const char *bytes = in;
+ const unsigned char *bytes = in;
__u32 w;
tap1 = r->poolinfo->tap1;
From: "Jason A. Donenfeld" <[email protected]>
commit 8b2d953b91e7f60200c24067ab17b77cc7bfd0d4 upstream.
This argument is always set to zero, as a result of us not caring about
keeping a certain amount reserved in the pool these days. So just remove
it and cleanup the function signatures.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 17 +++++++----------
1 file changed, 7 insertions(+), 10 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -520,7 +520,7 @@ struct entropy_store {
};
static ssize_t extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes, int min, int rsvd);
+ size_t nbytes, int min);
static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
size_t nbytes);
@@ -1040,7 +1040,7 @@ static void crng_reseed(struct crng_stat
} buf;
if (r) {
- num = extract_entropy(r, &buf, 32, 16, 0);
+ num = extract_entropy(r, &buf, 32, 16);
if (num == 0)
return;
} else {
@@ -1378,8 +1378,7 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
* This function decides how many bytes to actually take from the
* given pool, and also debits the entropy count accordingly.
*/
-static size_t account(struct entropy_store *r, size_t nbytes, int min,
- int reserved)
+static size_t account(struct entropy_store *r, size_t nbytes, int min)
{
int entropy_count, orig, have_bytes;
size_t ibytes, nfrac;
@@ -1393,7 +1392,7 @@ retry:
/* never pull more than available */
have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
- if ((have_bytes -= reserved) < 0)
+ if (have_bytes < 0)
have_bytes = 0;
ibytes = min_t(size_t, ibytes, have_bytes);
if (ibytes < min)
@@ -1499,15 +1498,13 @@ static ssize_t _extract_entropy(struct e
* returns it in a buffer.
*
* The min parameter specifies the minimum amount we can pull before
- * failing to avoid races that defeat catastrophic reseeding while the
- * reserved parameter indicates how much entropy we must leave in the
- * pool after each pull to avoid starving other readers.
+ * failing to avoid races that defeat catastrophic reseeding.
*/
static ssize_t extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes, int min, int reserved)
+ size_t nbytes, int min)
{
trace_extract_entropy(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
- nbytes = account(r, nbytes, min, reserved);
+ nbytes = account(r, nbytes, min);
return _extract_entropy(r, buf, nbytes);
}
From: "Jason A. Donenfeld" <[email protected]>
commit 28f425e573e906a4c15f8392cc2b1561ef448595 upstream.
When /dev/random was directly connected with entropy extraction, without
any expansion stage, extract_buf() was called for every 10 bytes of data
read from /dev/random. For that reason, RDRAND was used rather than
RDSEED. At the same time, crng_reseed() was still only called every 5
minutes, so there RDSEED made sense.
Those olden days were also a time when the entropy collector did not use
a cryptographic hash function, which meant most bets were off in terms
of real preimage resistance. For that reason too it didn't matter
_that_ much whether RDSEED was mixed in before or after entropy
extraction; both choices were sort of bad.
But now we have a cryptographic hash function at work, and with that we
get real preimage resistance. We also now only call extract_entropy()
every 5 minutes, rather than every 10 bytes. This allows us to do two
important things.
First, we can switch to using RDSEED in extract_entropy(), as Dominik
suggested. Second, we can ensure that RDSEED input always goes into the
cryptographic hash function with other things before being used
directly. This eliminates a category of attacks in which the CPU knows
the current state of the crng and knows that we're going to xor RDSEED
into it, and so it computes a malicious RDSEED. By going through our
hash function, it would require the CPU to compute a preimage on the
fly, which isn't going to happen.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Suggested-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 246 ++++++++++++--------------------------------------
1 file changed, 62 insertions(+), 184 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -324,14 +324,11 @@ static struct crng_state primary_crng =
* its value (from 0->1->2).
*/
static int crng_init = 0;
-static bool crng_need_final_init = false;
#define crng_ready() (likely(crng_init > 1))
static int crng_init_cnt = 0;
-static unsigned long crng_global_init_time = 0;
#define CRNG_INIT_CNT_THRESH (2 * CHACHA20_KEY_SIZE)
-static void _extract_crng(struct crng_state *crng, u8 out[CHACHA20_BLOCK_SIZE]);
-static void _crng_backtrack_protect(struct crng_state *crng,
- u8 tmp[CHACHA20_BLOCK_SIZE], int used);
+static void extract_crng(u8 out[CHACHA20_BLOCK_SIZE]);
+static void crng_backtrack_protect(u8 tmp[CHACHA20_BLOCK_SIZE], int used);
static void process_random_ready_list(void);
static void _get_random_bytes(void *buf, int nbytes);
@@ -366,7 +363,7 @@ static struct {
static void extract_entropy(void *buf, size_t nbytes);
-static void crng_reseed(struct crng_state *crng);
+static void crng_reseed(void);
/*
* This function adds bytes into the entropy "pool". It does not
@@ -465,7 +462,7 @@ static void credit_entropy_bits(int nbit
trace_credit_entropy_bits(nbits, entropy_count, _RET_IP_);
if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
- crng_reseed(&primary_crng);
+ crng_reseed();
}
/*********************************************************************
@@ -478,14 +475,6 @@ static void credit_entropy_bits(int nbit
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
-/*
- * Hack to deal with crazy userspace progams when they are all trying
- * to access /dev/urandom in parallel. The programs are almost
- * certainly doing something terribly wrong, but we'll work around
- * their brain damage.
- */
-static struct crng_state **crng_node_pool __read_mostly;
-
static void invalidate_batched_entropy(void);
static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
@@ -495,24 +484,6 @@ static int __init parse_trust_cpu(char *
}
early_param("random.trust_cpu", parse_trust_cpu);
-static bool crng_init_try_arch(struct crng_state *crng)
-{
- int i;
- bool arch_init = true;
- unsigned long rv;
-
- for (i = 4; i < 16; i++) {
- if (!arch_get_random_seed_long(&rv) &&
- !arch_get_random_long(&rv)) {
- rv = random_get_entropy();
- arch_init = false;
- }
- crng->state[i] ^= rv;
- }
-
- return arch_init;
-}
-
static bool __init crng_init_try_arch_early(void)
{
int i;
@@ -531,100 +502,17 @@ static bool __init crng_init_try_arch_ea
return arch_init;
}
-static void crng_initialize_secondary(struct crng_state *crng)
-{
- chacha_init_consts(crng->state);
- _get_random_bytes(&crng->state[4], sizeof(u32) * 12);
- crng_init_try_arch(crng);
- crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
-}
-
-static void __init crng_initialize_primary(void)
+static void __init crng_initialize(void)
{
extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
- numa_crng_init();
crng_init = 2;
pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
-static void crng_finalize_init(void)
-{
- if (!system_wq) {
- /* We can't call numa_crng_init until we have workqueues,
- * so mark this for processing later. */
- crng_need_final_init = true;
- return;
- }
-
- invalidate_batched_entropy();
- numa_crng_init();
- crng_init = 2;
- crng_need_final_init = false;
- process_random_ready_list();
- wake_up_interruptible(&crng_init_wait);
- kill_fasync(&fasync, SIGIO, POLL_IN);
- pr_notice("crng init done\n");
- if (unseeded_warning.missed) {
- pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
- unseeded_warning.missed);
- unseeded_warning.missed = 0;
- }
- if (urandom_warning.missed) {
- pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
- urandom_warning.missed);
- urandom_warning.missed = 0;
- }
-}
-
-static void do_numa_crng_init(struct work_struct *work)
-{
- int i;
- struct crng_state *crng;
- struct crng_state **pool;
-
- pool = kcalloc(nr_node_ids, sizeof(*pool), GFP_KERNEL | __GFP_NOFAIL);
- for_each_online_node(i) {
- crng = kmalloc_node(sizeof(struct crng_state),
- GFP_KERNEL | __GFP_NOFAIL, i);
- spin_lock_init(&crng->lock);
- crng_initialize_secondary(crng);
- pool[i] = crng;
- }
- /* pairs with READ_ONCE() in select_crng() */
- if (cmpxchg_release(&crng_node_pool, NULL, pool) != NULL) {
- for_each_node(i)
- kfree(pool[i]);
- kfree(pool);
- }
-}
-
-static DECLARE_WORK(numa_crng_init_work, do_numa_crng_init);
-
-static void numa_crng_init(void)
-{
- if (IS_ENABLED(CONFIG_NUMA))
- schedule_work(&numa_crng_init_work);
-}
-
-static struct crng_state *select_crng(void)
-{
- if (IS_ENABLED(CONFIG_NUMA)) {
- struct crng_state **pool;
- int nid = numa_node_id();
-
- /* pairs with cmpxchg_release() in do_numa_crng_init() */
- pool = READ_ONCE(crng_node_pool);
- if (pool && pool[nid])
- return pool[nid];
- }
-
- return &primary_crng;
-}
-
/*
* crng_fast_load() can be called by code in the interrupt service
* path. So we can't afford to dilly-dally. Returns the number of
@@ -753,73 +641,71 @@ static int crng_slow_load(const u8 *cp,
return 1;
}
-static void crng_reseed(struct crng_state *crng)
+static void crng_reseed(void)
{
unsigned long flags;
- int i;
+ int i, entropy_count;
union {
u8 block[CHACHA20_BLOCK_SIZE];
u32 key[8];
} buf;
- if (crng == &primary_crng) {
- int entropy_count;
- do {
- entropy_count = READ_ONCE(input_pool.entropy_count);
- if (entropy_count < POOL_MIN_BITS)
- return;
- } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
- extract_entropy(buf.key, sizeof(buf.key));
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
- } else {
- _extract_crng(&primary_crng, buf.block);
- _crng_backtrack_protect(&primary_crng, buf.block,
- CHACHA20_KEY_SIZE);
- }
- spin_lock_irqsave(&crng->lock, flags);
- for (i = 0; i < 8; i++) {
- unsigned long rv;
- if (!arch_get_random_seed_long(&rv) &&
- !arch_get_random_long(&rv))
- rv = random_get_entropy();
- crng->state[i + 4] ^= buf.key[i] ^ rv;
- }
+ do {
+ entropy_count = READ_ONCE(input_pool.entropy_count);
+ if (entropy_count < POOL_MIN_BITS)
+ return;
+ } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
+ extract_entropy(buf.key, sizeof(buf.key));
+ wake_up_interruptible(&random_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
+
+ spin_lock_irqsave(&primary_crng.lock, flags);
+ for (i = 0; i < 8; i++)
+ primary_crng.state[i + 4] ^= buf.key[i];
memzero_explicit(&buf, sizeof(buf));
- WRITE_ONCE(crng->init_time, jiffies);
- spin_unlock_irqrestore(&crng->lock, flags);
- if (crng == &primary_crng && crng_init < 2)
- crng_finalize_init();
+ WRITE_ONCE(primary_crng.init_time, jiffies);
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
+ if (crng_init < 2) {
+ invalidate_batched_entropy();
+ crng_init = 2;
+ process_random_ready_list();
+ wake_up_interruptible(&crng_init_wait);
+ kill_fasync(&fasync, SIGIO, POLL_IN);
+ pr_notice("crng init done\n");
+ if (unseeded_warning.missed) {
+ pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
+ unseeded_warning.missed);
+ unseeded_warning.missed = 0;
+ }
+ if (urandom_warning.missed) {
+ pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
+ urandom_warning.missed);
+ urandom_warning.missed = 0;
+ }
+ }
}
-static void _extract_crng(struct crng_state *crng, u8 out[CHACHA20_BLOCK_SIZE])
+static void extract_crng(u8 out[CHACHA20_BLOCK_SIZE])
{
unsigned long flags, init_time;
if (crng_ready()) {
- init_time = READ_ONCE(crng->init_time);
- if (time_after(READ_ONCE(crng_global_init_time), init_time) ||
- time_after(jiffies, init_time + CRNG_RESEED_INTERVAL))
- crng_reseed(crng);
- }
- spin_lock_irqsave(&crng->lock, flags);
- chacha20_block(&crng->state[0], out);
- if (crng->state[12] == 0)
- crng->state[13]++;
- spin_unlock_irqrestore(&crng->lock, flags);
-}
-
-static void extract_crng(u8 out[CHACHA20_BLOCK_SIZE])
-{
- _extract_crng(select_crng(), out);
+ init_time = READ_ONCE(primary_crng.init_time);
+ if (time_after(jiffies, init_time + CRNG_RESEED_INTERVAL))
+ crng_reseed();
+ }
+ spin_lock_irqsave(&primary_crng.lock, flags);
+ chacha20_block(&primary_crng.state[0], out);
+ if (primary_crng.state[12] == 0)
+ primary_crng.state[13]++;
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
}
/*
* Use the leftover bytes from the CRNG block output (if there is
* enough) to mutate the CRNG key to provide backtracking protection.
*/
-static void _crng_backtrack_protect(struct crng_state *crng,
- u8 tmp[CHACHA20_BLOCK_SIZE], int used)
+static void crng_backtrack_protect(u8 tmp[CHACHA20_BLOCK_SIZE], int used)
{
unsigned long flags;
u32 *s, *d;
@@ -830,17 +716,12 @@ static void _crng_backtrack_protect(stru
extract_crng(tmp);
used = 0;
}
- spin_lock_irqsave(&crng->lock, flags);
+ spin_lock_irqsave(&primary_crng.lock, flags);
s = (u32 *)&tmp[used];
- d = &crng->state[4];
+ d = &primary_crng.state[4];
for (i = 0; i < 8; i++)
*d++ ^= *s++;
- spin_unlock_irqrestore(&crng->lock, flags);
-}
-
-static void crng_backtrack_protect(u8 tmp[CHACHA20_BLOCK_SIZE], int used)
-{
- _crng_backtrack_protect(select_crng(), tmp, used);
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
}
static ssize_t extract_crng_user(void __user *buf, size_t nbytes)
@@ -1105,16 +986,17 @@ static void extract_entropy(void *buf, s
unsigned long flags;
u8 seed[BLAKE2S_HASH_SIZE], next_key[BLAKE2S_HASH_SIZE];
struct {
- unsigned long rdrand[32 / sizeof(long)];
+ unsigned long rdseed[32 / sizeof(long)];
size_t counter;
} block;
size_t i;
trace_extract_entropy(nbytes, input_pool.entropy_count);
- for (i = 0; i < ARRAY_SIZE(block.rdrand); ++i) {
- if (!arch_get_random_long(&block.rdrand[i]))
- block.rdrand[i] = random_get_entropy();
+ for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) {
+ if (!arch_get_random_seed_long(&block.rdseed[i]) &&
+ !arch_get_random_long(&block.rdseed[i]))
+ block.rdseed[i] = random_get_entropy();
}
spin_lock_irqsave(&input_pool.lock, flags);
@@ -1122,7 +1004,7 @@ static void extract_entropy(void *buf, s
/* seed = HASHPRF(last_key, entropy_input) */
blake2s_final(&input_pool.hash, seed);
- /* next_key = HASHPRF(seed, RDRAND || 0) */
+ /* next_key = HASHPRF(seed, RDSEED || 0) */
block.counter = 0;
blake2s(next_key, (u8 *)&block, seed, sizeof(next_key), sizeof(block), sizeof(seed));
blake2s_init_key(&input_pool.hash, BLAKE2S_HASH_SIZE, next_key, sizeof(next_key));
@@ -1132,7 +1014,7 @@ static void extract_entropy(void *buf, s
while (nbytes) {
i = min_t(size_t, nbytes, BLAKE2S_HASH_SIZE);
- /* output = HASHPRF(seed, RDRAND || ++counter) */
+ /* output = HASHPRF(seed, RDSEED || ++counter) */
++block.counter;
blake2s(buf, (u8 *)&block, seed, i, sizeof(block), sizeof(seed));
nbytes -= i;
@@ -1426,10 +1308,7 @@ static void __init init_std_data(void)
int __init rand_initialize(void)
{
init_std_data();
- if (crng_need_final_init)
- crng_finalize_init();
- crng_initialize_primary();
- crng_global_init_time = jiffies;
+ crng_initialize();
if (ratelimit_disable) {
urandom_warning.interval = 0;
unseeded_warning.interval = 0;
@@ -1599,8 +1478,7 @@ static long random_ioctl(struct file *f,
return -EPERM;
if (crng_init < 2)
return -ENODATA;
- crng_reseed(&primary_crng);
- WRITE_ONCE(crng_global_init_time, jiffies - 1);
+ crng_reseed();
return 0;
default:
return -EINVAL;
From: "Jason A. Donenfeld" <[email protected]>
commit 7b5164fb1279bf0251371848e40bae646b59b3a8 upstream.
This buffer may contain entropic data that shouldn't stick around longer
than needed, so zero out the temporary buffer at the end of write_pool().
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Jann Horn <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -552,6 +552,7 @@ static void crng_reseed(void)
int entropy_count;
unsigned long next_gen;
u8 key[CHACHA20_KEY_SIZE];
+ bool finalize_init = false;
/*
* First we make sure we have POOL_MIN_BITS of entropy in the pool,
@@ -579,12 +580,14 @@ static void crng_reseed(void)
++next_gen;
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
- spin_unlock_irqrestore(&base_crng.lock, flags);
- memzero_explicit(key, sizeof(key));
-
if (crng_init < 2) {
invalidate_batched_entropy();
crng_init = 2;
+ finalize_init = true;
+ }
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ memzero_explicit(key, sizeof(key));
+ if (finalize_init) {
process_random_ready_list();
wake_up_interruptible(&crng_init_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
@@ -1386,19 +1389,24 @@ static unsigned int random_poll(struct f
static int write_pool(const char __user *ubuf, size_t count)
{
size_t len;
+ int ret = 0;
u8 block[BLAKE2S_BLOCK_SIZE];
while (count) {
len = min(count, sizeof(block));
- if (copy_from_user(block, ubuf, len))
- return -EFAULT;
+ if (copy_from_user(block, ubuf, len)) {
+ ret = -EFAULT;
+ goto out;
+ }
count -= len;
ubuf += len;
mix_pool_bytes(block, len);
cond_resched();
}
- return 0;
+out:
+ memzero_explicit(block, sizeof(block));
+ return ret;
}
static ssize_t random_write(struct file *file, const char __user *buffer,
From: "Jason A. Donenfeld" <[email protected]>
commit 2ee25b6968b1b3c66ffa408de23d023c1bce81cf upstream.
RDRAND is not fast. RDRAND is actually quite slow. We've known this for
a while, which is why functions like get_random_u{32,64} were converted
to use batching of our ChaCha-based CRNG instead.
Yet CRNG extraction still includes a call to RDRAND, in the hot path of
every call to get_random_bytes(), /dev/urandom, and getrandom(2).
This call to RDRAND here seems quite superfluous. CRNG is already
extracting things based on a 256-bit key, based on good entropy, which
is then reseeded periodically, updated, backtrack-mutated, and so
forth. The CRNG extraction construction is something that we're already
relying on to be secure and solid. If it's not, that's a serious
problem, and it's unlikely that mixing in a measly 32 bits from RDRAND
is going to alleviate things.
And in the case where the CRNG doesn't have enough entropy yet, we're
already initializing the ChaCha key row with RDRAND in
crng_init_try_arch_early().
Removing the call to RDRAND improves performance on an i7-11850H by
370%. In other words, the vast majority of the work done by
extract_crng() prior to this commit was devoted to fetching 32 bits of
RDRAND.
Reviewed-by: Theodore Ts'o <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1074,7 +1074,7 @@ static void crng_reseed(struct crng_stat
static void _extract_crng(struct crng_state *crng,
__u32 out[CHACHA20_BLOCK_WORDS])
{
- unsigned long v, flags, init_time;
+ unsigned long flags, init_time;
if (crng_ready()) {
init_time = READ_ONCE(crng->init_time);
@@ -1084,8 +1084,6 @@ static void _extract_crng(struct crng_st
&input_pool : NULL);
}
spin_lock_irqsave(&crng->lock, flags);
- if (arch_get_random_long(&v))
- crng->state[14] ^= v;
chacha20_block(&crng->state[0], out);
if (crng->state[12] == 0)
crng->state[13]++;
From: "Jason A. Donenfeld" <[email protected]>
commit c30c575db4858f0bbe5e315ff2e529c782f33a1f upstream.
During crng_init == 0, we never credit entropy in add_interrupt_
randomness(), but instead dump it directly into the primary_crng. That's
fine, except for the fact that we then wind up throwing away that
entropy later when we switch to extracting from the input pool and
xoring into (and later in this series overwriting) the primary_crng key.
The two other early init sites -- add_hwgenerator_randomness()'s use
crng_fast_load() and add_device_ randomness()'s use of crng_slow_load()
-- always additionally give their inputs to the input pool. But not
add_interrupt_randomness().
This commit fixes that shortcoming by calling mix_pool_bytes() after
crng_fast_load() in add_interrupt_randomness(). That's partially
verboten on PREEMPT_RT, where it implies taking spinlock_t from an IRQ
handler. But this also only happens during early boot and then never
again after that. Plus it's a trylock so it has the same considerations
as calling crng_fast_load(), which we're already using.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Suggested-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 407 +++++++++++++++++++++++++++++---------------------
1 file changed, 237 insertions(+), 170 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -67,63 +67,19 @@
* Exported interfaces ---- kernel output
* --------------------------------------
*
- * The primary kernel interface is
+ * The primary kernel interfaces are:
*
* void get_random_bytes(void *buf, int nbytes);
- *
- * This interface will return the requested number of random bytes,
- * and place it in the requested buffer. This is equivalent to a
- * read from /dev/urandom.
- *
- * For less critical applications, there are the functions:
- *
* u32 get_random_u32()
* u64 get_random_u64()
* unsigned int get_random_int()
* unsigned long get_random_long()
*
- * These are produced by a cryptographic RNG seeded from get_random_bytes,
- * and so do not deplete the entropy pool as much. These are recommended
- * for most in-kernel operations *if the result is going to be stored in
- * the kernel*.
- *
- * Specifically, the get_random_int() family do not attempt to do
- * "anti-backtracking". If you capture the state of the kernel (e.g.
- * by snapshotting the VM), you can figure out previous get_random_int()
- * return values. But if the value is stored in the kernel anyway,
- * this is not a problem.
- *
- * It *is* safe to expose get_random_int() output to attackers (e.g. as
- * network cookies); given outputs 1..n, it's not feasible to predict
- * outputs 0 or n+1. The only concern is an attacker who breaks into
- * the kernel later; the get_random_int() engine is not reseeded as
- * often as the get_random_bytes() one.
- *
- * get_random_bytes() is needed for keys that need to stay secret after
- * they are erased from the kernel. For example, any key that will
- * be wrapped and stored encrypted. And session encryption keys: we'd
- * like to know that after the session is closed and the keys erased,
- * the plaintext is unrecoverable to someone who recorded the ciphertext.
- *
- * But for network ports/cookies, stack canaries, PRNG seeds, address
- * space layout randomization, session *authentication* keys, or other
- * applications where the sensitive data is stored in the kernel in
- * plaintext for as long as it's sensitive, the get_random_int() family
- * is just fine.
- *
- * Consider ASLR. We want to keep the address space secret from an
- * outside attacker while the process is running, but once the address
- * space is torn down, it's of no use to an attacker any more. And it's
- * stored in kernel data structures as long as it's alive, so worrying
- * about an attacker's ability to extrapolate it from the get_random_int()
- * CRNG is silly.
- *
- * Even some cryptographic keys are safe to generate with get_random_int().
- * In particular, keys for SipHash are generally fine. Here, knowledge
- * of the key authorizes you to do something to a kernel object (inject
- * packets to a network connection, or flood a hash table), and the
- * key is stored with the object being protected. Once it goes away,
- * we no longer care if anyone knows the key.
+ * These interfaces will return the requested number of random bytes
+ * into the given buffer or as a return value. This is equivalent to a
+ * read from /dev/urandom. The get_random_{u32,u64,int,long}() family
+ * of functions may be higher performance for one-off random integers,
+ * because they do a bit of buffering.
*
* prandom_u32()
* -------------
@@ -301,20 +257,6 @@ static struct fasync_struct *fasync;
static DEFINE_SPINLOCK(random_ready_list_lock);
static LIST_HEAD(random_ready_list);
-struct crng_state {
- u32 state[16];
- unsigned long init_time;
- spinlock_t lock;
-};
-
-static struct crng_state primary_crng = {
- .lock = __SPIN_LOCK_UNLOCKED(primary_crng.lock),
- .state[0] = CHACHA_CONSTANT_EXPA,
- .state[1] = CHACHA_CONSTANT_ND_3,
- .state[2] = CHACHA_CONSTANT_2_BY,
- .state[3] = CHACHA_CONSTANT_TE_K,
-};
-
/*
* crng_init = 0 --> Uninitialized
* 1 --> Initialized
@@ -326,9 +268,6 @@ static struct crng_state primary_crng =
static int crng_init = 0;
#define crng_ready() (likely(crng_init > 1))
static int crng_init_cnt = 0;
-#define CRNG_INIT_CNT_THRESH (2 * CHACHA20_KEY_SIZE)
-static void extract_crng(u8 out[CHACHA20_BLOCK_SIZE]);
-static void crng_backtrack_protect(u8 tmp[CHACHA20_BLOCK_SIZE], int used);
static void process_random_ready_list(void);
static void _get_random_bytes(void *buf, int nbytes);
@@ -471,7 +410,28 @@ static void credit_entropy_bits(int nbit
*
*********************************************************************/
-#define CRNG_RESEED_INTERVAL (300 * HZ)
+enum {
+ CRNG_RESEED_INTERVAL = 300 * HZ,
+ CRNG_INIT_CNT_THRESH = 2 * CHACHA20_KEY_SIZE
+};
+
+static struct {
+ u8 key[CHACHA20_KEY_SIZE] __aligned(__alignof__(long));
+ unsigned long birth;
+ unsigned long generation;
+ spinlock_t lock;
+} base_crng = {
+ .lock = __SPIN_LOCK_UNLOCKED(base_crng.lock)
+};
+
+struct crng {
+ u8 key[CHACHA20_KEY_SIZE];
+ unsigned long generation;
+};
+
+static DEFINE_PER_CPU(struct crng, crngs) = {
+ .generation = ULONG_MAX
+};
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
@@ -488,22 +448,22 @@ static size_t crng_fast_load(const u8 *c
u8 *p;
size_t ret = 0;
- if (!spin_trylock_irqsave(&primary_crng.lock, flags))
+ if (!spin_trylock_irqsave(&base_crng.lock, flags))
return 0;
if (crng_init != 0) {
- spin_unlock_irqrestore(&primary_crng.lock, flags);
+ spin_unlock_irqrestore(&base_crng.lock, flags);
return 0;
}
- p = (u8 *)&primary_crng.state[4];
+ p = base_crng.key;
while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
- p[crng_init_cnt % CHACHA20_KEY_SIZE] ^= *cp;
+ p[crng_init_cnt % sizeof(base_crng.key)] ^= *cp;
cp++; crng_init_cnt++; len--; ret++;
}
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
invalidate_batched_entropy();
crng_init = 1;
}
- spin_unlock_irqrestore(&primary_crng.lock, flags);
+ spin_unlock_irqrestore(&base_crng.lock, flags);
if (crng_init == 1)
pr_notice("fast init done\n");
return ret;
@@ -579,14 +539,14 @@ static int crng_slow_load(const u8 *cp,
unsigned long flags;
static u8 lfsr = 1;
u8 tmp;
- unsigned int i, max = CHACHA20_KEY_SIZE;
+ unsigned int i, max = sizeof(base_crng.key);
const u8 *src_buf = cp;
- u8 *dest_buf = (u8 *)&primary_crng.state[4];
+ u8 *dest_buf = base_crng.key;
- if (!spin_trylock_irqsave(&primary_crng.lock, flags))
+ if (!spin_trylock_irqsave(&base_crng.lock, flags))
return 0;
if (crng_init != 0) {
- spin_unlock_irqrestore(&primary_crng.lock, flags);
+ spin_unlock_irqrestore(&base_crng.lock, flags);
return 0;
}
if (len > max)
@@ -597,38 +557,50 @@ static int crng_slow_load(const u8 *cp,
lfsr >>= 1;
if (tmp & 1)
lfsr ^= 0xE1;
- tmp = dest_buf[i % CHACHA20_KEY_SIZE];
- dest_buf[i % CHACHA20_KEY_SIZE] ^= src_buf[i % len] ^ lfsr;
+ tmp = dest_buf[i % sizeof(base_crng.key)];
+ dest_buf[i % sizeof(base_crng.key)] ^= src_buf[i % len] ^ lfsr;
lfsr += (tmp << 3) | (tmp >> 5);
}
- spin_unlock_irqrestore(&primary_crng.lock, flags);
+ spin_unlock_irqrestore(&base_crng.lock, flags);
return 1;
}
static void crng_reseed(void)
{
unsigned long flags;
- int i, entropy_count;
- union {
- u8 block[CHACHA20_BLOCK_SIZE];
- u32 key[8];
- } buf;
+ int entropy_count;
+ unsigned long next_gen;
+ u8 key[CHACHA20_KEY_SIZE];
+ /*
+ * First we make sure we have POOL_MIN_BITS of entropy in the pool,
+ * and then we drain all of it. Only then can we extract a new key.
+ */
do {
entropy_count = READ_ONCE(input_pool.entropy_count);
if (entropy_count < POOL_MIN_BITS)
return;
} while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
- extract_entropy(buf.key, sizeof(buf.key));
+ extract_entropy(key, sizeof(key));
wake_up_interruptible(&random_write_wait);
kill_fasync(&fasync, SIGIO, POLL_OUT);
- spin_lock_irqsave(&primary_crng.lock, flags);
- for (i = 0; i < 8; i++)
- primary_crng.state[i + 4] ^= buf.key[i];
- memzero_explicit(&buf, sizeof(buf));
- WRITE_ONCE(primary_crng.init_time, jiffies);
- spin_unlock_irqrestore(&primary_crng.lock, flags);
+ /*
+ * We copy the new key into the base_crng, overwriting the old one,
+ * and update the generation counter. We avoid hitting ULONG_MAX,
+ * because the per-cpu crngs are initialized to ULONG_MAX, so this
+ * forces new CPUs that come online to always initialize.
+ */
+ spin_lock_irqsave(&base_crng.lock, flags);
+ memcpy(base_crng.key, key, sizeof(base_crng.key));
+ next_gen = base_crng.generation + 1;
+ if (next_gen == ULONG_MAX)
+ ++next_gen;
+ WRITE_ONCE(base_crng.generation, next_gen);
+ WRITE_ONCE(base_crng.birth, jiffies);
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ memzero_explicit(key, sizeof(key));
+
if (crng_init < 2) {
invalidate_batched_entropy();
crng_init = 2;
@@ -649,77 +621,143 @@ static void crng_reseed(void)
}
}
-static void extract_crng(u8 out[CHACHA20_BLOCK_SIZE])
+/*
+ * The general form here is based on a "fast key erasure RNG" from
+ * <https://blog.cr.yp.to/20170723-random.html>. It generates a ChaCha
+ * block using the provided key, and then immediately overwites that
+ * key with half the block. It returns the resultant ChaCha state to the
+ * user, along with the second half of the block containing 32 bytes of
+ * random data that may be used; random_data_len may not be greater than
+ * 32.
+ */
+static void crng_fast_key_erasure(u8 key[CHACHA20_KEY_SIZE],
+ u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)],
+ u8 *random_data, size_t random_data_len)
{
- unsigned long flags, init_time;
+ u8 first_block[CHACHA20_BLOCK_SIZE];
- if (crng_ready()) {
- init_time = READ_ONCE(primary_crng.init_time);
- if (time_after(jiffies, init_time + CRNG_RESEED_INTERVAL))
- crng_reseed();
- }
- spin_lock_irqsave(&primary_crng.lock, flags);
- chacha20_block(&primary_crng.state[0], out);
- if (primary_crng.state[12] == 0)
- primary_crng.state[13]++;
- spin_unlock_irqrestore(&primary_crng.lock, flags);
+ BUG_ON(random_data_len > 32);
+
+ chacha_init_consts(chacha_state);
+ memcpy(&chacha_state[4], key, CHACHA20_KEY_SIZE);
+ memset(&chacha_state[12], 0, sizeof(u32) * 4);
+ chacha20_block(chacha_state, first_block);
+
+ memcpy(key, first_block, CHACHA20_KEY_SIZE);
+ memcpy(random_data, first_block + CHACHA20_KEY_SIZE, random_data_len);
+ memzero_explicit(first_block, sizeof(first_block));
}
/*
- * Use the leftover bytes from the CRNG block output (if there is
- * enough) to mutate the CRNG key to provide backtracking protection.
+ * This function returns a ChaCha state that you may use for generating
+ * random data. It also returns up to 32 bytes on its own of random data
+ * that may be used; random_data_len may not be greater than 32.
*/
-static void crng_backtrack_protect(u8 tmp[CHACHA20_BLOCK_SIZE], int used)
+static void crng_make_state(u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)],
+ u8 *random_data, size_t random_data_len)
{
unsigned long flags;
- u32 *s, *d;
- int i;
+ struct crng *crng;
+
+ BUG_ON(random_data_len > 32);
+
+ /*
+ * For the fast path, we check whether we're ready, unlocked first, and
+ * then re-check once locked later. In the case where we're really not
+ * ready, we do fast key erasure with the base_crng directly, because
+ * this is what crng_{fast,slow}_load mutate during early init.
+ */
+ if (unlikely(!crng_ready())) {
+ bool ready;
+
+ spin_lock_irqsave(&base_crng.lock, flags);
+ ready = crng_ready();
+ if (!ready)
+ crng_fast_key_erasure(base_crng.key, chacha_state,
+ random_data, random_data_len);
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ if (!ready)
+ return;
+ }
+
+ /*
+ * If the base_crng is more than 5 minutes old, we reseed, which
+ * in turn bumps the generation counter that we check below.
+ */
+ if (unlikely(time_after(jiffies, READ_ONCE(base_crng.birth) + CRNG_RESEED_INTERVAL)))
+ crng_reseed();
+
+ local_irq_save(flags);
+ crng = raw_cpu_ptr(&crngs);
+
+ /*
+ * If our per-cpu crng is older than the base_crng, then it means
+ * somebody reseeded the base_crng. In that case, we do fast key
+ * erasure on the base_crng, and use its output as the new key
+ * for our per-cpu crng. This brings us up to date with base_crng.
+ */
+ if (unlikely(crng->generation != READ_ONCE(base_crng.generation))) {
+ spin_lock(&base_crng.lock);
+ crng_fast_key_erasure(base_crng.key, chacha_state,
+ crng->key, sizeof(crng->key));
+ crng->generation = base_crng.generation;
+ spin_unlock(&base_crng.lock);
+ }
+
+ /*
+ * Finally, when we've made it this far, our per-cpu crng has an up
+ * to date key, and we can do fast key erasure with it to produce
+ * some random data and a ChaCha state for the caller. All other
+ * branches of this function are "unlikely", so most of the time we
+ * should wind up here immediately.
+ */
+ crng_fast_key_erasure(crng->key, chacha_state, random_data, random_data_len);
+ local_irq_restore(flags);
+}
+
+static ssize_t get_random_bytes_user(void __user *buf, size_t nbytes)
+{
+ bool large_request = nbytes > 256;
+ ssize_t ret = 0, len;
+ u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
+ u8 output[CHACHA20_BLOCK_SIZE];
+
+ if (!nbytes)
+ return 0;
- used = round_up(used, sizeof(u32));
- if (used + CHACHA20_KEY_SIZE > CHACHA20_BLOCK_SIZE) {
- extract_crng(tmp);
- used = 0;
- }
- spin_lock_irqsave(&primary_crng.lock, flags);
- s = (u32 *)&tmp[used];
- d = &primary_crng.state[4];
- for (i = 0; i < 8; i++)
- *d++ ^= *s++;
- spin_unlock_irqrestore(&primary_crng.lock, flags);
-}
-
-static ssize_t extract_crng_user(void __user *buf, size_t nbytes)
-{
- ssize_t ret = 0, i = CHACHA20_BLOCK_SIZE;
- u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
- int large_request = (nbytes > 256);
+ len = min_t(ssize_t, 32, nbytes);
+ crng_make_state(chacha_state, output, len);
+
+ if (copy_to_user(buf, output, len))
+ return -EFAULT;
+ nbytes -= len;
+ buf += len;
+ ret += len;
while (nbytes) {
if (large_request && need_resched()) {
- if (signal_pending(current)) {
- if (ret == 0)
- ret = -ERESTARTSYS;
+ if (signal_pending(current))
break;
- }
schedule();
}
- extract_crng(tmp);
- i = min_t(int, nbytes, CHACHA20_BLOCK_SIZE);
- if (copy_to_user(buf, tmp, i)) {
+ chacha20_block(chacha_state, output);
+ if (unlikely(chacha_state[12] == 0))
+ ++chacha_state[13];
+
+ len = min_t(ssize_t, nbytes, CHACHA20_BLOCK_SIZE);
+ if (copy_to_user(buf, output, len)) {
ret = -EFAULT;
break;
}
- nbytes -= i;
- buf += i;
- ret += i;
+ nbytes -= len;
+ buf += len;
+ ret += len;
}
- crng_backtrack_protect(tmp, i);
-
- /* Wipe data just written to memory */
- memzero_explicit(tmp, sizeof(tmp));
+ memzero_explicit(chacha_state, sizeof(chacha_state));
+ memzero_explicit(output, sizeof(output));
return ret;
}
@@ -902,6 +940,10 @@ void add_interrupt_randomness(int irq)
crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)) > 0) {
fast_pool->count = 0;
fast_pool->last = now;
+ if (spin_trylock(&input_pool.lock)) {
+ _mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool));
+ spin_unlock(&input_pool.lock);
+ }
}
return;
}
@@ -1024,23 +1066,36 @@ static void _warn_unseeded_randomness(co
*/
static void _get_random_bytes(void *buf, int nbytes)
{
- u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
+ u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
+ u8 tmp[CHACHA20_BLOCK_SIZE];
+ ssize_t len;
trace_get_random_bytes(nbytes, _RET_IP_);
- while (nbytes >= CHACHA20_BLOCK_SIZE) {
- extract_crng(buf);
- buf += CHACHA20_BLOCK_SIZE;
+ if (!nbytes)
+ return;
+
+ len = min_t(ssize_t, 32, nbytes);
+ crng_make_state(chacha_state, buf, len);
+ nbytes -= len;
+ buf += len;
+
+ while (nbytes) {
+ if (nbytes < CHACHA20_BLOCK_SIZE) {
+ chacha20_block(chacha_state, tmp);
+ memcpy(buf, tmp, nbytes);
+ memzero_explicit(tmp, sizeof(tmp));
+ break;
+ }
+
+ chacha20_block(chacha_state, buf);
+ if (unlikely(chacha_state[12] == 0))
+ ++chacha_state[13];
nbytes -= CHACHA20_BLOCK_SIZE;
+ buf += CHACHA20_BLOCK_SIZE;
}
- if (nbytes > 0) {
- extract_crng(tmp);
- memcpy(buf, tmp, nbytes);
- crng_backtrack_protect(tmp, nbytes);
- } else
- crng_backtrack_protect(tmp, CHACHA20_BLOCK_SIZE);
- memzero_explicit(tmp, sizeof(tmp));
+ memzero_explicit(chacha_state, sizeof(chacha_state));
}
void get_random_bytes(void *buf, int nbytes)
@@ -1271,13 +1326,12 @@ int __init rand_initialize(void)
mix_pool_bytes(&now, sizeof(now));
mix_pool_bytes(utsname(), sizeof(*(utsname())));
- extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
+ extract_entropy(base_crng.key, sizeof(base_crng.key));
if (arch_init && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
crng_init = 2;
pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
- primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
if (ratelimit_disable) {
urandom_warning.interval = 0;
@@ -1309,7 +1363,7 @@ static ssize_t urandom_read_nowarn(struc
int ret;
nbytes = min_t(size_t, nbytes, INT_MAX >> 6);
- ret = extract_crng_user(buf, nbytes);
+ ret = get_random_bytes_user(buf, nbytes);
trace_urandom_read(8 * nbytes, 0, input_pool.entropy_count);
return ret;
}
@@ -1613,8 +1667,15 @@ static atomic_t batch_generation = ATOMI
struct batched_entropy {
union {
- u64 entropy_u64[CHACHA20_BLOCK_SIZE / sizeof(u64)];
- u32 entropy_u32[CHACHA20_BLOCK_SIZE / sizeof(u32)];
+ /*
+ * We make this 1.5x a ChaCha block, so that we get the
+ * remaining 32 bytes from fast key erasure, plus one full
+ * block from the detached ChaCha state. We can increase
+ * the size of this later if needed so long as we keep the
+ * formula of (integer_blocks + 0.5) * CHACHA20_BLOCK_SIZE.
+ */
+ u64 entropy_u64[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u64))];
+ u32 entropy_u32[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u32))];
};
unsigned int position;
int generation;
@@ -1622,13 +1683,13 @@ struct batched_entropy {
/*
* Get a random word for internal kernel use only. The quality of the random
- * number is good as /dev/urandom, but there is no backtrack protection, with
- * the goal of being quite fast and not depleting entropy. In order to ensure
- * that the randomness provided by this function is okay, the function
- * wait_for_random_bytes() should be called and return 0 at least once at any
- * point prior.
+ * number is good as /dev/urandom. In order to ensure that the randomness
+ * provided by this function is okay, the function wait_for_random_bytes()
+ * should be called and return 0 at least once at any point prior.
*/
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64);
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
+ .position = UINT_MAX
+};
u64 get_random_u64(void)
{
@@ -1644,20 +1705,24 @@ u64 get_random_u64(void)
batch = raw_cpu_ptr(&batched_entropy_u64);
next_gen = atomic_read(&batch_generation);
- if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0 ||
+ if (batch->position >= ARRAY_SIZE(batch->entropy_u64) ||
next_gen != batch->generation) {
- extract_crng((u8 *)batch->entropy_u64);
+ _get_random_bytes(batch->entropy_u64, sizeof(batch->entropy_u64));
batch->position = 0;
batch->generation = next_gen;
}
- ret = batch->entropy_u64[batch->position++];
+ ret = batch->entropy_u64[batch->position];
+ batch->entropy_u64[batch->position] = 0;
+ ++batch->position;
local_irq_restore(flags);
return ret;
}
EXPORT_SYMBOL(get_random_u64);
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32);
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
+ .position = UINT_MAX
+};
u32 get_random_u32(void)
{
@@ -1673,14 +1738,16 @@ u32 get_random_u32(void)
batch = raw_cpu_ptr(&batched_entropy_u32);
next_gen = atomic_read(&batch_generation);
- if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0 ||
+ if (batch->position >= ARRAY_SIZE(batch->entropy_u32) ||
next_gen != batch->generation) {
- extract_crng((u8 *)batch->entropy_u32);
+ _get_random_bytes(batch->entropy_u32, sizeof(batch->entropy_u32));
batch->position = 0;
batch->generation = next_gen;
}
- ret = batch->entropy_u32[batch->position++];
+ ret = batch->entropy_u32[batch->position];
+ batch->entropy_u32[batch->position] = 0;
+ ++batch->position;
local_irq_restore(flags);
return ret;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 18263c4e8e62f7329f38f5eadc568751242ca89c upstream.
The entropy estimator is calculated in terms of 1/8 bits, which means
there are various constants where things are shifted by 3. Move these
into our pool info enum with the other relevant constants. While we're
at it, move an English assertion about sizes into a proper BUILD_BUG_ON
so that the compiler can ensure this invariant.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 28 +++++++++++++---------------
1 file changed, 13 insertions(+), 15 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -360,16 +360,6 @@
/* #define ADD_INTERRUPT_BENCH */
/*
- * To allow fractional bits to be tracked, the entropy_count field is
- * denominated in units of 1/8th bits.
- *
- * 2*(POOL_ENTROPY_SHIFT + poolbitshift) must <= 31, or the multiply in
- * credit_entropy_bits() needs to be 64 bits wide.
- */
-#define POOL_ENTROPY_SHIFT 3
-#define POOL_ENTROPY_BITS() (input_pool.entropy_count >> POOL_ENTROPY_SHIFT)
-
-/*
* If the entropy count falls under this number of bits, then we
* should wake up processes which are selecting or polling on write
* access to /dev/random.
@@ -426,8 +416,13 @@ enum poolinfo {
POOL_WORDMASK = POOL_WORDS - 1,
POOL_BYTES = POOL_WORDS * sizeof(u32),
POOL_BITS = POOL_BYTES * 8,
- POOL_BITSHIFT = ilog2(POOL_WORDS) + 5,
- POOL_FRACBITS = POOL_WORDS << (POOL_ENTROPY_SHIFT + 5),
+ POOL_BITSHIFT = ilog2(POOL_BITS),
+
+ /* To allow fractional bits to be tracked, the entropy_count field is
+ * denominated in units of 1/8th bits. */
+ POOL_ENTROPY_SHIFT = 3,
+#define POOL_ENTROPY_BITS() (input_pool.entropy_count >> POOL_ENTROPY_SHIFT)
+ POOL_FRACBITS = POOL_BITS << POOL_ENTROPY_SHIFT,
/* x^128 + x^104 + x^76 + x^51 +x^25 + x + 1 */
POOL_TAP1 = 104,
@@ -653,6 +648,9 @@ static void credit_entropy_bits(int nbit
int entropy_count, entropy_bits, orig;
int nfrac = nbits << POOL_ENTROPY_SHIFT;
+ /* Ensure that the multiplication can avoid being 64 bits wide. */
+ BUILD_BUG_ON(2 * (POOL_ENTROPY_SHIFT + POOL_BITSHIFT) > 31);
+
if (!nbits)
return;
@@ -688,13 +686,13 @@ retry:
/* The +2 corresponds to the /4 in the denominator */
do {
- unsigned int anfrac = min(pnfrac, POOL_FRACBITS/2);
+ unsigned int anfrac = min(pnfrac, POOL_FRACBITS / 2);
unsigned int add =
- ((POOL_FRACBITS - entropy_count)*anfrac*3) >> s;
+ ((POOL_FRACBITS - entropy_count) * anfrac * 3) >> s;
entropy_count += add;
pnfrac -= anfrac;
- } while (unlikely(entropy_count < POOL_FRACBITS-2 && pnfrac));
+ } while (unlikely(entropy_count < POOL_FRACBITS - 2 && pnfrac));
}
if (WARN_ON(entropy_count < 0)) {
From: Schspa Shi <[email protected]>
commit c0a8a61e7abbf66729687ee63659ee25983fbb1e upstream.
s/or/for
Signed-off-by: Schspa Shi <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -101,7 +101,7 @@
* ===============================
*
* There are four exported interfaces; two for use within the kernel,
- * and two or use from userspace.
+ * and two for use from userspace.
*
* Exported interfaces ---- userspace output
* -----------------------------------------
From: "Jason A. Donenfeld" <[email protected]>
commit 1daf2f387652bf3a7044aea042f5023b3f6b189b upstream.
This has no real functional change, as crng_pre_init_inject() (and
before that, crng_slow_init()) always checks for == 0, not >= 2. So
correct the outer unlocked change to reflect that. Before this used
crng_ready(), which was not correct.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1016,7 +1016,7 @@ void add_device_randomness(const void *b
unsigned long time = random_get_entropy() ^ jiffies;
unsigned long flags;
- if (!crng_ready() && size)
+ if (crng_init == 0 && size)
crng_pre_init_inject(buf, size, false, false);
spin_lock_irqsave(&input_pool.lock, flags);
From: "Jason A. Donenfeld" <[email protected]>
commit da792c6d5f59a76c10a310c5d4c93428fd18f996 upstream.
crng_fast_load() and crng_slow_load() have different semantics:
- crng_fast_load() xors and accounts with crng_init_cnt.
- crng_slow_load() hashes and doesn't account.
However add_hwgenerator_randomness() can afford to hash (it's called
from a kthread), and it should account. Additionally, ones that can
afford to hash don't need to take a trylock but can take a normal lock.
So, we combine these into one function, crng_pre_init_inject(), which
allows us to control these in a uniform way. This will make it simpler
later to simplify this all down when the time comes for that.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 114 +++++++++++++++++++++++++-------------------------
1 file changed, 59 insertions(+), 55 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -385,7 +385,7 @@ static void crng_make_state(u32 chacha_s
* For the fast path, we check whether we're ready, unlocked first, and
* then re-check once locked later. In the case where we're really not
* ready, we do fast key erasure with the base_crng directly, because
- * this is what crng_{fast,slow}_load mutate during early init.
+ * this is what crng_pre_init_inject() mutates during early init.
*/
if (unlikely(!crng_ready())) {
bool ready;
@@ -436,72 +436,75 @@ static void crng_make_state(u32 chacha_s
}
/*
- * This function is for crng_init == 0 only.
+ * This function is for crng_init == 0 only. It loads entropy directly
+ * into the crng's key, without going through the input pool. It is,
+ * generally speaking, not very safe, but we use this only at early
+ * boot time when it's better to have something there rather than
+ * nothing.
+ *
+ * There are two paths, a slow one and a fast one. The slow one
+ * hashes the input along with the current key. The fast one simply
+ * xors it in, and should only be used from interrupt context.
+ *
+ * If account is set, then the crng_init_cnt counter is incremented.
+ * This shouldn't be set by functions like add_device_randomness(),
+ * where we can't trust the buffer passed to it is guaranteed to be
+ * unpredictable (so it might not have any entropy at all).
*
- * crng_fast_load() can be called by code in the interrupt service
- * path. So we can't afford to dilly-dally. Returns the number of
- * bytes processed from cp.
+ * Returns the number of bytes processed from input, which is bounded
+ * by CRNG_INIT_CNT_THRESH if account is true.
*/
-static size_t crng_fast_load(const void *cp, size_t len)
+static size_t crng_pre_init_inject(const void *input, size_t len,
+ bool fast, bool account)
{
static int crng_init_cnt = 0;
unsigned long flags;
- const u8 *src = (const u8 *)cp;
- size_t ret = 0;
- if (!spin_trylock_irqsave(&base_crng.lock, flags))
- return 0;
+ if (fast) {
+ if (!spin_trylock_irqsave(&base_crng.lock, flags))
+ return 0;
+ } else {
+ spin_lock_irqsave(&base_crng.lock, flags);
+ }
+
if (crng_init != 0) {
spin_unlock_irqrestore(&base_crng.lock, flags);
return 0;
}
- while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
- base_crng.key[crng_init_cnt % sizeof(base_crng.key)] ^= *src;
- src++; crng_init_cnt++; len--; ret++;
- }
- if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
- ++base_crng.generation;
- crng_init = 1;
- }
- spin_unlock_irqrestore(&base_crng.lock, flags);
- if (crng_init == 1)
- pr_notice("fast init done\n");
- return ret;
-}
-/*
- * This function is for crng_init == 0 only.
- *
- * crng_slow_load() is called by add_device_randomness, which has two
- * attributes. (1) We can't trust the buffer passed to it is
- * guaranteed to be unpredictable (so it might not have any entropy at
- * all), and (2) it doesn't have the performance constraints of
- * crng_fast_load().
- *
- * So, we simply hash the contents in with the current key. Finally,
- * we do *not* advance crng_init_cnt since buffer we may get may be
- * something like a fixed DMI table (for example), which might very
- * well be unique to the machine, but is otherwise unvarying.
- */
-static void crng_slow_load(const void *cp, size_t len)
-{
- unsigned long flags;
- struct blake2s_state hash;
+ if (account)
+ len = min_t(size_t, len, CRNG_INIT_CNT_THRESH - crng_init_cnt);
- blake2s_init(&hash, sizeof(base_crng.key));
-
- if (!spin_trylock_irqsave(&base_crng.lock, flags))
- return;
- if (crng_init != 0) {
- spin_unlock_irqrestore(&base_crng.lock, flags);
- return;
+ if (fast) {
+ const u8 *src = input;
+ size_t i;
+
+ for (i = 0; i < len; ++i)
+ base_crng.key[(crng_init_cnt + i) %
+ sizeof(base_crng.key)] ^= src[i];
+ } else {
+ struct blake2s_state hash;
+
+ blake2s_init(&hash, sizeof(base_crng.key));
+ blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
+ blake2s_update(&hash, input, len);
+ blake2s_final(&hash, base_crng.key);
+ }
+
+ if (account) {
+ crng_init_cnt += len;
+ if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
+ ++base_crng.generation;
+ crng_init = 1;
+ }
}
- blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
- blake2s_update(&hash, cp, len);
- blake2s_final(&hash, base_crng.key);
-
spin_unlock_irqrestore(&base_crng.lock, flags);
+
+ if (crng_init == 1)
+ pr_notice("fast init done\n");
+
+ return len;
}
static void _get_random_bytes(void *buf, size_t nbytes)
@@ -1014,7 +1017,7 @@ void add_device_randomness(const void *b
unsigned long flags;
if (!crng_ready() && size)
- crng_slow_load(buf, size);
+ crng_pre_init_inject(buf, size, false, false);
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(buf, size);
@@ -1131,7 +1134,7 @@ void add_hwgenerator_randomness(const vo
size_t entropy)
{
if (unlikely(crng_init == 0)) {
- size_t ret = crng_fast_load(buffer, count);
+ size_t ret = crng_pre_init_inject(buffer, count, false, true);
mix_pool_bytes(buffer, ret);
count -= ret;
buffer += ret;
@@ -1294,7 +1297,8 @@ void add_interrupt_randomness(int irq)
if (unlikely(crng_init == 0)) {
if (new_count >= 64 &&
- crng_fast_load(fast_pool->pool32, sizeof(fast_pool->pool32)) > 0) {
+ crng_pre_init_inject(fast_pool->pool32, sizeof(fast_pool->pool32),
+ true, true) > 0) {
atomic_set(&fast_pool->count, 0);
fast_pool->last = now;
if (spin_trylock(&input_pool.lock)) {
From: "Jason A. Donenfeld" <[email protected]>
commit d38bb0853589c939573ea50e9cb64f733e0e273d upstream.
Rather than using the userspace type, __uXX, switch to using uXX. And
rather than using variously chosen `char *` or `unsigned char *`, use
`u8 *` uniformly for things that aren't strings, in the case where we
are doing byte-by-byte traversal.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 114 +++++++++++++++++++++++++-------------------------
1 file changed, 57 insertions(+), 57 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -458,7 +458,7 @@ static DEFINE_SPINLOCK(random_ready_list
static LIST_HEAD(random_ready_list);
struct crng_state {
- __u32 state[16];
+ u32 state[16];
unsigned long init_time;
spinlock_t lock;
};
@@ -485,10 +485,9 @@ static bool crng_need_final_init = false
static int crng_init_cnt = 0;
static unsigned long crng_global_init_time = 0;
#define CRNG_INIT_CNT_THRESH (2*CHACHA20_KEY_SIZE)
-static void _extract_crng(struct crng_state *crng,
- __u8 out[CHACHA20_BLOCK_SIZE]);
+static void _extract_crng(struct crng_state *crng, u8 out[CHACHA20_BLOCK_SIZE]);
static void _crng_backtrack_protect(struct crng_state *crng,
- __u8 tmp[CHACHA20_BLOCK_SIZE], int used);
+ u8 tmp[CHACHA20_BLOCK_SIZE], int used);
static void process_random_ready_list(void);
static void _get_random_bytes(void *buf, int nbytes);
@@ -512,16 +511,16 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
struct entropy_store;
struct entropy_store {
/* read-only data: */
- __u32 *pool;
+ u32 *pool;
const char *name;
/* read-write data: */
spinlock_t lock;
- unsigned short add_ptr;
- unsigned short input_rotate;
+ u16 add_ptr;
+ u16 input_rotate;
int entropy_count;
unsigned int last_data_init:1;
- __u8 last_data[EXTRACT_SIZE];
+ u8 last_data[EXTRACT_SIZE];
};
static ssize_t extract_entropy(struct entropy_store *r, void *buf,
@@ -530,7 +529,7 @@ static ssize_t _extract_entropy(struct e
size_t nbytes, int fips);
static void crng_reseed(struct crng_state *crng, struct entropy_store *r);
-static __u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
+static u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
static struct entropy_store input_pool = {
.name = "input",
@@ -538,7 +537,7 @@ static struct entropy_store input_pool =
.pool = input_pool_data
};
-static __u32 const twist_table[8] = {
+static u32 const twist_table[8] = {
0x00000000, 0x3b6e20c8, 0x76dc4190, 0x4db26158,
0xedb88320, 0xd6d6a3e8, 0x9b64c2b0, 0xa00ae278 };
@@ -557,8 +556,8 @@ static void _mix_pool_bytes(struct entro
{
unsigned long i;
int input_rotate;
- const unsigned char *bytes = in;
- __u32 w;
+ const u8 *bytes = in;
+ u32 w;
input_rotate = r->input_rotate;
i = r->add_ptr;
@@ -611,10 +610,10 @@ static void mix_pool_bytes(struct entrop
}
struct fast_pool {
- __u32 pool[4];
+ u32 pool[4];
unsigned long last;
- unsigned short reg_idx;
- unsigned char count;
+ u16 reg_idx;
+ u8 count;
};
/*
@@ -624,8 +623,8 @@ struct fast_pool {
*/
static void fast_mix(struct fast_pool *f)
{
- __u32 a = f->pool[0], b = f->pool[1];
- __u32 c = f->pool[2], d = f->pool[3];
+ u32 a = f->pool[0], b = f->pool[1];
+ u32 c = f->pool[2], d = f->pool[3];
a += b; c += d;
b = rol32(b, 6); d = rol32(d, 27);
@@ -816,14 +815,14 @@ static bool __init crng_init_try_arch_ea
static void crng_initialize_secondary(struct crng_state *crng)
{
chacha_init_consts(crng->state);
- _get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
+ _get_random_bytes(&crng->state[4], sizeof(u32) * 12);
crng_init_try_arch(crng);
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
static void __init crng_initialize_primary(struct crng_state *crng)
{
- _extract_entropy(&input_pool, &crng->state[4], sizeof(__u32) * 12, 0);
+ _extract_entropy(&input_pool, &crng->state[4], sizeof(u32) * 12, 0);
if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
@@ -910,12 +909,14 @@ static struct crng_state *select_crng(vo
/*
* crng_fast_load() can be called by code in the interrupt service
- * path. So we can't afford to dilly-dally.
+ * path. So we can't afford to dilly-dally. Returns the number of
+ * bytes processed from cp.
*/
-static int crng_fast_load(const char *cp, size_t len)
+static size_t crng_fast_load(const u8 *cp, size_t len)
{
unsigned long flags;
- char *p;
+ u8 *p;
+ size_t ret = 0;
if (!spin_trylock_irqsave(&primary_crng.lock, flags))
return 0;
@@ -923,10 +924,10 @@ static int crng_fast_load(const char *cp
spin_unlock_irqrestore(&primary_crng.lock, flags);
return 0;
}
- p = (unsigned char *) &primary_crng.state[4];
+ p = (u8 *) &primary_crng.state[4];
while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
p[crng_init_cnt % CHACHA20_KEY_SIZE] ^= *cp;
- cp++; crng_init_cnt++; len--;
+ cp++; crng_init_cnt++; len--; ret++;
}
spin_unlock_irqrestore(&primary_crng.lock, flags);
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
@@ -934,7 +935,7 @@ static int crng_fast_load(const char *cp
crng_init = 1;
pr_notice("fast init done\n");
}
- return 1;
+ return ret;
}
#ifdef CONFIG_NUMA
@@ -1002,14 +1003,14 @@ static struct crng_state *select_crng(vo
* like a fixed DMI table (for example), which might very well be
* unique to the machine, but is otherwise unvarying.
*/
-static int crng_slow_load(const char *cp, size_t len)
+static int crng_slow_load(const u8 *cp, size_t len)
{
unsigned long flags;
- static unsigned char lfsr = 1;
- unsigned char tmp;
- unsigned i, max = CHACHA20_KEY_SIZE;
- const char * src_buf = cp;
- char * dest_buf = (char *) &primary_crng.state[4];
+ static u8 lfsr = 1;
+ u8 tmp;
+ unsigned int i, max = CHACHA20_KEY_SIZE;
+ const u8 * src_buf = cp;
+ u8 * dest_buf = (u8 *) &primary_crng.state[4];
if (!spin_trylock_irqsave(&primary_crng.lock, flags))
return 0;
@@ -1038,8 +1039,8 @@ static void crng_reseed(struct crng_stat
unsigned long flags;
int i, num;
union {
- __u8 block[CHACHA20_BLOCK_SIZE];
- __u32 key[8];
+ u8 block[CHACHA20_BLOCK_SIZE];
+ u32 key[8];
} buf;
if (r) {
@@ -1066,7 +1067,7 @@ static void crng_reseed(struct crng_stat
}
static void _extract_crng(struct crng_state *crng,
- __u8 out[CHACHA20_BLOCK_SIZE])
+ u8 out[CHACHA20_BLOCK_SIZE])
{
unsigned long flags, init_time;
@@ -1084,7 +1085,7 @@ static void _extract_crng(struct crng_st
spin_unlock_irqrestore(&crng->lock, flags);
}
-static void extract_crng(__u8 out[CHACHA20_BLOCK_SIZE])
+static void extract_crng(u8 out[CHACHA20_BLOCK_SIZE])
{
_extract_crng(select_crng(), out);
}
@@ -1094,26 +1095,26 @@ static void extract_crng(__u8 out[CHACHA
* enough) to mutate the CRNG key to provide backtracking protection.
*/
static void _crng_backtrack_protect(struct crng_state *crng,
- __u8 tmp[CHACHA20_BLOCK_SIZE], int used)
+ u8 tmp[CHACHA20_BLOCK_SIZE], int used)
{
unsigned long flags;
- __u32 *s, *d;
+ u32 *s, *d;
int i;
- used = round_up(used, sizeof(__u32));
+ used = round_up(used, sizeof(u32));
if (used + CHACHA20_KEY_SIZE > CHACHA20_BLOCK_SIZE) {
extract_crng(tmp);
used = 0;
}
spin_lock_irqsave(&crng->lock, flags);
- s = (__u32 *) &tmp[used];
+ s = (u32 *) &tmp[used];
d = &crng->state[4];
for (i=0; i < 8; i++)
*d++ ^= *s++;
spin_unlock_irqrestore(&crng->lock, flags);
}
-static void crng_backtrack_protect(__u8 tmp[CHACHA20_BLOCK_SIZE], int used)
+static void crng_backtrack_protect(u8 tmp[CHACHA20_BLOCK_SIZE], int used)
{
_crng_backtrack_protect(select_crng(), tmp, used);
}
@@ -1121,7 +1122,7 @@ static void crng_backtrack_protect(__u8
static ssize_t extract_crng_user(void __user *buf, size_t nbytes)
{
ssize_t ret = 0, i = CHACHA20_BLOCK_SIZE;
- __u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
+ u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
int large_request = (nbytes > 256);
while (nbytes) {
@@ -1209,8 +1210,8 @@ static void add_timer_randomness(struct
struct entropy_store *r;
struct {
long jiffies;
- unsigned cycles;
- unsigned num;
+ unsigned int cycles;
+ unsigned int num;
} sample;
long delta, delta2, delta3;
@@ -1292,15 +1293,15 @@ static void add_interrupt_bench(cycles_t
#define add_interrupt_bench(x)
#endif
-static __u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
+static u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
{
- __u32 *ptr = (__u32 *) regs;
+ u32 *ptr = (u32 *) regs;
unsigned int idx;
if (regs == NULL)
return 0;
idx = READ_ONCE(f->reg_idx);
- if (idx >= sizeof(struct pt_regs) / sizeof(__u32))
+ if (idx >= sizeof(struct pt_regs) / sizeof(u32))
idx = 0;
ptr += idx++;
WRITE_ONCE(f->reg_idx, idx);
@@ -1314,8 +1315,8 @@ void add_interrupt_randomness(int irq)
struct pt_regs *regs = get_irq_regs();
unsigned long now = jiffies;
cycles_t cycles = random_get_entropy();
- __u32 c_high, j_high;
- __u64 ip;
+ u32 c_high, j_high;
+ u64 ip;
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
@@ -1333,8 +1334,7 @@ void add_interrupt_randomness(int irq)
if (unlikely(crng_init == 0)) {
if ((fast_pool->count >= 64) &&
- crng_fast_load((char *) fast_pool->pool,
- sizeof(fast_pool->pool))) {
+ crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)) > 0) {
fast_pool->count = 0;
fast_pool->last = now;
}
@@ -1431,7 +1431,7 @@ retry:
*
* Note: we assume that .poolwords is a multiple of 16 words.
*/
-static void extract_buf(struct entropy_store *r, __u8 *out)
+static void extract_buf(struct entropy_store *r, u8 *out)
{
struct blake2s_state state __aligned(__alignof__(unsigned long));
u8 hash[BLAKE2S_HASH_SIZE];
@@ -1481,7 +1481,7 @@ static ssize_t _extract_entropy(struct e
size_t nbytes, int fips)
{
ssize_t ret = 0, i;
- __u8 tmp[EXTRACT_SIZE];
+ u8 tmp[EXTRACT_SIZE];
unsigned long flags;
while (nbytes) {
@@ -1519,7 +1519,7 @@ static ssize_t _extract_entropy(struct e
static ssize_t extract_entropy(struct entropy_store *r, void *buf,
size_t nbytes, int min, int reserved)
{
- __u8 tmp[EXTRACT_SIZE];
+ u8 tmp[EXTRACT_SIZE];
unsigned long flags;
/* if last_data isn't primed, we need EXTRACT_SIZE extra bytes */
@@ -1580,7 +1580,7 @@ static void _warn_unseeded_randomness(co
*/
static void _get_random_bytes(void *buf, int nbytes)
{
- __u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
+ u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
trace_get_random_bytes(nbytes, _RET_IP_);
@@ -1714,7 +1714,7 @@ EXPORT_SYMBOL(del_random_ready_callback)
int __must_check get_random_bytes_arch(void *buf, int nbytes)
{
int left = nbytes;
- char *p = buf;
+ u8 *p = buf;
trace_get_random_bytes_arch(left, _RET_IP_);
while (left) {
@@ -1856,7 +1856,7 @@ static int
write_pool(struct entropy_store *r, const char __user *buffer, size_t count)
{
size_t bytes;
- __u32 t, buf[16];
+ u32 t, buf[16];
const char __user *p = buffer;
while (count > 0) {
@@ -1866,7 +1866,7 @@ write_pool(struct entropy_store *r, cons
if (copy_from_user(&buf, p, bytes))
return -EFAULT;
- for (b = bytes ; b > 0 ; b -= sizeof(__u32), i++) {
+ for (b = bytes; b > 0; b -= sizeof(u32), i++) {
if (!arch_get_random_int(&t))
break;
buf[i] ^= t;
From: Stephen Boyd <[email protected]>
commit 59b569480dc8bb9dce57cdff133853a842dfd805 upstream.
Sebastian reports that after commit ff296293b353 ("random: Support freezable
kthreads in add_hwgenerator_randomness()") we can call might_sleep() when the
task state is TASK_INTERRUPTIBLE (state=1). This leads to the following warning.
do not call blocking ops when !TASK_RUNNING; state=1 set at [<00000000349d1489>] prepare_to_wait_event+0x5a/0x180
WARNING: CPU: 0 PID: 828 at kernel/sched/core.c:6741 __might_sleep+0x6f/0x80
Modules linked in:
CPU: 0 PID: 828 Comm: hwrng Not tainted 5.3.0-rc7-next-20190903+ #46
RIP: 0010:__might_sleep+0x6f/0x80
Call Trace:
kthread_freezable_should_stop+0x1b/0x60
add_hwgenerator_randomness+0xdd/0x130
hwrng_fillfn+0xbf/0x120
kthread+0x10c/0x140
ret_from_fork+0x27/0x50
We shouldn't call kthread_freezable_should_stop() from deep within the
wait_event code because the task state is still set as
TASK_INTERRUPTIBLE instead of TASK_RUNNING and
kthread_freezable_should_stop() will try to call into the freezer with
the task in the wrong state. Use wait_event_freezable() instead so that
it calls schedule() in the right place and tries to enter the freezer
when the task state is TASK_RUNNING instead.
Reported-by: Sebastian Andrzej Siewior <[email protected]>
Tested-by: Sebastian Andrzej Siewior <[email protected]>
Cc: Keerthy <[email protected]>
Fixes: ff296293b353 ("random: Support freezable kthreads in add_hwgenerator_randomness()")
Signed-off-by: Stephen Boyd <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -327,6 +327,7 @@
#include <linux/percpu.h>
#include <linux/cryptohash.h>
#include <linux/fips.h>
+#include <linux/freezer.h>
#include <linux/ptrace.h>
#include <linux/kmemcheck.h>
#include <linux/workqueue.h>
@@ -2473,7 +2474,6 @@ void add_hwgenerator_randomness(const ch
size_t entropy)
{
struct entropy_store *poolp = &input_pool;
- bool frozen = false;
if (unlikely(crng_init == 0)) {
crng_fast_load(buffer, count);
@@ -2484,13 +2484,11 @@ void add_hwgenerator_randomness(const ch
* We'll be woken up again once below random_write_wakeup_thresh,
* or when the calling thread is about to terminate.
*/
- wait_event_interruptible(random_write_wait,
- kthread_freezable_should_stop(&frozen) ||
+ wait_event_freezable(random_write_wait,
+ kthread_should_stop() ||
ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
- if (!frozen) {
- mix_pool_bytes(poolp, buffer, count);
- credit_entropy_bits(poolp, entropy);
- }
+ mix_pool_bytes(poolp, buffer, count);
+ credit_entropy_bits(poolp, entropy);
}
EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
From: Borislav Petkov <[email protected]>
commit 3fd57e7a9e66b9a8bcbf0560ff09e84d0b8de1bd upstream.
On Tue, Oct 01, 2019 at 10:14:40AM -0700, Linus Torvalds wrote:
> The previous state of the file didn't have that 0xa at the end, so you get that
>
>
> -EXPORT_SYMBOL_GPL(add_bootloader_randomness);
> \ No newline at end of file
> +EXPORT_SYMBOL_GPL(add_bootloader_randomness);
>
> which is "the '-' line doesn't have a newline, the '+' line does" marker.
Aaha, that makes total sense, thanks for explaining. Oh well, let's fix
it then so that people don't scratch heads like me.
Signed-off-by: Linus Torvalds <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2504,4 +2504,4 @@ void add_bootloader_randomness(const voi
else
add_device_randomness(buf, size);
}
-EXPORT_SYMBOL_GPL(add_bootloader_randomness);
\ No newline at end of file
+EXPORT_SYMBOL_GPL(add_bootloader_randomness);
From: Harald Freudenberger <[email protected]>
commit 2bbb6983887fefc8026beab01198d30f47b7bd22 upstream.
This patch rewoks the hwrng to always use the
rng source with best entropy quality.
On registation and unregistration the hwrng now
tries to choose the best (= highest quality value)
rng source. The handling of the internal list
of registered rng sources is now always sorted
by quality and the top most rng chosen.
Signed-off-by: Harald Freudenberger <[email protected]>
Reviewed-by: PrasannaKumar Muralidharan <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -28,6 +28,7 @@
static struct hwrng *current_rng;
static struct task_struct *hwrng_fill;
+/* list of registered rngs, sorted decending by quality */
static LIST_HEAD(rng_list);
/* Protects rng_list and current_rng */
static DEFINE_MUTEX(rng_mutex);
@@ -416,6 +417,7 @@ int hwrng_register(struct hwrng *rng)
{
int err = -EINVAL;
struct hwrng *old_rng, *tmp;
+ struct list_head *rng_list_ptr;
if (!rng->name || (!rng->data_read && !rng->read))
goto out;
@@ -431,14 +433,25 @@ int hwrng_register(struct hwrng *rng)
init_completion(&rng->cleanup_done);
complete(&rng->cleanup_done);
+ /* rng_list is sorted by decreasing quality */
+ list_for_each(rng_list_ptr, &rng_list) {
+ tmp = list_entry(rng_list_ptr, struct hwrng, list);
+ if (tmp->quality < rng->quality)
+ break;
+ }
+ list_add_tail(&rng->list, rng_list_ptr);
+
old_rng = current_rng;
err = 0;
- if (!old_rng) {
+ if (!old_rng || (rng->quality > old_rng->quality)) {
+ /*
+ * Set new rng as current as the new rng source
+ * provides better entropy quality.
+ */
err = set_current_rng(rng);
if (err)
goto out_unlock;
}
- list_add_tail(&rng->list, &rng_list);
if (old_rng && !rng->init) {
/*
@@ -465,12 +478,12 @@ void hwrng_unregister(struct hwrng *rng)
list_del(&rng->list);
if (current_rng == rng) {
drop_current_rng();
+ /* rng_list is sorted by quality, use the best (=first) one */
if (!list_empty(&rng_list)) {
- struct hwrng *tail;
-
- tail = list_entry(rng_list.prev, struct hwrng, list);
+ struct hwrng *new_rng;
- set_current_rng(tail);
+ new_rng = list_entry(rng_list.next, struct hwrng, list);
+ set_current_rng(new_rng);
}
}
From: Corentin LABBE <[email protected]>
commit fd50d71f94fb1c8614098949db068cd4c8dbb91d upstream.
This patch move the define for hwrng's miscdev minor number to
include/linux/miscdevice.h.
It's better that all minor number are in the same place.
Rename it to HWRNG_MINOR (from RNG_MISCDEV_MINOR) in he process since
no other miscdev define have MISCDEV in their name.
Signed-off-by: Corentin Labbe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 3 +--
include/linux/miscdevice.h | 1 +
2 files changed, 2 insertions(+), 2 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -26,7 +26,6 @@
#define RNG_MODULE_NAME "hw_random"
#define PFX RNG_MODULE_NAME ": "
-#define RNG_MISCDEV_MINOR 183 /* official */
static struct hwrng *current_rng;
static struct task_struct *hwrng_fill;
@@ -283,7 +282,7 @@ static const struct file_operations rng_
static const struct attribute_group *rng_dev_groups[];
static struct miscdevice rng_miscdev = {
- .minor = RNG_MISCDEV_MINOR,
+ .minor = HWRNG_MINOR,
.name = RNG_MODULE_NAME,
.nodename = "hwrng",
.fops = &rng_chrdev_ops,
--- a/include/linux/miscdevice.h
+++ b/include/linux/miscdevice.h
@@ -31,6 +31,7 @@
#define SGI_MMTIMER 153
#define STORE_QUEUE_MINOR 155 /* unused */
#define I2O_MINOR 166
+#define HWRNG_MINOR 183
#define MICROCODE_MINOR 184
#define VFIO_MINOR 196
#define TUN_MINOR 200
From: Corentin LABBE <[email protected]>
commit dd8014830d2b1fdf5328978ada706df3ec180c21 upstream.
checkpatch have lot of complaint about header.
Furthermore, the header have some offtopic/useless information.
This patch rewrite a proper header.
Signed-off-by: Corentin Labbe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 38 +++++++++-----------------------------
1 file changed, 9 insertions(+), 29 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -1,33 +1,13 @@
/*
- Added support for the AMD Geode LX RNG
- (c) Copyright 2004-2005 Advanced Micro Devices, Inc.
-
- derived from
-
- Hardware driver for the Intel/AMD/VIA Random Number Generators (RNG)
- (c) Copyright 2003 Red Hat Inc <[email protected]>
-
- derived from
-
- Hardware driver for the AMD 768 Random Number Generator (RNG)
- (c) Copyright 2001 Red Hat Inc <[email protected]>
-
- derived from
-
- Hardware driver for Intel i810 Random Number Generator (RNG)
- Copyright 2000,2001 Jeff Garzik <[email protected]>
- Copyright 2000,2001 Philipp Rumpf <[email protected]>
-
- Added generic RNG API
- Copyright 2006 Michael Buesch <[email protected]>
- Copyright 2005 (c) MontaVista Software, Inc.
-
- Please read Documentation/hw_random.txt for details on use.
-
- ----------------------------------------------------------
- This software may be used and distributed according to the terms
- of the GNU General Public License, incorporated herein by reference.
-
+ * hw_random/core.c: HWRNG core API
+ *
+ * Copyright 2006 Michael Buesch <[email protected]>
+ * Copyright 2005 (c) MontaVista Software, Inc.
+ *
+ * Please read Documentation/hw_random.txt for details on use.
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License, incorporated herein by reference.
*/
#include <linux/device.h>
From: Hsin-Yi Wang <[email protected]>
commit 428826f5358c922dc378830a1717b682c0823160 upstream.
Introducing a chosen node, rng-seed, which is an entropy that can be
passed to kernel called very early to increase initial device
randomness. Bootloader should provide this entropy and the value is
read from /chosen/rng-seed in DT.
Obtain of_fdt_crc32 for CRC check after early_init_dt_scan_nodes(),
since early_init_dt_scan_chosen() would modify fdt to erase rng-seed.
Add a new interface add_bootloader_randomness() for rng-seed use case.
Depends on whether the seed is trustworthy, rng seed would be passed to
add_hwgenerator_randomness(). Otherwise it would be passed to
add_device_randomness(). Decision is controlled by kernel config
RANDOM_TRUST_BOOTLOADER.
Signed-off-by: Hsin-Yi Wang <[email protected]>
Reviewed-by: Stephen Boyd <[email protected]>
Reviewed-by: Rob Herring <[email protected]>
Reviewed-by: Theodore Ts'o <[email protected]> # drivers/char/random.c
Signed-off-by: Will Deacon <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/Kconfig | 9 +++++++++
drivers/char/random.c | 14 ++++++++++++++
drivers/of/fdt.c | 14 ++++++++++++--
include/linux/random.h | 1 +
4 files changed, 36 insertions(+), 2 deletions(-)
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -609,3 +609,12 @@ config RANDOM_TRUST_CPU
has not installed a hidden back door to compromise the CPU's
random number generation facilities. This can also be configured
at boot with "random.trust_cpu=on/off".
+
+config RANDOM_TRUST_BOOTLOADER
+ bool "Trust the bootloader to initialize Linux's CRNG"
+ help
+ Some bootloaders can provide entropy to increase the kernel's initial
+ device randomness. Say Y here to assume the entropy provided by the
+ booloader is trustworthy so it will be added to the kernel's entropy
+ pool. Otherwise, say N here so it will be regarded as device input that
+ only mixes the entropy pool.
\ No newline at end of file
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2493,3 +2493,17 @@ void add_hwgenerator_randomness(const ch
}
}
EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
+
+/* Handle random seed passed by bootloader.
+ * If the seed is trustworthy, it would be regarded as hardware RNGs. Otherwise
+ * it would be regarded as device data.
+ * The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
+ */
+void add_bootloader_randomness(const void *buf, unsigned int size)
+{
+ if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
+ add_hwgenerator_randomness(buf, size, size * 8);
+ else
+ add_device_randomness(buf, size);
+}
+EXPORT_SYMBOL_GPL(add_bootloader_randomness);
\ No newline at end of file
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -27,6 +27,7 @@
#include <linux/debugfs.h>
#include <linux/serial_core.h>
#include <linux/sysfs.h>
+#include <linux/random.h>
#include <asm/setup.h> /* for COMMAND_LINE_SIZE */
#include <asm/page.h>
@@ -1068,6 +1069,7 @@ int __init early_init_dt_scan_chosen(uns
{
int l;
const char *p;
+ const void *rng_seed;
pr_debug("search \"chosen\", depth: %d, uname: %s\n", depth, uname);
@@ -1102,6 +1104,14 @@ int __init early_init_dt_scan_chosen(uns
pr_debug("Command line is: %s\n", (char*)data);
+ rng_seed = of_get_flat_dt_prop(node, "rng-seed", &l);
+ if (rng_seed && l > 0) {
+ add_bootloader_randomness(rng_seed, l);
+
+ /* try to clear seed so it won't be found. */
+ fdt_nop_property(initial_boot_params, node, "rng-seed");
+ }
+
/* break now */
return 1;
}
@@ -1203,8 +1213,6 @@ bool __init early_init_dt_verify(void *p
/* Setup flat device-tree pointer */
initial_boot_params = params;
- of_fdt_crc32 = crc32_be(~0, initial_boot_params,
- fdt_totalsize(initial_boot_params));
return true;
}
@@ -1230,6 +1238,8 @@ bool __init early_init_dt_scan(void *par
return false;
early_init_dt_scan_nodes();
+ of_fdt_crc32 = crc32_be(~0, initial_boot_params,
+ fdt_totalsize(initial_boot_params));
return true;
}
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -18,6 +18,7 @@ struct random_ready_callback {
};
extern void add_device_randomness(const void *, unsigned int);
+extern void add_bootloader_randomness(const void *, unsigned int);
#if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__)
static inline void add_latent_entropy(void)
From: "Jason A. Donenfeld" <[email protected]>
commit 58e1100fdc5990b0cc0d4beaf2562a92e621ac7d upstream.
random.c is a bit understaffed, and folks want more prompt reviews. I've
got the crypto background and the interest to do these reviews, and have
authored parts of the file already.
Cc: Theodore Ts'o <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10068,6 +10068,7 @@ F: drivers/block/brd.c
RANDOM NUMBER DRIVER
M: "Theodore Ts'o" <[email protected]>
+M: Jason A. Donenfeld <[email protected]>
S: Maintained
F: drivers/char/random.c
From: "Jason A. Donenfeld" <[email protected]>
commit d97c68d178fbf8aaaf21b69b446f2dfb13909316 upstream.
If CONFIG_RANDOM_TRUST_CPU is set, the RNG initializes using RDRAND.
But, the user can disable (or enable) this behavior by setting
`random.trust_cpu=0/1` on the kernel command line. This allows system
builders to do reasonable things while avoiding howls from tinfoil
hatters. (Or vice versa.)
CONFIG_RANDOM_TRUST_BOOTLOADER is basically the same thing, but regards
the seed passed via EFI or device tree, which might come from RDRAND or
a TPM or somewhere else. In order to allow distros to more easily enable
this while avoiding those same howls (or vice versa), this commit adds
the corresponding `random.trust_bootloader=0/1` toggle.
Cc: Theodore Ts'o <[email protected]>
Cc: Graham Christensen <[email protected]>
Reviewed-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Link: https://github.com/NixOS/nixpkgs/pull/165355
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/kernel-parameters.txt | 6 ++++++
drivers/char/Kconfig | 3 ++-
drivers/char/random.c | 8 +++++++-
3 files changed, 15 insertions(+), 2 deletions(-)
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -3583,6 +3583,12 @@ bytes respectively. Such letter suffixes
fully seed the kernel's CRNG. Default is controlled
by CONFIG_RANDOM_TRUST_CPU.
+ random.trust_bootloader={on,off}
+ [KNL] Enable or disable trusting the use of a
+ seed passed by the bootloader (if available) to
+ fully seed the kernel's CRNG. Default is controlled
+ by CONFIG_RANDOM_TRUST_BOOTLOADER.
+
rcu_nocbs= [KNL]
The argument is a cpu list, as described above.
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -617,4 +617,5 @@ config RANDOM_TRUST_BOOTLOADER
device randomness. Say Y here to assume the entropy provided by the
booloader is trustworthy so it will be added to the kernel's entropy
pool. Otherwise, say N here so it will be regarded as device input that
- only mixes the entropy pool.
\ No newline at end of file
+ only mixes the entropy pool. This can also be configured at boot with
+ "random.trust_bootloader=on/off".
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -941,11 +941,17 @@ static bool drain_entropy(void *buf, siz
**********************************************************************/
static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
+static bool trust_bootloader __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER);
static int __init parse_trust_cpu(char *arg)
{
return kstrtobool(arg, &trust_cpu);
}
+static int __init parse_trust_bootloader(char *arg)
+{
+ return kstrtobool(arg, &trust_bootloader);
+}
early_param("random.trust_cpu", parse_trust_cpu);
+early_param("random.trust_bootloader", parse_trust_bootloader);
/*
* The first collection of entropy occurs at system boot while interrupts
@@ -1153,7 +1159,7 @@ EXPORT_SYMBOL_GPL(add_hwgenerator_random
*/
void add_bootloader_randomness(const void *buf, size_t size)
{
- if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
+ if (trust_bootloader)
add_hwgenerator_randomness(buf, size, size * 8);
else
add_device_randomness(buf, size);
From: Jann Horn <[email protected]>
commit 6c8e11e08a5b74bb8a5cdd5cbc1e5143df0fba72 upstream.
At the moment, urandom_read() (used for /dev/urandom) resets crng_init_cnt
to zero when it is called at crng_init<2. This is inconsistent: We do it
for /dev/urandom reads, but not for the equivalent
getrandom(GRND_INSECURE).
(And worse, as Jason pointed out, we're only doing this as long as
maxwarn>0.)
crng_init_cnt is only read in crng_fast_load(); it is relevant at
crng_init==0 for determining when to switch to crng_init==1 (and where in
the RNG state array to write).
As far as I understand:
- crng_init==0 means "we have nothing, we might just be returning the same
exact numbers on every boot on every machine, we don't even have
non-cryptographic randomness; we should shove every bit of entropy we
can get into the RNG immediately"
- crng_init==1 means "well we have something, it might not be
cryptographic, but at least we're not gonna return the same data every
time or whatever, it's probably good enough for TCP and ASLR and stuff;
we now have time to build up actual cryptographic entropy in the input
pool"
- crng_init==2 means "this is supposed to be cryptographically secure now,
but we'll keep adding more entropy just to be sure".
The current code means that if someone is pulling data from /dev/urandom
fast enough at crng_init==0, we'll keep resetting crng_init_cnt, and we'll
never make forward progress to crng_init==1. It seems to be intended to
prevent an attacker from bruteforcing the contents of small individual RNG
inputs on the way from crng_init==0 to crng_init==1, but that's misguided;
crng_init==1 isn't supposed to provide proper cryptographic security
anyway, RNG users who care about getting secure RNG output have to wait
until crng_init==2.
This code was inconsistent, and it probably made things worse - just get
rid of it.
Signed-off-by: Jann Horn <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 ----
1 file changed, 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1821,7 +1821,6 @@ urandom_read_nowarn(struct file *file, c
static ssize_t
urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
{
- unsigned long flags;
static int maxwarn = 10;
if (!crng_ready() && maxwarn > 0) {
@@ -1829,9 +1828,6 @@ urandom_read(struct file *file, char __u
if (__ratelimit(&urandom_warning))
pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
current->comm, nbytes);
- spin_lock_irqsave(&primary_crng.lock, flags);
- crng_init_cnt = 0;
- spin_unlock_irqrestore(&primary_crng.lock, flags);
}
return urandom_read_nowarn(file, buf, nbytes, ppos);
From: "Jason A. Donenfeld" <[email protected]>
commit 66d7fb94e4ffe5acc589e0b2b4710aecc1f07a28 upstream.
The C implementation was originally based on Samuel Neves' public
domain reference implementation but has since been heavily modified
for the kernel. We're able to do compile-time optimizations by moving
some scaffolding around the final function into the header file.
Information: https://blake2.net/
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Samuel Neves <[email protected]>
Co-developed-by: Samuel Neves <[email protected]>
[ardb: - move from lib/zinc to lib/crypto
- remove simd handling
- rewrote selftest for better coverage
- use fixed digest length for blake2s_hmac() and rename to
blake2s256_hmac() ]
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
[Jason: for stable, skip kconfig and wire up directly, and skip the arch
hooks; optimized implementations need not be backported.]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/crypto/blake2s.h | 106 ++++++
include/crypto/internal/blake2s.h | 19 +
lib/Makefile | 2
lib/crypto/Makefile | 7
lib/crypto/blake2s-generic.c | 111 ++++++
lib/crypto/blake2s-selftest.c | 622 ++++++++++++++++++++++++++++++++++++++
lib/crypto/blake2s.c | 115 +++++++
7 files changed, 982 insertions(+)
create mode 100644 include/crypto/blake2s.h
create mode 100644 include/crypto/internal/blake2s.h
create mode 100644 lib/crypto/Makefile
create mode 100644 lib/crypto/blake2s-generic.c
create mode 100644 lib/crypto/blake2s-selftest.c
create mode 100644 lib/crypto/blake2s.c
--- /dev/null
+++ b/include/crypto/blake2s.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ */
+
+#ifndef BLAKE2S_H
+#define BLAKE2S_H
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
+
+#include <asm/bug.h>
+
+enum blake2s_lengths {
+ BLAKE2S_BLOCK_SIZE = 64,
+ BLAKE2S_HASH_SIZE = 32,
+ BLAKE2S_KEY_SIZE = 32,
+
+ BLAKE2S_128_HASH_SIZE = 16,
+ BLAKE2S_160_HASH_SIZE = 20,
+ BLAKE2S_224_HASH_SIZE = 28,
+ BLAKE2S_256_HASH_SIZE = 32,
+};
+
+struct blake2s_state {
+ u32 h[8];
+ u32 t[2];
+ u32 f[2];
+ u8 buf[BLAKE2S_BLOCK_SIZE];
+ unsigned int buflen;
+ unsigned int outlen;
+};
+
+enum blake2s_iv {
+ BLAKE2S_IV0 = 0x6A09E667UL,
+ BLAKE2S_IV1 = 0xBB67AE85UL,
+ BLAKE2S_IV2 = 0x3C6EF372UL,
+ BLAKE2S_IV3 = 0xA54FF53AUL,
+ BLAKE2S_IV4 = 0x510E527FUL,
+ BLAKE2S_IV5 = 0x9B05688CUL,
+ BLAKE2S_IV6 = 0x1F83D9ABUL,
+ BLAKE2S_IV7 = 0x5BE0CD19UL,
+};
+
+void blake2s_update(struct blake2s_state *state, const u8 *in, size_t inlen);
+void blake2s_final(struct blake2s_state *state, u8 *out);
+
+static inline void blake2s_init_param(struct blake2s_state *state,
+ const u32 param)
+{
+ *state = (struct blake2s_state){{
+ BLAKE2S_IV0 ^ param,
+ BLAKE2S_IV1,
+ BLAKE2S_IV2,
+ BLAKE2S_IV3,
+ BLAKE2S_IV4,
+ BLAKE2S_IV5,
+ BLAKE2S_IV6,
+ BLAKE2S_IV7,
+ }};
+}
+
+static inline void blake2s_init(struct blake2s_state *state,
+ const size_t outlen)
+{
+ blake2s_init_param(state, 0x01010000 | outlen);
+ state->outlen = outlen;
+}
+
+static inline void blake2s_init_key(struct blake2s_state *state,
+ const size_t outlen, const void *key,
+ const size_t keylen)
+{
+ WARN_ON(IS_ENABLED(DEBUG) && (!outlen || outlen > BLAKE2S_HASH_SIZE ||
+ !key || !keylen || keylen > BLAKE2S_KEY_SIZE));
+
+ blake2s_init_param(state, 0x01010000 | keylen << 8 | outlen);
+ memcpy(state->buf, key, keylen);
+ state->buflen = BLAKE2S_BLOCK_SIZE;
+ state->outlen = outlen;
+}
+
+static inline void blake2s(u8 *out, const u8 *in, const u8 *key,
+ const size_t outlen, const size_t inlen,
+ const size_t keylen)
+{
+ struct blake2s_state state;
+
+ WARN_ON(IS_ENABLED(DEBUG) && ((!in && inlen > 0) || !out || !outlen ||
+ outlen > BLAKE2S_HASH_SIZE || keylen > BLAKE2S_KEY_SIZE ||
+ (!key && keylen)));
+
+ if (keylen)
+ blake2s_init_key(&state, outlen, key, keylen);
+ else
+ blake2s_init(&state, outlen);
+
+ blake2s_update(&state, in, inlen);
+ blake2s_final(&state, out);
+}
+
+void blake2s256_hmac(u8 *out, const u8 *in, const u8 *key, const size_t inlen,
+ const size_t keylen);
+
+#endif /* BLAKE2S_H */
--- /dev/null
+++ b/include/crypto/internal/blake2s.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+
+#ifndef BLAKE2S_INTERNAL_H
+#define BLAKE2S_INTERNAL_H
+
+#include <crypto/blake2s.h>
+
+void blake2s_compress_generic(struct blake2s_state *state,const u8 *block,
+ size_t nblocks, const u32 inc);
+
+void blake2s_compress_arch(struct blake2s_state *state,const u8 *block,
+ size_t nblocks, const u32 inc);
+
+static inline void blake2s_set_lastblock(struct blake2s_state *state)
+{
+ state->f[0] = -1;
+}
+
+#endif /* BLAKE2S_INTERNAL_H */
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -234,3 +234,5 @@ KASAN_SANITIZE_ubsan.o := n
CFLAGS_ubsan.o := $(call cc-option, -fno-stack-protector) $(DISABLE_STACKLEAK_PLUGIN)
obj-$(CONFIG_SBITMAP) += sbitmap.o
+
+obj-y += crypto/
--- /dev/null
+++ b/lib/crypto/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-y += libblake2s.o
+libblake2s-y += blake2s.o blake2s-generic.o
+ifneq ($(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS),y)
+libblake2s-y += blake2s-selftest.o
+endif
--- /dev/null
+++ b/lib/crypto/blake2s-generic.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ *
+ * This is an implementation of the BLAKE2s hash and PRF functions.
+ *
+ * Information: https://blake2.net/
+ *
+ */
+
+#include <crypto/internal/blake2s.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/bug.h>
+#include <asm/unaligned.h>
+
+static const u8 blake2s_sigma[10][16] = {
+ { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 },
+ { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 },
+ { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 },
+ { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 },
+ { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 },
+ { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 },
+ { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 },
+ { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 },
+ { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13, 0 },
+};
+
+static inline void blake2s_increment_counter(struct blake2s_state *state,
+ const u32 inc)
+{
+ state->t[0] += inc;
+ state->t[1] += (state->t[0] < inc);
+}
+
+void blake2s_compress_generic(struct blake2s_state *state,const u8 *block,
+ size_t nblocks, const u32 inc)
+{
+ u32 m[16];
+ u32 v[16];
+ int i;
+
+ WARN_ON(IS_ENABLED(DEBUG) &&
+ (nblocks > 1 && inc != BLAKE2S_BLOCK_SIZE));
+
+ while (nblocks > 0) {
+ blake2s_increment_counter(state, inc);
+ memcpy(m, block, BLAKE2S_BLOCK_SIZE);
+ le32_to_cpu_array(m, ARRAY_SIZE(m));
+ memcpy(v, state->h, 32);
+ v[ 8] = BLAKE2S_IV0;
+ v[ 9] = BLAKE2S_IV1;
+ v[10] = BLAKE2S_IV2;
+ v[11] = BLAKE2S_IV3;
+ v[12] = BLAKE2S_IV4 ^ state->t[0];
+ v[13] = BLAKE2S_IV5 ^ state->t[1];
+ v[14] = BLAKE2S_IV6 ^ state->f[0];
+ v[15] = BLAKE2S_IV7 ^ state->f[1];
+
+#define G(r, i, a, b, c, d) do { \
+ a += b + m[blake2s_sigma[r][2 * i + 0]]; \
+ d = ror32(d ^ a, 16); \
+ c += d; \
+ b = ror32(b ^ c, 12); \
+ a += b + m[blake2s_sigma[r][2 * i + 1]]; \
+ d = ror32(d ^ a, 8); \
+ c += d; \
+ b = ror32(b ^ c, 7); \
+} while (0)
+
+#define ROUND(r) do { \
+ G(r, 0, v[0], v[ 4], v[ 8], v[12]); \
+ G(r, 1, v[1], v[ 5], v[ 9], v[13]); \
+ G(r, 2, v[2], v[ 6], v[10], v[14]); \
+ G(r, 3, v[3], v[ 7], v[11], v[15]); \
+ G(r, 4, v[0], v[ 5], v[10], v[15]); \
+ G(r, 5, v[1], v[ 6], v[11], v[12]); \
+ G(r, 6, v[2], v[ 7], v[ 8], v[13]); \
+ G(r, 7, v[3], v[ 4], v[ 9], v[14]); \
+} while (0)
+ ROUND(0);
+ ROUND(1);
+ ROUND(2);
+ ROUND(3);
+ ROUND(4);
+ ROUND(5);
+ ROUND(6);
+ ROUND(7);
+ ROUND(8);
+ ROUND(9);
+
+#undef G
+#undef ROUND
+
+ for (i = 0; i < 8; ++i)
+ state->h[i] ^= v[i] ^ v[i + 8];
+
+ block += BLAKE2S_BLOCK_SIZE;
+ --nblocks;
+ }
+}
+
+EXPORT_SYMBOL(blake2s_compress_generic);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("BLAKE2s hash function");
+MODULE_AUTHOR("Jason A. Donenfeld <[email protected]>");
--- /dev/null
+++ b/lib/crypto/blake2s-selftest.c
@@ -0,0 +1,622 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ */
+
+#include <crypto/blake2s.h>
+#include <linux/string.h>
+
+/*
+ * blake2s_testvecs[] generated with the program below (using libb2-dev and
+ * libssl-dev [OpenSSL])
+ *
+ * #include <blake2.h>
+ * #include <stdint.h>
+ * #include <stdio.h>
+ *
+ * #include <openssl/evp.h>
+ * #include <openssl/hmac.h>
+ *
+ * #define BLAKE2S_TESTVEC_COUNT 256
+ *
+ * static void print_vec(const uint8_t vec[], int len)
+ * {
+ * int i;
+ *
+ * printf(" { ");
+ * for (i = 0; i < len; i++) {
+ * if (i && (i % 12) == 0)
+ * printf("\n ");
+ * printf("0x%02x, ", vec[i]);
+ * }
+ * printf("},\n");
+ * }
+ *
+ * int main(void)
+ * {
+ * uint8_t key[BLAKE2S_KEYBYTES];
+ * uint8_t buf[BLAKE2S_TESTVEC_COUNT];
+ * uint8_t hash[BLAKE2S_OUTBYTES];
+ * int i, j;
+ *
+ * key[0] = key[1] = 1;
+ * for (i = 2; i < BLAKE2S_KEYBYTES; ++i)
+ * key[i] = key[i - 2] + key[i - 1];
+ *
+ * for (i = 0; i < BLAKE2S_TESTVEC_COUNT; ++i)
+ * buf[i] = (uint8_t)i;
+ *
+ * printf("static const u8 blake2s_testvecs[][BLAKE2S_HASH_SIZE] __initconst = {\n");
+ *
+ * for (i = 0; i < BLAKE2S_TESTVEC_COUNT; ++i) {
+ * int outlen = 1 + i % BLAKE2S_OUTBYTES;
+ * int keylen = (13 * i) % (BLAKE2S_KEYBYTES + 1);
+ *
+ * blake2s(hash, buf, key + BLAKE2S_KEYBYTES - keylen, outlen, i,
+ * keylen);
+ * print_vec(hash, outlen);
+ * }
+ * printf("};\n\n");
+ *
+ * printf("static const u8 blake2s_hmac_testvecs[][BLAKE2S_HASH_SIZE] __initconst = {\n");
+ *
+ * HMAC(EVP_blake2s256(), key, sizeof(key), buf, sizeof(buf), hash, NULL);
+ * print_vec(hash, BLAKE2S_OUTBYTES);
+ *
+ * HMAC(EVP_blake2s256(), buf, sizeof(buf), key, sizeof(key), hash, NULL);
+ * print_vec(hash, BLAKE2S_OUTBYTES);
+ *
+ * printf("};\n");
+ *
+ * return 0;
+ *}
+ */
+static const u8 blake2s_testvecs[][BLAKE2S_HASH_SIZE] __initconst = {
+ { 0xa1, },
+ { 0x7c, 0x89, },
+ { 0x74, 0x0e, 0xd4, },
+ { 0x47, 0x0c, 0x21, 0x15, },
+ { 0x18, 0xd6, 0x9c, 0xa6, 0xc4, },
+ { 0x13, 0x5d, 0x16, 0x63, 0x2e, 0xf9, },
+ { 0x2c, 0xb5, 0x04, 0xb7, 0x99, 0xe2, 0x73, },
+ { 0x9a, 0x0f, 0xd2, 0x39, 0xd6, 0x68, 0x1b, 0x92, },
+ { 0xc8, 0xde, 0x7a, 0xea, 0x2f, 0xf4, 0xd2, 0xe3, 0x2b, },
+ { 0x5b, 0xf9, 0x43, 0x52, 0x0c, 0x12, 0xba, 0xb5, 0x93, 0x9f, },
+ { 0xc6, 0x2c, 0x4e, 0x80, 0xfc, 0x32, 0x5b, 0x33, 0xb8, 0xb8, 0x0a, },
+ { 0xa7, 0x5c, 0xfd, 0x3a, 0xcc, 0xbf, 0x90, 0xca, 0xb7, 0x97, 0xde, 0xd8, },
+ { 0x66, 0xca, 0x3c, 0xc4, 0x19, 0xef, 0x92, 0x66, 0x3f, 0x21, 0x8f, 0xda,
+ 0xb7, },
+ { 0xba, 0xe5, 0xbb, 0x30, 0x25, 0x94, 0x6d, 0xc3, 0x89, 0x09, 0xc4, 0x25,
+ 0x52, 0x3e, },
+ { 0xa2, 0xef, 0x0e, 0x52, 0x0b, 0x5f, 0xa2, 0x01, 0x6d, 0x0a, 0x25, 0xbc,
+ 0x57, 0xe2, 0x27, },
+ { 0x4f, 0xe0, 0xf9, 0x52, 0x12, 0xda, 0x84, 0xb7, 0xab, 0xae, 0xb0, 0xa6,
+ 0x47, 0x2a, 0xc7, 0xf5, },
+ { 0x56, 0xe7, 0xa8, 0x1c, 0x4c, 0xca, 0xed, 0x90, 0x31, 0xec, 0x87, 0x43,
+ 0xe7, 0x72, 0x08, 0xec, 0xbe, },
+ { 0x7e, 0xdf, 0x80, 0x1c, 0x93, 0x33, 0xfd, 0x53, 0x44, 0xba, 0xfd, 0x96,
+ 0xe1, 0xbb, 0xb5, 0x65, 0xa5, 0x00, },
+ { 0xec, 0x6b, 0xed, 0xf7, 0x7b, 0x62, 0x1d, 0x7d, 0xf4, 0x82, 0xf3, 0x1e,
+ 0x18, 0xff, 0x2b, 0xc4, 0x06, 0x20, 0x2a, },
+ { 0x74, 0x98, 0xd7, 0x68, 0x63, 0xed, 0x87, 0xe4, 0x5d, 0x8d, 0x9e, 0x1d,
+ 0xfd, 0x2a, 0xbb, 0x86, 0xac, 0xe9, 0x2a, 0x89, },
+ { 0x89, 0xc3, 0x88, 0xce, 0x2b, 0x33, 0x1e, 0x10, 0xd1, 0x37, 0x20, 0x86,
+ 0x28, 0x43, 0x70, 0xd9, 0xfb, 0x96, 0xd9, 0xb5, 0xd3, },
+ { 0xcb, 0x56, 0x74, 0x41, 0x8d, 0x80, 0x01, 0x9a, 0x6b, 0x38, 0xe1, 0x41,
+ 0xad, 0x9c, 0x62, 0x74, 0xce, 0x35, 0xd5, 0x6c, 0x89, 0x6e, },
+ { 0x79, 0xaf, 0x94, 0x59, 0x99, 0x26, 0xe1, 0xc9, 0x34, 0xfe, 0x7c, 0x22,
+ 0xf7, 0x43, 0xd7, 0x65, 0xd4, 0x48, 0x18, 0xac, 0x3d, 0xfd, 0x93, },
+ { 0x85, 0x0d, 0xff, 0xb8, 0x3e, 0x87, 0x41, 0xb0, 0x95, 0xd3, 0x3d, 0x00,
+ 0x47, 0x55, 0x9e, 0xd2, 0x69, 0xea, 0xbf, 0xe9, 0x7a, 0x2d, 0x61, 0x45, },
+ { 0x03, 0xe0, 0x85, 0xec, 0x54, 0xb5, 0x16, 0x53, 0xa8, 0xc4, 0x71, 0xe9,
+ 0x6a, 0xe7, 0xcb, 0xc4, 0x15, 0x02, 0xfc, 0x34, 0xa4, 0xa4, 0x28, 0x13,
+ 0xd1, },
+ { 0xe3, 0x34, 0x4b, 0xe1, 0xd0, 0x4b, 0x55, 0x61, 0x8f, 0xc0, 0x24, 0x05,
+ 0xe6, 0xe0, 0x3d, 0x70, 0x24, 0x4d, 0xda, 0xb8, 0x91, 0x05, 0x29, 0x07,
+ 0x01, 0x3e, },
+ { 0x61, 0xff, 0x01, 0x72, 0xb1, 0x4d, 0xf6, 0xfe, 0xd1, 0xd1, 0x08, 0x74,
+ 0xe6, 0x91, 0x44, 0xeb, 0x61, 0xda, 0x40, 0xaf, 0xfc, 0x8c, 0x91, 0x6b,
+ 0xec, 0x13, 0xed, },
+ { 0xd4, 0x40, 0xd2, 0xa0, 0x7f, 0xc1, 0x58, 0x0c, 0x85, 0xa0, 0x86, 0xc7,
+ 0x86, 0xb9, 0x61, 0xc9, 0xea, 0x19, 0x86, 0x1f, 0xab, 0x07, 0xce, 0x37,
+ 0x72, 0x67, 0x09, 0xfc, },
+ { 0x9e, 0xf8, 0x18, 0x67, 0x93, 0x10, 0x9b, 0x39, 0x75, 0xe8, 0x8b, 0x38,
+ 0x82, 0x7d, 0xb8, 0xb7, 0xa5, 0xaf, 0xe6, 0x6a, 0x22, 0x5e, 0x1f, 0x9c,
+ 0x95, 0x29, 0x19, 0xf2, 0x4b, },
+ { 0xc8, 0x62, 0x25, 0xf5, 0x98, 0xc9, 0xea, 0xe5, 0x29, 0x3a, 0xd3, 0x22,
+ 0xeb, 0xeb, 0x07, 0x7c, 0x15, 0x07, 0xee, 0x15, 0x61, 0xbb, 0x05, 0x30,
+ 0x99, 0x7f, 0x11, 0xf6, 0x0a, 0x1d, },
+ { 0x68, 0x70, 0xf7, 0x90, 0xa1, 0x8b, 0x1f, 0x0f, 0xbb, 0xce, 0xd2, 0x0e,
+ 0x33, 0x1f, 0x7f, 0xa9, 0x78, 0xa8, 0xa6, 0x81, 0x66, 0xab, 0x8d, 0xcd,
+ 0x58, 0x55, 0x3a, 0x0b, 0x7a, 0xdb, 0xb5, },
+ { 0xdd, 0x35, 0xd2, 0xb4, 0xf6, 0xc7, 0xea, 0xab, 0x64, 0x24, 0x4e, 0xfe,
+ 0xe5, 0x3d, 0x4e, 0x95, 0x8b, 0x6d, 0x6c, 0xbc, 0xb0, 0xf8, 0x88, 0x61,
+ 0x09, 0xb7, 0x78, 0xa3, 0x31, 0xfe, 0xd9, 0x2f, },
+ { 0x0a, },
+ { 0x6e, 0xd4, },
+ { 0x64, 0xe9, 0xd1, },
+ { 0x30, 0xdd, 0x71, 0xef, },
+ { 0x11, 0xb5, 0x0c, 0x87, 0xc9, },
+ { 0x06, 0x1c, 0x6d, 0x04, 0x82, 0xd0, },
+ { 0x5c, 0x42, 0x0b, 0xee, 0xc5, 0x9c, 0xb2, },
+ { 0xe8, 0x29, 0xd6, 0xb4, 0x5d, 0xf7, 0x2b, 0x93, },
+ { 0x18, 0xca, 0x27, 0x72, 0x43, 0x39, 0x16, 0xbc, 0x6a, },
+ { 0x39, 0x8f, 0xfd, 0x64, 0xf5, 0x57, 0x23, 0xb0, 0x45, 0xf8, },
+ { 0xbb, 0x3a, 0x78, 0x6b, 0x02, 0x1d, 0x0b, 0x16, 0xe3, 0xb2, 0x9a, },
+ { 0xb8, 0xb4, 0x0b, 0xe5, 0xd4, 0x1d, 0x0d, 0x85, 0x49, 0x91, 0x35, 0xfa, },
+ { 0x6d, 0x48, 0x2a, 0x0c, 0x42, 0x08, 0xbd, 0xa9, 0x78, 0x6f, 0x18, 0xaf,
+ 0xe2, },
+ { 0x10, 0x45, 0xd4, 0x58, 0x88, 0xec, 0x4e, 0x1e, 0xf6, 0x14, 0x92, 0x64,
+ 0x7e, 0xb0, },
+ { 0x8b, 0x0b, 0x95, 0xee, 0x92, 0xc6, 0x3b, 0x91, 0xf1, 0x1e, 0xeb, 0x51,
+ 0x98, 0x0a, 0x8d, },
+ { 0xa3, 0x50, 0x4d, 0xa5, 0x1d, 0x03, 0x68, 0xe9, 0x57, 0x78, 0xd6, 0x04,
+ 0xf1, 0xc3, 0x94, 0xd8, },
+ { 0xb8, 0x66, 0x6e, 0xdd, 0x46, 0x15, 0xae, 0x3d, 0x83, 0x7e, 0xcf, 0xe7,
+ 0x2c, 0xe8, 0x8f, 0xc7, 0x34, },
+ { 0x2e, 0xc0, 0x1f, 0x29, 0xea, 0xf6, 0xb9, 0xe2, 0xc2, 0x93, 0xeb, 0x41,
+ 0x0d, 0xf0, 0x0a, 0x13, 0x0e, 0xa2, },
+ { 0x71, 0xb8, 0x33, 0xa9, 0x1b, 0xac, 0xf1, 0xb5, 0x42, 0x8f, 0x5e, 0x81,
+ 0x34, 0x43, 0xb7, 0xa4, 0x18, 0x5c, 0x47, },
+ { 0xda, 0x45, 0xb8, 0x2e, 0x82, 0x1e, 0xc0, 0x59, 0x77, 0x9d, 0xfa, 0xb4,
+ 0x1c, 0x5e, 0xa0, 0x2b, 0x33, 0x96, 0x5a, 0x58, },
+ { 0xe3, 0x09, 0x05, 0xa9, 0xeb, 0x48, 0x13, 0xad, 0x71, 0x88, 0x81, 0x9a,
+ 0x3e, 0x2c, 0xe1, 0x23, 0x99, 0x13, 0x35, 0x9f, 0xb5, },
+ { 0xb7, 0x86, 0x2d, 0x16, 0xe1, 0x04, 0x00, 0x47, 0x47, 0x61, 0x31, 0xfb,
+ 0x14, 0xac, 0xd8, 0xe9, 0xe3, 0x49, 0xbd, 0xf7, 0x9c, 0x3f, },
+ { 0x7f, 0xd9, 0x95, 0xa8, 0xa7, 0xa0, 0xcc, 0xba, 0xef, 0xb1, 0x0a, 0xa9,
+ 0x21, 0x62, 0x08, 0x0f, 0x1b, 0xff, 0x7b, 0x9d, 0xae, 0xb2, 0x95, },
+ { 0x85, 0x99, 0xea, 0x33, 0xe0, 0x56, 0xff, 0x13, 0xc6, 0x61, 0x8c, 0xf9,
+ 0x57, 0x05, 0x03, 0x11, 0xf9, 0xfb, 0x3a, 0xf7, 0xce, 0xbb, 0x52, 0x30, },
+ { 0xb2, 0x72, 0x9c, 0xf8, 0x77, 0x4e, 0x8f, 0x6b, 0x01, 0x6c, 0xff, 0x4e,
+ 0x4f, 0x02, 0xd2, 0xbc, 0xeb, 0x51, 0x28, 0x99, 0x50, 0xab, 0xc4, 0x42,
+ 0xe3, },
+ { 0x8b, 0x0a, 0xb5, 0x90, 0x8f, 0xf5, 0x7b, 0xdd, 0xba, 0x47, 0x37, 0xc9,
+ 0x2a, 0xd5, 0x4b, 0x25, 0x08, 0x8b, 0x02, 0x17, 0xa7, 0x9e, 0x6b, 0x6e,
+ 0xe3, 0x90, },
+ { 0x90, 0xdd, 0xf7, 0x75, 0xa7, 0xa3, 0x99, 0x5e, 0x5b, 0x7d, 0x75, 0xc3,
+ 0x39, 0x6b, 0xa0, 0xe2, 0x44, 0x53, 0xb1, 0x9e, 0xc8, 0xf1, 0x77, 0x10,
+ 0x58, 0x06, 0x9a, },
+ { 0x99, 0x52, 0xf0, 0x49, 0xa8, 0x8c, 0xec, 0xa6, 0x97, 0x32, 0x13, 0xb5,
+ 0xf7, 0xa3, 0x8e, 0xfb, 0x4b, 0x59, 0x31, 0x3d, 0x01, 0x59, 0x98, 0x5d,
+ 0x53, 0x03, 0x1a, 0x39, },
+ { 0x9f, 0xe0, 0xc2, 0xe5, 0x5d, 0x93, 0xd6, 0x9b, 0x47, 0x8f, 0x9b, 0xe0,
+ 0x26, 0x35, 0x84, 0x20, 0x1d, 0xc5, 0x53, 0x10, 0x0f, 0x22, 0xb9, 0xb5,
+ 0xd4, 0x36, 0xb1, 0xac, 0x73, },
+ { 0x30, 0x32, 0x20, 0x3b, 0x10, 0x28, 0xec, 0x1f, 0x4f, 0x9b, 0x47, 0x59,
+ 0xeb, 0x7b, 0xee, 0x45, 0xfb, 0x0c, 0x49, 0xd8, 0x3d, 0x69, 0xbd, 0x90,
+ 0x2c, 0xf0, 0x9e, 0x8d, 0xbf, 0xd5, },
+ { 0x2a, 0x37, 0x73, 0x7f, 0xf9, 0x96, 0x19, 0xaa, 0x25, 0xd8, 0x13, 0x28,
+ 0x01, 0x29, 0x89, 0xdf, 0x6e, 0x0c, 0x9b, 0x43, 0x44, 0x51, 0xe9, 0x75,
+ 0x26, 0x0c, 0xb7, 0x87, 0x66, 0x0b, 0x5f, },
+ { 0x23, 0xdf, 0x96, 0x68, 0x91, 0x86, 0xd0, 0x93, 0x55, 0x33, 0x24, 0xf6,
+ 0xba, 0x08, 0x75, 0x5b, 0x59, 0x11, 0x69, 0xb8, 0xb9, 0xe5, 0x2c, 0x77,
+ 0x02, 0xf6, 0x47, 0xee, 0x81, 0xdd, 0xb9, 0x06, },
+ { 0x9d, },
+ { 0x9d, 0x7d, },
+ { 0xfd, 0xc3, 0xda, },
+ { 0xe8, 0x82, 0xcd, 0x21, },
+ { 0xc3, 0x1d, 0x42, 0x4c, 0x74, },
+ { 0xe9, 0xda, 0xf1, 0xa2, 0xe5, 0x7c, },
+ { 0x52, 0xb8, 0x6f, 0x81, 0x5c, 0x3a, 0x4c, },
+ { 0x5b, 0x39, 0x26, 0xfc, 0x92, 0x5e, 0xe0, 0x49, },
+ { 0x59, 0xe4, 0x7c, 0x93, 0x1c, 0xf9, 0x28, 0x93, 0xde, },
+ { 0xde, 0xdf, 0xb2, 0x43, 0x61, 0x0b, 0x86, 0x16, 0x4c, 0x2e, },
+ { 0x14, 0x8f, 0x75, 0x51, 0xaf, 0xb9, 0xee, 0x51, 0x5a, 0xae, 0x23, },
+ { 0x43, 0x5f, 0x50, 0xd5, 0x70, 0xb0, 0x5b, 0x87, 0xf5, 0xd9, 0xb3, 0x6d, },
+ { 0x66, 0x0a, 0x64, 0x93, 0x79, 0x71, 0x94, 0x40, 0xb7, 0x68, 0x2d, 0xd3,
+ 0x63, },
+ { 0x15, 0x00, 0xc4, 0x0c, 0x7d, 0x1b, 0x10, 0xa9, 0x73, 0x1b, 0x90, 0x6f,
+ 0xe6, 0xa9, },
+ { 0x34, 0x75, 0xf3, 0x86, 0x8f, 0x56, 0xcf, 0x2a, 0x0a, 0xf2, 0x62, 0x0a,
+ 0xf6, 0x0e, 0x20, },
+ { 0xb1, 0xde, 0xc9, 0xf5, 0xdb, 0xf3, 0x2f, 0x4c, 0xd6, 0x41, 0x7d, 0x39,
+ 0x18, 0x3e, 0xc7, 0xc3, },
+ { 0xc5, 0x89, 0xb2, 0xf8, 0xb8, 0xc0, 0xa3, 0xb9, 0x3b, 0x10, 0x6d, 0x7c,
+ 0x92, 0xfc, 0x7f, 0x34, 0x41, },
+ { 0xc4, 0xd8, 0xef, 0xba, 0xef, 0xd2, 0xaa, 0xc5, 0x6c, 0x8e, 0x3e, 0xbb,
+ 0x12, 0xfc, 0x0f, 0x72, 0xbf, 0x0f, },
+ { 0xdd, 0x91, 0xd1, 0x15, 0x9e, 0x7d, 0xf8, 0xc1, 0xb9, 0x14, 0x63, 0x96,
+ 0xb5, 0xcb, 0x83, 0x1d, 0x35, 0x1c, 0xec, },
+ { 0xa9, 0xf8, 0x52, 0xc9, 0x67, 0x76, 0x2b, 0xad, 0xfb, 0xd8, 0x3a, 0xa6,
+ 0x74, 0x02, 0xae, 0xb8, 0x25, 0x2c, 0x63, 0x49, },
+ { 0x77, 0x1f, 0x66, 0x70, 0xfd, 0x50, 0x29, 0xaa, 0xeb, 0xdc, 0xee, 0xba,
+ 0x75, 0x98, 0xdc, 0x93, 0x12, 0x3f, 0xdc, 0x7c, 0x38, },
+ { 0xe2, 0xe1, 0x89, 0x5c, 0x37, 0x38, 0x6a, 0xa3, 0x40, 0xac, 0x3f, 0xb0,
+ 0xca, 0xfc, 0xa7, 0xf3, 0xea, 0xf9, 0x0f, 0x5d, 0x8e, 0x39, },
+ { 0x0f, 0x67, 0xc8, 0x38, 0x01, 0xb1, 0xb7, 0xb8, 0xa2, 0xe7, 0x0a, 0x6d,
+ 0xd2, 0x63, 0x69, 0x9e, 0xcc, 0xf0, 0xf2, 0xbe, 0x9b, 0x98, 0xdd, },
+ { 0x13, 0xe1, 0x36, 0x30, 0xfe, 0xc6, 0x01, 0x8a, 0xa1, 0x63, 0x96, 0x59,
+ 0xc2, 0xa9, 0x68, 0x3f, 0x58, 0xd4, 0x19, 0x0c, 0x40, 0xf3, 0xde, 0x02, },
+ { 0xa3, 0x9e, 0xce, 0xda, 0x42, 0xee, 0x8c, 0x6c, 0x5a, 0x7d, 0xdc, 0x89,
+ 0x02, 0x77, 0xdd, 0xe7, 0x95, 0xbb, 0xff, 0x0d, 0xa4, 0xb5, 0x38, 0x1e,
+ 0xaf, },
+ { 0x9a, 0xf6, 0xb5, 0x9a, 0x4f, 0xa9, 0x4f, 0x2c, 0x35, 0x3c, 0x24, 0xdc,
+ 0x97, 0x6f, 0xd9, 0xa1, 0x7d, 0x1a, 0x85, 0x0b, 0xf5, 0xda, 0x2e, 0xe7,
+ 0xb1, 0x1d, },
+ { 0x84, 0x1e, 0x8e, 0x3d, 0x45, 0xa5, 0xf2, 0x27, 0xf3, 0x31, 0xfe, 0xb9,
+ 0xfb, 0xc5, 0x45, 0x99, 0x99, 0xdd, 0x93, 0x43, 0x02, 0xee, 0x58, 0xaf,
+ 0xee, 0x6a, 0xbe, },
+ { 0x07, 0x2f, 0xc0, 0xa2, 0x04, 0xc4, 0xab, 0x7c, 0x26, 0xbb, 0xa8, 0xd8,
+ 0xe3, 0x1c, 0x75, 0x15, 0x64, 0x5d, 0x02, 0x6a, 0xf0, 0x86, 0xe9, 0xcd,
+ 0x5c, 0xef, 0xa3, 0x25, },
+ { 0x2f, 0x3b, 0x1f, 0xb5, 0x91, 0x8f, 0x86, 0xe0, 0xdc, 0x31, 0x48, 0xb6,
+ 0xa1, 0x8c, 0xfd, 0x75, 0xbb, 0x7d, 0x3d, 0xc1, 0xf0, 0x10, 0x9a, 0xd8,
+ 0x4b, 0x0e, 0xe3, 0x94, 0x9f, },
+ { 0x29, 0xbb, 0x8f, 0x6c, 0xd1, 0xf2, 0xb6, 0xaf, 0xe5, 0xe3, 0x2d, 0xdc,
+ 0x6f, 0xa4, 0x53, 0x88, 0xd8, 0xcf, 0x4d, 0x45, 0x42, 0x62, 0xdb, 0xdf,
+ 0xf8, 0x45, 0xc2, 0x13, 0xec, 0x35, },
+ { 0x06, 0x3c, 0xe3, 0x2c, 0x15, 0xc6, 0x43, 0x03, 0x81, 0xfb, 0x08, 0x76,
+ 0x33, 0xcb, 0x02, 0xc1, 0xba, 0x33, 0xe5, 0xe0, 0xd1, 0x92, 0xa8, 0x46,
+ 0x28, 0x3f, 0x3e, 0x9d, 0x2c, 0x44, 0x54, },
+ { 0xea, 0xbb, 0x96, 0xf8, 0xd1, 0x8b, 0x04, 0x11, 0x40, 0x78, 0x42, 0x02,
+ 0x19, 0xd1, 0xbc, 0x65, 0x92, 0xd3, 0xc3, 0xd6, 0xd9, 0x19, 0xe7, 0xc3,
+ 0x40, 0x97, 0xbd, 0xd4, 0xed, 0xfa, 0x5e, 0x28, },
+ { 0x02, },
+ { 0x52, 0xa8, },
+ { 0x38, 0x25, 0x0d, },
+ { 0xe3, 0x04, 0xd4, 0x92, },
+ { 0x97, 0xdb, 0xf7, 0x81, 0xca, },
+ { 0x8a, 0x56, 0x9d, 0x62, 0x56, 0xcc, },
+ { 0xa1, 0x8e, 0x3c, 0x72, 0x8f, 0x63, 0x03, },
+ { 0xf7, 0xf3, 0x39, 0x09, 0x0a, 0xa1, 0xbb, 0x23, },
+ { 0x6b, 0x03, 0xc0, 0xe9, 0xd9, 0x83, 0x05, 0x22, 0x01, },
+ { 0x1b, 0x4b, 0xf5, 0xd6, 0x4f, 0x05, 0x75, 0x91, 0x4c, 0x7f, },
+ { 0x4c, 0x8c, 0x25, 0x20, 0x21, 0xcb, 0xc2, 0x4b, 0x3a, 0x5b, 0x8d, },
+ { 0x56, 0xe2, 0x77, 0xa0, 0xb6, 0x9f, 0x81, 0xec, 0x83, 0x75, 0xc4, 0xf9, },
+ { 0x71, 0x70, 0x0f, 0xad, 0x4d, 0x35, 0x81, 0x9d, 0x88, 0x69, 0xf9, 0xaa,
+ 0xd3, },
+ { 0x50, 0x6e, 0x86, 0x6e, 0x43, 0xc0, 0xc2, 0x44, 0xc2, 0xe2, 0xa0, 0x1c,
+ 0xb7, 0x9a, },
+ { 0xe4, 0x7e, 0x72, 0xc6, 0x12, 0x8e, 0x7c, 0xfc, 0xbd, 0xe2, 0x08, 0x31,
+ 0x3d, 0x47, 0x3d, },
+ { 0x08, 0x97, 0x5b, 0x80, 0xae, 0xc4, 0x1d, 0x50, 0x77, 0xdf, 0x1f, 0xd0,
+ 0x24, 0xf0, 0x17, 0xc0, },
+ { 0x01, 0xb6, 0x29, 0xf4, 0xaf, 0x78, 0x5f, 0xb6, 0x91, 0xdd, 0x76, 0x76,
+ 0xd2, 0xfd, 0x0c, 0x47, 0x40, },
+ { 0xa1, 0xd8, 0x09, 0x97, 0x7a, 0xa6, 0xc8, 0x94, 0xf6, 0x91, 0x7b, 0xae,
+ 0x2b, 0x9f, 0x0d, 0x83, 0x48, 0xf7, },
+ { 0x12, 0xd5, 0x53, 0x7d, 0x9a, 0xb0, 0xbe, 0xd9, 0xed, 0xe9, 0x9e, 0xee,
+ 0x61, 0x5b, 0x42, 0xf2, 0xc0, 0x73, 0xc0, },
+ { 0xd5, 0x77, 0xd6, 0x5c, 0x6e, 0xa5, 0x69, 0x2b, 0x3b, 0x8c, 0xd6, 0x7d,
+ 0x1d, 0xbe, 0x2c, 0xa1, 0x02, 0x21, 0xcd, 0x29, },
+ { 0xa4, 0x98, 0x80, 0xca, 0x22, 0xcf, 0x6a, 0xab, 0x5e, 0x40, 0x0d, 0x61,
+ 0x08, 0x21, 0xef, 0xc0, 0x6c, 0x52, 0xb4, 0xb0, 0x53, },
+ { 0xbf, 0xaf, 0x8f, 0x3b, 0x7a, 0x97, 0x33, 0xe5, 0xca, 0x07, 0x37, 0xfd,
+ 0x15, 0xdf, 0xce, 0x26, 0x2a, 0xb1, 0xa7, 0x0b, 0xb3, 0xac, },
+ { 0x16, 0x22, 0xe1, 0xbc, 0x99, 0x4e, 0x01, 0xf0, 0xfa, 0xff, 0x8f, 0xa5,
+ 0x0c, 0x61, 0xb0, 0xad, 0xcc, 0xb1, 0xe1, 0x21, 0x46, 0xfa, 0x2e, },
+ { 0x11, 0x5b, 0x0b, 0x2b, 0xe6, 0x14, 0xc1, 0xd5, 0x4d, 0x71, 0x5e, 0x17,
+ 0xea, 0x23, 0xdd, 0x6c, 0xbd, 0x1d, 0xbe, 0x12, 0x1b, 0xee, 0x4c, 0x1a, },
+ { 0x40, 0x88, 0x22, 0xf3, 0x20, 0x6c, 0xed, 0xe1, 0x36, 0x34, 0x62, 0x2c,
+ 0x98, 0x83, 0x52, 0xe2, 0x25, 0xee, 0xe9, 0xf5, 0xe1, 0x17, 0xf0, 0x5c,
+ 0xae, },
+ { 0xc3, 0x76, 0x37, 0xde, 0x95, 0x8c, 0xca, 0x2b, 0x0c, 0x23, 0xe7, 0xb5,
+ 0x38, 0x70, 0x61, 0xcc, 0xff, 0xd3, 0x95, 0x7b, 0xf3, 0xff, 0x1f, 0x9d,
+ 0x59, 0x00, },
+ { 0x0c, 0x19, 0x52, 0x05, 0x22, 0x53, 0xcb, 0x48, 0xd7, 0x10, 0x0e, 0x7e,
+ 0x14, 0x69, 0xb5, 0xa2, 0x92, 0x43, 0xa3, 0x9e, 0x4b, 0x8f, 0x51, 0x2c,
+ 0x5a, 0x2c, 0x3b, },
+ { 0xe1, 0x9d, 0x70, 0x70, 0x28, 0xec, 0x86, 0x40, 0x55, 0x33, 0x56, 0xda,
+ 0x88, 0xca, 0xee, 0xc8, 0x6a, 0x20, 0xb1, 0xe5, 0x3d, 0x57, 0xf8, 0x3c,
+ 0x10, 0x07, 0x2a, 0xc4, },
+ { 0x0b, 0xae, 0xf1, 0xc4, 0x79, 0xee, 0x1b, 0x3d, 0x27, 0x35, 0x8d, 0x14,
+ 0xd6, 0xae, 0x4e, 0x3c, 0xe9, 0x53, 0x50, 0xb5, 0xcc, 0x0c, 0xf7, 0xdf,
+ 0xee, 0xa1, 0x74, 0xd6, 0x71, },
+ { 0xe6, 0xa4, 0xf4, 0x99, 0x98, 0xb9, 0x80, 0xea, 0x96, 0x7f, 0x4f, 0x33,
+ 0xcf, 0x74, 0x25, 0x6f, 0x17, 0x6c, 0xbf, 0xf5, 0x5c, 0x38, 0xd0, 0xff,
+ 0x96, 0xcb, 0x13, 0xf9, 0xdf, 0xfd, },
+ { 0xbe, 0x92, 0xeb, 0xba, 0x44, 0x2c, 0x24, 0x74, 0xd4, 0x03, 0x27, 0x3c,
+ 0x5d, 0x5b, 0x03, 0x30, 0x87, 0x63, 0x69, 0xe0, 0xb8, 0x94, 0xf4, 0x44,
+ 0x7e, 0xad, 0xcd, 0x20, 0x12, 0x16, 0x79, },
+ { 0x30, 0xf1, 0xc4, 0x8e, 0x05, 0x90, 0x2a, 0x97, 0x63, 0x94, 0x46, 0xff,
+ 0xce, 0xd8, 0x67, 0xa7, 0xac, 0x33, 0x8c, 0x95, 0xb7, 0xcd, 0xa3, 0x23,
+ 0x98, 0x9d, 0x76, 0x6c, 0x9d, 0xa8, 0xd6, 0x8a, },
+ { 0xbe, },
+ { 0x17, 0x6c, },
+ { 0x1a, 0x42, 0x4f, },
+ { 0xba, 0xaf, 0xb7, 0x65, },
+ { 0xc2, 0x63, 0x43, 0x6a, 0xea, },
+ { 0xe4, 0x4d, 0xad, 0xf2, 0x0b, 0x02, },
+ { 0x04, 0xc7, 0xc4, 0x7f, 0xa9, 0x2b, 0xce, },
+ { 0x66, 0xf6, 0x67, 0xcb, 0x03, 0x53, 0xc8, 0xf1, },
+ { 0x56, 0xa3, 0x60, 0x78, 0xc9, 0x5f, 0x70, 0x1b, 0x5e, },
+ { 0x99, 0xff, 0x81, 0x7c, 0x13, 0x3c, 0x29, 0x79, 0x4b, 0x65, },
+ { 0x51, 0x10, 0x50, 0x93, 0x01, 0x93, 0xb7, 0x01, 0xc9, 0x18, 0xb7, },
+ { 0x8e, 0x3c, 0x42, 0x1e, 0x5e, 0x7d, 0xc1, 0x50, 0x70, 0x1f, 0x00, 0x98, },
+ { 0x5f, 0xd9, 0x9b, 0xc8, 0xd7, 0xb2, 0x72, 0x62, 0x1a, 0x1e, 0xba, 0x92,
+ 0xe9, },
+ { 0x70, 0x2b, 0xba, 0xfe, 0xad, 0x5d, 0x96, 0x3f, 0x27, 0xc2, 0x41, 0x6d,
+ 0xc4, 0xb3, },
+ { 0xae, 0xe0, 0xd5, 0xd4, 0xc7, 0xae, 0x15, 0x5e, 0xdc, 0xdd, 0x33, 0x60,
+ 0xd7, 0xd3, 0x5e, },
+ { 0x79, 0x8e, 0xbc, 0x9e, 0x20, 0xb9, 0x19, 0x4b, 0x63, 0x80, 0xf3, 0x16,
+ 0xaf, 0x39, 0xbd, 0x92, },
+ { 0xc2, 0x0e, 0x85, 0xa0, 0x0b, 0x9a, 0xb0, 0xec, 0xde, 0x38, 0xd3, 0x10,
+ 0xd9, 0xa7, 0x66, 0x27, 0xcf, },
+ { 0x0e, 0x3b, 0x75, 0x80, 0x67, 0x14, 0x0c, 0x02, 0x90, 0xd6, 0xb3, 0x02,
+ 0x81, 0xf6, 0xa6, 0x87, 0xce, 0x58, },
+ { 0x79, 0xb5, 0xe9, 0x5d, 0x52, 0x4d, 0xf7, 0x59, 0xf4, 0x2e, 0x27, 0xdd,
+ 0xb3, 0xed, 0x57, 0x5b, 0x82, 0xea, 0x6f, },
+ { 0xa2, 0x97, 0xf5, 0x80, 0x02, 0x3d, 0xde, 0xa3, 0xf9, 0xf6, 0xab, 0xe3,
+ 0x57, 0x63, 0x7b, 0x9b, 0x10, 0x42, 0x6f, 0xf2, },
+ { 0x12, 0x7a, 0xfc, 0xb7, 0x67, 0x06, 0x0c, 0x78, 0x1a, 0xfe, 0x88, 0x4f,
+ 0xc6, 0xac, 0x52, 0x96, 0x64, 0x28, 0x97, 0x84, 0x06, },
+ { 0xc5, 0x04, 0x44, 0x6b, 0xb2, 0xa5, 0xa4, 0x66, 0xe1, 0x76, 0xa2, 0x51,
+ 0xf9, 0x59, 0x69, 0x97, 0x56, 0x0b, 0xbf, 0x50, 0xb3, 0x34, },
+ { 0x21, 0x32, 0x6b, 0x42, 0xb5, 0xed, 0x71, 0x8d, 0xf7, 0x5a, 0x35, 0xe3,
+ 0x90, 0xe2, 0xee, 0xaa, 0x89, 0xf6, 0xc9, 0x9c, 0x4d, 0x73, 0xf4, },
+ { 0x4c, 0xa6, 0x09, 0xf4, 0x48, 0xe7, 0x46, 0xbc, 0x49, 0xfc, 0xe5, 0xda,
+ 0xd1, 0x87, 0x13, 0x17, 0x4c, 0x59, 0x71, 0x26, 0x5b, 0x2c, 0x42, 0xb7, },
+ { 0x13, 0x63, 0xf3, 0x40, 0x02, 0xe5, 0xa3, 0x3a, 0x5e, 0x8e, 0xf8, 0xb6,
+ 0x8a, 0x49, 0x60, 0x76, 0x34, 0x72, 0x94, 0x73, 0xf6, 0xd9, 0x21, 0x6a,
+ 0x26, },
+ { 0xdf, 0x75, 0x16, 0x10, 0x1b, 0x5e, 0x81, 0xc3, 0xc8, 0xde, 0x34, 0x24,
+ 0xb0, 0x98, 0xeb, 0x1b, 0x8f, 0xa1, 0x9b, 0x05, 0xee, 0xa5, 0xe9, 0x35,
+ 0xf4, 0x1d, },
+ { 0xcd, 0x21, 0x93, 0x6e, 0x5b, 0xa0, 0x26, 0x2b, 0x21, 0x0e, 0xa0, 0xb9,
+ 0x1c, 0xb5, 0xbb, 0xb8, 0xf8, 0x1e, 0xff, 0x5c, 0xa8, 0xf9, 0x39, 0x46,
+ 0x4e, 0x29, 0x26, },
+ { 0x73, 0x7f, 0x0e, 0x3b, 0x0b, 0x5c, 0xf9, 0x60, 0xaa, 0x88, 0xa1, 0x09,
+ 0xb1, 0x5d, 0x38, 0x7b, 0x86, 0x8f, 0x13, 0x7a, 0x8d, 0x72, 0x7a, 0x98,
+ 0x1a, 0x5b, 0xff, 0xc9, },
+ { 0xd3, 0x3c, 0x61, 0x71, 0x44, 0x7e, 0x31, 0x74, 0x98, 0x9d, 0x9a, 0xd2,
+ 0x27, 0xf3, 0x46, 0x43, 0x42, 0x51, 0xd0, 0x5f, 0xe9, 0x1c, 0x5c, 0x69,
+ 0xbf, 0xf6, 0xbe, 0x3c, 0x40, },
+ { 0x31, 0x99, 0x31, 0x9f, 0xaa, 0x43, 0x2e, 0x77, 0x3e, 0x74, 0x26, 0x31,
+ 0x5e, 0x61, 0xf1, 0x87, 0xe2, 0xeb, 0x9b, 0xcd, 0xd0, 0x3a, 0xee, 0x20,
+ 0x7e, 0x10, 0x0a, 0x0b, 0x7e, 0xfa, },
+ { 0xa4, 0x27, 0x80, 0x67, 0x81, 0x2a, 0xa7, 0x62, 0xf7, 0x6e, 0xda, 0xd4,
+ 0x5c, 0x39, 0x74, 0xad, 0x7e, 0xbe, 0xad, 0xa5, 0x84, 0x7f, 0xa9, 0x30,
+ 0x5d, 0xdb, 0xe2, 0x05, 0x43, 0xf7, 0x1b, },
+ { 0x0b, 0x37, 0xd8, 0x02, 0xe1, 0x83, 0xd6, 0x80, 0xf2, 0x35, 0xc2, 0xb0,
+ 0x37, 0xef, 0xef, 0x5e, 0x43, 0x93, 0xf0, 0x49, 0x45, 0x0a, 0xef, 0xb5,
+ 0x76, 0x70, 0x12, 0x44, 0xc4, 0xdb, 0xf5, 0x7a, },
+ { 0x1f, },
+ { 0x82, 0x60, },
+ { 0xcc, 0xe3, 0x08, },
+ { 0x56, 0x17, 0xe4, 0x59, },
+ { 0xe2, 0xd7, 0x9e, 0xc4, 0x4c, },
+ { 0xb2, 0xad, 0xd3, 0x78, 0x58, 0x5a, },
+ { 0xce, 0x43, 0xb4, 0x02, 0x96, 0xab, 0x3c, },
+ { 0xe6, 0x05, 0x1a, 0x73, 0x22, 0x32, 0xbb, 0x77, },
+ { 0x23, 0xe7, 0xda, 0xfe, 0x2c, 0xef, 0x8c, 0x22, 0xec, },
+ { 0xe9, 0x8e, 0x55, 0x38, 0xd1, 0xd7, 0x35, 0x23, 0x98, 0xc7, },
+ { 0xb5, 0x81, 0x1a, 0xe5, 0xb5, 0xa5, 0xd9, 0x4d, 0xca, 0x41, 0xe7, },
+ { 0x41, 0x16, 0x16, 0x95, 0x8d, 0x9e, 0x0c, 0xea, 0x8c, 0x71, 0x9a, 0xc1, },
+ { 0x7c, 0x33, 0xc0, 0xa4, 0x00, 0x62, 0xea, 0x60, 0x67, 0xe4, 0x20, 0xbc,
+ 0x5b, },
+ { 0xdb, 0xb1, 0xdc, 0xfd, 0x08, 0xc0, 0xde, 0x82, 0xd1, 0xde, 0x38, 0xc0,
+ 0x90, 0x48, },
+ { 0x37, 0x18, 0x2e, 0x0d, 0x61, 0xaa, 0x61, 0xd7, 0x86, 0x20, 0x16, 0x60,
+ 0x04, 0xd9, 0xd5, },
+ { 0xb0, 0xcf, 0x2c, 0x4c, 0x5e, 0x5b, 0x4f, 0x2a, 0x23, 0x25, 0x58, 0x47,
+ 0xe5, 0x31, 0x06, 0x70, },
+ { 0x91, 0xa0, 0xa3, 0x86, 0x4e, 0xe0, 0x72, 0x38, 0x06, 0x67, 0x59, 0x5c,
+ 0x70, 0x25, 0xdb, 0x33, 0x27, },
+ { 0x44, 0x58, 0x66, 0xb8, 0x58, 0xc7, 0x13, 0xed, 0x4c, 0xc0, 0xf4, 0x9a,
+ 0x1e, 0x67, 0x75, 0x33, 0xb6, 0xb8, },
+ { 0x7f, 0x98, 0x4a, 0x8e, 0x50, 0xa2, 0x5c, 0xcd, 0x59, 0xde, 0x72, 0xb3,
+ 0x9d, 0xc3, 0x09, 0x8a, 0xab, 0x56, 0xf1, },
+ { 0x80, 0x96, 0x49, 0x1a, 0x59, 0xa2, 0xc5, 0xd5, 0xa7, 0x20, 0x8a, 0xb7,
+ 0x27, 0x62, 0x84, 0x43, 0xc6, 0xe1, 0x1b, 0x5d, },
+ { 0x6b, 0xb7, 0x2b, 0x26, 0x62, 0x14, 0x70, 0x19, 0x3d, 0x4d, 0xac, 0xac,
+ 0x63, 0x58, 0x5e, 0x94, 0xb5, 0xb7, 0xe8, 0xe8, 0xa2, },
+ { 0x20, 0xa8, 0xc0, 0xfd, 0x63, 0x3d, 0x6e, 0x98, 0xcf, 0x0c, 0x49, 0x98,
+ 0xe4, 0x5a, 0xfe, 0x8c, 0xaa, 0x70, 0x82, 0x1c, 0x7b, 0x74, },
+ { 0xc8, 0xe8, 0xdd, 0xdf, 0x69, 0x30, 0x01, 0xc2, 0x0f, 0x7e, 0x2f, 0x11,
+ 0xcc, 0x3e, 0x17, 0xa5, 0x69, 0x40, 0x3f, 0x0e, 0x79, 0x7f, 0xcf, },
+ { 0xdb, 0x61, 0xc0, 0xe2, 0x2e, 0x49, 0x07, 0x31, 0x1d, 0x91, 0x42, 0x8a,
+ 0xfc, 0x5e, 0xd3, 0xf8, 0x56, 0x1f, 0x2b, 0x73, 0xfd, 0x9f, 0xb2, 0x8e, },
+ { 0x0c, 0x89, 0x55, 0x0c, 0x1f, 0x59, 0x2c, 0x9d, 0x1b, 0x29, 0x1d, 0x41,
+ 0x1d, 0xe6, 0x47, 0x8f, 0x8c, 0x2b, 0xea, 0x8f, 0xf0, 0xff, 0x21, 0x70,
+ 0x88, },
+ { 0x12, 0x18, 0x95, 0xa6, 0x59, 0xb1, 0x31, 0x24, 0x45, 0x67, 0x55, 0xa4,
+ 0x1a, 0x2d, 0x48, 0x67, 0x1b, 0x43, 0x88, 0x2d, 0x8e, 0xa0, 0x70, 0xb3,
+ 0xc6, 0xbb, },
+ { 0xe7, 0xb1, 0x1d, 0xb2, 0x76, 0x4d, 0x68, 0x68, 0x68, 0x23, 0x02, 0x55,
+ 0x3a, 0xe2, 0xe5, 0xd5, 0x4b, 0x43, 0xf9, 0x34, 0x77, 0x5c, 0xa1, 0xf5,
+ 0x55, 0xfd, 0x4f, },
+ { 0x8c, 0x87, 0x5a, 0x08, 0x3a, 0x73, 0xad, 0x61, 0xe1, 0xe7, 0x99, 0x7e,
+ 0xf0, 0x5d, 0xe9, 0x5d, 0x16, 0x43, 0x80, 0x2f, 0xd0, 0x66, 0x34, 0xe2,
+ 0x42, 0x64, 0x3b, 0x1a, },
+ { 0x39, 0xc1, 0x99, 0xcf, 0x22, 0xbf, 0x16, 0x8f, 0x9f, 0x80, 0x7f, 0x95,
+ 0x0a, 0x05, 0x67, 0x27, 0xe7, 0x15, 0xdf, 0x9d, 0xb2, 0xfe, 0x1c, 0xb5,
+ 0x1d, 0x60, 0x8f, 0x8a, 0x1d, },
+ { 0x9b, 0x6e, 0x08, 0x09, 0x06, 0x73, 0xab, 0x68, 0x02, 0x62, 0x1a, 0xe4,
+ 0xd4, 0xdf, 0xc7, 0x02, 0x4c, 0x6a, 0x5f, 0xfd, 0x23, 0xac, 0xae, 0x6d,
+ 0x43, 0xa4, 0x7a, 0x50, 0x60, 0x3c, },
+ { 0x1d, 0xb4, 0xc6, 0xe1, 0xb1, 0x4b, 0xe3, 0xf2, 0xe2, 0x1a, 0x73, 0x1b,
+ 0xa0, 0x92, 0xa7, 0xf5, 0xff, 0x8f, 0x8b, 0x5d, 0xdf, 0xa8, 0x04, 0xb3,
+ 0xb0, 0xf7, 0xcc, 0x12, 0xfa, 0x35, 0x46, },
+ { 0x49, 0x45, 0x97, 0x11, 0x0f, 0x1c, 0x60, 0x8e, 0xe8, 0x47, 0x30, 0xcf,
+ 0x60, 0xa8, 0x71, 0xc5, 0x1b, 0xe9, 0x39, 0x4d, 0x49, 0xb6, 0x12, 0x1f,
+ 0x24, 0xab, 0x37, 0xff, 0x83, 0xc2, 0xe1, 0x3a, },
+ { 0x60, },
+ { 0x24, 0x26, },
+ { 0x47, 0xeb, 0xc9, },
+ { 0x4a, 0xd0, 0xbc, 0xf0, },
+ { 0x8e, 0x2b, 0xc9, 0x85, 0x3c, },
+ { 0xa2, 0x07, 0x15, 0xb8, 0x12, 0x74, },
+ { 0x0f, 0xdb, 0x5b, 0x33, 0x69, 0xfe, 0x4b, },
+ { 0xa2, 0x86, 0x54, 0xf4, 0xfd, 0xb2, 0xd4, 0xe6, },
+ { 0xbb, 0x84, 0x78, 0x49, 0x27, 0x8e, 0x61, 0xda, 0x60, },
+ { 0x04, 0xc3, 0xcd, 0xaa, 0x8f, 0xa7, 0x03, 0xc9, 0xf9, 0xb6, },
+ { 0xf8, 0x27, 0x1d, 0x61, 0xdc, 0x21, 0x42, 0xdd, 0xad, 0x92, 0x40, },
+ { 0x12, 0x87, 0xdf, 0xc2, 0x41, 0x45, 0x5a, 0x36, 0x48, 0x5b, 0x51, 0x2b, },
+ { 0xbb, 0x37, 0x5d, 0x1f, 0xf1, 0x68, 0x7a, 0xc4, 0xa5, 0xd2, 0xa4, 0x91,
+ 0x8d, },
+ { 0x5b, 0x27, 0xd1, 0x04, 0x54, 0x52, 0x9f, 0xa3, 0x47, 0x86, 0x33, 0x33,
+ 0xbf, 0xa0, },
+ { 0xcf, 0x04, 0xea, 0xf8, 0x03, 0x2a, 0x43, 0xff, 0xa6, 0x68, 0x21, 0x4c,
+ 0xd5, 0x4b, 0xed, },
+ { 0xaf, 0xb8, 0xbc, 0x63, 0x0f, 0x18, 0x4d, 0xe2, 0x7a, 0xdd, 0x46, 0x44,
+ 0xc8, 0x24, 0x0a, 0xb7, },
+ { 0x3e, 0xdc, 0x36, 0xe4, 0x89, 0xb1, 0xfa, 0xc6, 0x40, 0x93, 0x2e, 0x75,
+ 0xb2, 0x15, 0xd1, 0xb1, 0x10, },
+ { 0x6c, 0xd8, 0x20, 0x3b, 0x82, 0x79, 0xf9, 0xc8, 0xbc, 0x9d, 0xe0, 0x35,
+ 0xbe, 0x1b, 0x49, 0x1a, 0xbc, 0x3a, },
+ { 0x78, 0x65, 0x2c, 0xbe, 0x35, 0x67, 0xdc, 0x78, 0xd4, 0x41, 0xf6, 0xc9,
+ 0xde, 0xde, 0x1f, 0x18, 0x13, 0x31, 0x11, },
+ { 0x8a, 0x7f, 0xb1, 0x33, 0x8f, 0x0c, 0x3c, 0x0a, 0x06, 0x61, 0xf0, 0x47,
+ 0x29, 0x1b, 0x29, 0xbc, 0x1c, 0x47, 0xef, 0x7a, },
+ { 0x65, 0x91, 0xf1, 0xe6, 0xb3, 0x96, 0xd3, 0x8c, 0xc2, 0x4a, 0x59, 0x35,
+ 0x72, 0x8e, 0x0b, 0x9a, 0x87, 0xca, 0x34, 0x7b, 0x63, },
+ { 0x5f, 0x08, 0x87, 0x80, 0x56, 0x25, 0x89, 0x77, 0x61, 0x8c, 0x64, 0xa1,
+ 0x59, 0x6d, 0x59, 0x62, 0xe8, 0x4a, 0xc8, 0x58, 0x99, 0xd1, },
+ { 0x23, 0x87, 0x1d, 0xed, 0x6f, 0xf2, 0x91, 0x90, 0xe2, 0xfe, 0x43, 0x21,
+ 0xaf, 0x97, 0xc6, 0xbc, 0xd7, 0x15, 0xc7, 0x2d, 0x08, 0x77, 0x91, },
+ { 0x90, 0x47, 0x9a, 0x9e, 0x3a, 0xdf, 0xf3, 0xc9, 0x4c, 0x1e, 0xa7, 0xd4,
+ 0x6a, 0x32, 0x90, 0xfe, 0xb7, 0xb6, 0x7b, 0xfa, 0x96, 0x61, 0xfb, 0xa4, },
+ { 0xb1, 0x67, 0x60, 0x45, 0xb0, 0x96, 0xc5, 0x15, 0x9f, 0x4d, 0x26, 0xd7,
+ 0x9d, 0xf1, 0xf5, 0x6d, 0x21, 0x00, 0x94, 0x31, 0x64, 0x94, 0xd3, 0xa7,
+ 0xd3, },
+ { 0x02, 0x3e, 0xaf, 0xf3, 0x79, 0x73, 0xa5, 0xf5, 0xcc, 0x7a, 0x7f, 0xfb,
+ 0x79, 0x2b, 0x85, 0x8c, 0x88, 0x72, 0x06, 0xbe, 0xfe, 0xaf, 0xc1, 0x16,
+ 0xa6, 0xd6, },
+ { 0x2a, 0xb0, 0x1a, 0xe5, 0xaa, 0x6e, 0xb3, 0xae, 0x53, 0x85, 0x33, 0x80,
+ 0x75, 0xae, 0x30, 0xe6, 0xb8, 0x72, 0x42, 0xf6, 0x25, 0x4f, 0x38, 0x88,
+ 0x55, 0xd1, 0xa9, },
+ { 0x90, 0xd8, 0x0c, 0xc0, 0x93, 0x4b, 0x4f, 0x9e, 0x65, 0x6c, 0xa1, 0x54,
+ 0xa6, 0xf6, 0x6e, 0xca, 0xd2, 0xbb, 0x7e, 0x6a, 0x1c, 0xd3, 0xce, 0x46,
+ 0xef, 0xb0, 0x00, 0x8d, },
+ { 0xed, 0x9c, 0x49, 0xcd, 0xc2, 0xde, 0x38, 0x0e, 0xe9, 0x98, 0x6c, 0xc8,
+ 0x90, 0x9e, 0x3c, 0xd4, 0xd3, 0xeb, 0x88, 0x32, 0xc7, 0x28, 0xe3, 0x94,
+ 0x1c, 0x9f, 0x8b, 0xf3, 0xcb, },
+ { 0xac, 0xe7, 0x92, 0x16, 0xb4, 0x14, 0xa0, 0xe4, 0x04, 0x79, 0xa2, 0xf4,
+ 0x31, 0xe6, 0x0c, 0x26, 0xdc, 0xbf, 0x2f, 0x69, 0x1b, 0x55, 0x94, 0x67,
+ 0xda, 0x0c, 0xd7, 0x32, 0x1f, 0xef, },
+ { 0x68, 0x63, 0x85, 0x57, 0x95, 0x9e, 0x42, 0x27, 0x41, 0x43, 0x42, 0x02,
+ 0xa5, 0x78, 0xa7, 0xc6, 0x43, 0xc1, 0x6a, 0xba, 0x70, 0x80, 0xcd, 0x04,
+ 0xb6, 0x78, 0x76, 0x29, 0xf3, 0xe8, 0xa0, },
+ { 0xe6, 0xac, 0x8d, 0x9d, 0xf0, 0xc0, 0xf7, 0xf7, 0xe3, 0x3e, 0x4e, 0x28,
+ 0x0f, 0x59, 0xb2, 0x67, 0x9e, 0x84, 0x34, 0x42, 0x96, 0x30, 0x2b, 0xca,
+ 0x49, 0xb6, 0xc5, 0x9a, 0x84, 0x59, 0xa7, 0x81, },
+ { 0x7e, },
+ { 0x1e, 0x21, },
+ { 0x26, 0xd3, 0xdd, },
+ { 0x2c, 0xd4, 0xb3, 0x3d, },
+ { 0x86, 0x7b, 0x76, 0x3c, 0xf0, },
+ { 0x12, 0xc3, 0x70, 0x1d, 0x55, 0x18, },
+ { 0x96, 0xc2, 0xbd, 0x61, 0x55, 0xf4, 0x24, },
+ { 0x20, 0x51, 0xf7, 0x86, 0x58, 0x8f, 0x07, 0x2a, },
+ { 0x93, 0x15, 0xa8, 0x1d, 0xda, 0x97, 0xee, 0x0e, 0x6c, },
+ { 0x39, 0x93, 0xdf, 0xd5, 0x0e, 0xca, 0xdc, 0x7a, 0x92, 0xce, },
+ { 0x60, 0xd5, 0xfd, 0xf5, 0x1b, 0x26, 0x82, 0x26, 0x73, 0x02, 0xbc, },
+ { 0x98, 0xf2, 0x34, 0xe1, 0xf5, 0xfb, 0x00, 0xac, 0x10, 0x4a, 0x38, 0x9f, },
+ { 0xda, 0x3a, 0x92, 0x8a, 0xd0, 0xcd, 0x12, 0xcd, 0x15, 0xbb, 0xab, 0x77,
+ 0x66, },
+ { 0xa2, 0x92, 0x1a, 0xe5, 0xca, 0x0c, 0x30, 0x75, 0xeb, 0xaf, 0x00, 0x31,
+ 0x55, 0x66, },
+ { 0x06, 0xea, 0xfd, 0x3e, 0x86, 0x38, 0x62, 0x4e, 0xa9, 0x12, 0xa4, 0x12,
+ 0x43, 0xbf, 0xa1, },
+ { 0xe4, 0x71, 0x7b, 0x94, 0xdb, 0xa0, 0xd2, 0xff, 0x9b, 0xeb, 0xad, 0x8e,
+ 0x95, 0x8a, 0xc5, 0xed, },
+ { 0x25, 0x5a, 0x77, 0x71, 0x41, 0x0e, 0x7a, 0xe9, 0xed, 0x0c, 0x10, 0xef,
+ 0xf6, 0x2b, 0x3a, 0xba, 0x60, },
+ { 0xee, 0xe2, 0xa3, 0x67, 0x64, 0x1d, 0xc6, 0x04, 0xc4, 0xe1, 0x68, 0xd2,
+ 0x6e, 0xd2, 0x91, 0x75, 0x53, 0x07, },
+ { 0xe0, 0xf6, 0x4d, 0x8f, 0x68, 0xfc, 0x06, 0x7e, 0x18, 0x79, 0x7f, 0x2b,
+ 0x6d, 0xef, 0x46, 0x7f, 0xab, 0xb2, 0xad, },
+ { 0x3d, 0x35, 0x88, 0x9f, 0x2e, 0xcf, 0x96, 0x45, 0x07, 0x60, 0x71, 0x94,
+ 0x00, 0x8d, 0xbf, 0xf4, 0xef, 0x46, 0x2e, 0x3c, },
+ { 0x43, 0xcf, 0x98, 0xf7, 0x2d, 0xf4, 0x17, 0xe7, 0x8c, 0x05, 0x2d, 0x9b,
+ 0x24, 0xfb, 0x4d, 0xea, 0x4a, 0xec, 0x01, 0x25, 0x29, },
+ { 0x8e, 0x73, 0x9a, 0x78, 0x11, 0xfe, 0x48, 0xa0, 0x3b, 0x1a, 0x26, 0xdf,
+ 0x25, 0xe9, 0x59, 0x1c, 0x70, 0x07, 0x9f, 0xdc, 0xa0, 0xa6, },
+ { 0xe8, 0x47, 0x71, 0xc7, 0x3e, 0xdf, 0xb5, 0x13, 0xb9, 0x85, 0x13, 0xa8,
+ 0x54, 0x47, 0x6e, 0x59, 0x96, 0x09, 0x13, 0x5f, 0x82, 0x16, 0x0b, },
+ { 0xfb, 0xc0, 0x8c, 0x03, 0x21, 0xb3, 0xc4, 0xb5, 0x43, 0x32, 0x6c, 0xea,
+ 0x7f, 0xa8, 0x43, 0x91, 0xe8, 0x4e, 0x3f, 0xbf, 0x45, 0x58, 0x6a, 0xa3, },
+ { 0x55, 0xf8, 0xf3, 0x00, 0x76, 0x09, 0xef, 0x69, 0x5d, 0xd2, 0x8a, 0xf2,
+ 0x65, 0xc3, 0xcb, 0x9b, 0x43, 0xfd, 0xb1, 0x7e, 0x7f, 0xa1, 0x94, 0xb0,
+ 0xd7, },
+ { 0xaa, 0x13, 0xc1, 0x51, 0x40, 0x6d, 0x8d, 0x4c, 0x0a, 0x95, 0x64, 0x7b,
+ 0xd1, 0x96, 0xb6, 0x56, 0xb4, 0x5b, 0xcf, 0xd6, 0xd9, 0x15, 0x97, 0xdd,
+ 0xb6, 0xef, },
+ { 0xaf, 0xb7, 0x36, 0xb0, 0x04, 0xdb, 0xd7, 0x9c, 0x9a, 0x44, 0xc4, 0xf6,
+ 0x1f, 0x12, 0x21, 0x2d, 0x59, 0x30, 0x54, 0xab, 0x27, 0x61, 0xa3, 0x57,
+ 0xef, 0xf8, 0x53, },
+ { 0x97, 0x34, 0x45, 0x3e, 0xce, 0x7c, 0x35, 0xa2, 0xda, 0x9f, 0x4b, 0x46,
+ 0x6c, 0x11, 0x67, 0xff, 0x2f, 0x76, 0x58, 0x15, 0x71, 0xfa, 0x44, 0x89,
+ 0x89, 0xfd, 0xf7, 0x99, },
+ { 0x1f, 0xb1, 0x62, 0xeb, 0x83, 0xc5, 0x9c, 0x89, 0xf9, 0x2c, 0xd2, 0x03,
+ 0x61, 0xbc, 0xbb, 0xa5, 0x74, 0x0e, 0x9b, 0x7e, 0x82, 0x3e, 0x70, 0x0a,
+ 0xa9, 0x8f, 0x2b, 0x59, 0xfb, },
+ { 0xf8, 0xca, 0x5e, 0x3a, 0x4f, 0x9e, 0x10, 0x69, 0x10, 0xd5, 0x4c, 0xeb,
+ 0x1a, 0x0f, 0x3c, 0x6a, 0x98, 0xf5, 0xb0, 0x97, 0x5b, 0x37, 0x2f, 0x0d,
+ 0xbd, 0x42, 0x4b, 0x69, 0xa1, 0x82, },
+ { 0x12, 0x8c, 0x6d, 0x52, 0x08, 0xef, 0x74, 0xb2, 0xe6, 0xaa, 0xd3, 0xb0,
+ 0x26, 0xb0, 0xd9, 0x94, 0xb6, 0x11, 0x45, 0x0e, 0x36, 0x71, 0x14, 0x2d,
+ 0x41, 0x8c, 0x21, 0x53, 0x31, 0xe9, 0x68, },
+ { 0xee, 0xea, 0x0d, 0x89, 0x47, 0x7e, 0x72, 0xd1, 0xd8, 0xce, 0x58, 0x4c,
+ 0x94, 0x1f, 0x0d, 0x51, 0x08, 0xa3, 0xb6, 0x3d, 0xe7, 0x82, 0x46, 0x92,
+ 0xd6, 0x98, 0x6b, 0x07, 0x10, 0x65, 0x52, 0x65, },
+};
+
+static const u8 blake2s_hmac_testvecs[][BLAKE2S_HASH_SIZE] __initconst = {
+ { 0xce, 0xe1, 0x57, 0x69, 0x82, 0xdc, 0xbf, 0x43, 0xad, 0x56, 0x4c, 0x70,
+ 0xed, 0x68, 0x16, 0x96, 0xcf, 0xa4, 0x73, 0xe8, 0xe8, 0xfc, 0x32, 0x79,
+ 0x08, 0x0a, 0x75, 0x82, 0xda, 0x3f, 0x05, 0x11, },
+ { 0x77, 0x2f, 0x0c, 0x71, 0x41, 0xf4, 0x4b, 0x2b, 0xb3, 0xc6, 0xb6, 0xf9,
+ 0x60, 0xde, 0xe4, 0x52, 0x38, 0x66, 0xe8, 0xbf, 0x9b, 0x96, 0xc4, 0x9f,
+ 0x60, 0xd9, 0x24, 0x37, 0x99, 0xd6, 0xec, 0x31, },
+};
+
+bool __init blake2s_selftest(void)
+{
+ u8 key[BLAKE2S_KEY_SIZE];
+ u8 buf[ARRAY_SIZE(blake2s_testvecs)];
+ u8 hash[BLAKE2S_HASH_SIZE];
+ struct blake2s_state state;
+ bool success = true;
+ int i, l;
+
+ key[0] = key[1] = 1;
+ for (i = 2; i < sizeof(key); ++i)
+ key[i] = key[i - 2] + key[i - 1];
+
+ for (i = 0; i < sizeof(buf); ++i)
+ buf[i] = (u8)i;
+
+ for (i = l = 0; i < ARRAY_SIZE(blake2s_testvecs); l = (l + 37) % ++i) {
+ int outlen = 1 + i % BLAKE2S_HASH_SIZE;
+ int keylen = (13 * i) % (BLAKE2S_KEY_SIZE + 1);
+
+ blake2s(hash, buf, key + BLAKE2S_KEY_SIZE - keylen, outlen, i,
+ keylen);
+ if (memcmp(hash, blake2s_testvecs[i], outlen)) {
+ pr_err("blake2s self-test %d: FAIL\n", i + 1);
+ success = false;
+ }
+
+ if (!keylen)
+ blake2s_init(&state, outlen);
+ else
+ blake2s_init_key(&state, outlen,
+ key + BLAKE2S_KEY_SIZE - keylen,
+ keylen);
+
+ blake2s_update(&state, buf, l);
+ blake2s_update(&state, buf + l, i - l);
+ blake2s_final(&state, hash);
+ if (memcmp(hash, blake2s_testvecs[i], outlen)) {
+ pr_err("blake2s init/update/final self-test %d: FAIL\n",
+ i + 1);
+ success = false;
+ }
+ }
+
+ if (success) {
+ blake2s256_hmac(hash, buf, key, sizeof(buf), sizeof(key));
+ success &= !memcmp(hash, blake2s_hmac_testvecs[0], BLAKE2S_HASH_SIZE);
+
+ blake2s256_hmac(hash, key, buf, sizeof(key), sizeof(buf));
+ success &= !memcmp(hash, blake2s_hmac_testvecs[1], BLAKE2S_HASH_SIZE);
+
+ if (!success)
+ pr_err("blake2s256_hmac self-test: FAIL\n");
+ }
+
+ return success;
+}
--- /dev/null
+++ b/lib/crypto/blake2s.c
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ *
+ * This is an implementation of the BLAKE2s hash and PRF functions.
+ *
+ * Information: https://blake2.net/
+ *
+ */
+
+#include <crypto/internal/blake2s.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/bug.h>
+#include <asm/unaligned.h>
+
+bool blake2s_selftest(void);
+
+void blake2s_update(struct blake2s_state *state, const u8 *in, size_t inlen)
+{
+ const size_t fill = BLAKE2S_BLOCK_SIZE - state->buflen;
+
+ if (unlikely(!inlen))
+ return;
+ if (inlen > fill) {
+ memcpy(state->buf + state->buflen, in, fill);
+ blake2s_compress_generic(state, state->buf, 1,
+ BLAKE2S_BLOCK_SIZE);
+ state->buflen = 0;
+ in += fill;
+ inlen -= fill;
+ }
+ if (inlen > BLAKE2S_BLOCK_SIZE) {
+ const size_t nblocks = DIV_ROUND_UP(inlen, BLAKE2S_BLOCK_SIZE);
+ /* Hash one less (full) block than strictly possible */
+ blake2s_compress_generic(state, in, nblocks - 1,
+ BLAKE2S_BLOCK_SIZE);
+ in += BLAKE2S_BLOCK_SIZE * (nblocks - 1);
+ inlen -= BLAKE2S_BLOCK_SIZE * (nblocks - 1);
+ }
+ memcpy(state->buf + state->buflen, in, inlen);
+ state->buflen += inlen;
+}
+EXPORT_SYMBOL(blake2s_update);
+
+void blake2s_final(struct blake2s_state *state, u8 *out)
+{
+ WARN_ON(IS_ENABLED(DEBUG) && !out);
+ blake2s_set_lastblock(state);
+ memset(state->buf + state->buflen, 0,
+ BLAKE2S_BLOCK_SIZE - state->buflen); /* Padding */
+ blake2s_compress_generic(state, state->buf, 1, state->buflen);
+ cpu_to_le32_array(state->h, ARRAY_SIZE(state->h));
+ memcpy(out, state->h, state->outlen);
+ memzero_explicit(state, sizeof(*state));
+}
+EXPORT_SYMBOL(blake2s_final);
+
+void blake2s256_hmac(u8 *out, const u8 *in, const u8 *key, const size_t inlen,
+ const size_t keylen)
+{
+ struct blake2s_state state;
+ u8 x_key[BLAKE2S_BLOCK_SIZE] __aligned(__alignof__(u32)) = { 0 };
+ u8 i_hash[BLAKE2S_HASH_SIZE] __aligned(__alignof__(u32));
+ int i;
+
+ if (keylen > BLAKE2S_BLOCK_SIZE) {
+ blake2s_init(&state, BLAKE2S_HASH_SIZE);
+ blake2s_update(&state, key, keylen);
+ blake2s_final(&state, x_key);
+ } else
+ memcpy(x_key, key, keylen);
+
+ for (i = 0; i < BLAKE2S_BLOCK_SIZE; ++i)
+ x_key[i] ^= 0x36;
+
+ blake2s_init(&state, BLAKE2S_HASH_SIZE);
+ blake2s_update(&state, x_key, BLAKE2S_BLOCK_SIZE);
+ blake2s_update(&state, in, inlen);
+ blake2s_final(&state, i_hash);
+
+ for (i = 0; i < BLAKE2S_BLOCK_SIZE; ++i)
+ x_key[i] ^= 0x5c ^ 0x36;
+
+ blake2s_init(&state, BLAKE2S_HASH_SIZE);
+ blake2s_update(&state, x_key, BLAKE2S_BLOCK_SIZE);
+ blake2s_update(&state, i_hash, BLAKE2S_HASH_SIZE);
+ blake2s_final(&state, i_hash);
+
+ memcpy(out, i_hash, BLAKE2S_HASH_SIZE);
+ memzero_explicit(x_key, BLAKE2S_BLOCK_SIZE);
+ memzero_explicit(i_hash, BLAKE2S_HASH_SIZE);
+}
+EXPORT_SYMBOL(blake2s256_hmac);
+
+static int __init mod_init(void)
+{
+ if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
+ WARN_ON(!blake2s_selftest()))
+ return -ENODEV;
+ return 0;
+}
+
+static void __exit mod_exit(void)
+{
+}
+
+module_init(mod_init);
+module_exit(mod_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("BLAKE2s hash function");
+MODULE_AUTHOR("Jason A. Donenfeld <[email protected]>");
From: Fabio Estevam <[email protected]>
commit db61ffe3a71c697aaa91c42c862a5f7557a0e562 upstream.
Building arm allnodefconfig causes the following build warning:
drivers/char/random.c:318:12: warning: 'random_min_urandom_seed' defined but not used [-Wunused-variable]
Fix the warning by moving 'random_min_urandom_seed' declaration inside
the CONFIG_SYSCTL ifdef block, where it is actually used.
While at it, remove the comment prior to the variable declaration.
Signed-off-by: Fabio Estevam <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -314,11 +314,6 @@ static int random_read_wakeup_bits = 64;
static int random_write_wakeup_bits = 28 * OUTPUT_POOL_WORDS;
/*
- * Variable is currently unused by left for user space compatibility.
- */
-static int random_min_urandom_seed = 60;
-
-/*
* Originally, we used a primitive polynomial of degree .poolwords
* over GF(2). The taps for various sizes are defined below. They
* were chosen to be evenly spaced except for the last tap, which is 1
@@ -1957,6 +1952,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
static int min_read_thresh = 8, min_write_thresh;
static int max_read_thresh = OUTPUT_POOL_WORDS * 32;
static int max_write_thresh = INPUT_POOL_WORDS * 32;
+static int random_min_urandom_seed = 60;
static char sysctl_bootid[16];
/*
From: "Jason A. Donenfeld" <[email protected]>
commit da3951ebdcd1cb1d5c750e08cd05aee7b0c04d9a upstream.
When the interrupt handler does not have a valid cycle counter, it calls
get_reg() to read a register from the irq stack, in round-robin.
Currently it does this assuming that registers are 32-bit. This is
_probably_ the case, and probably all platforms without cycle counters
are in fact 32-bit platforms. But maybe not, and either way, it's not
quite correct. This commit fixes that to deal with `unsigned long`
rather than `u32`.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1257,15 +1257,15 @@ int random_online_cpu(unsigned int cpu)
}
#endif
-static u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
+static unsigned long get_reg(struct fast_pool *f, struct pt_regs *regs)
{
- u32 *ptr = (u32 *)regs;
+ unsigned long *ptr = (unsigned long *)regs;
unsigned int idx;
if (regs == NULL)
return 0;
idx = READ_ONCE(f->reg_idx);
- if (idx >= sizeof(struct pt_regs) / sizeof(u32))
+ if (idx >= sizeof(struct pt_regs) / sizeof(unsigned long))
idx = 0;
ptr += idx++;
WRITE_ONCE(f->reg_idx, idx);
From: Andy Lutomirski <[email protected]>
commit 4c8d062186d9923c09488716b2fb1b829b5b8006 upstream.
crng_init_wait is only used to wayt for crng_init to be set to 2, so
there's no point to waking it when crng_init is set to 1. Remove the
unnecessary wake_up_interruptible() call.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/6fbc0bfcbfc1fa2c76fd574f5b6f552b11be7fde.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 1 -
1 file changed, 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -963,7 +963,6 @@ static int crng_fast_load(const char *cp
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
invalidate_batched_entropy();
crng_init = 1;
- wake_up_interruptible(&crng_init_wait);
pr_notice("random: fast init done\n");
}
return 1;
From: "Jason A. Donenfeld" <[email protected]>
commit a07fdae346c35c6ba286af1c88e0effcfa330bf9 upstream.
Convert the current license into the SPDX notation of "(GPL-2.0 OR
BSD-3-Clause)". This infers GPL-2.0 from the text "ALTERNATIVELY, this
product may be distributed under the terms of the GNU General Public
License, in which case the provisions of the GPL are required INSTEAD OF
the above restrictions" and it infers BSD-3-Clause from the verbatim
BSD 3 clause license in the file.
Cc: Thomas Gleixner <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Reviewed-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 37 +------------------------------------
1 file changed, 1 insertion(+), 36 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1,44 +1,9 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
/*
- * random.c -- A strong random number generator
- *
* Copyright (C) 2017-2022 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
- *
* Copyright Matt Mackall <[email protected]>, 2003, 2004, 2005
- *
* Copyright Theodore Ts'o, 1994, 1995, 1996, 1997, 1998, 1999. All
* rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, and the entire permission notice in its entirety,
- * including the disclaimer of warranties.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- * 3. The name of the author may not be used to endorse or promote
- * products derived from this software without specific prior
- * written permission.
- *
- * ALTERNATIVELY, this product may be distributed under the terms of
- * the GNU General Public License, in which case the provisions of the GPL are
- * required INSTEAD OF the above restrictions. (This clause is
- * necessary due to a potential bad interaction between the GPL and
- * the restrictions contained in a BSD-style copyright.)
- *
- * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
- * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
- * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF
- * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
- * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
- * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
- * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
- * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH
- * DAMAGE.
*/
/*
From: "Jason A. Donenfeld" <[email protected]>
commit 8865bbe6ba1120e67f72201b7003a16202cd42be upstream.
PA-RISC defines a get_cycles() function, but it does not do the usual
`#define get_cycles get_cycles` dance, making it impossible for generic
code to see if an arch-specific function was defined. While the
get_cycles() ifdef is not currently used, the following timekeeping
patch in this series will depend on the macro existing (or not existing)
when defining random_get_entropy().
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Acked-by: Helge Deller <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/parisc/include/asm/timex.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--- a/arch/parisc/include/asm/timex.h
+++ b/arch/parisc/include/asm/timex.h
@@ -11,9 +11,10 @@
typedef unsigned long cycles_t;
-static inline cycles_t get_cycles (void)
+static inline cycles_t get_cycles(void)
{
return mfctl(16);
}
+#define get_cycles get_cycles
#endif
From: Yangtao Li <[email protected]>
commit 12cd53aff5ea0359b1dac91fcd9ddc7b9e646588 upstream.
Prefix all printk/pr_<level> messages with "random: " to make the
logging a bit more consistent.
Miscellanea:
o Convert a printks to pr_notice
o Whitespace to align to open parentheses
o Remove embedded "random: " from pr_* as pr_fmt adds it
Signed-off-by: Yangtao Li <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -307,6 +307,8 @@
* Eastlake, Steve Crocker, and Jeff Schiller.
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/utsname.h>
#include <linux/module.h>
#include <linux/kernel.h>
@@ -740,7 +742,7 @@ retry:
}
if (WARN_ON(entropy_count < 0)) {
- pr_warn("random: negative entropy/overflow: pool %s count %d\n",
+ pr_warn("negative entropy/overflow: pool %s count %d\n",
r->name, entropy_count);
entropy_count = 0;
} else if (entropy_count > pool_size)
@@ -833,7 +835,7 @@ static void crng_initialize(struct crng_
}
if (trust_cpu && arch_init) {
crng_init = 2;
- pr_notice("random: crng done (trusting CPU's manufacturer)\n");
+ pr_notice("crng done (trusting CPU's manufacturer)\n");
}
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
@@ -857,14 +859,12 @@ static void crng_finalize_init(struct cr
kill_fasync(&fasync, SIGIO, POLL_IN);
pr_notice("crng init done\n");
if (unseeded_warning.missed) {
- pr_notice("random: %d get_random_xx warning(s) missed "
- "due to ratelimiting\n",
+ pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
unseeded_warning.missed);
unseeded_warning.missed = 0;
}
if (urandom_warning.missed) {
- pr_notice("random: %d urandom warning(s) missed "
- "due to ratelimiting\n",
+ pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
urandom_warning.missed);
urandom_warning.missed = 0;
}
@@ -945,7 +945,7 @@ static int crng_fast_load(const char *cp
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
invalidate_batched_entropy();
crng_init = 1;
- pr_notice("random: fast init done\n");
+ pr_notice("fast init done\n");
}
return 1;
}
@@ -1432,7 +1432,7 @@ retry:
ibytes = 0;
if (WARN_ON(entropy_count < 0)) {
- pr_warn("random: negative entropy count: pool %s count %d\n",
+ pr_warn("negative entropy count: pool %s count %d\n",
r->name, entropy_count);
entropy_count = 0;
}
@@ -1857,9 +1857,8 @@ urandom_read(struct file *file, char __u
if (!crng_ready() && maxwarn > 0) {
maxwarn--;
if (__ratelimit(&urandom_warning))
- printk(KERN_NOTICE "random: %s: uninitialized "
- "urandom read (%zd bytes read)\n",
- current->comm, nbytes);
+ pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
+ current->comm, nbytes);
spin_lock_irqsave(&primary_crng.lock, flags);
crng_init_cnt = 0;
spin_unlock_irqrestore(&primary_crng.lock, flags);
From: Richard Henderson <[email protected]>
commit 5f2ed7f5b99b54389b74e53309677831ac9cb9d7 upstream.
Use the expansion of these macros directly in arch_get_random_*.
These symbols are currently part of the generic archrandom.h
interface, but are currently unused and can be removed.
Signed-off-by: Richard Henderson <[email protected]>
Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/x86/include/asm/archrandom.h | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
--- a/arch/x86/include/asm/archrandom.h
+++ b/arch/x86/include/asm/archrandom.h
@@ -86,10 +86,6 @@ static inline bool rdseed_int(unsigned i
return ok;
}
-/* Conditional execution based on CPU type */
-#define arch_has_random() static_cpu_has(X86_FEATURE_RDRAND)
-#define arch_has_random_seed() static_cpu_has(X86_FEATURE_RDSEED)
-
/*
* These are the generic interfaces; they must not be declared if the
* stubs in <linux/random.h> are to be invoked,
@@ -99,22 +95,22 @@ static inline bool rdseed_int(unsigned i
static inline bool arch_get_random_long(unsigned long *v)
{
- return arch_has_random() ? rdrand_long(v) : false;
+ return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false;
}
static inline bool arch_get_random_int(unsigned int *v)
{
- return arch_has_random() ? rdrand_int(v) : false;
+ return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false;
}
static inline bool arch_get_random_seed_long(unsigned long *v)
{
- return arch_has_random_seed() ? rdseed_long(v) : false;
+ return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false;
}
static inline bool arch_get_random_seed_int(unsigned int *v)
{
- return arch_has_random_seed() ? rdseed_int(v) : false;
+ return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false;
}
extern void x86_init_rdrand(struct cpuinfo_x86 *c);
From: "Jason A. Donenfeld" <[email protected]>
commit 14c174633f349cb41ea90c2c0aaddac157012f74 upstream.
These explicit tracepoints aren't really used and show sign of aging.
It's work to keep these up to date, and before I attempted to keep them
up to date, they weren't up to date, which indicates that they're not
really used. These days there are better ways of introspecting anyway.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 30 ------
include/trace/events/random.h | 194 ------------------------------------------
lib/random32.c | 2
3 files changed, 5 insertions(+), 221 deletions(-)
delete mode 100644 include/trace/events/random.h
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -238,9 +238,6 @@
#include <asm/irq_regs.h>
#include <asm/io.h>
-#define CREATE_TRACE_POINTS
-#include <trace/events/random.h>
-
enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
@@ -316,7 +313,6 @@ static void mix_pool_bytes(const void *i
{
unsigned long flags;
- trace_mix_pool_bytes(nbytes, _RET_IP_);
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(in, nbytes);
spin_unlock_irqrestore(&input_pool.lock, flags);
@@ -390,8 +386,6 @@ static void credit_entropy_bits(size_t n
entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
} while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
- trace_credit_entropy_bits(nbits, entropy_count, _RET_IP_);
-
if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
crng_reseed();
}
@@ -771,7 +765,6 @@ void add_device_randomness(const void *b
if (!crng_ready() && size)
crng_slow_load(buf, size);
- trace_add_device_randomness(size, _RET_IP_);
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(buf, size);
_mix_pool_bytes(&time, sizeof(time));
@@ -850,7 +843,6 @@ void add_input_randomness(unsigned int t
last_value = value;
add_timer_randomness(&input_timer_state,
(type << 4) ^ code ^ (code >> 4) ^ value);
- trace_add_input_randomness(input_pool.entropy_count);
}
EXPORT_SYMBOL_GPL(add_input_randomness);
@@ -930,7 +922,6 @@ void add_disk_randomness(struct gendisk
return;
/* first major is 1, so we get >= 0x200 here */
add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
- trace_add_disk_randomness(disk_devt(disk), input_pool.entropy_count);
}
EXPORT_SYMBOL_GPL(add_disk_randomness);
#endif
@@ -955,8 +946,6 @@ static void extract_entropy(void *buf, s
} block;
size_t i;
- trace_extract_entropy(nbytes, input_pool.entropy_count);
-
for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) {
if (!arch_get_random_seed_long(&block.rdseed[i]) &&
!arch_get_random_long(&block.rdseed[i]))
@@ -1028,8 +1017,6 @@ static void _get_random_bytes(void *buf,
u8 tmp[CHACHA20_BLOCK_SIZE];
size_t len;
- trace_get_random_bytes(nbytes, _RET_IP_);
-
if (!nbytes)
return;
@@ -1226,7 +1213,6 @@ size_t __must_check get_random_bytes_arc
size_t left = nbytes;
u8 *p = buf;
- trace_get_random_bytes_arch(left, _RET_IP_);
while (left) {
unsigned long v;
size_t chunk = min_t(size_t, left, sizeof(unsigned long));
@@ -1310,16 +1296,6 @@ void rand_initialize_disk(struct gendisk
}
#endif
-static ssize_t urandom_read_nowarn(struct file *file, char __user *buf,
- size_t nbytes, loff_t *ppos)
-{
- ssize_t ret;
-
- ret = get_random_bytes_user(buf, nbytes);
- trace_urandom_read(nbytes, input_pool.entropy_count);
- return ret;
-}
-
static ssize_t urandom_read(struct file *file, char __user *buf, size_t nbytes,
loff_t *ppos)
{
@@ -1332,7 +1308,7 @@ static ssize_t urandom_read(struct file
current->comm, nbytes);
}
- return urandom_read_nowarn(file, buf, nbytes, ppos);
+ return get_random_bytes_user(buf, nbytes);
}
static ssize_t random_read(struct file *file, char __user *buf, size_t nbytes,
@@ -1343,7 +1319,7 @@ static ssize_t random_read(struct file *
ret = wait_for_random_bytes();
if (ret != 0)
return ret;
- return urandom_read_nowarn(file, buf, nbytes, ppos);
+ return get_random_bytes_user(buf, nbytes);
}
static unsigned int random_poll(struct file *file, poll_table *wait)
@@ -1502,7 +1478,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
if (unlikely(ret))
return ret;
}
- return urandom_read_nowarn(NULL, buf, count, NULL);
+ return get_random_bytes_user(buf, count);
}
/********************************************************************
--- a/include/trace/events/random.h
+++ /dev/null
@@ -1,194 +0,0 @@
-#undef TRACE_SYSTEM
-#define TRACE_SYSTEM random
-
-#if !defined(_TRACE_RANDOM_H) || defined(TRACE_HEADER_MULTI_READ)
-#define _TRACE_RANDOM_H
-
-#include <linux/writeback.h>
-#include <linux/tracepoint.h>
-
-TRACE_EVENT(add_device_randomness,
- TP_PROTO(size_t bytes, unsigned long IP),
-
- TP_ARGS(bytes, IP),
-
- TP_STRUCT__entry(
- __field(size_t, bytes )
- __field(unsigned long, IP )
- ),
-
- TP_fast_assign(
- __entry->bytes = bytes;
- __entry->IP = IP;
- ),
-
- TP_printk("bytes %zu caller %pS",
- __entry->bytes, (void *)__entry->IP)
-);
-
-DECLARE_EVENT_CLASS(random__mix_pool_bytes,
- TP_PROTO(size_t bytes, unsigned long IP),
-
- TP_ARGS(bytes, IP),
-
- TP_STRUCT__entry(
- __field(size_t, bytes )
- __field(unsigned long, IP )
- ),
-
- TP_fast_assign(
- __entry->bytes = bytes;
- __entry->IP = IP;
- ),
-
- TP_printk("input pool: bytes %zu caller %pS",
- __entry->bytes, (void *)__entry->IP)
-);
-
-DEFINE_EVENT(random__mix_pool_bytes, mix_pool_bytes,
- TP_PROTO(size_t bytes, unsigned long IP),
-
- TP_ARGS(bytes, IP)
-);
-
-DEFINE_EVENT(random__mix_pool_bytes, mix_pool_bytes_nolock,
- TP_PROTO(int bytes, unsigned long IP),
-
- TP_ARGS(bytes, IP)
-);
-
-TRACE_EVENT(credit_entropy_bits,
- TP_PROTO(size_t bits, size_t entropy_count, unsigned long IP),
-
- TP_ARGS(bits, entropy_count, IP),
-
- TP_STRUCT__entry(
- __field(size_t, bits )
- __field(size_t, entropy_count )
- __field(unsigned long, IP )
- ),
-
- TP_fast_assign(
- __entry->bits = bits;
- __entry->entropy_count = entropy_count;
- __entry->IP = IP;
- ),
-
- TP_printk("input pool: bits %zu entropy_count %zu caller %pS",
- __entry->bits, __entry->entropy_count, (void *)__entry->IP)
-);
-
-TRACE_EVENT(add_input_randomness,
- TP_PROTO(size_t input_bits),
-
- TP_ARGS(input_bits),
-
- TP_STRUCT__entry(
- __field(size_t, input_bits )
- ),
-
- TP_fast_assign(
- __entry->input_bits = input_bits;
- ),
-
- TP_printk("input_pool_bits %zu", __entry->input_bits)
-);
-
-TRACE_EVENT(add_disk_randomness,
- TP_PROTO(dev_t dev, size_t input_bits),
-
- TP_ARGS(dev, input_bits),
-
- TP_STRUCT__entry(
- __field(dev_t, dev )
- __field(size_t, input_bits )
- ),
-
- TP_fast_assign(
- __entry->dev = dev;
- __entry->input_bits = input_bits;
- ),
-
- TP_printk("dev %d,%d input_pool_bits %zu", MAJOR(__entry->dev),
- MINOR(__entry->dev), __entry->input_bits)
-);
-
-DECLARE_EVENT_CLASS(random__get_random_bytes,
- TP_PROTO(size_t nbytes, unsigned long IP),
-
- TP_ARGS(nbytes, IP),
-
- TP_STRUCT__entry(
- __field(size_t, nbytes )
- __field(unsigned long, IP )
- ),
-
- TP_fast_assign(
- __entry->nbytes = nbytes;
- __entry->IP = IP;
- ),
-
- TP_printk("nbytes %zu caller %pS", __entry->nbytes, (void *)__entry->IP)
-);
-
-DEFINE_EVENT(random__get_random_bytes, get_random_bytes,
- TP_PROTO(size_t nbytes, unsigned long IP),
-
- TP_ARGS(nbytes, IP)
-);
-
-DEFINE_EVENT(random__get_random_bytes, get_random_bytes_arch,
- TP_PROTO(size_t nbytes, unsigned long IP),
-
- TP_ARGS(nbytes, IP)
-);
-
-DECLARE_EVENT_CLASS(random__extract_entropy,
- TP_PROTO(size_t nbytes, size_t entropy_count),
-
- TP_ARGS(nbytes, entropy_count),
-
- TP_STRUCT__entry(
- __field( size_t, nbytes )
- __field( size_t, entropy_count )
- ),
-
- TP_fast_assign(
- __entry->nbytes = nbytes;
- __entry->entropy_count = entropy_count;
- ),
-
- TP_printk("input pool: nbytes %zu entropy_count %zu",
- __entry->nbytes, __entry->entropy_count)
-);
-
-
-DEFINE_EVENT(random__extract_entropy, extract_entropy,
- TP_PROTO(size_t nbytes, size_t entropy_count),
-
- TP_ARGS(nbytes, entropy_count)
-);
-
-TRACE_EVENT(urandom_read,
- TP_PROTO(size_t nbytes, size_t entropy_count),
-
- TP_ARGS(nbytes, entropy_count),
-
- TP_STRUCT__entry(
- __field( size_t, nbytes )
- __field( size_t, entropy_count )
- ),
-
- TP_fast_assign(
- __entry->nbytes = nbytes;
- __entry->entropy_count = entropy_count;
- ),
-
- TP_printk("reading: nbytes %zu entropy_count %zu",
- __entry->nbytes, __entry->entropy_count)
-);
-
-#endif /* _TRACE_RANDOM_H */
-
-/* This part must be outside protection */
-#include <trace/define_trace.h>
--- a/lib/random32.c
+++ b/lib/random32.c
@@ -37,6 +37,8 @@
#include <linux/jiffies.h>
#include <linux/random.h>
#include <linux/sched.h>
+#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
/**
From: "Jason A. Donenfeld" <[email protected]>
commit 5b87adf30f1464477169a1d653e9baf8c012bbfe upstream.
We already had the POOL_* constants, so deduplicate the older INPUT_POOL
ones. As well, fold EXTRACT_SIZE into the poolinfo enum, since it's
related.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 17 ++++++-----------
1 file changed, 6 insertions(+), 11 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -360,13 +360,6 @@
/* #define ADD_INTERRUPT_BENCH */
/*
- * Configuration information
- */
-#define INPUT_POOL_SHIFT 12
-#define INPUT_POOL_WORDS (1 << (INPUT_POOL_SHIFT-5))
-#define EXTRACT_SIZE (BLAKE2S_HASH_SIZE / 2)
-
-/*
* To allow fractional bits to be tracked, the entropy_count field is
* denominated in units of 1/8th bits.
*
@@ -441,7 +434,9 @@ enum poolinfo {
POOL_TAP2 = 76,
POOL_TAP3 = 51,
POOL_TAP4 = 25,
- POOL_TAP5 = 1
+ POOL_TAP5 = 1,
+
+ EXTRACT_SIZE = BLAKE2S_HASH_SIZE / 2
};
/*
@@ -504,7 +499,7 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
*
**********************************************************************/
-static u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
+static u32 input_pool_data[POOL_WORDS] __latent_entropy;
static struct {
/* read-only data: */
@@ -2009,7 +2004,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
#include <linux/sysctl.h>
static int min_write_thresh;
-static int max_write_thresh = INPUT_POOL_WORDS * 32;
+static int max_write_thresh = POOL_BITS;
static int random_min_urandom_seed = 60;
static char sysctl_bootid[16];
@@ -2066,7 +2061,7 @@ static int proc_do_entropy(struct ctl_ta
return proc_dointvec(&fake_table, write, buffer, lenp, ppos);
}
-static int sysctl_poolsize = INPUT_POOL_WORDS * 32;
+static int sysctl_poolsize = POOL_BITS;
extern struct ctl_table random_table[];
struct ctl_table random_table[] = {
{
From: Sebastian Andrzej Siewior <[email protected]>
commit 72e5c740f6335e27253b8ff64d23d00337091535 upstream.
Avoid the READ_ONCE in commit 4a072c71f49b ("random: silence compiler
warnings and fix race") if we can leave the function after
arch_get_random_XXX().
Cc: Jason A. Donenfeld <[email protected]>
Signed-off-by: Sebastian Andrzej Siewior <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2161,7 +2161,7 @@ static DEFINE_PER_CPU(struct batched_ent
u64 get_random_u64(void)
{
u64 ret;
- bool use_lock = READ_ONCE(crng_init) < 2;
+ bool use_lock;
unsigned long flags = 0;
struct batched_entropy *batch;
static void *previous;
@@ -2177,6 +2177,7 @@ u64 get_random_u64(void)
warn_unseeded_randomness(&previous);
+ use_lock = READ_ONCE(crng_init) < 2;
batch = &get_cpu_var(batched_entropy_u64);
if (use_lock)
read_lock_irqsave(&batched_entropy_reset_lock, flags);
@@ -2196,7 +2197,7 @@ static DEFINE_PER_CPU(struct batched_ent
u32 get_random_u32(void)
{
u32 ret;
- bool use_lock = READ_ONCE(crng_init) < 2;
+ bool use_lock;
unsigned long flags = 0;
struct batched_entropy *batch;
static void *previous;
@@ -2206,6 +2207,7 @@ u32 get_random_u32(void)
warn_unseeded_randomness(&previous);
+ use_lock = READ_ONCE(crng_init) < 2;
batch = &get_cpu_var(batched_entropy_u32);
if (use_lock)
read_lock_irqsave(&batched_entropy_reset_lock, flags);
From: Andy Lutomirski <[email protected]>
commit c95ea0c69ffda19381c116db2be23c7e654dac98 upstream.
It has no effect any more, so remove it. We can revert this if
there is some user code that expects to be able to set this sysctl.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/a74ed2cf0b5a5451428a246a9239f5bc4e29358f.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 18 +-----------------
1 file changed, 1 insertion(+), 17 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -371,12 +371,6 @@
#define ENTROPY_BITS(r) ((r)->entropy_count >> ENTROPY_SHIFT)
/*
- * The minimum number of bits of entropy before we wake up a read on
- * /dev/random. Should be enough to do a significant reseed.
- */
-static int random_read_wakeup_bits = 64;
-
-/*
* If the entropy count falls under this number of bits, then we
* should wake up processes which are selecting or polling on write
* access to /dev/random.
@@ -2061,8 +2055,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
#include <linux/sysctl.h>
-static int min_read_thresh = 8, min_write_thresh;
-static int max_read_thresh = OUTPUT_POOL_WORDS * 32;
+static int min_write_thresh;
static int max_write_thresh = INPUT_POOL_WORDS * 32;
static int random_min_urandom_seed = 60;
static char sysctl_bootid[16];
@@ -2138,15 +2131,6 @@ struct ctl_table random_table[] = {
.data = &input_pool.entropy_count,
},
{
- .procname = "read_wakeup_threshold",
- .data = &random_read_wakeup_bits,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &min_read_thresh,
- .extra2 = &max_read_thresh,
- },
- {
.procname = "write_wakeup_threshold",
.data = &random_write_wakeup_bits,
.maxlen = sizeof(int),
From: Corentin LABBE <[email protected]>
commit 6bc17d90e62d16828d1a2113b54cfa4e04582fb6 upstream.
This patch fix the checkpatch warning "Please don't use multiple blank lines"
Signed-off-by: Corentin Labbe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 5 -----
1 file changed, 5 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -30,7 +30,6 @@
*/
-
#include <linux/device.h>
#include <linux/hw_random.h>
#include <linux/module.h>
@@ -45,12 +44,10 @@
#include <linux/err.h>
#include <asm/uaccess.h>
-
#define RNG_MODULE_NAME "hw_random"
#define PFX RNG_MODULE_NAME ": "
#define RNG_MISCDEV_MINOR 183 /* official */
-
static struct hwrng *current_rng;
static struct task_struct *hwrng_fill;
static LIST_HEAD(rng_list);
@@ -296,7 +293,6 @@ out_put:
goto out;
}
-
static const struct file_operations rng_chrdev_ops = {
.owner = THIS_MODULE,
.open = rng_dev_open,
@@ -314,7 +310,6 @@ static struct miscdevice rng_miscdev = {
.groups = rng_dev_groups,
};
-
static ssize_t hwrng_attr_current_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t len)
From: "Jason A. Donenfeld" <[email protected]>
commit 3bd4abc07a267e6a8b33d7f8717136e18f921c53 upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is suboptimal. Instead, fallback
to calling random_get_entropy_fallback(), which isn't extremely high
precision or guaranteed to be entropic, but is certainly better than
returning zero all the time.
If CONFIG_X86_TSC=n, then it's possible for the kernel to run on systems
without RDTSC, such as 486 and certain 586, so the fallback code is only
required for that case.
As well, fix up both the new function and the get_cycles() function from
which it was derived to use cpu_feature_enabled() rather than
boot_cpu_has(), and use !IS_ENABLED() instead of #ifndef.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: [email protected]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/x86/include/asm/timex.h | 9 +++++++++
arch/x86/include/asm/tsc.h | 7 +++----
2 files changed, 12 insertions(+), 4 deletions(-)
--- a/arch/x86/include/asm/timex.h
+++ b/arch/x86/include/asm/timex.h
@@ -4,6 +4,15 @@
#include <asm/processor.h>
#include <asm/tsc.h>
+static inline unsigned long random_get_entropy(void)
+{
+ if (!IS_ENABLED(CONFIG_X86_TSC) &&
+ !cpu_feature_enabled(X86_FEATURE_TSC))
+ return random_get_entropy_fallback();
+ return rdtsc();
+}
+#define random_get_entropy random_get_entropy
+
/* Assume we use the PIT time source for the clock tick */
#define CLOCK_TICK_RATE PIT_TICK_RATE
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -21,13 +21,12 @@ extern void disable_TSC(void);
static inline cycles_t get_cycles(void)
{
-#ifndef CONFIG_X86_TSC
- if (!boot_cpu_has(X86_FEATURE_TSC))
+ if (!IS_ENABLED(CONFIG_X86_TSC) &&
+ !cpu_feature_enabled(X86_FEATURE_TSC))
return 0;
-#endif
-
return rdtsc();
}
+#define get_cycles get_cycles
extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
From: Andy Lutomirski <[email protected]>
commit 90ea1c6436d26e62496616fb5891e00819ff4849 upstream.
There is no longer any interface to read data from the blocking
pool, so remove it.
This enables quite a bit of code deletion, much of which will be
done in subsequent patches.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/511225a224bf0a291149d3c0b8b45393cd03ab96.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 106 --------------------------------------------------
1 file changed, 106 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -471,7 +471,6 @@ static const struct poolinfo {
/*
* Static global variables
*/
-static DECLARE_WAIT_QUEUE_HEAD(random_read_wait);
static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
static struct fasync_struct *fasync;
@@ -533,7 +532,6 @@ struct entropy_store {
__u32 *pool;
const char *name;
struct entropy_store *pull;
- struct work_struct push_work;
/* read-write data: */
unsigned long last_pulled;
@@ -552,9 +550,7 @@ static ssize_t _extract_entropy(struct e
size_t nbytes, int fips);
static void crng_reseed(struct crng_state *crng, struct entropy_store *r);
-static void push_to_pool(struct work_struct *work);
static __u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
-static __u32 blocking_pool_data[OUTPUT_POOL_WORDS] __latent_entropy;
static struct entropy_store input_pool = {
.poolinfo = &poolinfo_table[0],
@@ -563,16 +559,6 @@ static struct entropy_store input_pool =
.pool = input_pool_data
};
-static struct entropy_store blocking_pool = {
- .poolinfo = &poolinfo_table[1],
- .name = "blocking",
- .pull = &input_pool,
- .lock = __SPIN_LOCK_UNLOCKED(blocking_pool.lock),
- .pool = blocking_pool_data,
- .push_work = __WORK_INITIALIZER(blocking_pool.push_work,
- push_to_pool),
-};
-
static __u32 const twist_table[8] = {
0x00000000, 0x3b6e20c8, 0x76dc4190, 0x4db26158,
0xedb88320, 0xd6d6a3e8, 0x9b64c2b0, 0xa00ae278 };
@@ -768,15 +754,11 @@ retry:
entropy_count = 0;
} else if (entropy_count > pool_size)
entropy_count = pool_size;
- if ((r == &blocking_pool) && !r->initialized &&
- (entropy_count >> ENTROPY_SHIFT) > 128)
- has_initialized = 1;
if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
goto retry;
if (has_initialized) {
r->initialized = 1;
- wake_up_interruptible(&random_read_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
}
@@ -785,7 +767,6 @@ retry:
if (r == &input_pool) {
int entropy_bits = entropy_count >> ENTROPY_SHIFT;
- struct entropy_store *other = &blocking_pool;
if (crng_init < 2) {
if (entropy_bits < 128)
@@ -793,27 +774,6 @@ retry:
crng_reseed(&primary_crng, r);
entropy_bits = r->entropy_count >> ENTROPY_SHIFT;
}
-
- /* initialize the blocking pool if necessary */
- if (entropy_bits >= random_read_wakeup_bits &&
- !other->initialized) {
- schedule_work(&other->push_work);
- return;
- }
-
- /* should we wake readers? */
- if (entropy_bits >= random_read_wakeup_bits &&
- wq_has_sleeper(&random_read_wait)) {
- wake_up_interruptible(&random_read_wait);
- }
- /* If the input pool is getting full, and the blocking
- * pool has room, send some entropy to the blocking
- * pool.
- */
- if (!work_pending(&other->push_work) &&
- (ENTROPY_BITS(r) > 6 * r->poolinfo->poolbytes) &&
- (ENTROPY_BITS(other) <= 6 * other->poolinfo->poolbytes))
- schedule_work(&other->push_work);
}
}
@@ -1491,22 +1451,6 @@ static void _xfer_secondary_pool(struct
}
/*
- * Used as a workqueue function so that when the input pool is getting
- * full, we can "spill over" some entropy to the output pools. That
- * way the output pools can store some of the excess entropy instead
- * of letting it go to waste.
- */
-static void push_to_pool(struct work_struct *work)
-{
- struct entropy_store *r = container_of(work, struct entropy_store,
- push_work);
- BUG_ON(!r);
- _xfer_secondary_pool(r, random_read_wakeup_bits/8);
- trace_push_to_pool(r->name, r->entropy_count >> ENTROPY_SHIFT,
- r->pull->entropy_count >> ENTROPY_SHIFT);
-}
-
-/*
* This function decides how many bytes to actually take from the
* given pool, and also debits the entropy count accordingly.
*/
@@ -1684,54 +1628,6 @@ static ssize_t extract_entropy(struct en
return _extract_entropy(r, buf, nbytes, fips_enabled);
}
-/*
- * This function extracts randomness from the "entropy pool", and
- * returns it in a userspace buffer.
- */
-static ssize_t extract_entropy_user(struct entropy_store *r, void __user *buf,
- size_t nbytes)
-{
- ssize_t ret = 0, i;
- __u8 tmp[EXTRACT_SIZE];
- int large_request = (nbytes > 256);
-
- trace_extract_entropy_user(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
- if (!r->initialized && r->pull) {
- xfer_secondary_pool(r, ENTROPY_BITS(r->pull)/8);
- if (!r->initialized)
- return 0;
- }
- xfer_secondary_pool(r, nbytes);
- nbytes = account(r, nbytes, 0, 0);
-
- while (nbytes) {
- if (large_request && need_resched()) {
- if (signal_pending(current)) {
- if (ret == 0)
- ret = -ERESTARTSYS;
- break;
- }
- schedule();
- }
-
- extract_buf(r, tmp);
- i = min_t(int, nbytes, EXTRACT_SIZE);
- if (copy_to_user(buf, tmp, i)) {
- ret = -EFAULT;
- break;
- }
-
- nbytes -= i;
- buf += i;
- ret += i;
- }
-
- /* Wipe data just returned from memory */
- memzero_explicit(tmp, sizeof(tmp));
-
- return ret;
-}
-
#define warn_unseeded_randomness(previous) \
_warn_unseeded_randomness(__func__, (void *) _RET_IP_, (previous))
@@ -1961,7 +1857,6 @@ static void __init init_std_data(struct
int __init rand_initialize(void)
{
init_std_data(&input_pool);
- init_std_data(&blocking_pool);
if (crng_need_final_init)
crng_finalize_init(&primary_crng);
crng_initialize(&primary_crng);
@@ -2132,7 +2027,6 @@ static long random_ioctl(struct file *f,
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
input_pool.entropy_count = 0;
- blocking_pool.entropy_count = 0;
return 0;
case RNDRESEEDCRNG:
if (!capable(CAP_SYS_ADMIN))
From: George Spelvin <[email protected]>
commit 92e507d216139b356a375afbda2824e85235e748 upstream.
Explain what these functions are for and when they offer
an advantage over get_random_bytes().
(We still need documentation on rng_is_initialized(), the
random_ready_callback system, and early boot in general.)
Signed-off-by: George Spelvin <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 83 +++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 76 insertions(+), 7 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -101,15 +101,13 @@
* Exported interfaces ---- output
* ===============================
*
- * There are three exported interfaces; the first is one designed to
- * be used from within the kernel:
+ * There are four exported interfaces; two for use within the kernel,
+ * and two or use from userspace.
*
- * void get_random_bytes(void *buf, int nbytes);
- *
- * This interface will return the requested number of random bytes,
- * and place it in the requested buffer.
+ * Exported interfaces ---- userspace output
+ * -----------------------------------------
*
- * The two other interfaces are two character devices /dev/random and
+ * The userspace interfaces are two character devices /dev/random and
* /dev/urandom. /dev/random is suitable for use when very high
* quality randomness is desired (for example, for key generation or
* one-time pads), as it will only return a maximum of the number of
@@ -122,6 +120,77 @@
* this will result in random numbers that are merely cryptographically
* strong. For many applications, however, this is acceptable.
*
+ * Exported interfaces ---- kernel output
+ * --------------------------------------
+ *
+ * The primary kernel interface is
+ *
+ * void get_random_bytes(void *buf, int nbytes);
+ *
+ * This interface will return the requested number of random bytes,
+ * and place it in the requested buffer. This is equivalent to a
+ * read from /dev/urandom.
+ *
+ * For less critical applications, there are the functions:
+ *
+ * u32 get_random_u32()
+ * u64 get_random_u64()
+ * unsigned int get_random_int()
+ * unsigned long get_random_long()
+ *
+ * These are produced by a cryptographic RNG seeded from get_random_bytes,
+ * and so do not deplete the entropy pool as much. These are recommended
+ * for most in-kernel operations *if the result is going to be stored in
+ * the kernel*.
+ *
+ * Specifically, the get_random_int() family do not attempt to do
+ * "anti-backtracking". If you capture the state of the kernel (e.g.
+ * by snapshotting the VM), you can figure out previous get_random_int()
+ * return values. But if the value is stored in the kernel anyway,
+ * this is not a problem.
+ *
+ * It *is* safe to expose get_random_int() output to attackers (e.g. as
+ * network cookies); given outputs 1..n, it's not feasible to predict
+ * outputs 0 or n+1. The only concern is an attacker who breaks into
+ * the kernel later; the get_random_int() engine is not reseeded as
+ * often as the get_random_bytes() one.
+ *
+ * get_random_bytes() is needed for keys that need to stay secret after
+ * they are erased from the kernel. For example, any key that will
+ * be wrapped and stored encrypted. And session encryption keys: we'd
+ * like to know that after the session is closed and the keys erased,
+ * the plaintext is unrecoverable to someone who recorded the ciphertext.
+ *
+ * But for network ports/cookies, stack canaries, PRNG seeds, address
+ * space layout randomization, session *authentication* keys, or other
+ * applications where the sensitive data is stored in the kernel in
+ * plaintext for as long as it's sensitive, the get_random_int() family
+ * is just fine.
+ *
+ * Consider ASLR. We want to keep the address space secret from an
+ * outside attacker while the process is running, but once the address
+ * space is torn down, it's of no use to an attacker any more. And it's
+ * stored in kernel data structures as long as it's alive, so worrying
+ * about an attacker's ability to extrapolate it from the get_random_int()
+ * CRNG is silly.
+ *
+ * Even some cryptographic keys are safe to generate with get_random_int().
+ * In particular, keys for SipHash are generally fine. Here, knowledge
+ * of the key authorizes you to do something to a kernel object (inject
+ * packets to a network connection, or flood a hash table), and the
+ * key is stored with the object being protected. Once it goes away,
+ * we no longer care if anyone knows the key.
+ *
+ * prandom_u32()
+ * -------------
+ *
+ * For even weaker applications, see the pseudorandom generator
+ * prandom_u32(), prandom_max(), and prandom_bytes(). If the random
+ * numbers aren't security-critical at all, these are *far* cheaper.
+ * Useful for self-tests, random error simulation, randomized backoffs,
+ * and any other application where you trust that nobody is trying to
+ * maliciously mess with you by guessing the "random" numbers.
+ *
* Exported interfaces ---- input
* ==============================
*
From: "Jason A. Donenfeld" <[email protected]>
commit 9f9eff85a008b095eafc5f4ecbaf5aca689271c1 upstream.
This commit addresses one of the lower hanging fruits of the RNG: its
usage of SHA1.
BLAKE2s is generally faster, and certainly more secure, than SHA1, which
has [1] been [2] really [3] very [4] broken [5]. Additionally, the
current construction in the RNG doesn't use the full SHA1 function, as
specified, and allows overwriting the IV with RDRAND output in an
undocumented way, even in the case when RDRAND isn't set to "trusted",
which means potential malicious IV choices. And its short length means
that keeping only half of it secret when feeding back into the mixer
gives us only 2^80 bits of forward secrecy. In other words, not only is
the choice of hash function dated, but the use of it isn't really great
either.
This commit aims to fix both of these issues while also keeping the
general structure and semantics as close to the original as possible.
Specifically:
a) Rather than overwriting the hash IV with RDRAND, we put it into
BLAKE2's documented "salt" and "personal" fields, which were
specifically created for this type of usage.
b) Since this function feeds the full hash result back into the
entropy collector, we only return from it half the length of the
hash, just as it was done before. This increases the
construction's forward secrecy from 2^80 to a much more
comfortable 2^128.
c) Rather than using the raw "sha1_transform" function alone, we
instead use the full proper BLAKE2s function, with finalization.
This also has the advantage of supplying 16 bytes at a time rather than
SHA1's 10 bytes, which, in addition to having a faster compression
function to begin with, means faster extraction in general. On an Intel
i7-11850H, this commit makes initial seeding around 131% faster.
BLAKE2s itself has the nice property of internally being based on the
ChaCha permutation, which the RNG is already using for expansion, so
there shouldn't be any issue with newness, funkiness, or surprising CPU
behavior, since it's based on something already in use.
[1] https://eprint.iacr.org/2005/010.pdf
[2] https://www.iacr.org/archive/crypto2005/36210017/36210017.pdf
[3] https://eprint.iacr.org/2015/967.pdf
[4] https://shattered.io/static/shattered.pdf
[5] https://www.usenix.org/system/files/sec20-leurent.pdf
Reviewed-by: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Greg Kroah-Hartman <[email protected]>
Reviewed-by: Jean-Philippe Aumasson <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 70 +++++++++++++++++++++-----------------------------
1 file changed, 30 insertions(+), 40 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1,8 +1,7 @@
/*
* random.c -- A strong random number generator
*
- * Copyright (C) 2017 Jason A. Donenfeld <[email protected]>. All
- * Rights Reserved.
+ * Copyright (C) 2017-2022 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
*
* Copyright Matt Mackall <[email protected]>, 2003, 2004, 2005
*
@@ -78,12 +77,12 @@
* an *estimate* of how many bits of randomness have been stored into
* the random number generator's internal state.
*
- * When random bytes are desired, they are obtained by taking the SHA
- * hash of the contents of the "entropy pool". The SHA hash avoids
+ * When random bytes are desired, they are obtained by taking the BLAKE2s
+ * hash of the contents of the "entropy pool". The BLAKE2s hash avoids
* exposing the internal state of the entropy pool. It is believed to
* be computationally infeasible to derive any useful information
- * about the input of SHA from its output. Even if it is possible to
- * analyze SHA in some clever way, as long as the amount of data
+ * about the input of BLAKE2s from its output. Even if it is possible to
+ * analyze BLAKE2s in some clever way, as long as the amount of data
* returned from the generator is less than the inherent entropy in
* the pool, the output data is totally unpredictable. For this
* reason, the routine decreases its internal estimate of how many
@@ -93,7 +92,7 @@
* If this estimate goes to zero, the routine can still generate
* random numbers; however, an attacker may (at least in theory) be
* able to infer the future output of the generator from prior
- * outputs. This requires successful cryptanalysis of SHA, which is
+ * outputs. This requires successful cryptanalysis of BLAKE2s, which is
* not believed to be feasible, but there is a remote possibility.
* Nonetheless, these numbers should be useful for the vast majority
* of purposes.
@@ -349,6 +348,7 @@
#include <linux/completion.h>
#include <linux/uuid.h>
#include <crypto/chacha20.h>
+#include <crypto/blake2s.h>
#include <asm/processor.h>
#include <asm/uaccess.h>
@@ -368,10 +368,7 @@
#define INPUT_POOL_WORDS (1 << (INPUT_POOL_SHIFT-5))
#define OUTPUT_POOL_SHIFT 10
#define OUTPUT_POOL_WORDS (1 << (OUTPUT_POOL_SHIFT-5))
-#define EXTRACT_SIZE 10
-
-
-#define LONGS(x) (((x) + sizeof(unsigned long) - 1)/sizeof(unsigned long))
+#define EXTRACT_SIZE (BLAKE2S_HASH_SIZE / 2)
/*
* To allow fractional bits to be tracked, the entropy_count field is
@@ -407,7 +404,7 @@ static int random_write_wakeup_bits = 28
* Thanks to Colin Plumb for suggesting this.
*
* The mixing operation is much less sensitive than the output hash,
- * where we use SHA-1. All that we want of mixing operation is that
+ * where we use BLAKE2s. All that we want of mixing operation is that
* it be a good non-cryptographic hash; i.e. it not produce collisions
* when fed "random" data of the sort we expect to see. As long as
* the pool state differs for different inputs, we have preserved the
@@ -1449,56 +1446,49 @@ retry:
*/
static void extract_buf(struct entropy_store *r, __u8 *out)
{
- int i;
- union {
- __u32 w[5];
- unsigned long l[LONGS(20)];
- } hash;
- __u32 workspace[SHA_WORKSPACE_WORDS];
+ struct blake2s_state state __aligned(__alignof__(unsigned long));
+ u8 hash[BLAKE2S_HASH_SIZE];
+ unsigned long *salt;
unsigned long flags;
+ blake2s_init(&state, sizeof(hash));
+
/*
* If we have an architectural hardware random number
- * generator, use it for SHA's initial vector
+ * generator, use it for BLAKE2's salt & personal fields.
*/
- sha_init(hash.w);
- for (i = 0; i < LONGS(20); i++) {
+ for (salt = (unsigned long *)&state.h[4];
+ salt < (unsigned long *)&state.h[8]; ++salt) {
unsigned long v;
if (!arch_get_random_long(&v))
break;
- hash.l[i] = v;
+ *salt ^= v;
}
- /* Generate a hash across the pool, 16 words (512 bits) at a time */
+ /* Generate a hash across the pool */
spin_lock_irqsave(&r->lock, flags);
- for (i = 0; i < r->poolinfo->poolwords; i += 16)
- sha_transform(hash.w, (__u8 *)(r->pool + i), workspace);
+ blake2s_update(&state, (const u8 *)r->pool,
+ r->poolinfo->poolwords * sizeof(*r->pool));
+ blake2s_final(&state, hash); /* final zeros out state */
/*
* We mix the hash back into the pool to prevent backtracking
* attacks (where the attacker knows the state of the pool
* plus the current outputs, and attempts to find previous
- * ouputs), unless the hash function can be inverted. By
- * mixing at least a SHA1 worth of hash data back, we make
+ * outputs), unless the hash function can be inverted. By
+ * mixing at least a hash worth of hash data back, we make
* brute-forcing the feedback as hard as brute-forcing the
* hash.
*/
- __mix_pool_bytes(r, hash.w, sizeof(hash.w));
+ __mix_pool_bytes(r, hash, sizeof(hash));
spin_unlock_irqrestore(&r->lock, flags);
- memzero_explicit(workspace, sizeof(workspace));
-
- /*
- * In case the hash function has some recognizable output
- * pattern, we fold it in half. Thus, we always feed back
- * twice as much data as we output.
+ /* Note that EXTRACT_SIZE is half of hash size here, because above
+ * we've dumped the full length back into mixer. By reducing the
+ * amount that we emit, we retain a level of forward secrecy.
*/
- hash.w[0] ^= hash.w[3];
- hash.w[1] ^= hash.w[4];
- hash.w[2] ^= rol32(hash.w[2], 16);
-
- memcpy(out, &hash, EXTRACT_SIZE);
- memzero_explicit(&hash, sizeof(hash));
+ memcpy(out, hash, EXTRACT_SIZE);
+ memzero_explicit(hash, sizeof(hash));
}
static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
From: "Jason A. Donenfeld" <[email protected]>
commit aba120cc101788544aa3e2c30c8da88513892350 upstream.
The fast key erasure RNG design relies on the key that's used to be used
and then discarded. We do this, making judicious use of
memzero_explicit(). However, reads to /dev/urandom and calls to
getrandom() involve a copy_to_user(), and userspace can use FUSE or
userfaultfd, or make a massive call, dynamically remap memory addresses
as it goes, and set the process priority to idle, in order to keep a
kernel stack alive indefinitely. By probing
/proc/sys/kernel/random/entropy_avail to learn when the crng key is
refreshed, a malicious userspace could mount this attack every 5 minutes
thereafter, breaking the crng's forward secrecy.
In order to fix this, we just overwrite the stack's key with the first
32 bytes of the "free" fast key erasure output. If we're returning <= 32
bytes to the user, then we can still return those bytes directly, so
that short reads don't become slower. And for long reads, the difference
is hopefully lost in the amortization, so it doesn't change much, with
that amortization helping variously for medium reads.
We don't need to do this for get_random_bytes() and the various
kernel-space callers, and later, if we ever switch to always batching,
this won't be necessary either, so there's no need to change the API of
these functions.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Jann Horn <[email protected]>
Fixes: c92e040d575a ("random: add backtracking protection to the CRNG")
Fixes: 186873c549df ("random: use simpler fast key erasure flow on per-cpu keys")
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 35 +++++++++++++++++++++++------------
1 file changed, 23 insertions(+), 12 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -533,19 +533,29 @@ static ssize_t get_random_bytes_user(voi
if (!nbytes)
return 0;
- len = min_t(size_t, 32, nbytes);
- crng_make_state(chacha_state, output, len);
-
- if (copy_to_user(buf, output, len))
- return -EFAULT;
- nbytes -= len;
- buf += len;
- ret += len;
+ /*
+ * Immediately overwrite the ChaCha key at index 4 with random
+ * bytes, in case userspace causes copy_to_user() below to sleep
+ * forever, so that we still retain forward secrecy in that case.
+ */
+ crng_make_state(chacha_state, (u8 *)&chacha_state[4], CHACHA_KEY_SIZE);
+ /*
+ * However, if we're doing a read of len <= 32, we don't need to
+ * use chacha_state after, so we can simply return those bytes to
+ * the user directly.
+ */
+ if (nbytes <= CHACHA_KEY_SIZE) {
+ ret = copy_to_user(buf, &chacha_state[4], nbytes) ? -EFAULT : nbytes;
+ goto out_zero_chacha;
+ }
- while (nbytes) {
+ do {
if (large_request && need_resched()) {
- if (signal_pending(current))
+ if (signal_pending(current)) {
+ if (!ret)
+ ret = -ERESTARTSYS;
break;
+ }
schedule();
}
@@ -562,10 +572,11 @@ static ssize_t get_random_bytes_user(voi
nbytes -= len;
buf += len;
ret += len;
- }
+ } while (nbytes);
- memzero_explicit(chacha_state, sizeof(chacha_state));
memzero_explicit(output, sizeof(output));
+out_zero_chacha:
+ memzero_explicit(chacha_state, sizeof(chacha_state));
return ret;
}
From: Dominik Brodowski <[email protected]>
commit 7c2fe2b32bf76441ff5b7a425b384e5f75aa530a upstream.
crng_init is protected by primary_crng->lock, so keep holding that lock
when incrementing crng_init from 0 to 1 in crng_fast_load(). The call to
pr_notice() can wait until the lock is released; this code path cannot
be reached twice, as crng_fast_load() aborts early if crng_init > 0.
Signed-off-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -647,12 +647,13 @@ static size_t crng_fast_load(const u8 *c
p[crng_init_cnt % CHACHA20_KEY_SIZE] ^= *cp;
cp++; crng_init_cnt++; len--; ret++;
}
- spin_unlock_irqrestore(&primary_crng.lock, flags);
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
invalidate_batched_entropy();
crng_init = 1;
- pr_notice("fast init done\n");
}
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
+ if (crng_init == 1)
+ pr_notice("fast init done\n");
return ret;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 2e3df523256cb9836de8441e9c791a796759bb3c upstream.
S390x defines a get_cycles() function, but it does not do the usual
`#define get_cycles get_cycles` dance, making it impossible for generic
code to see if an arch-specific function was defined. While the
get_cycles() ifdef is not currently used, the following timekeeping
patch in this series will depend on the macro existing (or not existing)
when defining random_get_entropy().
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Vasily Gorbik <[email protected]>
Cc: Alexander Gordeev <[email protected]>
Cc: Christian Borntraeger <[email protected]>
Cc: Sven Schnelle <[email protected]>
Acked-by: Heiko Carstens <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/s390/include/asm/timex.h | 1 +
1 file changed, 1 insertion(+)
--- a/arch/s390/include/asm/timex.h
+++ b/arch/s390/include/asm/timex.h
@@ -168,6 +168,7 @@ static inline cycles_t get_cycles(void)
{
return (cycles_t) get_tod_clock() >> 2;
}
+#define get_cycles get_cycles
int get_phys_clock(unsigned long long *clock);
void init_cpu_timer(void);
From: "Jason A. Donenfeld" <[email protected]>
commit c2a7de4feb6e09f23af7accc0f882a8fa92e7ae5 upstream.
Taking spinlocks from IRQ context is generally problematic for
PREEMPT_RT. That is, in part, why we take trylocks instead. However, a
spin_try_lock() is also problematic since another spin_lock() invocation
can potentially PI-boost the wrong task, as the spin_try_lock() is
invoked from an IRQ-context, so the task on CPU (random task or idle) is
not the actual owner.
Additionally, by deferring the crng pre-init loading to the worker, we
can use the cryptographic hash function rather than xor, which is
perhaps a meaningful difference when considering this data has only been
through the relatively weak fast_mix() function.
The biggest downside of this approach is that the pre-init loading is
now deferred until later, which means things that need random numbers
after interrupts are enabled, but before workqueues are running -- or
before this particular worker manages to run -- are going to get into
trouble. Hopefully in the real world, this window is rather small,
especially since this code won't run until 64 interrupts had occurred.
Cc: Sultan Alsawaf <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Eric Biggers <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Acked-by: Sebastian Andrzej Siewior <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 65 ++++++++++++++------------------------------------
1 file changed, 19 insertions(+), 46 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -442,10 +442,6 @@ static void crng_make_state(u32 chacha_s
* boot time when it's better to have something there rather than
* nothing.
*
- * There are two paths, a slow one and a fast one. The slow one
- * hashes the input along with the current key. The fast one simply
- * xors it in, and should only be used from interrupt context.
- *
* If account is set, then the crng_init_cnt counter is incremented.
* This shouldn't be set by functions like add_device_randomness(),
* where we can't trust the buffer passed to it is guaranteed to be
@@ -454,19 +450,15 @@ static void crng_make_state(u32 chacha_s
* Returns the number of bytes processed from input, which is bounded
* by CRNG_INIT_CNT_THRESH if account is true.
*/
-static size_t crng_pre_init_inject(const void *input, size_t len,
- bool fast, bool account)
+static size_t crng_pre_init_inject(const void *input, size_t len, bool account)
{
static int crng_init_cnt = 0;
+ struct blake2s_state hash;
unsigned long flags;
- if (fast) {
- if (!spin_trylock_irqsave(&base_crng.lock, flags))
- return 0;
- } else {
- spin_lock_irqsave(&base_crng.lock, flags);
- }
+ blake2s_init(&hash, sizeof(base_crng.key));
+ spin_lock_irqsave(&base_crng.lock, flags);
if (crng_init != 0) {
spin_unlock_irqrestore(&base_crng.lock, flags);
return 0;
@@ -475,21 +467,9 @@ static size_t crng_pre_init_inject(const
if (account)
len = min_t(size_t, len, CRNG_INIT_CNT_THRESH - crng_init_cnt);
- if (fast) {
- const u8 *src = input;
- size_t i;
-
- for (i = 0; i < len; ++i)
- base_crng.key[(crng_init_cnt + i) %
- sizeof(base_crng.key)] ^= src[i];
- } else {
- struct blake2s_state hash;
-
- blake2s_init(&hash, sizeof(base_crng.key));
- blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
- blake2s_update(&hash, input, len);
- blake2s_final(&hash, base_crng.key);
- }
+ blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
+ blake2s_update(&hash, input, len);
+ blake2s_final(&hash, base_crng.key);
if (account) {
crng_init_cnt += len;
@@ -1030,7 +1010,7 @@ void add_device_randomness(const void *b
unsigned long flags, now = jiffies;
if (crng_init == 0 && size)
- crng_pre_init_inject(buf, size, false, false);
+ crng_pre_init_inject(buf, size, false);
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(&cycles, sizeof(cycles));
@@ -1151,7 +1131,7 @@ void add_hwgenerator_randomness(const vo
size_t entropy)
{
if (unlikely(crng_init == 0)) {
- size_t ret = crng_pre_init_inject(buffer, count, false, true);
+ size_t ret = crng_pre_init_inject(buffer, count, true);
mix_pool_bytes(buffer, ret);
count -= ret;
buffer += ret;
@@ -1291,8 +1271,14 @@ static void mix_interrupt_randomness(str
fast_pool->last = jiffies;
local_irq_enable();
- mix_pool_bytes(pool, sizeof(pool));
- credit_entropy_bits(1);
+ if (unlikely(crng_init == 0)) {
+ crng_pre_init_inject(pool, sizeof(pool), true);
+ mix_pool_bytes(pool, sizeof(pool));
+ } else {
+ mix_pool_bytes(pool, sizeof(pool));
+ credit_entropy_bits(1);
+ }
+
memzero_explicit(pool, sizeof(pool));
}
@@ -1325,24 +1311,11 @@ void add_interrupt_randomness(int irq)
fast_mix(fast_pool->pool32);
new_count = ++fast_pool->count;
- if (unlikely(crng_init == 0)) {
- if (new_count >= 64 &&
- crng_pre_init_inject(fast_pool->pool32, sizeof(fast_pool->pool32),
- true, true) > 0) {
- fast_pool->count = 0;
- fast_pool->last = now;
- if (spin_trylock(&input_pool.lock)) {
- _mix_pool_bytes(&fast_pool->pool32, sizeof(fast_pool->pool32));
- spin_unlock(&input_pool.lock);
- }
- }
- return;
- }
-
if (new_count & MIX_INFLIGHT)
return;
- if (new_count < 64 && !time_after(now, fast_pool->last + HZ))
+ if (new_count < 64 && (!time_after(now, fast_pool->last + HZ) ||
+ unlikely(crng_init == 0)))
return;
if (unlikely(!fast_pool->mix.func))
From: Yangtao Li <[email protected]>
commit 12faac30d157970fdbfa171bbeb1fb88350303b1 upstream.
Signed-off-by: Yangtao Li <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -763,7 +763,7 @@ retry:
if (entropy_bits < 128)
return;
crng_reseed(&primary_crng, r);
- entropy_bits = r->entropy_count >> ENTROPY_SHIFT;
+ entropy_bits = ENTROPY_BITS(r);
}
}
}
@@ -1446,8 +1446,7 @@ retry:
goto retry;
trace_debit_entropy(r->name, 8 * ibytes);
- if (ibytes &&
- (r->entropy_count >> ENTROPY_SHIFT) < random_write_wakeup_bits) {
+ if (ibytes && ENTROPY_BITS(r) < random_write_wakeup_bits) {
wake_up_interruptible(&random_write_wait);
kill_fasync(&fasync, SIGIO, POLL_OUT);
}
From: "Jason A. Donenfeld" <[email protected]>
commit 1366992e16bddd5e2d9a561687f367f9f802e2e4 upstream.
The addition of random_get_entropy_fallback() provides access to
whichever time source has the highest frequency, which is useful for
gathering entropy on platforms without available cycle counters. It's
not necessarily as good as being able to quickly access a cycle counter
that the CPU has, but it's still something, even when it falls back to
being jiffies-based.
In the event that a given arch does not define get_cycles(), falling
back to the get_cycles() default implementation that returns 0 is really
not the best we can do. Instead, at least calling
random_get_entropy_fallback() would be preferable, because that always
needs to return _something_, even falling back to jiffies eventually.
It's not as though random_get_entropy_fallback() is super high precision
or guaranteed to be entropic, but basically anything that's not zero all
the time is better than returning zero all the time.
Finally, since random_get_entropy_fallback() is used during extremely
early boot when randomizing freelists in mm_init(), it can be called
before timekeeping has been initialized. In that case there really is
nothing we can do; jiffies hasn't even started ticking yet. So just give
up and return 0.
Suggested-by: Thomas Gleixner <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/timex.h | 8 ++++++++
kernel/time/timekeeping.c | 16 ++++++++++++++++
2 files changed, 24 insertions(+)
--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -62,6 +62,8 @@
#include <linux/types.h>
#include <linux/param.h>
+unsigned long random_get_entropy_fallback(void);
+
#include <asm/timex.h>
#ifndef random_get_entropy
@@ -74,8 +76,14 @@
*
* By default we use get_cycles() for this purpose, but individual
* architectures may override this in their asm/timex.h header file.
+ * If a given arch does not have get_cycles(), then we fallback to
+ * using random_get_entropy_fallback().
*/
+#ifdef get_cycles
#define random_get_entropy() ((unsigned long)get_cycles())
+#else
+#define random_get_entropy() random_get_entropy_fallback()
+#endif
#endif
/*
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -19,6 +19,7 @@
#include <linux/clocksource.h>
#include <linux/jiffies.h>
#include <linux/time.h>
+#include <linux/timex.h>
#include <linux/tick.h>
#include <linux/stop_machine.h>
#include <linux/pvclock_gtod.h>
@@ -2270,6 +2271,21 @@ ktime_t ktime_get_update_offsets_now(uns
}
/**
+ * random_get_entropy_fallback - Returns the raw clock source value,
+ * used by random.c for platforms with no valid random_get_entropy().
+ */
+unsigned long random_get_entropy_fallback(void)
+{
+ struct tk_read_base *tkr = &tk_core.timekeeper.tkr_mono;
+ struct clocksource *clock = READ_ONCE(tkr->clock);
+
+ if (unlikely(timekeeping_suspended || !clock))
+ return 0;
+ return clock->read(clock);
+}
+EXPORT_SYMBOL_GPL(random_get_entropy_fallback);
+
+/**
* do_adjtimex() - Accessor function to NTP __do_adjtimex function
*/
int do_adjtimex(struct timex *txc)
From: "Jason A. Donenfeld" <[email protected]>
commit af704c856e888fb044b058d731d61b46eeec499d upstream.
At boot time, EFI calls add_bootloader_randomness(), which in turn calls
add_hwgenerator_randomness(). Currently add_hwgenerator_randomness()
feeds the first 64 bytes of randomness to the "fast init"
non-crypto-grade phase. But if add_hwgenerator_randomness() gets called
with more than POOL_MIN_BITS of entropy, there's no point in passing it
off to the "fast init" stage, since that's enough entropy to bootstrap
the real RNG. The "fast init" stage is just there to provide _something_
in the case where we don't have enough entropy to properly bootstrap the
RNG. But if we do have enough entropy to bootstrap the RNG, the current
logic doesn't serve a purpose. So, in the case where we're passed
greater than or equal to POOL_MIN_BITS of entropy, this commit makes us
skip the "fast init" phase.
Cc: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1121,7 +1121,7 @@ void rand_initialize_disk(struct gendisk
void add_hwgenerator_randomness(const void *buffer, size_t count,
size_t entropy)
{
- if (unlikely(crng_init == 0)) {
+ if (unlikely(crng_init == 0 && entropy < POOL_MIN_BITS)) {
size_t ret = crng_pre_init_inject(buffer, count, true);
mix_pool_bytes(buffer, ret);
count -= ret;
From: Ard Biesheuvel <[email protected]>
commit 390596c9959c2a4f5b456df339f0604df3d55fe0 upstream.
When reseeding the CRNG periodically, arch_get_random_seed_long() is
called to obtain entropy from an architecture specific source if one
is implemented. In most cases, these are special instructions, but in
some cases, such as on ARM, we may want to back this using firmware
calls, which are considerably more expensive.
Another call to arch_get_random_seed_long() exists in the CRNG driver,
in add_interrupt_randomness(), which collects entropy by capturing
inter-interrupt timing and relying on interrupt jitter to provide
random bits. This is done by keeping a per-CPU state, and mixing in
the IRQ number, the cycle counter and the return address every time an
interrupt is taken, and mixing this per-CPU state into the entropy pool
every 64 invocations, or at least once per second. The entropy that is
gathered this way is credited as 1 bit of entropy. Every time this
happens, arch_get_random_seed_long() is invoked, and the result is
mixed in as well, and also credited with 1 bit of entropy.
This means that arch_get_random_seed_long() is called at least once
per second on every CPU, which seems excessive, and doesn't really
scale, especially in a virtualization scenario where CPUs may be
oversubscribed: in cases where arch_get_random_seed_long() is backed
by an instruction that actually goes back to a shared hardware entropy
source (such as RNDRRS on ARM), we will end up hitting it hundreds of
times per second.
So let's drop the call to arch_get_random_seed_long() from
add_interrupt_randomness(), and instead, rely on crng_reseed() to call
the arch hook to get random seed material from the platform.
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Andre Przywara <[email protected]>
Tested-by: Andre Przywara <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Acked-by: Marc Zyngier <[email protected]>
Reviewed-by: Jason A. Donenfeld <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Will Deacon <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 15 +--------------
1 file changed, 1 insertion(+), 14 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1331,8 +1331,6 @@ void add_interrupt_randomness(int irq, i
cycles_t cycles = random_get_entropy();
__u32 c_high, j_high;
__u64 ip;
- unsigned long seed;
- int credit = 0;
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
@@ -1368,23 +1366,12 @@ void add_interrupt_randomness(int irq, i
fast_pool->last = now;
__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
-
- /*
- * If we have architectural seed generator, produce a seed and
- * add it to the pool. For the sake of paranoia don't let the
- * architectural seed generator dominate the input from the
- * interrupt noise.
- */
- if (arch_get_random_seed_long(&seed)) {
- __mix_pool_bytes(r, &seed, sizeof(seed));
- credit = 1;
- }
spin_unlock(&r->lock);
fast_pool->count = 0;
/* award one bit for the contents of the fast pool */
- credit_entropy_bits(r, credit + 1);
+ credit_entropy_bits(r, 1);
}
EXPORT_SYMBOL_GPL(add_interrupt_randomness);
From: Jann Horn <[email protected]>
commit 1448769c9cdb69ad65287f4f7ab58bc5f2f5d7ba upstream.
signal_pending() checks TIF_NOTIFY_SIGNAL and TIF_SIGPENDING, which
signal that the task should bail out of the syscall when possible. This
is a separate concept from need_resched(), which checks
TIF_NEED_RESCHED, signaling that the task should preempt.
In particular, with the current code, the signal_pending() bailout
probably won't work reliably.
Change this to look like other functions that read lots of data, such as
read_zero().
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Signed-off-by: Jann Horn <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -550,13 +550,13 @@ static ssize_t get_random_bytes_user(voi
}
do {
- if (large_request && need_resched()) {
+ if (large_request) {
if (signal_pending(current)) {
if (!ret)
ret = -ERESTARTSYS;
break;
}
- schedule();
+ cond_resched();
}
chacha20_block(chacha_state, output);
From: Eric Biggers <[email protected]>
[ Upstream commit a5e9f557098e54af44ade5d501379be18435bfbf ]
In commit 9f480faec58c ("crypto: chacha20 - Fix keystream alignment for
chacha20_block()"), I had missed that chacha20_block() can be called
directly on the buffer passed to get_random_bytes(), which can have any
alignment. So, while my commit didn't break anything, it didn't fully
solve the alignment problems.
Revert my solution and just update chacha20_block() to use
put_unaligned_le32(), so the output buffer need not be aligned.
This is simpler, and on many CPUs it's the same speed.
But, I kept the 'tmp' buffers in extract_crng_user() and
_get_random_bytes() 4-byte aligned, since that alignment is actually
needed for _crng_backtrack_protect() too.
Reported-by: Stephan Müller <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Eric Biggers <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/chacha20_generic.c | 7 ++++---
drivers/char/random.c | 24 ++++++++++++------------
include/crypto/chacha20.h | 3 +--
lib/chacha20.c | 6 +++---
4 files changed, 20 insertions(+), 20 deletions(-)
--- a/crypto/chacha20_generic.c
+++ b/crypto/chacha20_generic.c
@@ -23,20 +23,21 @@ static inline u32 le32_to_cpuvp(const vo
static void chacha20_docrypt(u32 *state, u8 *dst, const u8 *src,
unsigned int bytes)
{
- u32 stream[CHACHA20_BLOCK_WORDS];
+ /* aligned to potentially speed up crypto_xor() */
+ u8 stream[CHACHA20_BLOCK_SIZE] __aligned(sizeof(long));
if (dst != src)
memcpy(dst, src, bytes);
while (bytes >= CHACHA20_BLOCK_SIZE) {
chacha20_block(state, stream);
- crypto_xor(dst, (const u8 *)stream, CHACHA20_BLOCK_SIZE);
+ crypto_xor(dst, stream, CHACHA20_BLOCK_SIZE);
bytes -= CHACHA20_BLOCK_SIZE;
dst += CHACHA20_BLOCK_SIZE;
}
if (bytes) {
chacha20_block(state, stream);
- crypto_xor(dst, (const u8 *)stream, bytes);
+ crypto_xor(dst, stream, bytes);
}
}
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -486,9 +486,9 @@ static int crng_init_cnt = 0;
static unsigned long crng_global_init_time = 0;
#define CRNG_INIT_CNT_THRESH (2*CHACHA20_KEY_SIZE)
static void _extract_crng(struct crng_state *crng,
- __u32 out[CHACHA20_BLOCK_WORDS]);
+ __u8 out[CHACHA20_BLOCK_SIZE]);
static void _crng_backtrack_protect(struct crng_state *crng,
- __u32 tmp[CHACHA20_BLOCK_WORDS], int used);
+ __u8 tmp[CHACHA20_BLOCK_SIZE], int used);
static void process_random_ready_list(void);
static void _get_random_bytes(void *buf, int nbytes);
@@ -1038,7 +1038,7 @@ static void crng_reseed(struct crng_stat
unsigned long flags;
int i, num;
union {
- __u32 block[CHACHA20_BLOCK_WORDS];
+ __u8 block[CHACHA20_BLOCK_SIZE];
__u32 key[8];
} buf;
@@ -1066,7 +1066,7 @@ static void crng_reseed(struct crng_stat
}
static void _extract_crng(struct crng_state *crng,
- __u32 out[CHACHA20_BLOCK_WORDS])
+ __u8 out[CHACHA20_BLOCK_SIZE])
{
unsigned long flags, init_time;
@@ -1084,7 +1084,7 @@ static void _extract_crng(struct crng_st
spin_unlock_irqrestore(&crng->lock, flags);
}
-static void extract_crng(__u32 out[CHACHA20_BLOCK_WORDS])
+static void extract_crng(__u8 out[CHACHA20_BLOCK_SIZE])
{
_extract_crng(select_crng(), out);
}
@@ -1094,7 +1094,7 @@ static void extract_crng(__u32 out[CHACH
* enough) to mutate the CRNG key to provide backtracking protection.
*/
static void _crng_backtrack_protect(struct crng_state *crng,
- __u32 tmp[CHACHA20_BLOCK_WORDS], int used)
+ __u8 tmp[CHACHA20_BLOCK_SIZE], int used)
{
unsigned long flags;
__u32 *s, *d;
@@ -1106,14 +1106,14 @@ static void _crng_backtrack_protect(stru
used = 0;
}
spin_lock_irqsave(&crng->lock, flags);
- s = &tmp[used / sizeof(__u32)];
+ s = (__u32 *) &tmp[used];
d = &crng->state[4];
for (i=0; i < 8; i++)
*d++ ^= *s++;
spin_unlock_irqrestore(&crng->lock, flags);
}
-static void crng_backtrack_protect(__u32 tmp[CHACHA20_BLOCK_WORDS], int used)
+static void crng_backtrack_protect(__u8 tmp[CHACHA20_BLOCK_SIZE], int used)
{
_crng_backtrack_protect(select_crng(), tmp, used);
}
@@ -1121,7 +1121,7 @@ static void crng_backtrack_protect(__u32
static ssize_t extract_crng_user(void __user *buf, size_t nbytes)
{
ssize_t ret = 0, i = CHACHA20_BLOCK_SIZE;
- __u32 tmp[CHACHA20_BLOCK_WORDS];
+ __u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
int large_request = (nbytes > 256);
while (nbytes) {
@@ -1580,7 +1580,7 @@ static void _warn_unseeded_randomness(co
*/
static void _get_random_bytes(void *buf, int nbytes)
{
- __u32 tmp[CHACHA20_BLOCK_WORDS];
+ __u8 tmp[CHACHA20_BLOCK_SIZE] __aligned(4);
trace_get_random_bytes(nbytes, _RET_IP_);
@@ -2167,7 +2167,7 @@ u64 get_random_u64(void)
batch = raw_cpu_ptr(&batched_entropy_u64);
spin_lock_irqsave(&batch->batch_lock, flags);
if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) {
- extract_crng((__u32 *)batch->entropy_u64);
+ extract_crng((u8 *)batch->entropy_u64);
batch->position = 0;
}
ret = batch->entropy_u64[batch->position++];
@@ -2191,7 +2191,7 @@ u32 get_random_u32(void)
batch = raw_cpu_ptr(&batched_entropy_u32);
spin_lock_irqsave(&batch->batch_lock, flags);
if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) {
- extract_crng(batch->entropy_u32);
+ extract_crng((u8 *)batch->entropy_u32);
batch->position = 0;
}
ret = batch->entropy_u32[batch->position++];
--- a/include/crypto/chacha20.h
+++ b/include/crypto/chacha20.h
@@ -11,13 +11,12 @@
#define CHACHA20_IV_SIZE 16
#define CHACHA20_KEY_SIZE 32
#define CHACHA20_BLOCK_SIZE 64
-#define CHACHA20_BLOCK_WORDS (CHACHA20_BLOCK_SIZE / sizeof(u32))
struct chacha20_ctx {
u32 key[8];
};
-void chacha20_block(u32 *state, u32 *stream);
+void chacha20_block(u32 *state, u8 *stream);
void crypto_chacha20_init(u32 *state, struct chacha20_ctx *ctx, u8 *iv);
int crypto_chacha20_setkey(struct crypto_tfm *tfm, const u8 *key,
unsigned int keysize);
--- a/lib/chacha20.c
+++ b/lib/chacha20.c
@@ -21,9 +21,9 @@ static inline u32 rotl32(u32 v, u8 n)
return (v << n) | (v >> (sizeof(v) * 8 - n));
}
-void chacha20_block(u32 *state, u32 *stream)
+void chacha20_block(u32 *state, u8 *stream)
{
- u32 x[16], *out = stream;
+ u32 x[16];
int i;
for (i = 0; i < ARRAY_SIZE(x); i++)
@@ -72,7 +72,7 @@ void chacha20_block(u32 *state, u32 *str
}
for (i = 0; i < ARRAY_SIZE(x); i++)
- out[i] = cpu_to_le32(x[i] + state[i]);
+ put_unaligned_le32(x[i] + state[i], &stream[i * sizeof(u32)]);
state[12]++;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 7782cfeca7d420e8bb707613d4cfb0f7ff29bb3a upstream.
Accoriding to the kernel style guide, having `extern` on functions in
headers is old school and deprecated, and doesn't add anything. So remove
them from random.h, and tidy up the file a little bit too.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/random.h | 67 +++++++++++++++++++------------------------------
1 file changed, 27 insertions(+), 40 deletions(-)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -12,13 +12,12 @@
struct notifier_block;
-extern void add_device_randomness(const void *, size_t);
-extern void add_bootloader_randomness(const void *, size_t);
-extern void add_input_randomness(unsigned int type, unsigned int code,
- unsigned int value) __latent_entropy;
-extern void add_interrupt_randomness(int irq) __latent_entropy;
-extern void add_hwgenerator_randomness(const void *buffer, size_t count,
- size_t entropy);
+void add_device_randomness(const void *, size_t);
+void add_bootloader_randomness(const void *, size_t);
+void add_input_randomness(unsigned int type, unsigned int code,
+ unsigned int value) __latent_entropy;
+void add_interrupt_randomness(int irq) __latent_entropy;
+void add_hwgenerator_randomness(const void *buffer, size_t count, size_t entropy);
#if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__)
static inline void add_latent_entropy(void)
@@ -26,21 +25,11 @@ static inline void add_latent_entropy(vo
add_device_randomness((const void *)&latent_entropy, sizeof(latent_entropy));
}
#else
-static inline void add_latent_entropy(void) {}
-#endif
-
-extern void get_random_bytes(void *buf, size_t nbytes);
-extern int wait_for_random_bytes(void);
-extern int __init random_init(const char *command_line);
-extern bool rng_is_initialized(void);
-extern int register_random_ready_notifier(struct notifier_block *nb);
-extern int unregister_random_ready_notifier(struct notifier_block *nb);
-extern size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes);
-
-#ifndef MODULE
-extern const struct file_operations random_fops, urandom_fops;
+static inline void add_latent_entropy(void) { }
#endif
+void get_random_bytes(void *buf, size_t nbytes);
+size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes);
u32 get_random_u32(void);
u64 get_random_u64(void);
static inline unsigned int get_random_int(void)
@@ -56,6 +45,14 @@ static inline unsigned long get_random_l
#endif
}
+unsigned long randomize_page(unsigned long start, unsigned long range);
+
+int __init random_init(const char *command_line);
+bool rng_is_initialized(void);
+int wait_for_random_bytes(void);
+int register_random_ready_notifier(struct notifier_block *nb);
+int unregister_random_ready_notifier(struct notifier_block *nb);
+
/* Calls wait_for_random_bytes() and then calls get_random_bytes(buf, nbytes).
* Returns the result of the call to wait_for_random_bytes. */
static inline int get_random_bytes_wait(void *buf, size_t nbytes)
@@ -79,8 +76,6 @@ declare_get_random_var_wait(int)
declare_get_random_var_wait(long)
#undef declare_get_random_var
-unsigned long randomize_page(unsigned long start, unsigned long range);
-
/*
* This is designed to be standalone for just prandom
* users, but for now we include it from <linux/random.h>
@@ -91,22 +86,10 @@ unsigned long randomize_page(unsigned lo
#ifdef CONFIG_ARCH_RANDOM
# include <asm/archrandom.h>
#else
-static inline bool __must_check arch_get_random_long(unsigned long *v)
-{
- return false;
-}
-static inline bool __must_check arch_get_random_int(unsigned int *v)
-{
- return false;
-}
-static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
-{
- return false;
-}
-static inline bool __must_check arch_get_random_seed_int(unsigned int *v)
-{
- return false;
-}
+static inline bool __must_check arch_get_random_long(unsigned long *v) { return false; }
+static inline bool __must_check arch_get_random_int(unsigned int *v) { return false; }
+static inline bool __must_check arch_get_random_seed_long(unsigned long *v) { return false; }
+static inline bool __must_check arch_get_random_seed_int(unsigned int *v) { return false; }
#endif
/*
@@ -130,8 +113,12 @@ static inline bool __init arch_get_rando
#endif
#ifdef CONFIG_SMP
-extern int random_prepare_cpu(unsigned int cpu);
-extern int random_online_cpu(unsigned int cpu);
+int random_prepare_cpu(unsigned int cpu);
+int random_online_cpu(unsigned int cpu);
+#endif
+
+#ifndef MODULE
+extern const struct file_operations random_fops, urandom_fops;
#endif
#endif /* _LINUX_RANDOM_H */
From: "Jason A. Donenfeld" <[email protected]>
commit e3d2c5e79a999aa4e7d6f0127e16d3da5a4ff70d upstream.
crng_init represents a state machine, with three states, and various
rules for transitions. For the longest time, we've been managing these
with "0", "1", and "2", and expecting people to figure it out. To make
the code more obvious, replace these with proper enum values
representing the transition, and then redocument what each of these
states mean.
Reviewed-by: Dominik Brodowski <[email protected]>
Cc: Joe Perches <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 38 +++++++++++++++++++-------------------
1 file changed, 19 insertions(+), 19 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -71,16 +71,16 @@
*********************************************************************/
/*
- * crng_init = 0 --> Uninitialized
- * 1 --> Initialized
- * 2 --> Initialized from input_pool
- *
* crng_init is protected by base_crng->lock, and only increases
- * its value (from 0->1->2).
+ * its value (from empty->early->ready).
*/
-static int crng_init = 0;
-#define crng_ready() (likely(crng_init > 1))
-/* Various types of waiters for crng_init->2 transition. */
+static enum {
+ CRNG_EMPTY = 0, /* Little to no entropy collected */
+ CRNG_EARLY = 1, /* At least POOL_EARLY_BITS collected */
+ CRNG_READY = 2 /* Fully initialized with POOL_READY_BITS collected */
+} crng_init = CRNG_EMPTY;
+#define crng_ready() (likely(crng_init >= CRNG_READY))
+/* Various types of waiters for crng_init->CRNG_READY transition. */
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
static struct fasync_struct *fasync;
static DEFINE_SPINLOCK(random_ready_chain_lock);
@@ -283,7 +283,7 @@ static void crng_reseed(void)
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
if (!crng_ready()) {
- crng_init = 2;
+ crng_init = CRNG_READY;
finalize_init = true;
}
spin_unlock_irqrestore(&base_crng.lock, flags);
@@ -377,7 +377,7 @@ static void crng_make_state(u32 chacha_s
* For the fast path, we check whether we're ready, unlocked first, and
* then re-check once locked later. In the case where we're really not
* ready, we do fast key erasure with the base_crng directly, extracting
- * when crng_init==0.
+ * when crng_init is CRNG_EMPTY.
*/
if (!crng_ready()) {
bool ready;
@@ -385,7 +385,7 @@ static void crng_make_state(u32 chacha_s
spin_lock_irqsave(&base_crng.lock, flags);
ready = crng_ready();
if (!ready) {
- if (crng_init == 0)
+ if (crng_init == CRNG_EMPTY)
extract_entropy(base_crng.key, sizeof(base_crng.key));
crng_fast_key_erasure(base_crng.key, chacha_state,
random_data, random_data_len);
@@ -736,8 +736,8 @@ EXPORT_SYMBOL(get_random_bytes_arch);
enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
- POOL_INIT_BITS = POOL_BITS, /* No point in settling for less. */
- POOL_FAST_INIT_BITS = POOL_INIT_BITS / 2
+ POOL_READY_BITS = POOL_BITS, /* When crng_init->CRNG_READY */
+ POOL_EARLY_BITS = POOL_READY_BITS / 2 /* When crng_init->CRNG_EARLY */
};
static struct {
@@ -832,13 +832,13 @@ static void credit_init_bits(size_t nbit
init_bits = min_t(unsigned int, POOL_BITS, orig + add);
} while (cmpxchg(&input_pool.init_bits, orig, init_bits) != orig);
- if (!crng_ready() && init_bits >= POOL_INIT_BITS)
+ if (!crng_ready() && init_bits >= POOL_READY_BITS)
crng_reseed();
- else if (unlikely(crng_init == 0 && init_bits >= POOL_FAST_INIT_BITS)) {
+ else if (unlikely(crng_init == CRNG_EMPTY && init_bits >= POOL_EARLY_BITS)) {
spin_lock_irqsave(&base_crng.lock, flags);
- if (crng_init == 0) {
+ if (crng_init == CRNG_EMPTY) {
extract_entropy(base_crng.key, sizeof(base_crng.key));
- crng_init = 1;
+ crng_init = CRNG_EARLY;
}
spin_unlock_irqrestore(&base_crng.lock, flags);
}
@@ -1511,7 +1511,7 @@ const struct file_operations urandom_fop
*
* - write_wakeup_threshold - the amount of entropy in the input pool
* below which write polls to /dev/random will unblock, requesting
- * more entropy, tied to the POOL_INIT_BITS constant. It is writable
+ * more entropy, tied to the POOL_READY_BITS constant. It is writable
* to avoid breaking old userspaces, but writing to it does not
* change any behavior of the RNG.
*
@@ -1526,7 +1526,7 @@ const struct file_operations urandom_fop
#include <linux/sysctl.h>
static int sysctl_random_min_urandom_seed = CRNG_RESEED_INTERVAL / HZ;
-static int sysctl_random_write_wakeup_bits = POOL_INIT_BITS;
+static int sysctl_random_write_wakeup_bits = POOL_READY_BITS;
static int sysctl_poolsize = POOL_BITS;
static u8 sysctl_bootid[UUID_SIZE];
From: "Jason A. Donenfeld" <[email protected]>
commit e3c1c4fd9e6d14059ed93ebfe15e1c57793b1a05 upstream.
In 1448769c9cdb ("random: check for signal_pending() outside of
need_resched() check"), Jann pointed out that we previously were only
checking the TIF_NOTIFY_SIGNAL and TIF_SIGPENDING flags if the process
had TIF_NEED_RESCHED set, which meant in practice, super long reads to
/dev/[u]random would delay signal handling by a long time. I tried this
using the below program, and indeed I wasn't able to interrupt a
/dev/urandom read until after several megabytes had been read. The bug
he fixed has always been there, and so code that reads from /dev/urandom
without checking the return value of read() has mostly worked for a long
time, for most sizes, not just for <= 256.
Maybe it makes sense to keep that code working. The reason it was so
small prior, ignoring the fact that it didn't work anyway, was likely
because /dev/random used to block, and that could happen for pretty
large lengths of time while entropy was gathered. But now, it's just a
chacha20 call, which is extremely fast and is just operating on pure
data, without having to wait for some external event. In that sense,
/dev/[u]random is a lot more like /dev/zero.
Taking a page out of /dev/zero's read_zero() function, it always returns
at least one chunk, and then checks for signals after each chunk. Chunk
sizes there are of length PAGE_SIZE. Let's just copy the same thing for
/dev/[u]random, and check for signals and cond_resched() for every
PAGE_SIZE amount of data. This makes the behavior more consistent with
expectations, and should mitigate the impact of Jann's fix for the
age-old signal check bug.
---- test program ----
#include <unistd.h>
#include <signal.h>
#include <stdio.h>
#include <sys/random.h>
static unsigned char x[~0U];
static void handle(int) { }
int main(int argc, char *argv[])
{
pid_t pid = getpid(), child;
signal(SIGUSR1, handle);
if (!(child = fork())) {
for (;;)
kill(pid, SIGUSR1);
}
pause();
printf("interrupted after reading %zd bytes\n", getrandom(x, sizeof(x), 0));
kill(child, SIGTERM);
return 0;
}
Cc: Jann Horn <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 41 ++++++++++++++++++++---------------------
1 file changed, 20 insertions(+), 21 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -524,9 +524,7 @@ EXPORT_SYMBOL(get_random_bytes);
static ssize_t get_random_bytes_user(void __user *buf, size_t nbytes)
{
- bool large_request = nbytes > 256;
- ssize_t ret = 0;
- size_t len;
+ size_t len, left, ret = 0;
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
u8 output[CHACHA20_BLOCK_SIZE];
@@ -538,46 +536,47 @@ static ssize_t get_random_bytes_user(voi
* bytes, in case userspace causes copy_to_user() below to sleep
* forever, so that we still retain forward secrecy in that case.
*/
- crng_make_state(chacha_state, (u8 *)&chacha_state[4], CHACHA_KEY_SIZE);
+ crng_make_state(chacha_state, (u8 *)&chacha_state[4], CHACHA20_KEY_SIZE);
/*
* However, if we're doing a read of len <= 32, we don't need to
* use chacha_state after, so we can simply return those bytes to
* the user directly.
*/
- if (nbytes <= CHACHA_KEY_SIZE) {
- ret = copy_to_user(buf, &chacha_state[4], nbytes) ? -EFAULT : nbytes;
+ if (nbytes <= CHACHA20_KEY_SIZE) {
+ ret = nbytes - copy_to_user(buf, &chacha_state[4], nbytes);
goto out_zero_chacha;
}
- do {
- if (large_request) {
- if (signal_pending(current)) {
- if (!ret)
- ret = -ERESTARTSYS;
- break;
- }
- cond_resched();
- }
-
+ for (;;) {
chacha20_block(chacha_state, output);
if (unlikely(chacha_state[12] == 0))
++chacha_state[13];
len = min_t(size_t, nbytes, CHACHA20_BLOCK_SIZE);
- if (copy_to_user(buf, output, len)) {
- ret = -EFAULT;
+ left = copy_to_user(buf, output, len);
+ if (left) {
+ ret += len - left;
break;
}
- nbytes -= len;
buf += len;
ret += len;
- } while (nbytes);
+ nbytes -= len;
+ if (!nbytes)
+ break;
+
+ BUILD_BUG_ON(PAGE_SIZE % CHACHA20_BLOCK_SIZE != 0);
+ if (ret % PAGE_SIZE == 0) {
+ if (signal_pending(current))
+ break;
+ cond_resched();
+ }
+ }
memzero_explicit(output, sizeof(output));
out_zero_chacha:
memzero_explicit(chacha_state, sizeof(chacha_state));
- return ret;
+ return ret ? ret : -EFAULT;
}
/*
From: Jens Axboe <[email protected]>
commit 22b0a222af4df8ee9bb8e07013ab44da9511b047 upstream.
Now that the read side has been converted to fix a regression with
splice, convert the write side as well to have some symmetry in the
interface used (and help deprecate ->write()).
Signed-off-by: Jens Axboe <[email protected]>
[Jason: cleaned up random_ioctl a bit, require full writes in
RNDADDENTROPY since it's crediting entropy, simplify control flow of
write_pool(), and incorporate suggestions from Al.]
Cc: Al Viro <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 67 ++++++++++++++++++++++++++------------------------
1 file changed, 35 insertions(+), 32 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1252,39 +1252,31 @@ static unsigned int random_poll(struct f
return crng_ready() ? POLLIN | POLLRDNORM : POLLOUT | POLLWRNORM;
}
-static int write_pool(const char __user *ubuf, size_t len)
+static ssize_t write_pool(struct iov_iter *iter)
{
- size_t block_len;
- int ret = 0;
u8 block[BLAKE2S_BLOCK_SIZE];
+ ssize_t ret = 0;
+ size_t copied;
- while (len) {
- block_len = min(len, sizeof(block));
- if (copy_from_user(block, ubuf, block_len)) {
- ret = -EFAULT;
- goto out;
- }
- len -= block_len;
- ubuf += block_len;
- mix_pool_bytes(block, block_len);
+ if (unlikely(!iov_iter_count(iter)))
+ return 0;
+
+ for (;;) {
+ copied = copy_from_iter(block, sizeof(block), iter);
+ ret += copied;
+ mix_pool_bytes(block, copied);
+ if (!iov_iter_count(iter) || copied != sizeof(block))
+ break;
cond_resched();
}
-out:
memzero_explicit(block, sizeof(block));
- return ret;
+ return ret ? ret : -EFAULT;
}
-static ssize_t random_write(struct file *file, const char __user *ubuf,
- size_t len, loff_t *ppos)
+static ssize_t random_write_iter(struct kiocb *kiocb, struct iov_iter *iter)
{
- int ret;
-
- ret = write_pool(ubuf, len);
- if (ret)
- return ret;
-
- return (ssize_t)len;
+ return write_pool(iter);
}
static ssize_t urandom_read_iter(struct kiocb *kiocb, struct iov_iter *iter)
@@ -1316,9 +1308,8 @@ static ssize_t random_read_iter(struct k
static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
{
- int size, ent_count;
int __user *p = (int __user *)arg;
- int retval;
+ int ent_count;
switch (cmd) {
case RNDGETENTCNT:
@@ -1335,20 +1326,32 @@ static long random_ioctl(struct file *f,
return -EINVAL;
credit_init_bits(ent_count);
return 0;
- case RNDADDENTROPY:
+ case RNDADDENTROPY: {
+ struct iov_iter iter;
+ struct iovec iov;
+ ssize_t ret;
+ int len;
+
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
if (get_user(ent_count, p++))
return -EFAULT;
if (ent_count < 0)
return -EINVAL;
- if (get_user(size, p++))
+ if (get_user(len, p++))
+ return -EFAULT;
+ ret = import_single_range(WRITE, p, len, &iov, &iter);
+ if (unlikely(ret))
+ return ret;
+ ret = write_pool(&iter);
+ if (unlikely(ret < 0))
+ return ret;
+ /* Since we're crediting, enforce that it was all written into the pool. */
+ if (unlikely(ret != len))
return -EFAULT;
- retval = write_pool((const char __user *)p, size);
- if (retval < 0)
- return retval;
credit_init_bits(ent_count);
return 0;
+ }
case RNDZAPENTCNT:
case RNDCLEARPOOL:
/* No longer has any effect. */
@@ -1374,7 +1377,7 @@ static int random_fasync(int fd, struct
const struct file_operations random_fops = {
.read_iter = random_read_iter,
- .write = random_write,
+ .write_iter = random_write_iter,
.poll = random_poll,
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
@@ -1383,7 +1386,7 @@ const struct file_operations random_fops
const struct file_operations urandom_fops = {
.read_iter = urandom_read_iter,
- .write = random_write,
+ .write_iter = random_write_iter,
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
.llseek = noop_llseek,
From: "Jason A. Donenfeld" <[email protected]>
commit 8717627d6ac53251ee012c3c7aca392f29f38a42 upstream.
This reverts 35a33ff3807d ("random: use memmove instead of memcpy for
remaining 32 bytes"), which was made on a totally bogus basis. The thing
it was worried about overlapping came from the stack, not from one of
its arguments, as Eric pointed out.
But the fact that this confusion even happened draws attention to the
fact that it's a bit non-obvious that the random_data parameter can
alias chacha_state, and in fact should do so when the caller can't rely
on the stack being cleared in a timely manner. So this commit documents
that.
Reported-by: Eric Biggers <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 7 +++++++
1 file changed, 7 insertions(+)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -319,6 +319,13 @@ static void crng_reseed(void)
* the resultant ChaCha state to the user, along with the second
* half of the block containing 32 bytes of random data that may
* be used; random_data_len may not be greater than 32.
+ *
+ * The returned ChaCha state contains within it a copy of the old
+ * key value, at index 4, so the state should always be zeroed out
+ * immediately after using in order to maintain forward secrecy.
+ * If the state cannot be erased in a timely manner, then it is
+ * safer to set the random_data parameter to &chacha_state[4] so
+ * that this function overwrites it before returning.
*/
static void crng_fast_key_erasure(u8 key[CHACHA20_KEY_SIZE],
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)],
From: "Jason A. Donenfeld" <[email protected]>
commit 5ad7dd882e45d7fe432c32e896e2aaa0b21746ea upstream.
randomize_page is an mm function. It is documented like one. It contains
the history of one. It has the naming convention of one. It looks
just like another very similar function in mm, randomize_stack_top().
And it has always been maintained and updated by mm people. There is no
need for it to be in random.c. In the "which shape does not look like
the other ones" test, pointing to randomize_page() is correct.
So move randomize_page() into mm/util.c, right next to the similar
randomize_stack_top() function.
This commit contains no actual code changes.
Cc: Andrew Morton <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 238 ++++++++++++++++---------------------------------
include/linux/mm.h | 2
include/linux/random.h | 2
mm/util.c | 33 ++++++
4 files changed, 117 insertions(+), 158 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -52,6 +52,7 @@
#include <linux/completion.h>
#include <linux/uuid.h>
#include <linux/siphash.h>
+#include <linux/uio.h>
#include <crypto/chacha20.h>
#include <crypto/blake2s.h>
#include <asm/processor.h>
@@ -447,13 +448,13 @@ void get_random_bytes(void *buf, size_t
}
EXPORT_SYMBOL(get_random_bytes);
-static ssize_t get_random_bytes_user(void __user *ubuf, size_t len)
+static ssize_t get_random_bytes_user(struct iov_iter *iter)
{
- size_t block_len, left, ret = 0;
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
- u8 output[CHACHA20_BLOCK_SIZE];
+ u8 block[CHACHA20_BLOCK_SIZE];
+ size_t ret = 0, copied;
- if (!len)
+ if (unlikely(!iov_iter_count(iter)))
return 0;
/*
@@ -467,30 +468,22 @@ static ssize_t get_random_bytes_user(voi
* use chacha_state after, so we can simply return those bytes to
* the user directly.
*/
- if (len <= CHACHA20_KEY_SIZE) {
- ret = len - copy_to_user(ubuf, &chacha_state[4], len);
+ if (iov_iter_count(iter) <= CHACHA20_KEY_SIZE) {
+ ret = copy_to_iter(&chacha_state[4], CHACHA20_KEY_SIZE, iter);
goto out_zero_chacha;
}
for (;;) {
- chacha20_block(chacha_state, output);
+ chacha20_block(chacha_state, block);
if (unlikely(chacha_state[12] == 0))
++chacha_state[13];
- block_len = min_t(size_t, len, CHACHA20_BLOCK_SIZE);
- left = copy_to_user(ubuf, output, block_len);
- if (left) {
- ret += block_len - left;
+ copied = copy_to_iter(block, sizeof(block), iter);
+ ret += copied;
+ if (!iov_iter_count(iter) || copied != sizeof(block))
break;
- }
- ubuf += block_len;
- ret += block_len;
- len -= block_len;
- if (!len)
- break;
-
- BUILD_BUG_ON(PAGE_SIZE % CHACHA20_BLOCK_SIZE != 0);
+ BUILD_BUG_ON(PAGE_SIZE % sizeof(block) != 0);
if (ret % PAGE_SIZE == 0) {
if (signal_pending(current))
break;
@@ -498,7 +491,7 @@ static ssize_t get_random_bytes_user(voi
}
}
- memzero_explicit(output, sizeof(output));
+ memzero_explicit(block, sizeof(block));
out_zero_chacha:
memzero_explicit(chacha_state, sizeof(chacha_state));
return ret ? ret : -EFAULT;
@@ -510,96 +503,60 @@ out_zero_chacha:
* provided by this function is okay, the function wait_for_random_bytes()
* should be called and return 0 at least once at any point prior.
*/
-struct batched_entropy {
- union {
- /*
- * We make this 1.5x a ChaCha block, so that we get the
- * remaining 32 bytes from fast key erasure, plus one full
- * block from the detached ChaCha state. We can increase
- * the size of this later if needed so long as we keep the
- * formula of (integer_blocks + 0.5) * CHACHA20_BLOCK_SIZE.
- */
- u64 entropy_u64[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u64))];
- u32 entropy_u32[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u32))];
- };
- unsigned long generation;
- unsigned int position;
-};
-
-
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
- .position = UINT_MAX
-};
-
-u64 get_random_u64(void)
-{
- u64 ret;
- unsigned long flags;
- struct batched_entropy *batch;
- unsigned long next_gen;
-
- warn_unseeded_randomness();
-
- if (!crng_ready()) {
- _get_random_bytes(&ret, sizeof(ret));
- return ret;
- }
-
- local_irq_save(flags);
- batch = raw_cpu_ptr(&batched_entropy_u64);
-
- next_gen = READ_ONCE(base_crng.generation);
- if (batch->position >= ARRAY_SIZE(batch->entropy_u64) ||
- next_gen != batch->generation) {
- _get_random_bytes(batch->entropy_u64, sizeof(batch->entropy_u64));
- batch->position = 0;
- batch->generation = next_gen;
- }
-
- ret = batch->entropy_u64[batch->position];
- batch->entropy_u64[batch->position] = 0;
- ++batch->position;
- local_irq_restore(flags);
- return ret;
-}
-EXPORT_SYMBOL(get_random_u64);
-
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
- .position = UINT_MAX
-};
-
-u32 get_random_u32(void)
-{
- u32 ret;
- unsigned long flags;
- struct batched_entropy *batch;
- unsigned long next_gen;
- warn_unseeded_randomness();
+#define DEFINE_BATCHED_ENTROPY(type) \
+struct batch_ ##type { \
+ /* \
+ * We make this 1.5x a ChaCha block, so that we get the \
+ * remaining 32 bytes from fast key erasure, plus one full \
+ * block from the detached ChaCha state. We can increase \
+ * the size of this later if needed so long as we keep the \
+ * formula of (integer_blocks + 0.5) * CHACHA20_BLOCK_SIZE. \
+ */ \
+ type entropy[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(type))]; \
+ unsigned long generation; \
+ unsigned int position; \
+}; \
+ \
+static DEFINE_PER_CPU(struct batch_ ##type, batched_entropy_ ##type) = { \
+ .position = UINT_MAX \
+}; \
+ \
+type get_random_ ##type(void) \
+{ \
+ type ret; \
+ unsigned long flags; \
+ struct batch_ ##type *batch; \
+ unsigned long next_gen; \
+ \
+ warn_unseeded_randomness(); \
+ \
+ if (!crng_ready()) { \
+ _get_random_bytes(&ret, sizeof(ret)); \
+ return ret; \
+ } \
+ \
+ local_irq_save(flags); \
+ batch = raw_cpu_ptr(&batched_entropy_##type); \
+ \
+ next_gen = READ_ONCE(base_crng.generation); \
+ if (batch->position >= ARRAY_SIZE(batch->entropy) || \
+ next_gen != batch->generation) { \
+ _get_random_bytes(batch->entropy, sizeof(batch->entropy)); \
+ batch->position = 0; \
+ batch->generation = next_gen; \
+ } \
+ \
+ ret = batch->entropy[batch->position]; \
+ batch->entropy[batch->position] = 0; \
+ ++batch->position; \
+ local_irq_restore(flags); \
+ return ret; \
+} \
+EXPORT_SYMBOL(get_random_ ##type);
- if (!crng_ready()) {
- _get_random_bytes(&ret, sizeof(ret));
- return ret;
- }
-
- local_irq_save(flags);
- batch = raw_cpu_ptr(&batched_entropy_u32);
-
- next_gen = READ_ONCE(base_crng.generation);
- if (batch->position >= ARRAY_SIZE(batch->entropy_u32) ||
- next_gen != batch->generation) {
- _get_random_bytes(batch->entropy_u32, sizeof(batch->entropy_u32));
- batch->position = 0;
- batch->generation = next_gen;
- }
-
- ret = batch->entropy_u32[batch->position];
- batch->entropy_u32[batch->position] = 0;
- ++batch->position;
- local_irq_restore(flags);
- return ret;
-}
-EXPORT_SYMBOL(get_random_u32);
+DEFINE_BATCHED_ENTROPY(u64)
+DEFINE_BATCHED_ENTROPY(u32)
#ifdef CONFIG_SMP
/*
@@ -620,38 +577,6 @@ int __cold random_prepare_cpu(unsigned i
}
#endif
-/**
- * randomize_page - Generate a random, page aligned address
- * @start: The smallest acceptable address the caller will take.
- * @range: The size of the area, starting at @start, within which the
- * random address must fall.
- *
- * If @start + @range would overflow, @range is capped.
- *
- * NOTE: Historical use of randomize_range, which this replaces, presumed that
- * @start was already page aligned. We now align it regardless.
- *
- * Return: A page aligned address within [start, start + range). On error,
- * @start is returned.
- */
-unsigned long randomize_page(unsigned long start, unsigned long range)
-{
- if (!PAGE_ALIGNED(start)) {
- range -= PAGE_ALIGN(start) - start;
- start = PAGE_ALIGN(start);
- }
-
- if (start > ULONG_MAX - range)
- range = ULONG_MAX - start;
-
- range >>= PAGE_SHIFT;
-
- if (range == 0)
- return start;
-
- return start + (get_random_long() % range << PAGE_SHIFT);
-}
-
/*
* This function will use the architecture-specific hardware random
* number generator if it is available. It is not recommended for
@@ -1293,6 +1218,10 @@ static void __cold try_to_generate_entro
SYSCALL_DEFINE3(getrandom, char __user *, ubuf, size_t, len, unsigned int, flags)
{
+ struct iov_iter iter;
+ struct iovec iov;
+ int ret;
+
if (flags & ~(GRND_NONBLOCK | GRND_RANDOM | GRND_INSECURE))
return -EINVAL;
@@ -1303,19 +1232,18 @@ SYSCALL_DEFINE3(getrandom, char __user *
if ((flags & (GRND_INSECURE | GRND_RANDOM)) == (GRND_INSECURE | GRND_RANDOM))
return -EINVAL;
- if (len > INT_MAX)
- len = INT_MAX;
-
if (!crng_ready() && !(flags & GRND_INSECURE)) {
- int ret;
-
if (flags & GRND_NONBLOCK)
return -EAGAIN;
ret = wait_for_random_bytes();
if (unlikely(ret))
return ret;
}
- return get_random_bytes_user(ubuf, len);
+
+ ret = import_single_range(READ, ubuf, len, &iov, &iter);
+ if (unlikely(ret))
+ return ret;
+ return get_random_bytes_user(&iter);
}
static unsigned int random_poll(struct file *file, poll_table *wait)
@@ -1359,8 +1287,7 @@ static ssize_t random_write(struct file
return (ssize_t)len;
}
-static ssize_t urandom_read(struct file *file, char __user *ubuf,
- size_t len, loff_t *ppos)
+static ssize_t urandom_read_iter(struct kiocb *kiocb, struct iov_iter *iter)
{
static int maxwarn = 10;
@@ -1369,23 +1296,22 @@ static ssize_t urandom_read(struct file
++urandom_warning.missed;
else if (ratelimit_disable || __ratelimit(&urandom_warning)) {
--maxwarn;
- pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
- current->comm, len);
+ pr_notice("%s: uninitialized urandom read (%zu bytes read)\n",
+ current->comm, iov_iter_count(iter));
}
}
- return get_random_bytes_user(ubuf, len);
+ return get_random_bytes_user(iter);
}
-static ssize_t random_read(struct file *file, char __user *ubuf,
- size_t len, loff_t *ppos)
+static ssize_t random_read_iter(struct kiocb *kiocb, struct iov_iter *iter)
{
int ret;
ret = wait_for_random_bytes();
if (ret != 0)
return ret;
- return get_random_bytes_user(ubuf, len);
+ return get_random_bytes_user(iter);
}
static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
@@ -1447,7 +1373,7 @@ static int random_fasync(int fd, struct
}
const struct file_operations random_fops = {
- .read = random_read,
+ .read_iter = random_read_iter,
.write = random_write,
.poll = random_poll,
.unlocked_ioctl = random_ioctl,
@@ -1456,7 +1382,7 @@ const struct file_operations random_fops
};
const struct file_operations urandom_fops = {
- .read = urandom_read,
+ .read_iter = urandom_read_iter,
.write = random_write,
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2059,6 +2059,8 @@ extern int install_special_mapping(struc
unsigned long addr, unsigned long len,
unsigned long flags, struct page **pages);
+unsigned long randomize_page(unsigned long start, unsigned long range);
+
extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
extern unsigned long mmap_region(struct file *file, unsigned long addr,
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -45,8 +45,6 @@ static inline unsigned long get_random_l
#endif
}
-unsigned long randomize_page(unsigned long start, unsigned long range);
-
int __init random_init(const char *command_line);
bool rng_is_initialized(void);
int wait_for_random_bytes(void);
--- a/mm/util.c
+++ b/mm/util.c
@@ -11,6 +11,7 @@
#include <linux/mman.h>
#include <linux/hugetlb.h>
#include <linux/vmalloc.h>
+#include <linux/random.h>
#include <asm/sections.h>
#include <asm/uaccess.h>
@@ -261,6 +262,38 @@ int vma_is_stack_for_current(struct vm_a
return (vma->vm_start <= KSTK_ESP(t) && vma->vm_end >= KSTK_ESP(t));
}
+/**
+ * randomize_page - Generate a random, page aligned address
+ * @start: The smallest acceptable address the caller will take.
+ * @range: The size of the area, starting at @start, within which the
+ * random address must fall.
+ *
+ * If @start + @range would overflow, @range is capped.
+ *
+ * NOTE: Historical use of randomize_range, which this replaces, presumed that
+ * @start was already page aligned. We now align it regardless.
+ *
+ * Return: A page aligned address within [start, start + range). On error,
+ * @start is returned.
+ */
+unsigned long randomize_page(unsigned long start, unsigned long range)
+{
+ if (!PAGE_ALIGNED(start)) {
+ range -= PAGE_ALIGN(start) - start;
+ start = PAGE_ALIGN(start);
+ }
+
+ if (start > ULONG_MAX - range)
+ range = ULONG_MAX - start;
+
+ range >>= PAGE_SHIFT;
+
+ if (range == 0)
+ return start;
+
+ return start + (get_random_long() % range << PAGE_SHIFT);
+}
+
#if defined(CONFIG_MMU) && !defined(HAVE_ARCH_PICK_MMAP_LAYOUT)
void arch_pick_mmap_layout(struct mm_struct *mm)
{
From: "Jason A. Donenfeld" <[email protected]>
commit 1754abb3e7583c570666fa1e1ee5b317e88c89a0 upstream.
Prior, the "input_pool_data" array needed no real initialization, and so
it was easy to mark it with __latent_entropy to populate it during
compile-time. In switching to using a hash function, this required us to
specifically initialize it to some specific state, which means we
dropped the __latent_entropy attribute. An unfortunate side effect was
this meant the pool was no longer seeded using compile-time random data.
In order to bring this back, we declare an array in rand_initialize()
with __latent_entropy and call mix_pool_bytes() on that at init, which
accomplishes the same thing as before. We make this __initconst, so that
it doesn't take up space at runtime after init.
Fixes: 6e8ec2552c7d ("random: use computational hash for entropy extraction")
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 5 +++++
1 file changed, 5 insertions(+)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -968,6 +968,11 @@ int __init rand_initialize(void)
bool arch_init = true;
unsigned long rv;
+#if defined(LATENT_ENTROPY_PLUGIN)
+ static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy;
+ _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed));
+#endif
+
for (i = 0; i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
if (!arch_get_random_seed_long_early(&rv) &&
!arch_get_random_long_early(&rv)) {
From: "Jason A. Donenfeld" <[email protected]>
commit 39e0f991a62ed5efabd20711a7b6e7da92603170 upstream.
add_bootloader_randomness() and the variables it touches are only used
during __init and not after, so mark these as __init. At the same time,
unexport this, since it's only called by other __init code that's
built-in.
Cc: [email protected]
Fixes: 428826f5358c ("fdt: add support for rng-seed")
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 7 +++----
include/linux/random.h | 2 +-
2 files changed, 4 insertions(+), 5 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -786,8 +786,8 @@ static void __cold _credit_init_bits(siz
*
**********************************************************************/
-static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
-static bool trust_bootloader __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER);
+static bool trust_cpu __initdata = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
+static bool trust_bootloader __initdata = IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER);
static int __init parse_trust_cpu(char *arg)
{
return kstrtobool(arg, &trust_cpu);
@@ -883,13 +883,12 @@ EXPORT_SYMBOL_GPL(add_hwgenerator_random
* Handle random seed passed by bootloader, and credit it if
* CONFIG_RANDOM_TRUST_BOOTLOADER is set.
*/
-void __cold add_bootloader_randomness(const void *buf, size_t len)
+void __init add_bootloader_randomness(const void *buf, size_t len)
{
mix_pool_bytes(buf, len);
if (trust_bootloader)
credit_init_bits(len * 8);
}
-EXPORT_SYMBOL_GPL(add_bootloader_randomness);
struct fast_pool {
struct work_struct mix;
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -13,7 +13,7 @@
struct notifier_block;
void add_device_randomness(const void *buf, size_t len);
-void add_bootloader_randomness(const void *buf, size_t len);
+void __init add_bootloader_randomness(const void *buf, size_t len);
void add_input_randomness(unsigned int type, unsigned int code,
unsigned int value) __latent_entropy;
void add_interrupt_randomness(int irq) __latent_entropy;
From: Ding Xiang <[email protected]>
commit bc75a6eb856cb1507fa907bf6c1eda91b3fef52f upstream.
Since dx_make_map() may return -EFSCORRUPTED now, so change "count" to
be a signed integer so we can correctly check for an error code returned
by dx_make_map().
Fixes: 46c116b920eb ("ext4: verify dir block before splitting it")
Cc: [email protected]
Signed-off-by: Ding Xiang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
fs/ext4/namei.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -1726,7 +1726,8 @@ static struct ext4_dir_entry_2 *do_split
struct dx_hash_info *hinfo)
{
unsigned blocksize = dir->i_sb->s_blocksize;
- unsigned count, continued;
+ unsigned continued;
+ int count;
struct buffer_head *bh2;
ext4_lblk_t newblock;
u32 hash2;
From: Nicolai Stange <[email protected]>
commit ce8ce31b2c5c8b18667784b8c515650c65d57b4e upstream.
There are two different randomness sources the DRBGs are getting seeded
from, namely the jitterentropy source (if enabled) and get_random_bytes().
At initial DRBG seeding time during boot, the latter might not have
collected sufficient entropy for seeding itself yet and thus, the DRBG
implementation schedules a reseed work from a random_ready_callback once
that has happened. This is particularly important for the !->pr DRBG
instances, for which (almost) no further reseeds are getting triggered
during their lifetime.
Because collecting data from the jitterentropy source is a rather expensive
operation, the aforementioned asynchronously scheduled reseed work
restricts itself to get_random_bytes() only. That is, it in some sense
amends the initial DRBG seed derived from jitterentropy output at full
(estimated) entropy with fresh randomness obtained from get_random_bytes()
once that has been seeded with sufficient entropy itself.
With the advent of rng_is_initialized(), there is no real need for doing
the reseed operation from an asynchronously scheduled work anymore and a
subsequent patch will make it synchronous by moving it next to related
logic already present in drbg_generate().
However, for tracking whether a full reseed including the jitterentropy
source is required or a "partial" reseed involving only get_random_bytes()
would be sufficient already, the boolean struct drbg_state's ->seeded
member must become a tristate value.
Prepare for this by introducing the new enum drbg_seed_state and change
struct drbg_state's ->seeded member's type from bool to that type.
For facilitating review, enum drbg_seed_state is made to only contain
two members corresponding to the former ->seeded values of false and true
resp. at this point: DRBG_SEED_STATE_UNSEEDED and DRBG_SEED_STATE_FULL. A
third one for tracking the intermediate state of "seeded from jitterentropy
only" will be introduced with a subsequent patch.
There is no change in behaviour at this point.
Signed-off-by: Nicolai Stange <[email protected]>
Reviewed-by: Stephan Müller <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 19 ++++++++++---------
include/crypto/drbg.h | 7 ++++++-
2 files changed, 16 insertions(+), 10 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1044,7 +1044,7 @@ static inline int __drbg_seed(struct drb
if (ret)
return ret;
- drbg->seeded = true;
+ drbg->seeded = DRBG_SEED_STATE_FULL;
/* 10.1.1.2 / 10.1.1.3 step 5 */
drbg->reseed_ctr = 1;
@@ -1089,14 +1089,14 @@ static void drbg_async_seed(struct work_
if (ret)
goto unlock;
- /* Set seeded to false so that if __drbg_seed fails the
- * next generate call will trigger a reseed.
+ /* Reset ->seeded so that if __drbg_seed fails the next
+ * generate call will trigger a reseed.
*/
- drbg->seeded = false;
+ drbg->seeded = DRBG_SEED_STATE_UNSEEDED;
__drbg_seed(drbg, &seedlist, true);
- if (drbg->seeded)
+ if (drbg->seeded == DRBG_SEED_STATE_FULL)
drbg->reseed_threshold = drbg_max_requests(drbg);
unlock:
@@ -1385,13 +1385,14 @@ static int drbg_generate(struct drbg_sta
* here. The spec is a bit convoluted here, we make it simpler.
*/
if (drbg->reseed_threshold < drbg->reseed_ctr)
- drbg->seeded = false;
+ drbg->seeded = DRBG_SEED_STATE_UNSEEDED;
- if (drbg->pr || !drbg->seeded) {
+ if (drbg->pr || drbg->seeded == DRBG_SEED_STATE_UNSEEDED) {
pr_devel("DRBG: reseeding before generation (prediction "
"resistance: %s, state %s)\n",
drbg->pr ? "true" : "false",
- drbg->seeded ? "seeded" : "unseeded");
+ (drbg->seeded == DRBG_SEED_STATE_FULL ?
+ "seeded" : "unseeded"));
/* 9.3.1 steps 7.1 through 7.3 */
len = drbg_seed(drbg, addtl, true);
if (len)
@@ -1576,7 +1577,7 @@ static int drbg_instantiate(struct drbg_
if (!drbg->core) {
drbg->core = &drbg_cores[coreref];
drbg->pr = pr;
- drbg->seeded = false;
+ drbg->seeded = DRBG_SEED_STATE_UNSEEDED;
drbg->reseed_threshold = drbg_max_requests(drbg);
ret = drbg_alloc_state(drbg);
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -105,6 +105,11 @@ struct drbg_test_data {
struct drbg_string *testentropy; /* TEST PARAMETER: test entropy */
};
+enum drbg_seed_state {
+ DRBG_SEED_STATE_UNSEEDED,
+ DRBG_SEED_STATE_FULL,
+};
+
struct drbg_state {
struct mutex drbg_mutex; /* lock around DRBG */
unsigned char *V; /* internal state 10.1.1.1 1a) */
@@ -129,7 +134,7 @@ struct drbg_state {
struct completion ctr_completion; /* CTR mode async handler */
int ctr_async_err; /* CTR mode async error */
- bool seeded; /* DRBG fully seeded? */
+ enum drbg_seed_state seeded; /* DRBG fully seeded? */
bool pr; /* Prediction resistance enabled? */
bool fips_primed; /* Continuous test primed? */
unsigned char *prev; /* FIPS 140-2 continuous test value */
From: Colin Ian King <[email protected]>
commit 912288442cb2f431bf3c8cb097a5de83bc6dbac1 upstream.
Currently the header size calculations are using an assignment
operator instead of a += operator when accumulating the header
size leading to incorrect sizes. Fix this by using the correct
operator.
Addresses-Coverity: ("Unused value")
Fixes: 302d3deb2068 ("xprtrdma: Prevent inline overflow")
Signed-off-by: Colin Ian King <[email protected]>
Reviewed-by: Chuck Lever <[email protected]>
Signed-off-by: Anna Schumaker <[email protected]>
[bwh: Backported to 4.9: adjust context]
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/sunrpc/xprtrdma/rpc_rdma.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/net/sunrpc/xprtrdma/rpc_rdma.c
+++ b/net/sunrpc/xprtrdma/rpc_rdma.c
@@ -75,7 +75,7 @@ static unsigned int rpcrdma_max_call_hea
/* Maximum Read list size */
maxsegs += 2; /* segment for head and tail buffers */
- size = maxsegs * sizeof(struct rpcrdma_read_chunk);
+ size += maxsegs * sizeof(struct rpcrdma_read_chunk);
/* Minimal Read chunk size */
size += sizeof(__be32); /* segment count */
@@ -101,7 +101,7 @@ static unsigned int rpcrdma_max_reply_he
/* Maximum Write list size */
maxsegs += 2; /* segment for head and tail buffers */
- size = sizeof(__be32); /* segment count */
+ size += sizeof(__be32); /* segment count */
size += maxsegs * sizeof(struct rpcrdma_segment);
size += sizeof(__be32); /* list discriminator */
From: James Chapman <[email protected]>
commit d02ba2a6110c530a32926af8ad441111774d2893 upstream.
pppol2tp_release uses call_rcu to put the final ref on its socket. But
the session object doesn't hold a ref on the session socket so may be
freed while the pppol2tp_put_sk RCU callback is scheduled. Fix this by
having the session hold a ref on its socket until the session is
destroyed. It is this ref that is dropped via call_rcu.
Sessions are also deleted via l2tp_tunnel_closeall. This must now also put
the final ref via call_rcu. So move the call_rcu call site into
pppol2tp_session_close so that this happens in both destroy paths. A
common destroy path should really be implemented, perhaps with
l2tp_tunnel_closeall calling l2tp_session_delete like pppol2tp_release
does, but this will be looked at later.
ODEBUG: activate active (active state 1) object type: rcu_head hint: (null)
WARNING: CPU: 3 PID: 13407 at lib/debugobjects.c:291 debug_print_object+0x166/0x220
Modules linked in:
CPU: 3 PID: 13407 Comm: syzbot_19c09769 Not tainted 4.16.0-rc2+ #38
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
RIP: 0010:debug_print_object+0x166/0x220
RSP: 0018:ffff880013647a00 EFLAGS: 00010082
RAX: dffffc0000000008 RBX: 0000000000000003 RCX: ffffffff814d3333
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88001a59f6d0
RBP: ffff880013647a40 R08: 0000000000000000 R09: 0000000000000001
R10: ffff8800136479a8 R11: 0000000000000000 R12: 0000000000000001
R13: ffffffff86161420 R14: ffffffff85648b60 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88001a580000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020e77000 CR3: 0000000006022000 CR4: 00000000000006e0
Call Trace:
debug_object_activate+0x38b/0x530
? debug_object_assert_init+0x3b0/0x3b0
? __mutex_unlock_slowpath+0x85/0x8b0
? pppol2tp_session_destruct+0x110/0x110
__call_rcu.constprop.66+0x39/0x890
? __call_rcu.constprop.66+0x39/0x890
call_rcu_sched+0x17/0x20
pppol2tp_release+0x2c7/0x440
? fcntl_setlk+0xca0/0xca0
? sock_alloc_file+0x340/0x340
sock_release+0x92/0x1e0
sock_close+0x1b/0x20
__fput+0x296/0x6e0
____fput+0x1a/0x20
task_work_run+0x127/0x1a0
do_exit+0x7f9/0x2ce0
? SYSC_connect+0x212/0x310
? mm_update_next_owner+0x690/0x690
? up_read+0x1f/0x40
? __do_page_fault+0x3c8/0xca0
do_group_exit+0x10d/0x330
? do_group_exit+0x330/0x330
SyS_exit_group+0x22/0x30
do_syscall_64+0x1e0/0x730
? trace_hardirqs_off_thunk+0x1a/0x1c
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x7f362e471259
RSP: 002b:00007ffe389abe08 EFLAGS: 00000202 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f362e471259
RDX: 00007f362e471259 RSI: 000000000000002e RDI: 0000000000000000
RBP: 00007ffe389abe30 R08: 0000000000000000 R09: 00007f362e944270
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000400b60
R13: 00007ffe389abf50 R14: 0000000000000000 R15: 0000000000000000
Code: 8d 3c dd a0 8f 64 85 48 89 fa 48 c1 ea 03 80 3c 02 00 75 7b 48 8b 14 dd a0 8f 64 85 4c 89 f6 48 c7 c7 20 85 64 85 e
8 2a 55 14 ff <0f> 0b 83 05 ad 2a 68 04 01 48 83 c4 18 5b 41 5c 41 5d 41 5e 41
Fixes: ee40fb2e1eb5b ("l2tp: protect sock pointer of struct pppol2tp_session with RCU")
Signed-off-by: James Chapman <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
Cc: Lee Jones <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/l2tp/l2tp_ppp.c | 52 +++++++++++++++++++++++++++-------------------------
1 file changed, 27 insertions(+), 25 deletions(-)
--- a/net/l2tp/l2tp_ppp.c
+++ b/net/l2tp/l2tp_ppp.c
@@ -435,10 +435,28 @@ abort:
* Session (and tunnel control) socket create/destroy.
*****************************************************************************/
+static void pppol2tp_put_sk(struct rcu_head *head)
+{
+ struct pppol2tp_session *ps;
+
+ ps = container_of(head, typeof(*ps), rcu);
+ sock_put(ps->__sk);
+}
+
/* Called by l2tp_core when a session socket is being closed.
*/
static void pppol2tp_session_close(struct l2tp_session *session)
{
+ struct pppol2tp_session *ps;
+
+ ps = l2tp_session_priv(session);
+ mutex_lock(&ps->sk_lock);
+ ps->__sk = rcu_dereference_protected(ps->sk,
+ lockdep_is_held(&ps->sk_lock));
+ RCU_INIT_POINTER(ps->sk, NULL);
+ if (ps->__sk)
+ call_rcu(&ps->rcu, pppol2tp_put_sk);
+ mutex_unlock(&ps->sk_lock);
}
/* Really kill the session socket. (Called from sock_put() if
@@ -458,14 +476,6 @@ static void pppol2tp_session_destruct(st
}
}
-static void pppol2tp_put_sk(struct rcu_head *head)
-{
- struct pppol2tp_session *ps;
-
- ps = container_of(head, typeof(*ps), rcu);
- sock_put(ps->__sk);
-}
-
/* Called when the PPPoX socket (session) is closed.
*/
static int pppol2tp_release(struct socket *sock)
@@ -489,26 +499,17 @@ static int pppol2tp_release(struct socke
sock_orphan(sk);
sock->sk = NULL;
+ /* If the socket is associated with a session,
+ * l2tp_session_delete will call pppol2tp_session_close which
+ * will drop the session's ref on the socket.
+ */
session = pppol2tp_sock_to_session(sk);
-
- if (session != NULL) {
- struct pppol2tp_session *ps;
-
+ if (session) {
l2tp_session_delete(session);
-
- ps = l2tp_session_priv(session);
- mutex_lock(&ps->sk_lock);
- ps->__sk = rcu_dereference_protected(ps->sk,
- lockdep_is_held(&ps->sk_lock));
- RCU_INIT_POINTER(ps->sk, NULL);
- mutex_unlock(&ps->sk_lock);
- call_rcu(&ps->rcu, pppol2tp_put_sk);
-
- /* Rely on the sock_put() call at the end of the function for
- * dropping the reference held by pppol2tp_sock_to_session().
- * The last reference will be dropped by pppol2tp_put_sk().
- */
+ /* drop the ref obtained by pppol2tp_sock_to_session */
+ sock_put(sk);
}
+
release_sock(sk);
/* This will delete the session context via
@@ -817,6 +818,7 @@ static int pppol2tp_connect(struct socke
out_no_ppp:
/* This is how we get the session context from the socket. */
+ sock_hold(sk);
sk->sk_user_data = session;
rcu_assign_pointer(ps->sk, sk);
mutex_unlock(&ps->sk_lock);
From: Adam Ford <[email protected]>
[ Upstream commit d1f5272c0f7d2e53c6f2480f46725442776f5f78 ]
If the audio CODEC is playing sound when the system is suspended,
it can be left in a state which throws the following error:
wm8962 3-001a: ASoC: error at soc_component_read_no_lock on wm8962.3-001a: -16
Once this error has occurred, the audio will not work again until rebooted.
Fix this by configuring SET_SYSTEM_SLEEP_PM_OPS.
Signed-off-by: Adam Ford <[email protected]>
Acked-by: Charles Keepax <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Mark Brown <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
sound/soc/codecs/wm8962.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/sound/soc/codecs/wm8962.c b/sound/soc/codecs/wm8962.c
index 0e8008d38161..d46881f96c16 100644
--- a/sound/soc/codecs/wm8962.c
+++ b/sound/soc/codecs/wm8962.c
@@ -3861,6 +3861,7 @@ static int wm8962_runtime_suspend(struct device *dev)
#endif
static const struct dev_pm_ops wm8962_pm = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
SET_RUNTIME_PM_OPS(wm8962_runtime_suspend, wm8962_runtime_resume, NULL)
};
--
2.35.1
From: Miklos Szeredi <[email protected]>
commit 0c4bcfdecb1ac0967619ee7ff44871d93c08c909 upstream.
In FOPEN_DIRECT_IO mode, fuse_file_write_iter() calls
fuse_direct_write_iter(), which normally calls fuse_direct_io(), which then
imports the write buffer with fuse_get_user_pages(), which uses
iov_iter_get_pages() to grab references to userspace pages instead of
actually copying memory.
On the filesystem device side, these pages can then either be read to
userspace (via fuse_dev_read()), or splice()d over into a pipe using
fuse_dev_splice_read() as pipe buffers with &nosteal_pipe_buf_ops.
This is wrong because after fuse_dev_do_read() unlocks the FUSE request,
the userspace filesystem can mark the request as completed, causing write()
to return. At that point, the userspace filesystem should no longer have
access to the pipe buffer.
Fix by copying pages coming from the user address space to new pipe
buffers.
Reported-by: Jann Horn <[email protected]>
Fixes: c3021629a0d8 ("fuse: support splice() reading from fuse device")
Cc: <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>
Signed-off-by: Zach O'Keefe <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
fs/fuse/dev.c | 12 +++++++++++-
fs/fuse/file.c | 1 +
fs/fuse/fuse_i.h | 2 ++
3 files changed, 14 insertions(+), 1 deletion(-)
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -992,7 +992,17 @@ static int fuse_copy_page(struct fuse_co
while (count) {
if (cs->write && cs->pipebufs && page) {
- return fuse_ref_page(cs, page, offset, count);
+ /*
+ * Can't control lifetime of pipe buffers, so always
+ * copy user pages.
+ */
+ if (cs->req->user_pages) {
+ err = fuse_copy_fill(cs);
+ if (err)
+ return err;
+ } else {
+ return fuse_ref_page(cs, page, offset, count);
+ }
} else if (!cs->len) {
if (cs->move_pages && page &&
offset == 0 && count == PAGE_SIZE) {
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1319,6 +1319,7 @@ static int fuse_get_user_pages(struct fu
(PAGE_SIZE - ret) & (PAGE_SIZE - 1);
}
+ req->user_pages = true;
if (write)
req->in.argpages = 1;
else
--- a/fs/fuse/fuse_i.h
+++ b/fs/fuse/fuse_i.h
@@ -310,6 +310,8 @@ struct fuse_req {
/** refcount */
atomic_t count;
+ bool user_pages;
+
/** Unique ID for the interrupt request */
u64 intr_unique;
From: Halil Pasic <[email protected]>
commit ddbd89deb7d32b1fbb879f48d68fda1a8ac58e8e upstream.
The problem I'm addressing was discovered by the LTP test covering
cve-2018-1000204.
A short description of what happens follows:
1) The test case issues a command code 00 (TEST UNIT READY) via the SG_IO
interface with: dxfer_len == 524288, dxdfer_dir == SG_DXFER_FROM_DEV
and a corresponding dxferp. The peculiar thing about this is that TUR
is not reading from the device.
2) In sg_start_req() the invocation of blk_rq_map_user() effectively
bounces the user-space buffer. As if the device was to transfer into
it. Since commit a45b599ad808 ("scsi: sg: allocate with __GFP_ZERO in
sg_build_indirect()") we make sure this first bounce buffer is
allocated with GFP_ZERO.
3) For the rest of the story we keep ignoring that we have a TUR, so the
device won't touch the buffer we prepare as if the we had a
DMA_FROM_DEVICE type of situation. My setup uses a virtio-scsi device
and the buffer allocated by SG is mapped by the function
virtqueue_add_split() which uses DMA_FROM_DEVICE for the "in" sgs (here
scatter-gather and not scsi generics). This mapping involves bouncing
via the swiotlb (we need swiotlb to do virtio in protected guest like
s390 Secure Execution, or AMD SEV).
4) When the SCSI TUR is done, we first copy back the content of the second
(that is swiotlb) bounce buffer (which most likely contains some
previous IO data), to the first bounce buffer, which contains all
zeros. Then we copy back the content of the first bounce buffer to
the user-space buffer.
5) The test case detects that the buffer, which it zero-initialized,
ain't all zeros and fails.
One can argue that this is an swiotlb problem, because without swiotlb
we leak all zeros, and the swiotlb should be transparent in a sense that
it does not affect the outcome (if all other participants are well
behaved).
Copying the content of the original buffer into the swiotlb buffer is
the only way I can think of to make swiotlb transparent in such
scenarios. So let's do just that if in doubt, but allow the driver
to tell us that the whole mapped buffer is going to be overwritten,
in which case we can preserve the old behavior and avoid the performance
impact of the extra bounce.
Signed-off-by: Halil Pasic <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>
[OP: backport to 4.14: apply swiotlb_tbl_map_single() changes in lib/swiotlb.c]
Signed-off-by: Ovidiu Panait <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
[bwh: Backported to 4.9: adjust context]
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/DMA-attributes.txt | 10 ++++++++++
include/linux/dma-mapping.h | 7 +++++++
lib/swiotlb.c | 3 ++-
3 files changed, 19 insertions(+), 1 deletion(-)
--- a/Documentation/DMA-attributes.txt
+++ b/Documentation/DMA-attributes.txt
@@ -143,3 +143,13 @@ So, this provides a way for drivers to a
where allocation failures are not a problem, and shouldn't bother the logs.
NOTE: At the moment DMA_ATTR_NO_WARN is only implemented on PowerPC.
+
+DMA_ATTR_PRIVILEGED
+-------------------
+
+Some advanced peripherals such as remote processors and GPUs perform
+accesses to DMA buffers in both privileged "supervisor" and unprivileged
+"user" modes. This attribute is used to indicate to the DMA-mapping
+subsystem that the buffer is fully accessible at the elevated privilege
+level (and ideally inaccessible or at least read-only at the
+lesser-privileged levels).
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -61,6 +61,13 @@
* allocation failure reports (similarly to __GFP_NOWARN).
*/
#define DMA_ATTR_NO_WARN (1UL << 8)
+/*
+ * This is a hint to the DMA-mapping subsystem that the device is expected
+ * to overwrite the entire mapped size, thus the caller does not require any
+ * of the previous buffer contents to be preserved. This allows
+ * bounce-buffering implementations to optimise DMA_FROM_DEVICE transfers.
+ */
+#define DMA_ATTR_OVERWRITE (1UL << 10)
/*
* A dma_addr_t can hold any valid DMA or bus address for the platform.
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -532,7 +532,8 @@ found:
*/
for (i = 0; i < nslots; i++)
io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
- if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)
+ if (!(attrs & DMA_ATTR_OVERWRITE) || dir == DMA_TO_DEVICE ||
+ dir == DMA_BIDIRECTIONAL)
swiotlb_bounce(orig_addr, tlb_addr, size, DMA_TO_DEVICE);
return tlb_addr;
From: "Jason A. Donenfeld" <[email protected]>
commit 6e8ec2552c7d13991148e551e3325a624d73fac6 upstream.
The current 4096-bit LFSR used for entropy collection had a few
desirable attributes for the context in which it was created. For
example, the state was huge, which meant that /dev/random would be able
to output quite a bit of accumulated entropy before blocking. It was
also, in its time, quite fast at accumulating entropy byte-by-byte,
which matters given the varying contexts in which mix_pool_bytes() is
called. And its diffusion was relatively high, which meant that changes
would ripple across several words of state rather quickly.
However, it also suffers from a few security vulnerabilities. In
particular, inputs learned by an attacker can be undone, but moreover,
if the state of the pool leaks, its contents can be controlled and
entirely zeroed out. I've demonstrated this attack with this SMT2
script, <https://xn--4db.cc/5o9xO8pb>, which Boolector/CaDiCal solves in
a matter of seconds on a single core of my laptop, resulting in little
proof of concept C demonstrators such as <https://xn--4db.cc/jCkvvIaH/c>.
For basically all recent formal models of RNGs, these attacks represent
a significant cryptographic flaw. But how does this manifest
practically? If an attacker has access to the system to such a degree
that he can learn the internal state of the RNG, arguably there are
other lower hanging vulnerabilities -- side-channel, infoleak, or
otherwise -- that might have higher priority. On the other hand, seed
files are frequently used on systems that have a hard time generating
much entropy on their own, and these seed files, being files, often leak
or are duplicated and distributed accidentally, or are even seeded over
the Internet intentionally, where their contents might be recorded or
tampered with. Seen this way, an otherwise quasi-implausible
vulnerability is a bit more practical than initially thought.
Another aspect of the current mix_pool_bytes() function is that, while
its performance was arguably competitive for the time in which it was
created, it's no longer considered so. This patch improves performance
significantly: on a high-end CPU, an i7-11850H, it improves performance
of mix_pool_bytes() by 225%, and on a low-end CPU, a Cortex-A7, it
improves performance by 103%.
This commit replaces the LFSR of mix_pool_bytes() with a straight-
forward cryptographic hash function, BLAKE2s, which is already in use
for pool extraction. Universal hashing with a secret seed was considered
too, something along the lines of <https://eprint.iacr.org/2013/338>,
but the requirement for a secret seed makes for a chicken & egg problem.
Instead we go with a formally proven scheme using a computational hash
function, described in sections 5.1, 6.4, and B.1.8 of
<https://eprint.iacr.org/2019/198>.
BLAKE2s outputs 256 bits, which should give us an appropriate amount of
min-entropy accumulation, and a wide enough margin of collision
resistance against active attacks. mix_pool_bytes() becomes a simple
call to blake2s_update(), for accumulation, while the extraction step
becomes a blake2s_final() to generate a seed, with which we can then do
a HKDF-like or BLAKE2X-like expansion, the first part of which we fold
back as an init key for subsequent blake2s_update()s, and the rest we
produce to the caller. This then is provided to our CRNG like usual. In
that expansion step, we make opportunistic use of 32 bytes of RDRAND
output, just as before. We also always reseed the crng with 32 bytes,
unconditionally, or not at all, rather than sometimes with 16 as before,
as we don't win anything by limiting beyond the 16 byte threshold.
Going for a hash function as an entropy collector is a conservative,
proven approach. The result of all this is a much simpler and much less
bespoke construction than what's there now, which not only plugs a
vulnerability but also improves performance considerably.
Cc: Theodore Ts'o <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Greg Kroah-Hartman <[email protected]>
Reviewed-by: Jean-Philippe Aumasson <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 304 +++++++++-----------------------------------------
1 file changed, 55 insertions(+), 249 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -42,61 +42,6 @@
*/
/*
- * (now, with legal B.S. out of the way.....)
- *
- * This routine gathers environmental noise from device drivers, etc.,
- * and returns good random numbers, suitable for cryptographic use.
- * Besides the obvious cryptographic uses, these numbers are also good
- * for seeding TCP sequence numbers, and other places where it is
- * desirable to have numbers which are not only random, but hard to
- * predict by an attacker.
- *
- * Theory of operation
- * ===================
- *
- * Computers are very predictable devices. Hence it is extremely hard
- * to produce truly random numbers on a computer --- as opposed to
- * pseudo-random numbers, which can easily generated by using a
- * algorithm. Unfortunately, it is very easy for attackers to guess
- * the sequence of pseudo-random number generators, and for some
- * applications this is not acceptable. So instead, we must try to
- * gather "environmental noise" from the computer's environment, which
- * must be hard for outside attackers to observe, and use that to
- * generate random numbers. In a Unix environment, this is best done
- * from inside the kernel.
- *
- * Sources of randomness from the environment include inter-keyboard
- * timings, inter-interrupt timings from some interrupts, and other
- * events which are both (a) non-deterministic and (b) hard for an
- * outside observer to measure. Randomness from these sources are
- * added to an "entropy pool", which is mixed using a CRC-like function.
- * This is not cryptographically strong, but it is adequate assuming
- * the randomness is not chosen maliciously, and it is fast enough that
- * the overhead of doing it on every interrupt is very reasonable.
- * As random bytes are mixed into the entropy pool, the routines keep
- * an *estimate* of how many bits of randomness have been stored into
- * the random number generator's internal state.
- *
- * When random bytes are desired, they are obtained by taking the BLAKE2s
- * hash of the contents of the "entropy pool". The BLAKE2s hash avoids
- * exposing the internal state of the entropy pool. It is believed to
- * be computationally infeasible to derive any useful information
- * about the input of BLAKE2s from its output. Even if it is possible to
- * analyze BLAKE2s in some clever way, as long as the amount of data
- * returned from the generator is less than the inherent entropy in
- * the pool, the output data is totally unpredictable. For this
- * reason, the routine decreases its internal estimate of how many
- * bits of "true randomness" are contained in the entropy pool as it
- * outputs random numbers.
- *
- * If this estimate goes to zero, the routine can still generate
- * random numbers; however, an attacker may (at least in theory) be
- * able to infer the future output of the generator from prior
- * outputs. This requires successful cryptanalysis of BLAKE2s, which is
- * not believed to be feasible, but there is a remote possibility.
- * Nonetheless, these numbers should be useful for the vast majority
- * of purposes.
- *
* Exported interfaces ---- output
* ===============================
*
@@ -298,23 +243,6 @@
*
* mknod /dev/random c 1 8
* mknod /dev/urandom c 1 9
- *
- * Acknowledgements:
- * =================
- *
- * Ideas for constructing this random number generator were derived
- * from Pretty Good Privacy's random number generator, and from private
- * discussions with Phil Karn. Colin Plumb provided a faster random
- * number generator, which speed up the mixing function of the entropy
- * pool, taken from PGPfone. Dale Worley has also contributed many
- * useful ideas and suggestions to improve this driver.
- *
- * Any flaws in the design are solely my responsibility, and should
- * not be attributed to the Phil, Colin, or any of authors of PGP.
- *
- * Further background information on this topic may be obtained from
- * RFC 1750, "Randomness Recommendations for Security", by Donald
- * Eastlake, Steve Crocker, and Jeff Schiller.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
@@ -359,79 +287,15 @@
/* #define ADD_INTERRUPT_BENCH */
-/*
- * If the entropy count falls under this number of bits, then we
- * should wake up processes which are selecting or polling on write
- * access to /dev/random.
- */
-static int random_write_wakeup_bits = 28 * (1 << 5);
-
-/*
- * Originally, we used a primitive polynomial of degree .poolwords
- * over GF(2). The taps for various sizes are defined below. They
- * were chosen to be evenly spaced except for the last tap, which is 1
- * to get the twisting happening as fast as possible.
- *
- * For the purposes of better mixing, we use the CRC-32 polynomial as
- * well to make a (modified) twisted Generalized Feedback Shift
- * Register. (See M. Matsumoto & Y. Kurita, 1992. Twisted GFSR
- * generators. ACM Transactions on Modeling and Computer Simulation
- * 2(3):179-194. Also see M. Matsumoto & Y. Kurita, 1994. Twisted
- * GFSR generators II. ACM Transactions on Modeling and Computer
- * Simulation 4:254-266)
- *
- * Thanks to Colin Plumb for suggesting this.
- *
- * The mixing operation is much less sensitive than the output hash,
- * where we use BLAKE2s. All that we want of mixing operation is that
- * it be a good non-cryptographic hash; i.e. it not produce collisions
- * when fed "random" data of the sort we expect to see. As long as
- * the pool state differs for different inputs, we have preserved the
- * input entropy and done a good job. The fact that an intelligent
- * attacker can construct inputs that will produce controlled
- * alterations to the pool's state is not important because we don't
- * consider such inputs to contribute any randomness. The only
- * property we need with respect to them is that the attacker can't
- * increase his/her knowledge of the pool's state. Since all
- * additions are reversible (knowing the final state and the input,
- * you can reconstruct the initial state), if an attacker has any
- * uncertainty about the initial state, he/she can only shuffle that
- * uncertainty about, but never cause any collisions (which would
- * decrease the uncertainty).
- *
- * Our mixing functions were analyzed by Lacharme, Roeck, Strubel, and
- * Videau in their paper, "The Linux Pseudorandom Number Generator
- * Revisited" (see: http://eprint.iacr.org/2012/251.pdf). In their
- * paper, they point out that we are not using a true Twisted GFSR,
- * since Matsumoto & Kurita used a trinomial feedback polynomial (that
- * is, with only three taps, instead of the six that we are using).
- * As a result, the resulting polynomial is neither primitive nor
- * irreducible, and hence does not have a maximal period over
- * GF(2**32). They suggest a slight change to the generator
- * polynomial which improves the resulting TGFSR polynomial to be
- * irreducible, which we have made here.
- */
enum poolinfo {
- POOL_WORDS = 128,
- POOL_WORDMASK = POOL_WORDS - 1,
- POOL_BYTES = POOL_WORDS * sizeof(u32),
- POOL_BITS = POOL_BYTES * 8,
+ POOL_BITS = BLAKE2S_HASH_SIZE * 8,
POOL_BITSHIFT = ilog2(POOL_BITS),
/* To allow fractional bits to be tracked, the entropy_count field is
* denominated in units of 1/8th bits. */
POOL_ENTROPY_SHIFT = 3,
#define POOL_ENTROPY_BITS() (input_pool.entropy_count >> POOL_ENTROPY_SHIFT)
- POOL_FRACBITS = POOL_BITS << POOL_ENTROPY_SHIFT,
-
- /* x^128 + x^104 + x^76 + x^51 +x^25 + x + 1 */
- POOL_TAP1 = 104,
- POOL_TAP2 = 76,
- POOL_TAP3 = 51,
- POOL_TAP4 = 25,
- POOL_TAP5 = 1,
-
- EXTRACT_SIZE = BLAKE2S_HASH_SIZE / 2
+ POOL_FRACBITS = POOL_BITS << POOL_ENTROPY_SHIFT
};
/*
@@ -439,6 +303,12 @@ enum poolinfo {
*/
static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
static struct fasync_struct *fasync;
+/*
+ * If the entropy count falls under this number of bits, then we
+ * should wake up processes which are selecting or polling on write
+ * access to /dev/random.
+ */
+static int random_write_wakeup_bits = POOL_BITS * 3 / 4;
static DEFINE_SPINLOCK(random_ready_list_lock);
static LIST_HEAD(random_ready_list);
@@ -494,73 +364,31 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
*
**********************************************************************/
-static u32 input_pool_data[POOL_WORDS] __latent_entropy;
-
static struct {
+ struct blake2s_state hash;
spinlock_t lock;
- u16 add_ptr;
- u16 input_rotate;
int entropy_count;
} input_pool = {
+ .hash.h = { BLAKE2S_IV0 ^ (0x01010000 | BLAKE2S_HASH_SIZE),
+ BLAKE2S_IV1, BLAKE2S_IV2, BLAKE2S_IV3, BLAKE2S_IV4,
+ BLAKE2S_IV5, BLAKE2S_IV6, BLAKE2S_IV7 },
+ .hash.outlen = BLAKE2S_HASH_SIZE,
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
};
-static ssize_t extract_entropy(void *buf, size_t nbytes, int min);
-static ssize_t _extract_entropy(void *buf, size_t nbytes);
+static bool extract_entropy(void *buf, size_t nbytes, int min);
+static void _extract_entropy(void *buf, size_t nbytes);
static void crng_reseed(struct crng_state *crng, bool use_input_pool);
-static const u32 twist_table[8] = {
- 0x00000000, 0x3b6e20c8, 0x76dc4190, 0x4db26158,
- 0xedb88320, 0xd6d6a3e8, 0x9b64c2b0, 0xa00ae278 };
-
/*
* This function adds bytes into the entropy "pool". It does not
* update the entropy estimate. The caller should call
* credit_entropy_bits if this is appropriate.
- *
- * The pool is stirred with a primitive polynomial of the appropriate
- * degree, and then twisted. We twist by three bits at a time because
- * it's cheap to do so and helps slightly in the expected case where
- * the entropy is concentrated in the low-order bits.
*/
static void _mix_pool_bytes(const void *in, int nbytes)
{
- unsigned long i;
- int input_rotate;
- const u8 *bytes = in;
- u32 w;
-
- input_rotate = input_pool.input_rotate;
- i = input_pool.add_ptr;
-
- /* mix one byte at a time to simplify size handling and churn faster */
- while (nbytes--) {
- w = rol32(*bytes++, input_rotate);
- i = (i - 1) & POOL_WORDMASK;
-
- /* XOR in the various taps */
- w ^= input_pool_data[i];
- w ^= input_pool_data[(i + POOL_TAP1) & POOL_WORDMASK];
- w ^= input_pool_data[(i + POOL_TAP2) & POOL_WORDMASK];
- w ^= input_pool_data[(i + POOL_TAP3) & POOL_WORDMASK];
- w ^= input_pool_data[(i + POOL_TAP4) & POOL_WORDMASK];
- w ^= input_pool_data[(i + POOL_TAP5) & POOL_WORDMASK];
-
- /* Mix the result back in with a twist */
- input_pool_data[i] = (w >> 3) ^ twist_table[w & 7];
-
- /*
- * Normally, we add 7 bits of rotation to the pool.
- * At the beginning of the pool, add an extra 7 bits
- * rotation, so that successive passes spread the
- * input bits across the pool evenly.
- */
- input_rotate = (input_rotate + (i ? 7 : 14)) & 31;
- }
-
- input_pool.input_rotate = input_rotate;
- input_pool.add_ptr = i;
+ blake2s_update(&input_pool.hash, in, nbytes);
}
static void __mix_pool_bytes(const void *in, int nbytes)
@@ -1004,15 +832,14 @@ static int crng_slow_load(const u8 *cp,
static void crng_reseed(struct crng_state *crng, bool use_input_pool)
{
unsigned long flags;
- int i, num;
+ int i;
union {
u8 block[CHACHA20_BLOCK_SIZE];
u32 key[8];
} buf;
if (use_input_pool) {
- num = extract_entropy(&buf, 32, 16);
- if (num == 0)
+ if (!extract_entropy(&buf, 32, 16))
return;
} else {
_extract_crng(&primary_crng, buf.block);
@@ -1380,74 +1207,48 @@ retry:
}
/*
- * This function does the actual extraction for extract_entropy.
- *
- * Note: we assume that .poolwords is a multiple of 16 words.
+ * This is an HKDF-like construction for using the hashed collected entropy
+ * as a PRF key, that's then expanded block-by-block.
*/
-static void extract_buf(u8 *out)
+static void _extract_entropy(void *buf, size_t nbytes)
{
- struct blake2s_state state __aligned(__alignof__(unsigned long));
- u8 hash[BLAKE2S_HASH_SIZE];
- unsigned long *salt;
unsigned long flags;
-
- blake2s_init(&state, sizeof(hash));
-
- /*
- * If we have an architectural hardware random number
- * generator, use it for BLAKE2's salt & personal fields.
- */
- for (salt = (unsigned long *)&state.h[4];
- salt < (unsigned long *)&state.h[8]; ++salt) {
- unsigned long v;
- if (!arch_get_random_long(&v))
- break;
- *salt ^= v;
+ u8 seed[BLAKE2S_HASH_SIZE], next_key[BLAKE2S_HASH_SIZE];
+ struct {
+ unsigned long rdrand[32 / sizeof(long)];
+ size_t counter;
+ } block;
+ size_t i;
+
+ for (i = 0; i < ARRAY_SIZE(block.rdrand); ++i) {
+ if (!arch_get_random_long(&block.rdrand[i]))
+ block.rdrand[i] = random_get_entropy();
}
- /* Generate a hash across the pool */
spin_lock_irqsave(&input_pool.lock, flags);
- blake2s_update(&state, (const u8 *)input_pool_data, POOL_BYTES);
- blake2s_final(&state, hash); /* final zeros out state */
- /*
- * We mix the hash back into the pool to prevent backtracking
- * attacks (where the attacker knows the state of the pool
- * plus the current outputs, and attempts to find previous
- * outputs), unless the hash function can be inverted. By
- * mixing at least a hash worth of hash data back, we make
- * brute-forcing the feedback as hard as brute-forcing the
- * hash.
- */
- __mix_pool_bytes(hash, sizeof(hash));
- spin_unlock_irqrestore(&input_pool.lock, flags);
+ /* seed = HASHPRF(last_key, entropy_input) */
+ blake2s_final(&input_pool.hash, seed);
- /* Note that EXTRACT_SIZE is half of hash size here, because above
- * we've dumped the full length back into mixer. By reducing the
- * amount that we emit, we retain a level of forward secrecy.
- */
- memcpy(out, hash, EXTRACT_SIZE);
- memzero_explicit(hash, sizeof(hash));
-}
+ /* next_key = HASHPRF(seed, RDRAND || 0) */
+ block.counter = 0;
+ blake2s(next_key, (u8 *)&block, seed, sizeof(next_key), sizeof(block), sizeof(seed));
+ blake2s_init_key(&input_pool.hash, BLAKE2S_HASH_SIZE, next_key, sizeof(next_key));
-static ssize_t _extract_entropy(void *buf, size_t nbytes)
-{
- ssize_t ret = 0, i;
- u8 tmp[EXTRACT_SIZE];
+ spin_unlock_irqrestore(&input_pool.lock, flags);
+ memzero_explicit(next_key, sizeof(next_key));
while (nbytes) {
- extract_buf(tmp);
- i = min_t(int, nbytes, EXTRACT_SIZE);
- memcpy(buf, tmp, i);
+ i = min_t(size_t, nbytes, BLAKE2S_HASH_SIZE);
+ /* output = HASHPRF(seed, RDRAND || ++counter) */
+ ++block.counter;
+ blake2s(buf, (u8 *)&block, seed, i, sizeof(block), sizeof(seed));
nbytes -= i;
buf += i;
- ret += i;
}
- /* Wipe data just returned from memory */
- memzero_explicit(tmp, sizeof(tmp));
-
- return ret;
+ memzero_explicit(seed, sizeof(seed));
+ memzero_explicit(&block, sizeof(block));
}
/*
@@ -1455,13 +1256,18 @@ static ssize_t _extract_entropy(void *bu
* returns it in a buffer.
*
* The min parameter specifies the minimum amount we can pull before
- * failing to avoid races that defeat catastrophic reseeding.
+ * failing to avoid races that defeat catastrophic reseeding. If we
+ * have less than min entropy available, we return false and buf is
+ * not filled.
*/
-static ssize_t extract_entropy(void *buf, size_t nbytes, int min)
+static bool extract_entropy(void *buf, size_t nbytes, int min)
{
trace_extract_entropy(nbytes, POOL_ENTROPY_BITS(), _RET_IP_);
- nbytes = account(nbytes, min);
- return _extract_entropy(buf, nbytes);
+ if (account(nbytes, min)) {
+ _extract_entropy(buf, nbytes);
+ return true;
+ }
+ return false;
}
#define warn_unseeded_randomness(previous) \
@@ -1725,7 +1531,7 @@ static void __init init_std_data(void)
unsigned long rv;
mix_pool_bytes(&now, sizeof(now));
- for (i = POOL_BYTES; i > 0; i -= sizeof(rv)) {
+ for (i = BLAKE2S_BLOCK_SIZE; i > 0; i -= sizeof(rv)) {
if (!arch_get_random_seed_long(&rv) &&
!arch_get_random_long(&rv))
rv = random_get_entropy();
From: "Stephan M?ller" <[email protected]>
commit 819966c06b759022e9932f328284314d9272b9f3 upstream.
The Jitter RNG is unconditionally allocated as a seed source follwoing
the patch 97f2650e5040. Thus, the instance must always be deallocated.
Reported-by: [email protected]
Fixes: 97f2650e5040 ("crypto: drbg - always seeded with SP800-90B ...")
Signed-off-by: Stephan Mueller <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1646,10 +1646,12 @@ static int drbg_uninstantiate(struct drb
if (drbg->random_ready.notifier_call) {
unregister_random_ready_notifier(&drbg->random_ready);
cancel_work_sync(&drbg->seed_work);
- crypto_free_rng(drbg->jent);
- drbg->jent = NULL;
}
+ if (!IS_ERR_OR_NULL(drbg->jent))
+ crypto_free_rng(drbg->jent);
+ drbg->jent = NULL;
+
if (drbg->d_ops)
drbg->d_ops->crypto_fini(drbg);
drbg_dealloc_state(drbg);
From: Eric Dumazet <[email protected]>
commit c579bd1b4021c42ae247108f1e6f73dd3f08600c upstream.
Even when implementing RFC 6056 3.3.4 (Algorithm 4: Double-Hash
Port Selection Algorithm), a patient attacker could still be able
to collect enough state from an otherwise idle host.
Idea of this patch is to inject some noise, in the
cases __inet_hash_connect() found a candidate in the first
attempt.
This noise should not significantly reduce the collision
avoidance, and should be zero if connection table
is already well used.
Note that this is not implementing RFC 6056 3.3.5
because we think Algorithm 5 could hurt typical
workloads.
Signed-off-by: Eric Dumazet <[email protected]>
Cc: David Dworken <[email protected]>
Cc: Willem de Bruijn <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 5 +++++
1 file changed, 5 insertions(+)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -642,6 +642,11 @@ next_port:
return -EADDRNOTAVAIL;
ok:
+ /* If our first attempt found a candidate, skip next candidate
+ * in 1/16 of cases to add some noise.
+ */
+ if (!i && !(prandom_u32() % 16))
+ i = 2;
WRITE_ONCE(table_perturb[index], READ_ONCE(table_perturb[index]) + i + 2);
/* Head lock still held and bh's disabled */
From: Nicolai Stange <[email protected]>
commit 074bcd4000e0d812bc253f86fedc40f81ed59ccc upstream.
get_random_bytes() usually hasn't full entropy available by the time DRBG
instances are first getting seeded from it during boot. Thus, the DRBG
implementation registers random_ready_callbacks which would in turn
schedule some work for reseeding the DRBGs once get_random_bytes() has
sufficient entropy available.
For reference, the relevant history around handling DRBG (re)seeding in
the context of a not yet fully seeded get_random_bytes() is:
commit 16b369a91d0d ("random: Blocking API for accessing
nonblocking_pool")
commit 4c7879907edd ("crypto: drbg - add async seeding operation")
commit 205a525c3342 ("random: Add callback API for random pool
readiness")
commit 57225e679788 ("crypto: drbg - Use callback API for random
readiness")
commit c2719503f5e1 ("random: Remove kernel blocking API")
However, some time later, the initialization state of get_random_bytes()
has been made queryable via rng_is_initialized() introduced with commit
9a47249d444d ("random: Make crng state queryable"). This primitive now
allows for streamlining the DRBG reseeding from get_random_bytes() by
replacing that aforementioned asynchronous work scheduling from
random_ready_callbacks with some simpler, synchronous code in
drbg_generate() next to the related logic already present therein. Apart
from improving overall code readability, this change will also enable DRBG
users to rely on wait_for_random_bytes() for ensuring that the initial
seeding has completed, if desired.
The previous patches already laid the grounds by making drbg_seed() to
record at each DRBG instance whether it was being seeded at a time when
rng_is_initialized() still had been false as indicated by
->seeded == DRBG_SEED_STATE_PARTIAL.
All that remains to be done now is to make drbg_generate() check for this
condition, determine whether rng_is_initialized() has flipped to true in
the meanwhile and invoke a reseed from get_random_bytes() if so.
Make this move:
- rename the former drbg_async_seed() work handler, i.e. the one in charge
of reseeding a DRBG instance from get_random_bytes(), to
"drbg_seed_from_random()",
- change its signature as appropriate, i.e. make it take a struct
drbg_state rather than a work_struct and change its return type from
"void" to "int" in order to allow for passing error information from
e.g. its __drbg_seed() invocation onwards to callers,
- make drbg_generate() invoke this drbg_seed_from_random() once it
encounters a DRBG instance with ->seeded == DRBG_SEED_STATE_PARTIAL by
the time rng_is_initialized() has flipped to true and
- prune everything related to the former, random_ready_callback based
mechanism.
As drbg_seed_from_random() is now getting invoked from drbg_generate() with
the ->drbg_mutex being held, it must not attempt to recursively grab it
once again. Remove the corresponding mutex operations from what is now
drbg_seed_from_random(). Furthermore, as drbg_seed_from_random() can now
report errors directly to its caller, there's no need for it to temporarily
switch the DRBG's ->seeded state to DRBG_SEED_STATE_UNSEEDED so that a
failure of the subsequently invoked __drbg_seed() will get signaled to
drbg_generate(). Don't do it then.
Signed-off-by: Nicolai Stange <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
[Jason: for stable, undid the modifications for the backport of 5acd3548.]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 61 +++++++++-----------------------------------------
drivers/char/random.c | 2 -
include/crypto/drbg.h | 2 -
3 files changed, 11 insertions(+), 54 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1087,12 +1087,10 @@ static inline int drbg_get_random_bytes(
return 0;
}
-static void drbg_async_seed(struct work_struct *work)
+static int drbg_seed_from_random(struct drbg_state *drbg)
{
struct drbg_string data;
LIST_HEAD(seedlist);
- struct drbg_state *drbg = container_of(work, struct drbg_state,
- seed_work);
unsigned int entropylen = drbg_sec_strength(drbg->core->flags);
unsigned char entropy[32];
int ret;
@@ -1103,23 +1101,15 @@ static void drbg_async_seed(struct work_
drbg_string_fill(&data, entropy, entropylen);
list_add_tail(&data.list, &seedlist);
- mutex_lock(&drbg->drbg_mutex);
-
ret = drbg_get_random_bytes(drbg, entropy, entropylen);
if (ret)
- goto unlock;
-
- /* Reset ->seeded so that if __drbg_seed fails the next
- * generate call will trigger a reseed.
- */
- drbg->seeded = DRBG_SEED_STATE_UNSEEDED;
+ goto out;
- __drbg_seed(drbg, &seedlist, true, DRBG_SEED_STATE_FULL);
-
-unlock:
- mutex_unlock(&drbg->drbg_mutex);
+ ret = __drbg_seed(drbg, &seedlist, true, DRBG_SEED_STATE_FULL);
+out:
memzero_explicit(entropy, entropylen);
+ return ret;
}
/*
@@ -1420,6 +1410,11 @@ static int drbg_generate(struct drbg_sta
goto err;
/* 9.3.1 step 7.4 */
addtl = NULL;
+ } else if (rng_is_initialized() &&
+ drbg->seeded == DRBG_SEED_STATE_PARTIAL) {
+ len = drbg_seed_from_random(drbg);
+ if (len)
+ goto err;
}
if (addtl && 0 < addtl->len)
@@ -1512,44 +1507,15 @@ static int drbg_generate_long(struct drb
return 0;
}
-static int drbg_schedule_async_seed(struct notifier_block *nb, unsigned long action, void *data)
-{
- struct drbg_state *drbg = container_of(nb, struct drbg_state,
- random_ready);
-
- schedule_work(&drbg->seed_work);
- return 0;
-}
-
static int drbg_prepare_hrng(struct drbg_state *drbg)
{
- int err;
-
/* We do not need an HRNG in test mode. */
if (list_empty(&drbg->test_data.list))
return 0;
drbg->jent = crypto_alloc_rng("jitterentropy_rng", 0, 0);
- INIT_WORK(&drbg->seed_work, drbg_async_seed);
-
- drbg->random_ready.notifier_call = drbg_schedule_async_seed;
- err = register_random_ready_notifier(&drbg->random_ready);
-
- switch (err) {
- case 0:
- break;
-
- case -EALREADY:
- err = 0;
- /* fall through */
-
- default:
- drbg->random_ready.notifier_call = NULL;
- return err;
- }
-
- return err;
+ return 0;
}
/*
@@ -1643,11 +1609,6 @@ free_everything:
*/
static int drbg_uninstantiate(struct drbg_state *drbg)
{
- if (drbg->random_ready.notifier_call) {
- unregister_random_ready_notifier(&drbg->random_ready);
- cancel_work_sync(&drbg->seed_work);
- }
-
if (!IS_ERR_OR_NULL(drbg->jent))
crypto_free_rng(drbg->jent);
drbg->jent = NULL;
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -158,7 +158,6 @@ int __cold register_random_ready_notifie
spin_unlock_irqrestore(&random_ready_chain_lock, flags);
return ret;
}
-EXPORT_SYMBOL(register_random_ready_notifier);
/*
* Delete a previously registered readiness callback function.
@@ -173,7 +172,6 @@ int __cold unregister_random_ready_notif
spin_unlock_irqrestore(&random_ready_chain_lock, flags);
return ret;
}
-EXPORT_SYMBOL(unregister_random_ready_notifier);
static void __cold process_random_ready_list(void)
{
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -139,12 +139,10 @@ struct drbg_state {
bool pr; /* Prediction resistance enabled? */
bool fips_primed; /* Continuous test primed? */
unsigned char *prev; /* FIPS 140-2 continuous test value */
- struct work_struct seed_work; /* asynchronous seeding support */
struct crypto_rng *jent;
const struct drbg_state_ops *d_ops;
const struct drbg_core *core;
struct drbg_string test_data;
- struct notifier_block random_ready;
};
static inline __u8 drbg_statelen(struct drbg_state *drbg)
From: "Jason A. Donenfeld" <[email protected]>
commit 9c3ddde3f811aabbb83778a2a615bf141b4909ef upstream.
If the bootloader supplies sufficient material and crng_reseed() is called
very early on, but not too early that wqs aren't available yet, then we
might transition to crng_init==2 before rand_initialize()'s call to
crng_initialize_primary() made. Then, when crng_initialize_primary() is
called, if we're trusting the CPU's RDRAND instructions, we'll
needlessly reinitialize the RNG and emit a message about it. This is
mostly harmless, as numa_crng_init() will allocate and then free what it
just allocated, and excessive calls to invalidate_batched_entropy()
aren't so harmful. But it is funky and the extra message is confusing,
so avoid the re-initialization all together by checking for crng_init <
2 in crng_initialize_primary(), just as we already do in crng_reseed().
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -829,7 +829,7 @@ static void __init crng_initialize_prima
{
memcpy(&crng->state[0], "expand 32-byte k", 16);
_extract_entropy(&input_pool, &crng->state[4], sizeof(__u32) * 12, 0);
- if (crng_init_try_arch_early(crng) && trust_cpu) {
+ if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
crng_init = 2;
From: "Jason A. Donenfeld" <[email protected]>
commit 6071a6c0fba2d747742cadcbb3ba26ed756ed73b upstream.
This really adds nothing at all useful.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/random.h | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -1,8 +1,5 @@
-/*
- * include/linux/random.h
- *
- * Include file for the random number generator.
- */
+/* SPDX-License-Identifier: GPL-2.0 */
+
#ifndef _LINUX_RANDOM_H
#define _LINUX_RANDOM_H
From: Vasily Gorbik <[email protected]>
commit 7e756f423af808b6571fed3144747db2ef7fa1c5 upstream.
Some architectures set up CFLAGS for linux decompressor phase from
scratch and do not include GCC_PLUGINS_CFLAGS. Since "latent_entropy"
variable declaration is generated by the plugin code itself including
linux/random.h in decompressor code then would cause a build
error. E.g. on s390:
In file included from ./include/linux/net.h:22,
from ./include/linux/skbuff.h:29,
from ./include/linux/if_ether.h:23,
from ./arch/s390/include/asm/diag.h:12,
from arch/s390/boot/startup.c:8:
./include/linux/random.h: In function 'add_latent_entropy':
./include/linux/random.h:26:39: error: 'latent_entropy' undeclared
(first use in this function); did you mean 'add_latent_entropy'?
26 | add_device_randomness((const void *)&latent_entropy,
| ^~~~~~~~~~~~~~
| add_latent_entropy
./include/linux/random.h:26:39: note: each undeclared identifier is
reported only once for each function it appears in
The build error is triggered by commit a80313ff91ab ("s390/kernel:
introduce .dma sections") which made it into 5.2 merge window.
To address that avoid using CONFIG_GCC_PLUGIN_LATENT_ENTROPY in
favour of LATENT_ENTROPY_PLUGIN definition which is defined as a
part of gcc plugins cflags and hence reflect more accurately when gcc
plugin is active. Besides that it is also used for similar purpose in
linux/compiler-gcc.h for latent_entropy attribute definition.
Signed-off-by: Vasily Gorbik <[email protected]>
Acked-by: Kees Cook <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/random.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -19,7 +19,7 @@ struct random_ready_callback {
extern void add_device_randomness(const void *, unsigned int);
-#if defined(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) && !defined(__CHECKER__)
+#if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__)
static inline void add_latent_entropy(void)
{
add_device_randomness((const void *)&latent_entropy,
From: Eric Biggers <[email protected]>
commit bbda6e0f1303953c855ee3669655a81b69fbe899 upstream.
Address the following checkpatch warning:
WARNING: Use #include <linux/bug.h> instead of <asm/bug.h>
Signed-off-by: Eric Biggers <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/crypto/blake2s.h | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
--- a/include/crypto/blake2s.h
+++ b/include/crypto/blake2s.h
@@ -6,12 +6,11 @@
#ifndef BLAKE2S_H
#define BLAKE2S_H
+#include <linux/bug.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
-#include <asm/bug.h>
-
enum blake2s_lengths {
BLAKE2S_BLOCK_SIZE = 64,
BLAKE2S_HASH_SIZE = 32,
From: Andy Lutomirski <[email protected]>
commit 30c08efec8884fb106b8e57094baa51bb4c44e32 upstream.
This patch changes the read semantics of /dev/random to be the same
as /dev/urandom except that reads will block until the CRNG is
ready.
None of the cleanups that this enables have been done yet. As a
result, this gives a warning about an unused function.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/5e6ac8831c6cf2e56a7a4b39616d1732b2bdd06c.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 54 ++++++++++++--------------------------------------
1 file changed, 13 insertions(+), 41 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -355,7 +355,6 @@
#define INPUT_POOL_WORDS (1 << (INPUT_POOL_SHIFT-5))
#define OUTPUT_POOL_SHIFT 10
#define OUTPUT_POOL_WORDS (1 << (OUTPUT_POOL_SHIFT-5))
-#define SEC_XFER_SIZE 512
#define EXTRACT_SIZE 10
@@ -805,7 +804,6 @@ retry:
if (entropy_bits >= random_read_wakeup_bits &&
wq_has_sleeper(&random_read_wait)) {
wake_up_interruptible(&random_read_wait);
- kill_fasync(&fasync, SIGIO, POLL_IN);
}
/* If the input pool is getting full, and the blocking
* pool has room, send some entropy to the blocking
@@ -1978,43 +1976,6 @@ void rand_initialize_disk(struct gendisk
#endif
static ssize_t
-_random_read(int nonblock, char __user *buf, size_t nbytes)
-{
- ssize_t n;
-
- if (nbytes == 0)
- return 0;
-
- nbytes = min_t(size_t, nbytes, SEC_XFER_SIZE);
- while (1) {
- n = extract_entropy_user(&blocking_pool, buf, nbytes);
- if (n < 0)
- return n;
- trace_random_read(n*8, (nbytes-n)*8,
- ENTROPY_BITS(&blocking_pool),
- ENTROPY_BITS(&input_pool));
- if (n > 0)
- return n;
-
- /* Pool is (near) empty. Maybe wait and retry. */
- if (nonblock)
- return -EAGAIN;
-
- wait_event_interruptible(random_read_wait,
- blocking_pool.initialized &&
- (ENTROPY_BITS(&input_pool) >= random_read_wakeup_bits));
- if (signal_pending(current))
- return -ERESTARTSYS;
- }
-}
-
-static ssize_t
-random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
-{
- return _random_read(file->f_flags & O_NONBLOCK, buf, nbytes);
-}
-
-static ssize_t
urandom_read_nowarn(struct file *file, char __user *buf, size_t nbytes,
loff_t *ppos)
{
@@ -2046,15 +2007,26 @@ urandom_read(struct file *file, char __u
return urandom_read_nowarn(file, buf, nbytes, ppos);
}
+static ssize_t
+random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
+{
+ int ret;
+
+ ret = wait_for_random_bytes();
+ if (ret != 0)
+ return ret;
+ return urandom_read_nowarn(file, buf, nbytes, ppos);
+}
+
static unsigned int
random_poll(struct file *file, poll_table * wait)
{
unsigned int mask;
- poll_wait(file, &random_read_wait, wait);
+ poll_wait(file, &crng_init_wait, wait);
poll_wait(file, &random_write_wait, wait);
mask = 0;
- if (ENTROPY_BITS(&input_pool) >= random_read_wakeup_bits)
+ if (crng_ready())
mask |= POLLIN | POLLRDNORM;
if (ENTROPY_BITS(&input_pool) < random_write_wakeup_bits)
mask |= POLLOUT | POLLWRNORM;
From: "Jason A. Donenfeld" <[email protected]>
commit 5f1bb112006b104b3e2a1e1b39bbb9b2617581e6 upstream.
This pulls all of the readiness waiting-focused functions into the first
labeled section.
No functional changes.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 1066 ++++++++++++++++++++++++--------------------------
1 file changed, 527 insertions(+), 539 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -202,126 +202,144 @@
#include <asm/irq_regs.h>
#include <asm/io.h>
-enum {
- POOL_BITS = BLAKE2S_HASH_SIZE * 8,
- POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
-};
-
-/*
- * Static global variables
- */
-static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
-static struct fasync_struct *fasync;
-
-static DEFINE_SPINLOCK(random_ready_list_lock);
-static LIST_HEAD(random_ready_list);
+/*********************************************************************
+ *
+ * Initialization and readiness waiting.
+ *
+ * Much of the RNG infrastructure is devoted to various dependencies
+ * being able to wait until the RNG has collected enough entropy and
+ * is ready for safe consumption.
+ *
+ *********************************************************************/
/*
* crng_init = 0 --> Uninitialized
* 1 --> Initialized
* 2 --> Initialized from input_pool
*
- * crng_init is protected by primary_crng->lock, and only increases
+ * crng_init is protected by base_crng->lock, and only increases
* its value (from 0->1->2).
*/
static int crng_init = 0;
#define crng_ready() (likely(crng_init > 1))
-static int crng_init_cnt = 0;
-static void process_random_ready_list(void);
-static void _get_random_bytes(void *buf, size_t nbytes);
+/* Various types of waiters for crng_init->2 transition. */
+static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
+static struct fasync_struct *fasync;
+static DEFINE_SPINLOCK(random_ready_list_lock);
+static LIST_HEAD(random_ready_list);
+/* Control how we warn userspace. */
static struct ratelimit_state unseeded_warning =
RATELIMIT_STATE_INIT("warn_unseeded_randomness", HZ, 3);
static struct ratelimit_state urandom_warning =
RATELIMIT_STATE_INIT("warn_urandom_randomness", HZ, 3);
-
static int ratelimit_disable __read_mostly;
-
module_param_named(ratelimit_disable, ratelimit_disable, int, 0644);
MODULE_PARM_DESC(ratelimit_disable, "Disable random ratelimit suppression");
-/**********************************************************************
- *
- * OS independent entropy store. Here are the functions which handle
- * storing entropy in an entropy pool.
+/*
+ * Returns whether or not the input pool has been seeded and thus guaranteed
+ * to supply cryptographically secure random numbers. This applies to: the
+ * /dev/urandom device, the get_random_bytes function, and the get_random_{u32,
+ * ,u64,int,long} family of functions.
*
- **********************************************************************/
-
-static struct {
- struct blake2s_state hash;
- spinlock_t lock;
- unsigned int entropy_count;
-} input_pool = {
- .hash.h = { BLAKE2S_IV0 ^ (0x01010000 | BLAKE2S_HASH_SIZE),
- BLAKE2S_IV1, BLAKE2S_IV2, BLAKE2S_IV3, BLAKE2S_IV4,
- BLAKE2S_IV5, BLAKE2S_IV6, BLAKE2S_IV7 },
- .hash.outlen = BLAKE2S_HASH_SIZE,
- .lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
-};
-
-static void extract_entropy(void *buf, size_t nbytes);
-static bool drain_entropy(void *buf, size_t nbytes);
+ * Returns: true if the input pool has been seeded.
+ * false if the input pool has not been seeded.
+ */
+bool rng_is_initialized(void)
+{
+ return crng_ready();
+}
+EXPORT_SYMBOL(rng_is_initialized);
-static void crng_reseed(void);
+/* Used by wait_for_random_bytes(), and considered an entropy collector, below. */
+static void try_to_generate_entropy(void);
/*
- * This function adds bytes into the entropy "pool". It does not
- * update the entropy estimate. The caller should call
- * credit_entropy_bits if this is appropriate.
+ * Wait for the input pool to be seeded and thus guaranteed to supply
+ * cryptographically secure random numbers. This applies to: the /dev/urandom
+ * device, the get_random_bytes function, and the get_random_{u32,u64,int,long}
+ * family of functions. Using any of these functions without first calling
+ * this function forfeits the guarantee of security.
+ *
+ * Returns: 0 if the input pool has been seeded.
+ * -ERESTARTSYS if the function was interrupted by a signal.
*/
-static void _mix_pool_bytes(const void *in, size_t nbytes)
+int wait_for_random_bytes(void)
{
- blake2s_update(&input_pool.hash, in, nbytes);
-}
+ if (likely(crng_ready()))
+ return 0;
-static void mix_pool_bytes(const void *in, size_t nbytes)
-{
- unsigned long flags;
+ do {
+ int ret;
+ ret = wait_event_interruptible_timeout(crng_init_wait, crng_ready(), HZ);
+ if (ret)
+ return ret > 0 ? 0 : ret;
- spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(in, nbytes);
- spin_unlock_irqrestore(&input_pool.lock, flags);
-}
+ try_to_generate_entropy();
+ } while (!crng_ready());
-struct fast_pool {
- union {
- u32 pool32[4];
- u64 pool64[2];
- };
- unsigned long last;
- u16 reg_idx;
- u8 count;
-};
+ return 0;
+}
+EXPORT_SYMBOL(wait_for_random_bytes);
/*
- * This is a fast mixing routine used by the interrupt randomness
- * collector. It's hardcoded for an 128 bit pool and assumes that any
- * locks that might be needed are taken by the caller.
+ * Add a callback function that will be invoked when the input
+ * pool is initialised.
+ *
+ * returns: 0 if callback is successfully added
+ * -EALREADY if pool is already initialised (callback not called)
+ * -ENOENT if module for callback is not alive
*/
-static void fast_mix(u32 pool[4])
+int add_random_ready_callback(struct random_ready_callback *rdy)
{
- u32 a = pool[0], b = pool[1];
- u32 c = pool[2], d = pool[3];
+ struct module *owner;
+ unsigned long flags;
+ int err = -EALREADY;
- a += b; c += d;
- b = rol32(b, 6); d = rol32(d, 27);
- d ^= a; b ^= c;
+ if (crng_ready())
+ return err;
- a += b; c += d;
- b = rol32(b, 16); d = rol32(d, 14);
- d ^= a; b ^= c;
+ owner = rdy->owner;
+ if (!try_module_get(owner))
+ return -ENOENT;
- a += b; c += d;
- b = rol32(b, 6); d = rol32(d, 27);
- d ^= a; b ^= c;
+ spin_lock_irqsave(&random_ready_list_lock, flags);
+ if (crng_ready())
+ goto out;
- a += b; c += d;
- b = rol32(b, 16); d = rol32(d, 14);
- d ^= a; b ^= c;
+ owner = NULL;
- pool[0] = a; pool[1] = b;
- pool[2] = c; pool[3] = d;
+ list_add(&rdy->list, &random_ready_list);
+ err = 0;
+
+out:
+ spin_unlock_irqrestore(&random_ready_list_lock, flags);
+
+ module_put(owner);
+
+ return err;
}
+EXPORT_SYMBOL(add_random_ready_callback);
+
+/*
+ * Delete a previously registered readiness callback function.
+ */
+void del_random_ready_callback(struct random_ready_callback *rdy)
+{
+ unsigned long flags;
+ struct module *owner = NULL;
+
+ spin_lock_irqsave(&random_ready_list_lock, flags);
+ if (!list_empty(&rdy->list)) {
+ list_del_init(&rdy->list);
+ owner = rdy->owner;
+ }
+ spin_unlock_irqrestore(&random_ready_list_lock, flags);
+
+ module_put(owner);
+}
+EXPORT_SYMBOL(del_random_ready_callback);
static void process_random_ready_list(void)
{
@@ -339,27 +357,51 @@ static void process_random_ready_list(vo
spin_unlock_irqrestore(&random_ready_list_lock, flags);
}
-static void credit_entropy_bits(size_t nbits)
+#define warn_unseeded_randomness(previous) \
+ _warn_unseeded_randomness(__func__, (void *)_RET_IP_, (previous))
+
+static void _warn_unseeded_randomness(const char *func_name, void *caller, void **previous)
{
- unsigned int entropy_count, orig, add;
+#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM
+ const bool print_once = false;
+#else
+ static bool print_once __read_mostly;
+#endif
- if (!nbits)
+ if (print_once || crng_ready() ||
+ (previous && (caller == READ_ONCE(*previous))))
return;
-
- add = min_t(size_t, nbits, POOL_BITS);
-
- do {
- orig = READ_ONCE(input_pool.entropy_count);
- entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
- } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
-
- if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
- crng_reseed();
+ WRITE_ONCE(*previous, caller);
+#ifndef CONFIG_WARN_ALL_UNSEEDED_RANDOM
+ print_once = true;
+#endif
+ if (__ratelimit(&unseeded_warning))
+ printk_deferred(KERN_NOTICE "random: %s called from %pS with crng_init=%d\n",
+ func_name, caller, crng_init);
}
+
/*********************************************************************
*
- * CRNG using CHACHA20
+ * Fast key erasure RNG, the "crng".
+ *
+ * These functions expand entropy from the entropy extractor into
+ * long streams for external consumption using the "fast key erasure"
+ * RNG described at <https://blog.cr.yp.to/20170723-random.html>.
+ *
+ * There are a few exported interfaces for use by other drivers:
+ *
+ * void get_random_bytes(void *buf, size_t nbytes)
+ * u32 get_random_u32()
+ * u64 get_random_u64()
+ * unsigned int get_random_int()
+ * unsigned long get_random_long()
+ *
+ * These interfaces will return the requested number of random bytes
+ * into the given buffer or as a return value. This is equivalent to
+ * a read from /dev/urandom. The integer family of functions may be
+ * higher performance for one-off random integers, because they do a
+ * bit of buffering.
*
*********************************************************************/
@@ -386,123 +428,14 @@ static DEFINE_PER_CPU(struct crng, crngs
.generation = ULONG_MAX
};
-static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
-
-/*
- * crng_fast_load() can be called by code in the interrupt service
- * path. So we can't afford to dilly-dally. Returns the number of
- * bytes processed from cp.
- */
-static size_t crng_fast_load(const void *cp, size_t len)
-{
- unsigned long flags;
- const u8 *src = (const u8 *)cp;
- size_t ret = 0;
-
- if (!spin_trylock_irqsave(&base_crng.lock, flags))
- return 0;
- if (crng_init != 0) {
- spin_unlock_irqrestore(&base_crng.lock, flags);
- return 0;
- }
- while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
- base_crng.key[crng_init_cnt % sizeof(base_crng.key)] ^= *src;
- src++; crng_init_cnt++; len--; ret++;
- }
- if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
- ++base_crng.generation;
- crng_init = 1;
- }
- spin_unlock_irqrestore(&base_crng.lock, flags);
- if (crng_init == 1)
- pr_notice("fast init done\n");
- return ret;
-}
-
-#ifdef CONFIG_NUMA
-static void do_numa_crng_init(struct work_struct *work)
-{
- int i;
- struct crng_state *crng;
- struct crng_state **pool;
-
- pool = kcalloc(nr_node_ids, sizeof(*pool), GFP_KERNEL|__GFP_NOFAIL);
- for_each_online_node(i) {
- crng = kmalloc_node(sizeof(struct crng_state),
- GFP_KERNEL | __GFP_NOFAIL, i);
- spin_lock_init(&crng->lock);
- crng_initialize(crng);
- pool[i] = crng;
- }
- /* pairs with READ_ONCE() in select_crng() */
- if (cmpxchg_release(&crng_node_pool, NULL, pool) != NULL) {
- for_each_node(i)
- kfree(pool[i]);
- kfree(pool);
- }
-}
-
-static DECLARE_WORK(numa_crng_init_work, do_numa_crng_init);
-
-static void numa_crng_init(void)
-{
- schedule_work(&numa_crng_init_work);
-}
-
-static struct crng_state *select_crng(void)
-{
- struct crng_state **pool;
- int nid = numa_node_id();
-
- /* pairs with cmpxchg_release() in do_numa_crng_init() */
- pool = READ_ONCE(crng_node_pool);
- if (pool && pool[nid])
- return pool[nid];
-
- return &primary_crng;
-}
-#else
-static void numa_crng_init(void) {}
-
-static struct crng_state *select_crng(void)
-{
- return &primary_crng;
-}
-#endif
+/* Used by crng_reseed() to extract a new seed from the input pool. */
+static bool drain_entropy(void *buf, size_t nbytes);
/*
- * crng_slow_load() is called by add_device_randomness, which has two
- * attributes. (1) We can't trust the buffer passed to it is
- * guaranteed to be unpredictable (so it might not have any entropy at
- * all), and (2) it doesn't have the performance constraints of
- * crng_fast_load().
- *
- * So, we simply hash the contents in with the current key. Finally,
- * we do *not* advance crng_init_cnt since buffer we may get may be
- * something like a fixed DMI table (for example), which might very
- * well be unique to the machine, but is otherwise unvarying.
+ * This extracts a new crng key from the input pool, but only if there is a
+ * sufficient amount of entropy available, in order to mitigate bruteforcing
+ * of newly added bits.
*/
-static void crng_slow_load(const void *cp, size_t len)
-{
- unsigned long flags;
- struct blake2s_state hash;
-
- blake2s_init(&hash, sizeof(base_crng.key));
-
- if (!spin_trylock_irqsave(&base_crng.lock, flags))
- return;
- if (crng_init != 0) {
- spin_unlock_irqrestore(&base_crng.lock, flags);
- return;
- }
-
- blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
- blake2s_update(&hash, cp, len);
- blake2s_final(&hash, base_crng.key);
-
- spin_unlock_irqrestore(&base_crng.lock, flags);
-}
-
static void crng_reseed(void)
{
unsigned long flags;
@@ -552,13 +485,11 @@ static void crng_reseed(void)
}
/*
- * The general form here is based on a "fast key erasure RNG" from
- * <https://blog.cr.yp.to/20170723-random.html>. It generates a ChaCha
- * block using the provided key, and then immediately overwites that
- * key with half the block. It returns the resultant ChaCha state to the
- * user, along with the second half of the block containing 32 bytes of
- * random data that may be used; random_data_len may not be greater than
- * 32.
+ * This generates a ChaCha block using the provided key, and then
+ * immediately overwites that key with half the block. It returns
+ * the resultant ChaCha state to the user, along with the second
+ * half of the block containing 32 bytes of random data that may
+ * be used; random_data_len may not be greater than 32.
*/
static void crng_fast_key_erasure(u8 key[CHACHA20_KEY_SIZE],
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)],
@@ -645,6 +576,126 @@ static void crng_make_state(u32 chacha_s
local_irq_restore(flags);
}
+/*
+ * This function is for crng_init == 0 only.
+ *
+ * crng_fast_load() can be called by code in the interrupt service
+ * path. So we can't afford to dilly-dally. Returns the number of
+ * bytes processed from cp.
+ */
+static size_t crng_fast_load(const void *cp, size_t len)
+{
+ static int crng_init_cnt = 0;
+ unsigned long flags;
+ const u8 *src = (const u8 *)cp;
+ size_t ret = 0;
+
+ if (!spin_trylock_irqsave(&base_crng.lock, flags))
+ return 0;
+ if (crng_init != 0) {
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ return 0;
+ }
+ while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
+ base_crng.key[crng_init_cnt % sizeof(base_crng.key)] ^= *src;
+ src++; crng_init_cnt++; len--; ret++;
+ }
+ if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
+ ++base_crng.generation;
+ crng_init = 1;
+ }
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ if (crng_init == 1)
+ pr_notice("fast init done\n");
+ return ret;
+}
+
+/*
+ * This function is for crng_init == 0 only.
+ *
+ * crng_slow_load() is called by add_device_randomness, which has two
+ * attributes. (1) We can't trust the buffer passed to it is
+ * guaranteed to be unpredictable (so it might not have any entropy at
+ * all), and (2) it doesn't have the performance constraints of
+ * crng_fast_load().
+ *
+ * So, we simply hash the contents in with the current key. Finally,
+ * we do *not* advance crng_init_cnt since buffer we may get may be
+ * something like a fixed DMI table (for example), which might very
+ * well be unique to the machine, but is otherwise unvarying.
+ */
+static void crng_slow_load(const void *cp, size_t len)
+{
+ unsigned long flags;
+ struct blake2s_state hash;
+
+ blake2s_init(&hash, sizeof(base_crng.key));
+
+ if (!spin_trylock_irqsave(&base_crng.lock, flags))
+ return;
+ if (crng_init != 0) {
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ return;
+ }
+
+ blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
+ blake2s_update(&hash, cp, len);
+ blake2s_final(&hash, base_crng.key);
+
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+}
+
+static void _get_random_bytes(void *buf, size_t nbytes)
+{
+ u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
+ u8 tmp[CHACHA20_BLOCK_SIZE];
+ size_t len;
+
+ if (!nbytes)
+ return;
+
+ len = min_t(size_t, 32, nbytes);
+ crng_make_state(chacha_state, buf, len);
+ nbytes -= len;
+ buf += len;
+
+ while (nbytes) {
+ if (nbytes < CHACHA20_BLOCK_SIZE) {
+ chacha20_block(chacha_state, tmp);
+ memcpy(buf, tmp, nbytes);
+ memzero_explicit(tmp, sizeof(tmp));
+ break;
+ }
+
+ chacha20_block(chacha_state, buf);
+ if (unlikely(chacha_state[12] == 0))
+ ++chacha_state[13];
+ nbytes -= CHACHA20_BLOCK_SIZE;
+ buf += CHACHA20_BLOCK_SIZE;
+ }
+
+ memzero_explicit(chacha_state, sizeof(chacha_state));
+}
+
+/*
+ * This function is the exported kernel interface. It returns some
+ * number of good random numbers, suitable for key generation, seeding
+ * TCP sequence numbers, etc. It does not rely on the hardware random
+ * number generator. For random bytes direct from the hardware RNG
+ * (when available), use get_random_bytes_arch(). In order to ensure
+ * that the randomness provided by this function is okay, the function
+ * wait_for_random_bytes() should be called and return 0 at least once
+ * at any point prior.
+ */
+void get_random_bytes(void *buf, size_t nbytes)
+{
+ static void *previous;
+
+ warn_unseeded_randomness(&previous);
+ _get_random_bytes(buf, nbytes);
+}
+EXPORT_SYMBOL(get_random_bytes);
+
static ssize_t get_random_bytes_user(void __user *buf, size_t nbytes)
{
bool large_request = nbytes > 256;
@@ -692,6 +743,265 @@ static ssize_t get_random_bytes_user(voi
return ret;
}
+/*
+ * Batched entropy returns random integers. The quality of the random
+ * number is good as /dev/urandom. In order to ensure that the randomness
+ * provided by this function is okay, the function wait_for_random_bytes()
+ * should be called and return 0 at least once at any point prior.
+ */
+struct batched_entropy {
+ union {
+ /*
+ * We make this 1.5x a ChaCha block, so that we get the
+ * remaining 32 bytes from fast key erasure, plus one full
+ * block from the detached ChaCha state. We can increase
+ * the size of this later if needed so long as we keep the
+ * formula of (integer_blocks + 0.5) * CHACHA20_BLOCK_SIZE.
+ */
+ u64 entropy_u64[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u64))];
+ u32 entropy_u32[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u32))];
+ };
+ unsigned long generation;
+ unsigned int position;
+};
+
+
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
+ .position = UINT_MAX
+};
+
+u64 get_random_u64(void)
+{
+ u64 ret;
+ unsigned long flags;
+ struct batched_entropy *batch;
+ static void *previous;
+ unsigned long next_gen;
+
+ warn_unseeded_randomness(&previous);
+
+ local_irq_save(flags);
+ batch = raw_cpu_ptr(&batched_entropy_u64);
+
+ next_gen = READ_ONCE(base_crng.generation);
+ if (batch->position >= ARRAY_SIZE(batch->entropy_u64) ||
+ next_gen != batch->generation) {
+ _get_random_bytes(batch->entropy_u64, sizeof(batch->entropy_u64));
+ batch->position = 0;
+ batch->generation = next_gen;
+ }
+
+ ret = batch->entropy_u64[batch->position];
+ batch->entropy_u64[batch->position] = 0;
+ ++batch->position;
+ local_irq_restore(flags);
+ return ret;
+}
+EXPORT_SYMBOL(get_random_u64);
+
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
+ .position = UINT_MAX
+};
+
+u32 get_random_u32(void)
+{
+ u32 ret;
+ unsigned long flags;
+ struct batched_entropy *batch;
+ static void *previous;
+ unsigned long next_gen;
+
+ warn_unseeded_randomness(&previous);
+
+ local_irq_save(flags);
+ batch = raw_cpu_ptr(&batched_entropy_u32);
+
+ next_gen = READ_ONCE(base_crng.generation);
+ if (batch->position >= ARRAY_SIZE(batch->entropy_u32) ||
+ next_gen != batch->generation) {
+ _get_random_bytes(batch->entropy_u32, sizeof(batch->entropy_u32));
+ batch->position = 0;
+ batch->generation = next_gen;
+ }
+
+ ret = batch->entropy_u32[batch->position];
+ batch->entropy_u32[batch->position] = 0;
+ ++batch->position;
+ local_irq_restore(flags);
+ return ret;
+}
+EXPORT_SYMBOL(get_random_u32);
+
+/**
+ * randomize_page - Generate a random, page aligned address
+ * @start: The smallest acceptable address the caller will take.
+ * @range: The size of the area, starting at @start, within which the
+ * random address must fall.
+ *
+ * If @start + @range would overflow, @range is capped.
+ *
+ * NOTE: Historical use of randomize_range, which this replaces, presumed that
+ * @start was already page aligned. We now align it regardless.
+ *
+ * Return: A page aligned address within [start, start + range). On error,
+ * @start is returned.
+ */
+unsigned long randomize_page(unsigned long start, unsigned long range)
+{
+ if (!PAGE_ALIGNED(start)) {
+ range -= PAGE_ALIGN(start) - start;
+ start = PAGE_ALIGN(start);
+ }
+
+ if (start > ULONG_MAX - range)
+ range = ULONG_MAX - start;
+
+ range >>= PAGE_SHIFT;
+
+ if (range == 0)
+ return start;
+
+ return start + (get_random_long() % range << PAGE_SHIFT);
+}
+
+/*
+ * This function will use the architecture-specific hardware random
+ * number generator if it is available. It is not recommended for
+ * use. Use get_random_bytes() instead. It returns the number of
+ * bytes filled in.
+ */
+size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes)
+{
+ size_t left = nbytes;
+ u8 *p = buf;
+
+ while (left) {
+ unsigned long v;
+ size_t chunk = min_t(size_t, left, sizeof(unsigned long));
+
+ if (!arch_get_random_long(&v))
+ break;
+
+ memcpy(p, &v, chunk);
+ p += chunk;
+ left -= chunk;
+ }
+
+ return nbytes - left;
+}
+EXPORT_SYMBOL(get_random_bytes_arch);
+
+enum {
+ POOL_BITS = BLAKE2S_HASH_SIZE * 8,
+ POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
+};
+
+/*
+ * Static global variables
+ */
+static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
+
+/**********************************************************************
+ *
+ * OS independent entropy store. Here are the functions which handle
+ * storing entropy in an entropy pool.
+ *
+ **********************************************************************/
+
+static struct {
+ struct blake2s_state hash;
+ spinlock_t lock;
+ unsigned int entropy_count;
+} input_pool = {
+ .hash.h = { BLAKE2S_IV0 ^ (0x01010000 | BLAKE2S_HASH_SIZE),
+ BLAKE2S_IV1, BLAKE2S_IV2, BLAKE2S_IV3, BLAKE2S_IV4,
+ BLAKE2S_IV5, BLAKE2S_IV6, BLAKE2S_IV7 },
+ .hash.outlen = BLAKE2S_HASH_SIZE,
+ .lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
+};
+
+static void extract_entropy(void *buf, size_t nbytes);
+static bool drain_entropy(void *buf, size_t nbytes);
+
+static void crng_reseed(void);
+
+/*
+ * This function adds bytes into the entropy "pool". It does not
+ * update the entropy estimate. The caller should call
+ * credit_entropy_bits if this is appropriate.
+ */
+static void _mix_pool_bytes(const void *in, size_t nbytes)
+{
+ blake2s_update(&input_pool.hash, in, nbytes);
+}
+
+static void mix_pool_bytes(const void *in, size_t nbytes)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(in, nbytes);
+ spin_unlock_irqrestore(&input_pool.lock, flags);
+}
+
+struct fast_pool {
+ union {
+ u32 pool32[4];
+ u64 pool64[2];
+ };
+ unsigned long last;
+ u16 reg_idx;
+ u8 count;
+};
+
+/*
+ * This is a fast mixing routine used by the interrupt randomness
+ * collector. It's hardcoded for an 128 bit pool and assumes that any
+ * locks that might be needed are taken by the caller.
+ */
+static void fast_mix(u32 pool[4])
+{
+ u32 a = pool[0], b = pool[1];
+ u32 c = pool[2], d = pool[3];
+
+ a += b; c += d;
+ b = rol32(b, 6); d = rol32(d, 27);
+ d ^= a; b ^= c;
+
+ a += b; c += d;
+ b = rol32(b, 16); d = rol32(d, 14);
+ d ^= a; b ^= c;
+
+ a += b; c += d;
+ b = rol32(b, 6); d = rol32(d, 27);
+ d ^= a; b ^= c;
+
+ a += b; c += d;
+ b = rol32(b, 16); d = rol32(d, 14);
+ d ^= a; b ^= c;
+
+ pool[0] = a; pool[1] = b;
+ pool[2] = c; pool[3] = d;
+}
+
+static void credit_entropy_bits(size_t nbits)
+{
+ unsigned int entropy_count, orig, add;
+
+ if (!nbits)
+ return;
+
+ add = min_t(size_t, nbits, POOL_BITS);
+
+ do {
+ orig = READ_ONCE(input_pool.entropy_count);
+ entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
+ } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
+
+ if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
+ crng_reseed();
+}
+
/*********************************************************************
*
* Entropy input management
@@ -959,80 +1269,6 @@ static bool drain_entropy(void *buf, siz
return true;
}
-#define warn_unseeded_randomness(previous) \
- _warn_unseeded_randomness(__func__, (void *)_RET_IP_, (previous))
-
-static void _warn_unseeded_randomness(const char *func_name, void *caller, void **previous)
-{
-#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM
- const bool print_once = false;
-#else
- static bool print_once __read_mostly;
-#endif
-
- if (print_once || crng_ready() ||
- (previous && (caller == READ_ONCE(*previous))))
- return;
- WRITE_ONCE(*previous, caller);
-#ifndef CONFIG_WARN_ALL_UNSEEDED_RANDOM
- print_once = true;
-#endif
- if (__ratelimit(&unseeded_warning))
- pr_notice("random: %s called from %pS with crng_init=%d\n",
- func_name, caller, crng_init);
-}
-
-/*
- * This function is the exported kernel interface. It returns some
- * number of good random numbers, suitable for key generation, seeding
- * TCP sequence numbers, etc. It does not rely on the hardware random
- * number generator. For random bytes direct from the hardware RNG
- * (when available), use get_random_bytes_arch(). In order to ensure
- * that the randomness provided by this function is okay, the function
- * wait_for_random_bytes() should be called and return 0 at least once
- * at any point prior.
- */
-static void _get_random_bytes(void *buf, size_t nbytes)
-{
- u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
- u8 tmp[CHACHA20_BLOCK_SIZE];
- size_t len;
-
- if (!nbytes)
- return;
-
- len = min_t(size_t, 32, nbytes);
- crng_make_state(chacha_state, buf, len);
- nbytes -= len;
- buf += len;
-
- while (nbytes) {
- if (nbytes < CHACHA20_BLOCK_SIZE) {
- chacha20_block(chacha_state, tmp);
- memcpy(buf, tmp, nbytes);
- memzero_explicit(tmp, sizeof(tmp));
- break;
- }
-
- chacha20_block(chacha_state, buf);
- if (unlikely(chacha_state[12] == 0))
- ++chacha_state[13];
- nbytes -= CHACHA20_BLOCK_SIZE;
- buf += CHACHA20_BLOCK_SIZE;
- }
-
- memzero_explicit(chacha_state, sizeof(chacha_state));
-}
-
-void get_random_bytes(void *buf, size_t nbytes)
-{
- static void *previous;
-
- warn_unseeded_randomness(&previous);
- _get_random_bytes(buf, nbytes);
-}
-EXPORT_SYMBOL(get_random_bytes);
-
/*
* Each time the timer fires, we expect that we got an unpredictable
* jump in the cycle counter. Even if the timer is running on another
@@ -1082,134 +1318,6 @@ static void try_to_generate_entropy(void
mix_pool_bytes(&stack.now, sizeof(stack.now));
}
-/*
- * Wait for the urandom pool to be seeded and thus guaranteed to supply
- * cryptographically secure random numbers. This applies to: the /dev/urandom
- * device, the get_random_bytes function, and the get_random_{u32,u64,int,long}
- * family of functions. Using any of these functions without first calling
- * this function forfeits the guarantee of security.
- *
- * Returns: 0 if the urandom pool has been seeded.
- * -ERESTARTSYS if the function was interrupted by a signal.
- */
-int wait_for_random_bytes(void)
-{
- if (likely(crng_ready()))
- return 0;
-
- do {
- int ret;
- ret = wait_event_interruptible_timeout(crng_init_wait, crng_ready(), HZ);
- if (ret)
- return ret > 0 ? 0 : ret;
-
- try_to_generate_entropy();
- } while (!crng_ready());
-
- return 0;
-}
-EXPORT_SYMBOL(wait_for_random_bytes);
-
-/*
- * Returns whether or not the urandom pool has been seeded and thus guaranteed
- * to supply cryptographically secure random numbers. This applies to: the
- * /dev/urandom device, the get_random_bytes function, and the get_random_{u32,
- * ,u64,int,long} family of functions.
- *
- * Returns: true if the urandom pool has been seeded.
- * false if the urandom pool has not been seeded.
- */
-bool rng_is_initialized(void)
-{
- return crng_ready();
-}
-EXPORT_SYMBOL(rng_is_initialized);
-
-/*
- * Add a callback function that will be invoked when the nonblocking
- * pool is initialised.
- *
- * returns: 0 if callback is successfully added
- * -EALREADY if pool is already initialised (callback not called)
- * -ENOENT if module for callback is not alive
- */
-int add_random_ready_callback(struct random_ready_callback *rdy)
-{
- struct module *owner;
- unsigned long flags;
- int err = -EALREADY;
-
- if (crng_ready())
- return err;
-
- owner = rdy->owner;
- if (!try_module_get(owner))
- return -ENOENT;
-
- spin_lock_irqsave(&random_ready_list_lock, flags);
- if (crng_ready())
- goto out;
-
- owner = NULL;
-
- list_add(&rdy->list, &random_ready_list);
- err = 0;
-
-out:
- spin_unlock_irqrestore(&random_ready_list_lock, flags);
-
- module_put(owner);
-
- return err;
-}
-EXPORT_SYMBOL(add_random_ready_callback);
-
-/*
- * Delete a previously registered readiness callback function.
- */
-void del_random_ready_callback(struct random_ready_callback *rdy)
-{
- unsigned long flags;
- struct module *owner = NULL;
-
- spin_lock_irqsave(&random_ready_list_lock, flags);
- if (!list_empty(&rdy->list)) {
- list_del_init(&rdy->list);
- owner = rdy->owner;
- }
- spin_unlock_irqrestore(&random_ready_list_lock, flags);
-
- module_put(owner);
-}
-EXPORT_SYMBOL(del_random_ready_callback);
-
-/*
- * This function will use the architecture-specific hardware random
- * number generator if it is available. It is not recommended for
- * use. Use get_random_bytes() instead. It returns the number of
- * bytes filled in.
- */
-size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes)
-{
- size_t left = nbytes;
- u8 *p = buf;
-
- while (left) {
- unsigned long v;
- size_t chunk = min_t(size_t, left, sizeof(unsigned long));
-
- if (!arch_get_random_long(&v))
- break;
-
- memcpy(p, &v, chunk);
- p += chunk;
- left -= chunk;
- }
-
- return nbytes - left;
-}
-EXPORT_SYMBOL(get_random_bytes_arch);
-
static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
static int __init parse_trust_cpu(char *arg)
{
@@ -1560,126 +1668,6 @@ struct ctl_table random_table[] = {
};
#endif /* CONFIG_SYSCTL */
-struct batched_entropy {
- union {
- /*
- * We make this 1.5x a ChaCha block, so that we get the
- * remaining 32 bytes from fast key erasure, plus one full
- * block from the detached ChaCha state. We can increase
- * the size of this later if needed so long as we keep the
- * formula of (integer_blocks + 0.5) * CHACHA20_BLOCK_SIZE.
- */
- u64 entropy_u64[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u64))];
- u32 entropy_u32[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u32))];
- };
- unsigned long generation;
- unsigned int position;
-};
-
-/*
- * Get a random word for internal kernel use only. The quality of the random
- * number is good as /dev/urandom. In order to ensure that the randomness
- * provided by this function is okay, the function wait_for_random_bytes()
- * should be called and return 0 at least once at any point prior.
- */
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
- .position = UINT_MAX
-};
-
-u64 get_random_u64(void)
-{
- u64 ret;
- unsigned long flags;
- struct batched_entropy *batch;
- static void *previous;
- unsigned long next_gen;
-
- warn_unseeded_randomness(&previous);
-
- local_irq_save(flags);
- batch = raw_cpu_ptr(&batched_entropy_u64);
-
- next_gen = READ_ONCE(base_crng.generation);
- if (batch->position >= ARRAY_SIZE(batch->entropy_u64) ||
- next_gen != batch->generation) {
- _get_random_bytes(batch->entropy_u64, sizeof(batch->entropy_u64));
- batch->position = 0;
- batch->generation = next_gen;
- }
-
- ret = batch->entropy_u64[batch->position];
- batch->entropy_u64[batch->position] = 0;
- ++batch->position;
- local_irq_restore(flags);
- return ret;
-}
-EXPORT_SYMBOL(get_random_u64);
-
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
- .position = UINT_MAX
-};
-
-u32 get_random_u32(void)
-{
- u32 ret;
- unsigned long flags;
- struct batched_entropy *batch;
- static void *previous;
- unsigned long next_gen;
-
- warn_unseeded_randomness(&previous);
-
- local_irq_save(flags);
- batch = raw_cpu_ptr(&batched_entropy_u32);
-
- next_gen = READ_ONCE(base_crng.generation);
- if (batch->position >= ARRAY_SIZE(batch->entropy_u32) ||
- next_gen != batch->generation) {
- _get_random_bytes(batch->entropy_u32, sizeof(batch->entropy_u32));
- batch->position = 0;
- batch->generation = next_gen;
- }
-
- ret = batch->entropy_u32[batch->position];
- batch->entropy_u32[batch->position] = 0;
- ++batch->position;
- local_irq_restore(flags);
- return ret;
-}
-EXPORT_SYMBOL(get_random_u32);
-
-/**
- * randomize_page - Generate a random, page aligned address
- * @start: The smallest acceptable address the caller will take.
- * @range: The size of the area, starting at @start, within which the
- * random address must fall.
- *
- * If @start + @range would overflow, @range is capped.
- *
- * NOTE: Historical use of randomize_range, which this replaces, presumed that
- * @start was already page aligned. We now align it regardless.
- *
- * Return: A page aligned address within [start, start + range). On error,
- * @start is returned.
- */
-unsigned long randomize_page(unsigned long start, unsigned long range)
-{
- if (!PAGE_ALIGNED(start)) {
- range -= PAGE_ALIGN(start) - start;
- start = PAGE_ALIGN(start);
- }
-
- if (start > ULONG_MAX - range)
- range = ULONG_MAX - start;
-
- range >>= PAGE_SHIFT;
-
- if (range == 0)
- return start;
-
- return start + (get_random_long() % range << PAGE_SHIFT);
-}
-
/* Interface for in-kernel drivers of true hardware RNGs.
* Those devices may produce endless random bits and will be throttled
* when our pool is full.
From: Harald Freudenberger <[email protected]>
commit 10a515ddb5f19a1ff0b9882c430b4427843169f3 upstream.
When a user chooses a rng source via sysfs attribute
this rng should be sticky, even when other sources
with better quality to register. This patch introduces
a simple way to remember the user's choice. This is
reflected by a new sysfs attribute file 'rng_selected'
which shows if the current rng has been chosen by
userspace. The new attribute file shows '1' for user
selected rng and '0' otherwise.
Signed-off-by: Harald Freudenberger <[email protected]>
Reviewed-by: PrasannaKumar Muralidharan <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -27,6 +27,8 @@
#define RNG_MODULE_NAME "hw_random"
static struct hwrng *current_rng;
+/* the current rng has been explicitly chosen by user via sysfs */
+static int cur_rng_set_by_user;
static struct task_struct *hwrng_fill;
/* list of registered rngs, sorted decending by quality */
static LIST_HEAD(rng_list);
@@ -303,6 +305,7 @@ static ssize_t hwrng_attr_current_store(
list_for_each_entry(rng, &rng_list, list) {
if (sysfs_streq(rng->name, buf)) {
err = 0;
+ cur_rng_set_by_user = 1;
if (rng != current_rng)
err = set_current_rng(rng);
break;
@@ -351,16 +354,27 @@ static ssize_t hwrng_attr_available_show
return strlen(buf);
}
+static ssize_t hwrng_attr_selected_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%d\n", cur_rng_set_by_user);
+}
+
static DEVICE_ATTR(rng_current, S_IRUGO | S_IWUSR,
hwrng_attr_current_show,
hwrng_attr_current_store);
static DEVICE_ATTR(rng_available, S_IRUGO,
hwrng_attr_available_show,
NULL);
+static DEVICE_ATTR(rng_selected, S_IRUGO,
+ hwrng_attr_selected_show,
+ NULL);
static struct attribute *rng_dev_attrs[] = {
&dev_attr_rng_current.attr,
&dev_attr_rng_available.attr,
+ &dev_attr_rng_selected.attr,
NULL
};
@@ -443,10 +457,12 @@ int hwrng_register(struct hwrng *rng)
old_rng = current_rng;
err = 0;
- if (!old_rng || (rng->quality > old_rng->quality)) {
+ if (!old_rng ||
+ (!cur_rng_set_by_user && rng->quality > old_rng->quality)) {
/*
* Set new rng as current as the new rng source
- * provides better entropy quality.
+ * provides better entropy quality and was not
+ * chosen by userspace.
*/
err = set_current_rng(rng);
if (err)
@@ -478,6 +494,7 @@ void hwrng_unregister(struct hwrng *rng)
list_del(&rng->list);
if (current_rng == rng) {
drop_current_rng();
+ cur_rng_set_by_user = 0;
/* rng_list is sorted by quality, use the best (=first) one */
if (!list_empty(&rng_list)) {
struct hwrng *new_rng;
From: "Jason A. Donenfeld" <[email protected]>
commit ff8a8f59c99f6a7c656387addc4d9f2247d75077 upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is really not the best we can do.
Instead, at least calling random_get_entropy_fallback() would be
preferable, because that always needs to return _something_, even
falling back to jiffies eventually. It's not as though
random_get_entropy_fallback() is super high precision or guaranteed to
be entropic, but basically anything that's not zero all the time is
better than returning zero all the time.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Reviewed-by: Russell King (Oracle) <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/arm/include/asm/timex.h | 1 +
1 file changed, 1 insertion(+)
--- a/arch/arm/include/asm/timex.h
+++ b/arch/arm/include/asm/timex.h
@@ -14,5 +14,6 @@
typedef unsigned long cycles_t;
#define get_cycles() ({ cycles_t c; read_current_timer(&c) ? 0 : c; })
+#define random_get_entropy() (((unsigned long)get_cycles()) ?: random_get_entropy_fallback())
#endif
From: Yangtao Li <[email protected]>
commit 727d499a6f4f29b6abdb635032f5e53e5905aedb upstream.
s/entimate/estimate
Signed-off-by: Yangtao Li <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1263,7 +1263,7 @@ static void add_timer_randomness(struct
/*
* delta is now minimum absolute delta.
* Round down by 1 bit on general principles,
- * and limit entropy entimate to 12 bits.
+ * and limit entropy estimate to 12 bits.
*/
credit_entropy_bits(r, min_t(int, fls(delta>>1), 11));
}
From: Eric Biggers <[email protected]>
commit 5d58ea3a31cc98b9fa563f6921d3d043bf0103d1 upstream.
The primary_crng is always reseeded from the input_pool, while the NUMA
crngs are always reseeded from the primary_crng. Remove the redundant
'use_input_pool' parameter from crng_reseed() and just directly check
whether the crng is the primary_crng.
Signed-off-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -366,7 +366,7 @@ static struct {
static void extract_entropy(void *buf, size_t nbytes);
-static void crng_reseed(struct crng_state *crng, bool use_input_pool);
+static void crng_reseed(struct crng_state *crng);
/*
* This function adds bytes into the entropy "pool". It does not
@@ -465,7 +465,7 @@ static void credit_entropy_bits(int nbit
trace_credit_entropy_bits(nbits, entropy_count, _RET_IP_);
if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
- crng_reseed(&primary_crng, true);
+ crng_reseed(&primary_crng);
}
/*********************************************************************
@@ -752,7 +752,7 @@ static int crng_slow_load(const u8 *cp,
return 1;
}
-static void crng_reseed(struct crng_state *crng, bool use_input_pool)
+static void crng_reseed(struct crng_state *crng)
{
unsigned long flags;
int i;
@@ -761,7 +761,7 @@ static void crng_reseed(struct crng_stat
u32 key[8];
} buf;
- if (use_input_pool) {
+ if (crng == &primary_crng) {
int entropy_count;
do {
entropy_count = READ_ONCE(input_pool.entropy_count);
@@ -799,7 +799,7 @@ static void _extract_crng(struct crng_st
init_time = READ_ONCE(crng->init_time);
if (time_after(READ_ONCE(crng_global_init_time), init_time) ||
time_after(jiffies, init_time + CRNG_RESEED_INTERVAL))
- crng_reseed(crng, crng == &primary_crng);
+ crng_reseed(crng);
}
spin_lock_irqsave(&crng->lock, flags);
chacha20_block(&crng->state[0], out);
@@ -1598,7 +1598,7 @@ static long random_ioctl(struct file *f,
return -EPERM;
if (crng_init < 2)
return -ENODATA;
- crng_reseed(&primary_crng, true);
+ crng_reseed(&primary_crng);
WRITE_ONCE(crng_global_init_time, jiffies - 1);
return 0;
default:
From: "Jason A. Donenfeld" <[email protected]>
commit cbe89e5a375a51bbb952929b93fa973416fea74e upstream.
It's too hard to keep the batches synchronized, and pointless anyway,
since in !crng_ready(), we're updating the base_crng key really often,
where batching only hurts. So instead, if the crng isn't ready, just
call into get_random_bytes(). At this stage nothing is performance
critical anyhow.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 158 ++++++++++++++++++--------------------------------
1 file changed, 59 insertions(+), 99 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -234,10 +234,7 @@ static void _warn_unseeded_randomness(co
*
*********************************************************************/
-enum {
- CRNG_RESEED_INTERVAL = 300 * HZ,
- CRNG_INIT_CNT_THRESH = 2 * CHACHA20_KEY_SIZE
-};
+enum { CRNG_RESEED_INTERVAL = 300 * HZ };
static struct {
u8 key[CHACHA20_KEY_SIZE] __aligned(__alignof__(long));
@@ -259,6 +256,8 @@ static DEFINE_PER_CPU(struct crng, crngs
/* Used by crng_reseed() to extract a new seed from the input pool. */
static bool drain_entropy(void *buf, size_t nbytes);
+/* Used by crng_make_state() to extract a new seed when crng_init==0. */
+static void extract_entropy(void *buf, size_t nbytes);
/*
* This extracts a new crng key from the input pool, but only if there is a
@@ -383,17 +382,20 @@ static void crng_make_state(u32 chacha_s
/*
* For the fast path, we check whether we're ready, unlocked first, and
* then re-check once locked later. In the case where we're really not
- * ready, we do fast key erasure with the base_crng directly, because
- * this is what crng_pre_init_inject() mutates during early init.
+ * ready, we do fast key erasure with the base_crng directly, extracting
+ * when crng_init==0.
*/
if (!crng_ready()) {
bool ready;
spin_lock_irqsave(&base_crng.lock, flags);
ready = crng_ready();
- if (!ready)
+ if (!ready) {
+ if (crng_init == 0)
+ extract_entropy(base_crng.key, sizeof(base_crng.key));
crng_fast_key_erasure(base_crng.key, chacha_state,
random_data, random_data_len);
+ }
spin_unlock_irqrestore(&base_crng.lock, flags);
if (!ready)
return;
@@ -434,50 +436,6 @@ static void crng_make_state(u32 chacha_s
local_irq_restore(flags);
}
-/*
- * This function is for crng_init == 0 only. It loads entropy directly
- * into the crng's key, without going through the input pool. It is,
- * generally speaking, not very safe, but we use this only at early
- * boot time when it's better to have something there rather than
- * nothing.
- *
- * If account is set, then the crng_init_cnt counter is incremented.
- * This shouldn't be set by functions like add_device_randomness(),
- * where we can't trust the buffer passed to it is guaranteed to be
- * unpredictable (so it might not have any entropy at all).
- */
-static void crng_pre_init_inject(const void *input, size_t len, bool account)
-{
- static int crng_init_cnt = 0;
- struct blake2s_state hash;
- unsigned long flags;
-
- blake2s_init(&hash, sizeof(base_crng.key));
-
- spin_lock_irqsave(&base_crng.lock, flags);
- if (crng_init != 0) {
- spin_unlock_irqrestore(&base_crng.lock, flags);
- return;
- }
-
- blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
- blake2s_update(&hash, input, len);
- blake2s_final(&hash, base_crng.key);
-
- if (account) {
- crng_init_cnt += min_t(size_t, len, CRNG_INIT_CNT_THRESH - crng_init_cnt);
- if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
- ++base_crng.generation;
- crng_init = 1;
- }
- }
-
- spin_unlock_irqrestore(&base_crng.lock, flags);
-
- if (crng_init == 1)
- pr_notice("fast init done\n");
-}
-
static void _get_random_bytes(void *buf, size_t nbytes)
{
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
@@ -623,6 +581,11 @@ u64 get_random_u64(void)
warn_unseeded_randomness(&previous);
+ if (!crng_ready()) {
+ _get_random_bytes(&ret, sizeof(ret));
+ return ret;
+ }
+
local_irq_save(flags);
batch = raw_cpu_ptr(&batched_entropy_u64);
@@ -656,6 +619,11 @@ u32 get_random_u32(void)
warn_unseeded_randomness(&previous);
+ if (!crng_ready()) {
+ _get_random_bytes(&ret, sizeof(ret));
+ return ret;
+ }
+
local_irq_save(flags);
batch = raw_cpu_ptr(&batched_entropy_u32);
@@ -777,7 +745,8 @@ EXPORT_SYMBOL(get_random_bytes_arch);
enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
- POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
+ POOL_MIN_BITS = POOL_BITS, /* No point in settling for less. */
+ POOL_FAST_INIT_BITS = POOL_MIN_BITS / 2
};
/* For notifying userspace should write into /dev/random. */
@@ -814,24 +783,6 @@ static void mix_pool_bytes(const void *i
spin_unlock_irqrestore(&input_pool.lock, flags);
}
-static void credit_entropy_bits(size_t nbits)
-{
- unsigned int entropy_count, orig, add;
-
- if (!nbits)
- return;
-
- add = min_t(size_t, nbits, POOL_BITS);
-
- do {
- orig = READ_ONCE(input_pool.entropy_count);
- entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
- } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
-
- if (!crng_ready() && entropy_count >= POOL_MIN_BITS)
- crng_reseed();
-}
-
/*
* This is an HKDF-like construction for using the hashed collected entropy
* as a PRF key, that's then expanded block-by-block.
@@ -897,6 +848,33 @@ static bool drain_entropy(void *buf, siz
return true;
}
+static void credit_entropy_bits(size_t nbits)
+{
+ unsigned int entropy_count, orig, add;
+ unsigned long flags;
+
+ if (!nbits)
+ return;
+
+ add = min_t(size_t, nbits, POOL_BITS);
+
+ do {
+ orig = READ_ONCE(input_pool.entropy_count);
+ entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
+ } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
+
+ if (!crng_ready() && entropy_count >= POOL_MIN_BITS)
+ crng_reseed();
+ else if (unlikely(crng_init == 0 && entropy_count >= POOL_FAST_INIT_BITS)) {
+ spin_lock_irqsave(&base_crng.lock, flags);
+ if (crng_init == 0) {
+ extract_entropy(base_crng.key, sizeof(base_crng.key));
+ crng_init = 1;
+ }
+ spin_unlock_irqrestore(&base_crng.lock, flags);
+ }
+}
+
/**********************************************************************
*
@@ -939,9 +917,9 @@ static bool drain_entropy(void *buf, siz
* entropy as specified by the caller. If the entropy pool is full it will
* block until more entropy is needed.
*
- * add_bootloader_randomness() is the same as add_hwgenerator_randomness() or
- * add_device_randomness(), depending on whether or not the configuration
- * option CONFIG_RANDOM_TRUST_BOOTLOADER is set.
+ * add_bootloader_randomness() is called by bootloader drivers, such as EFI
+ * and device tree, and credits its input depending on whether or not the
+ * configuration option CONFIG_RANDOM_TRUST_BOOTLOADER is set.
*
* add_interrupt_randomness() uses the interrupt timing as random
* inputs to the entropy pool. Using the cycle counters and the irq source
@@ -1021,9 +999,6 @@ void add_device_randomness(const void *b
unsigned long entropy = random_get_entropy();
unsigned long flags;
- if (crng_init == 0 && size)
- crng_pre_init_inject(buf, size, false);
-
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(&entropy, sizeof(entropy));
_mix_pool_bytes(buf, size);
@@ -1139,12 +1114,6 @@ void rand_initialize_disk(struct gendisk
void add_hwgenerator_randomness(const void *buffer, size_t count,
size_t entropy)
{
- if (unlikely(crng_init == 0 && entropy < POOL_MIN_BITS)) {
- crng_pre_init_inject(buffer, count, true);
- mix_pool_bytes(buffer, count);
- return;
- }
-
/*
* Throttle writing if we're above the trickle threshold.
* We'll be woken up again once below POOL_MIN_BITS, when
@@ -1152,7 +1121,7 @@ void add_hwgenerator_randomness(const vo
* CRNG_RESEED_INTERVAL has elapsed.
*/
wait_event_interruptible_timeout(random_write_wait,
- !system_wq || kthread_should_stop() ||
+ kthread_should_stop() ||
input_pool.entropy_count < POOL_MIN_BITS,
CRNG_RESEED_INTERVAL);
mix_pool_bytes(buffer, count);
@@ -1161,17 +1130,14 @@ void add_hwgenerator_randomness(const vo
EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
/*
- * Handle random seed passed by bootloader.
- * If the seed is trustworthy, it would be regarded as hardware RNGs. Otherwise
- * it would be regarded as device data.
- * The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
+ * Handle random seed passed by bootloader, and credit it if
+ * CONFIG_RANDOM_TRUST_BOOTLOADER is set.
*/
void add_bootloader_randomness(const void *buf, size_t size)
{
+ mix_pool_bytes(buf, size);
if (trust_bootloader)
- add_hwgenerator_randomness(buf, size, size * 8);
- else
- add_device_randomness(buf, size);
+ credit_entropy_bits(size * 8);
}
EXPORT_SYMBOL_GPL(add_bootloader_randomness);
@@ -1271,13 +1237,8 @@ static void mix_interrupt_randomness(str
fast_pool->last = jiffies;
local_irq_enable();
- if (unlikely(crng_init == 0)) {
- crng_pre_init_inject(pool, sizeof(pool), true);
- mix_pool_bytes(pool, sizeof(pool));
- } else {
- mix_pool_bytes(pool, sizeof(pool));
- credit_entropy_bits(1);
- }
+ mix_pool_bytes(pool, sizeof(pool));
+ credit_entropy_bits(1);
memzero_explicit(pool, sizeof(pool));
}
@@ -1299,8 +1260,7 @@ void add_interrupt_randomness(int irq)
if (new_count & MIX_INFLIGHT)
return;
- if (new_count < 64 && (!time_is_before_jiffies(fast_pool->last + HZ) ||
- unlikely(crng_init == 0)))
+ if (new_count < 64 && !time_is_before_jiffies(fast_pool->last + HZ))
return;
if (unlikely(!fast_pool->mix.func))
From: Mark Rutland <[email protected]>
commit ab9a7e27044b87ff2be47b8f8e095400e7fccc44 upstream.
As crng_initialize_secondary() is only called by do_numa_crng_init(),
and the latter is under ifdeffery for CONFIG_NUMA, when CONFIG_NUMA is
not selected the compiler will warn that the former is unused:
| drivers/char/random.c:820:13: warning: 'crng_initialize_secondary' defined but not used [-Wunused-function]
| 820 | static void crng_initialize_secondary(struct crng_state *crng)
| | ^~~~~~~~~~~~~~~~~~~~~~~~~
Stephen reports that this happens for x86_64 noallconfig builds.
We could move crng_initialize_secondary() and crng_init_try_arch() under
the CONFIG_NUMA ifdeffery, but this has the unfortunate property of
separating them from crng_initialize_primary() and
crng_init_try_arch_early() respectively. Instead, let's mark
crng_initialize_secondary() as __maybe_unused.
Link: https://lore.kernel.org/r/[email protected]
Fixes: 5cbe0f13b51a ("random: split primary/secondary crng init paths")
Reported-by: Stephen Rothwell <[email protected]>
Signed-off-by: Mark Rutland <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -801,7 +801,7 @@ static bool crng_init_try_arch(struct cr
return arch_init;
}
-static void crng_initialize_secondary(struct crng_state *crng)
+static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
{
memcpy(&crng->state[0], "expand 32-byte k", 16);
_get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
From: Miaoqian Lin <[email protected]>
[ Upstream commit f4b98e314888cc51486421bcf6d52852452ea48b ]
of_find_matching_node_and_match() returns a node pointer with refcount
incremented, we should use of_node_put() on it when not need anymore.
Add missing of_node_put() to avoid refcount leak.
Fixes: 82b0a434b436 ("irqchip/gic/realview: Support more RealView DCC variants")
Signed-off-by: Miaoqian Lin <[email protected]>
Signed-off-by: Marc Zyngier <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/irqchip/irq-gic-realview.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/irqchip/irq-gic-realview.c b/drivers/irqchip/irq-gic-realview.c
index 54c296401525..61024882c685 100644
--- a/drivers/irqchip/irq-gic-realview.c
+++ b/drivers/irqchip/irq-gic-realview.c
@@ -56,6 +56,7 @@ realview_gic_of_init(struct device_node *node, struct device_node *parent)
/* The PB11MPCore GIC needs to be configured in the syscon */
map = syscon_node_to_regmap(np);
+ of_node_put(np);
if (!IS_ERR(map)) {
/* new irq mode with no DCC */
regmap_write(map, REALVIEW_SYS_LOCK_OFFSET,
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 1ce6c8d68f8ac587f54d0a271ac594d3d51f3efb upstream.
get_random_bytes_user() checks for signals after producing a PAGE_SIZE
worth of output, just like /dev/zero does. write_pool() is doing
basically the same work (actually, slightly more expensive), and so
should stop to check for signals in the same way. Let's also name it
write_pool_user() to match get_random_bytes_user(), so this won't be
misused in the future.
Before this patch, massive writes to /dev/urandom would tie up the
process for an extremely long time and make it unterminatable. After, it
can be successfully interrupted. The following test program can be used
to see this works as intended:
#include <unistd.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
static unsigned char x[~0U];
static void handle(int) { }
int main(int argc, char *argv[])
{
pid_t pid = getpid(), child;
int fd;
signal(SIGUSR1, handle);
if (!(child = fork())) {
for (;;)
kill(pid, SIGUSR1);
}
fd = open("/dev/urandom", O_WRONLY);
pause();
printf("interrupted after writing %zd bytes\n", write(fd, x, sizeof(x)));
close(fd);
kill(child, SIGTERM);
return 0;
}
Result before: "interrupted after writing 2147479552 bytes"
Result after: "interrupted after writing 4096 bytes"
Cc: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1252,7 +1252,7 @@ static unsigned int random_poll(struct f
return crng_ready() ? POLLIN | POLLRDNORM : POLLOUT | POLLWRNORM;
}
-static ssize_t write_pool(struct iov_iter *iter)
+static ssize_t write_pool_user(struct iov_iter *iter)
{
u8 block[BLAKE2S_BLOCK_SIZE];
ssize_t ret = 0;
@@ -1267,7 +1267,13 @@ static ssize_t write_pool(struct iov_ite
mix_pool_bytes(block, copied);
if (!iov_iter_count(iter) || copied != sizeof(block))
break;
- cond_resched();
+
+ BUILD_BUG_ON(PAGE_SIZE % sizeof(block) != 0);
+ if (ret % PAGE_SIZE == 0) {
+ if (signal_pending(current))
+ break;
+ cond_resched();
+ }
}
memzero_explicit(block, sizeof(block));
@@ -1276,7 +1282,7 @@ static ssize_t write_pool(struct iov_ite
static ssize_t random_write_iter(struct kiocb *kiocb, struct iov_iter *iter)
{
- return write_pool(iter);
+ return write_pool_user(iter);
}
static ssize_t urandom_read_iter(struct kiocb *kiocb, struct iov_iter *iter)
@@ -1343,7 +1349,7 @@ static long random_ioctl(struct file *f,
ret = import_single_range(WRITE, p, len, &iov, &iter);
if (unlikely(ret))
return ret;
- ret = write_pool(&iter);
+ ret = write_pool_user(&iter);
if (unlikely(ret < 0))
return ret;
/* Since we're crediting, enforce that it was all written into the pool. */
From: Kees Cook <[email protected]>
commit d55535232c3dbde9a523a9d10d68670f5fe5dec3 upstream.
Right now rand_initialize() is run as an early_initcall(), but it only
depends on timekeeping_init() (for mixing ktime_get_real() into the
pools). However, the call to boot_init_stack_canary() for stack canary
initialization runs earlier, which triggers a warning at boot:
random: get_random_bytes called from start_kernel+0x357/0x548 with crng_init=0
Instead, this moves rand_initialize() to after timekeeping_init(), and moves
canary initialization here as well.
Note that this warning may still remain for machines that do not have
UEFI RNG support (which initializes the RNG pools during setup_arch()),
or for x86 machines without RDRAND (or booting without "random.trust=on"
or CONFIG_RANDOM_TRUST_CPU=y).
Signed-off-by: Kees Cook <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 5 ++---
include/linux/random.h | 1 +
init/main.c | 14 ++++++++++++++
3 files changed, 17 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1842,7 +1842,7 @@ EXPORT_SYMBOL(get_random_bytes_arch);
* data into the pool to prepare it for use. The pool is not cleared
* as that can only decrease the entropy in the pool.
*/
-static void init_std_data(struct entropy_store *r)
+static void __init init_std_data(struct entropy_store *r)
{
int i;
ktime_t now = ktime_get_real();
@@ -1869,7 +1869,7 @@ static void init_std_data(struct entropy
* take care not to overwrite the precious per platform data
* we were given.
*/
-static int rand_initialize(void)
+int __init rand_initialize(void)
{
init_std_data(&input_pool);
init_std_data(&blocking_pool);
@@ -1881,7 +1881,6 @@ static int rand_initialize(void)
}
return 0;
}
-early_initcall(rand_initialize);
#ifdef CONFIG_BLOCK
void rand_initialize_disk(struct gendisk *disk)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -35,6 +35,7 @@ extern void add_interrupt_randomness(int
extern void get_random_bytes(void *buf, int nbytes);
extern int wait_for_random_bytes(void);
+extern int __init rand_initialize(void);
extern bool rng_is_initialized(void);
extern int add_random_ready_callback(struct random_ready_callback *rdy);
extern void del_random_ready_callback(struct random_ready_callback *rdy);
--- a/init/main.c
+++ b/init/main.c
@@ -570,6 +570,20 @@ asmlinkage __visible void __init start_k
hrtimers_init();
softirq_init();
timekeeping_init();
+
+ /*
+ * For best initial stack canary entropy, prepare it after:
+ * - setup_arch() for any UEFI RNG entropy and boot cmdline access
+ * - timekeeping_init() for ktime entropy used in rand_initialize()
+ * - rand_initialize() to get any arch-specific entropy like RDRAND
+ * - add_latent_entropy() to get any latent entropy
+ * - adding command line entropy
+ */
+ rand_initialize();
+ add_latent_entropy();
+ add_device_randomness(command_line, strlen(command_line));
+ boot_init_stack_canary();
+
time_init();
sched_clock_postinit();
printk_nmi_init();
From: Richard Henderson <[email protected]>
commit cbac004995a0ce8453bdc555fab579e2bdb842a6 upstream.
These symbols are currently part of the generic archrandom.h
interface, but are currently unused and can be removed.
Signed-off-by: Richard Henderson <[email protected]>
Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/powerpc/include/asm/archrandom.h | 10 ----------
1 file changed, 10 deletions(-)
--- a/arch/powerpc/include/asm/archrandom.h
+++ b/arch/powerpc/include/asm/archrandom.h
@@ -33,16 +33,6 @@ static inline int arch_get_random_seed_i
return rc;
}
-
-static inline int arch_has_random(void)
-{
- return 0;
-}
-
-static inline int arch_has_random_seed(void)
-{
- return !!ppc_md.get_random_seed;
-}
#endif /* CONFIG_ARCH_RANDOM */
#ifdef CONFIG_PPC_POWERNV
From: James Chapman <[email protected]>
commit 225eb26489d05c679a4c4197ffcb81c81e9dcaf4 upstream.
Previously, if a ppp session was closed, we called inet_shutdown to mark
the socket as unconnected such that userspace would get errors and
then close the socket. This could race with userspace closing the
socket. Instead, leave userspace to close the socket in its own time
(our session will be detached anyway).
BUG: KASAN: use-after-free in inet_shutdown+0x5d/0x1c0
Read of size 4 at addr ffff880010ea3ac0 by task syzbot_347bd5ac/8296
CPU: 3 PID: 8296 Comm: syzbot_347bd5ac Not tainted 4.16.0-rc1+ #91
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Call Trace:
dump_stack+0x101/0x157
? inet_shutdown+0x5d/0x1c0
print_address_description+0x78/0x260
? inet_shutdown+0x5d/0x1c0
kasan_report+0x240/0x360
__asan_load4+0x78/0x80
inet_shutdown+0x5d/0x1c0
? pppol2tp_show+0x80/0x80
pppol2tp_session_close+0x68/0xb0
l2tp_tunnel_closeall+0x199/0x210
? udp_v6_flush_pending_frames+0x90/0x90
l2tp_udp_encap_destroy+0x6b/0xc0
? l2tp_tunnel_del_work+0x2e0/0x2e0
udpv6_destroy_sock+0x8c/0x90
sk_common_release+0x47/0x190
udp_lib_close+0x15/0x20
inet_release+0x85/0xd0
inet6_release+0x43/0x60
sock_release+0x53/0x100
? sock_alloc_file+0x260/0x260
sock_close+0x1b/0x20
__fput+0x19f/0x380
____fput+0x1a/0x20
task_work_run+0xd2/0x110
exit_to_usermode_loop+0x18d/0x190
do_syscall_64+0x389/0x3b0
entry_SYSCALL_64_after_hwframe+0x26/0x9b
RIP: 0033:0x7fe240a45259
RSP: 002b:00007fe241132df8 EFLAGS: 00000297 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fe240a45259
RDX: 00007fe240a45259 RSI: 0000000000000000 RDI: 00000000000000a5
RBP: 00007fe241132e20 R08: 00007fe241133700 R09: 0000000000000000
R10: 00007fe241133700 R11: 0000000000000297 R12: 0000000000000000
R13: 00007ffc49aff84f R14: 0000000000000000 R15: 00007fe241141040
Allocated by task 8331:
save_stack+0x43/0xd0
kasan_kmalloc+0xad/0xe0
kasan_slab_alloc+0x12/0x20
kmem_cache_alloc+0x144/0x3e0
sock_alloc_inode+0x22/0x130
alloc_inode+0x3d/0xf0
new_inode_pseudo+0x1c/0x90
sock_alloc+0x30/0x110
__sock_create+0xaa/0x4c0
SyS_socket+0xbe/0x130
do_syscall_64+0x128/0x3b0
entry_SYSCALL_64_after_hwframe+0x26/0x9b
Freed by task 8314:
save_stack+0x43/0xd0
__kasan_slab_free+0x11a/0x170
kasan_slab_free+0xe/0x10
kmem_cache_free+0x88/0x2b0
sock_destroy_inode+0x49/0x50
destroy_inode+0x77/0xb0
evict+0x285/0x340
iput+0x429/0x530
dentry_unlink_inode+0x28c/0x2c0
__dentry_kill+0x1e3/0x2f0
dput.part.21+0x500/0x560
dput+0x24/0x30
__fput+0x2aa/0x380
____fput+0x1a/0x20
task_work_run+0xd2/0x110
exit_to_usermode_loop+0x18d/0x190
do_syscall_64+0x389/0x3b0
entry_SYSCALL_64_after_hwframe+0x26/0x9b
Fixes: fd558d186df2c ("l2tp: Split pppol2tp patch into separate l2tp and ppp parts")
Signed-off-by: James Chapman <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
Cc: Lee Jones <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/l2tp/l2tp_ppp.c | 10 ----------
1 file changed, 10 deletions(-)
--- a/net/l2tp/l2tp_ppp.c
+++ b/net/l2tp/l2tp_ppp.c
@@ -439,16 +439,6 @@ abort:
*/
static void pppol2tp_session_close(struct l2tp_session *session)
{
- struct sock *sk;
-
- BUG_ON(session->magic != L2TP_SESSION_MAGIC);
-
- sk = pppol2tp_session_get_sock(session);
- if (sk) {
- if (sk->sk_socket)
- inet_shutdown(sk->sk_socket, SEND_SHUTDOWN);
- sock_put(sk);
- }
}
/* Really kill the session socket. (Called from sock_put() if
From: "Jason A. Donenfeld" <[email protected]>
commit 77fc95f8c0dc9e1f8e620ec14d2fb65028fb7adc upstream.
Rather than accounting in bytes and multiplying (shifting), we can just
account in bits and avoid the shift. The main motivation for this is
there are other patches in flux that expand this code a bit, and
avoiding the duplication of "* 8" everywhere makes things a bit clearer.
Cc: [email protected]
Fixes: 12e45a2a6308 ("random: credit architectural init the exact amount")
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -810,7 +810,7 @@ early_param("random.trust_bootloader", p
int __init random_init(const char *command_line)
{
ktime_t now = ktime_get_real();
- unsigned int i, arch_bytes;
+ unsigned int i, arch_bits;
unsigned long entropy;
#if defined(LATENT_ENTROPY_PLUGIN)
@@ -818,12 +818,12 @@ int __init random_init(const char *comma
_mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed));
#endif
- for (i = 0, arch_bytes = BLAKE2S_BLOCK_SIZE;
+ for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8;
i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) {
if (!arch_get_random_seed_long_early(&entropy) &&
!arch_get_random_long_early(&entropy)) {
entropy = random_get_entropy();
- arch_bytes -= sizeof(entropy);
+ arch_bits -= sizeof(entropy) * 8;
}
_mix_pool_bytes(&entropy, sizeof(entropy));
}
@@ -835,7 +835,7 @@ int __init random_init(const char *comma
if (crng_ready())
crng_reseed();
else if (trust_cpu)
- _credit_init_bits(arch_bytes * 8);
+ _credit_init_bits(arch_bits);
return 0;
}
From: "Jason A. Donenfeld" <[email protected]>
commit b2f408fe403800c91a49f6589d95b6759ce1b30b upstream.
In the irq handler, we fill out 16 bytes differently on 32-bit and
64-bit platforms, and for 32-bit vs 64-bit cycle counters, which doesn't
always correspond with the bitness of the platform. Whether or not you
like this strangeness, it is a matter of fact. But it might not be a
fact you well realized until now, because the code that loaded the irq
info into 4 32-bit words was quite confusing. Instead, this commit
makes everything explicit by having separate (compile-time) branches for
32-bit and 64-bit types.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 49 ++++++++++++++++++++++++++++---------------------
1 file changed, 28 insertions(+), 21 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -284,7 +284,10 @@ static void mix_pool_bytes(const void *i
}
struct fast_pool {
- u32 pool[4];
+ union {
+ u32 pool32[4];
+ u64 pool64[2];
+ };
unsigned long last;
u16 reg_idx;
u8 count;
@@ -295,10 +298,10 @@ struct fast_pool {
* collector. It's hardcoded for an 128 bit pool and assumes that any
* locks that might be needed are taken by the caller.
*/
-static void fast_mix(struct fast_pool *f)
+static void fast_mix(u32 pool[4])
{
- u32 a = f->pool[0], b = f->pool[1];
- u32 c = f->pool[2], d = f->pool[3];
+ u32 a = pool[0], b = pool[1];
+ u32 c = pool[2], d = pool[3];
a += b; c += d;
b = rol32(b, 6); d = rol32(d, 27);
@@ -316,9 +319,8 @@ static void fast_mix(struct fast_pool *f
b = rol32(b, 16); d = rol32(d, 14);
d ^= a; b ^= c;
- f->pool[0] = a; f->pool[1] = b;
- f->pool[2] = c; f->pool[3] = d;
- f->count++;
+ pool[0] = a; pool[1] = b;
+ pool[2] = c; pool[3] = d;
}
static void process_random_ready_list(void)
@@ -834,29 +836,34 @@ void add_interrupt_randomness(int irq)
struct pt_regs *regs = get_irq_regs();
unsigned long now = jiffies;
cycles_t cycles = random_get_entropy();
- u32 c_high, j_high;
- u64 ip;
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
- c_high = (sizeof(cycles) > 4) ? cycles >> 32 : 0;
- j_high = (sizeof(now) > 4) ? now >> 32 : 0;
- fast_pool->pool[0] ^= cycles ^ j_high ^ irq;
- fast_pool->pool[1] ^= now ^ c_high;
- ip = regs ? instruction_pointer(regs) : _RET_IP_;
- fast_pool->pool[2] ^= ip;
- fast_pool->pool[3] ^=
- (sizeof(ip) > 4) ? ip >> 32 : get_reg(fast_pool, regs);
- fast_mix(fast_pool);
+ if (sizeof(cycles) == 8)
+ fast_pool->pool64[0] ^= cycles ^ rol64(now, 32) ^ irq;
+ else {
+ fast_pool->pool32[0] ^= cycles ^ irq;
+ fast_pool->pool32[1] ^= now;
+ }
+
+ if (sizeof(unsigned long) == 8)
+ fast_pool->pool64[1] ^= regs ? instruction_pointer(regs) : _RET_IP_;
+ else {
+ fast_pool->pool32[2] ^= regs ? instruction_pointer(regs) : _RET_IP_;
+ fast_pool->pool32[3] ^= get_reg(fast_pool, regs);
+ }
+
+ fast_mix(fast_pool->pool32);
+ ++fast_pool->count;
if (unlikely(crng_init == 0)) {
if (fast_pool->count >= 64 &&
- crng_fast_load(fast_pool->pool, sizeof(fast_pool->pool)) > 0) {
+ crng_fast_load(fast_pool->pool32, sizeof(fast_pool->pool32)) > 0) {
fast_pool->count = 0;
fast_pool->last = now;
if (spin_trylock(&input_pool.lock)) {
- _mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool));
+ _mix_pool_bytes(&fast_pool->pool32, sizeof(fast_pool->pool32));
spin_unlock(&input_pool.lock);
}
}
@@ -870,7 +877,7 @@ void add_interrupt_randomness(int irq)
return;
fast_pool->last = now;
- _mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool));
+ _mix_pool_bytes(&fast_pool->pool32, sizeof(fast_pool->pool32));
spin_unlock(&input_pool.lock);
fast_pool->count = 0;
From: Wentao Wang <[email protected]>
[ Upstream commit cf71d59c2eceadfcde0fb52e237990a0909880d7 ]
vcpuHint has been expanded to 16 bit on host to enable routing to more
CPUs. Guest side should align with the change. This change has been tested
with hosts with 8-bit and 16-bit vcpuHint, on both platforms host side can
get correct value.
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Wentao Wang <[email protected]>
Signed-off-by: Martin K. Petersen <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/scsi/vmw_pvscsi.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/vmw_pvscsi.h b/drivers/scsi/vmw_pvscsi.h
index d41292ef85f2..98ad17cb6643 100644
--- a/drivers/scsi/vmw_pvscsi.h
+++ b/drivers/scsi/vmw_pvscsi.h
@@ -333,8 +333,8 @@ struct PVSCSIRingReqDesc {
u8 tag;
u8 bus;
u8 target;
- u8 vcpuHint;
- u8 unused[59];
+ u16 vcpuHint;
+ u8 unused[58];
} __packed;
/*
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 3e504d2026eb6c8762cd6040ae57db166516824a upstream.
Rather than waiting a full second in an interruptable waiter before
trying to generate entropy, try to generate entropy first and wait
second. While waiting one second might give an extra second for getting
entropy from elsewhere, we're already pretty late in the init process
here, and whatever else is generating entropy will still continue to
contribute. This has implications on signal handling: we call
try_to_generate_entropy() from wait_for_random_bytes(), and
wait_for_random_bytes() always uses wait_event_interruptible_timeout()
when waiting, since it's called by userspace code in restartable
contexts, where signals can pend. Since try_to_generate_entropy() now
runs first, if a signal is pending, it's necessary for
try_to_generate_entropy() to check for signals, since it won't hit the
wait until after try_to_generate_entropy() has returned. And even before
this change, when entering a busy loop in try_to_generate_entropy(), we
should have been checking to see if any signals are pending, so that a
process doesn't get stuck in that loop longer than expected.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -128,10 +128,11 @@ int wait_for_random_bytes(void)
{
while (!crng_ready()) {
int ret;
+
+ try_to_generate_entropy();
ret = wait_event_interruptible_timeout(crng_init_wait, crng_ready(), HZ);
if (ret)
return ret > 0 ? 0 : ret;
- try_to_generate_entropy();
}
return 0;
}
@@ -1367,7 +1368,7 @@ static void try_to_generate_entropy(void
return;
__setup_timer_on_stack(&stack.timer, entropy_timer, 0, 0);
- while (!crng_ready()) {
+ while (!crng_ready() && !signal_pending(current)) {
if (!timer_pending(&stack.timer))
mod_timer(&stack.timer, jiffies + 1);
mix_pool_bytes(&stack.cycles, sizeof(stack.cycles));
From: Andy Lutomirski <[email protected]>
commit 75551dbf112c992bc6c99a972990b3f272247e23 upstream.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/d5473b56cf1fa900ca4bd2b3fc1e5b8874399919.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 11 +++++++++--
include/uapi/linux/random.h | 2 ++
2 files changed, 11 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2187,7 +2187,14 @@ SYSCALL_DEFINE3(getrandom, char __user *
{
int ret;
- if (flags & ~(GRND_NONBLOCK|GRND_RANDOM))
+ if (flags & ~(GRND_NONBLOCK|GRND_RANDOM|GRND_INSECURE))
+ return -EINVAL;
+
+ /*
+ * Requesting insecure and blocking randomness at the same time makes
+ * no sense.
+ */
+ if ((flags & (GRND_INSECURE|GRND_RANDOM)) == (GRND_INSECURE|GRND_RANDOM))
return -EINVAL;
if (count > INT_MAX)
@@ -2196,7 +2203,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
if (flags & GRND_RANDOM)
return _random_read(flags & GRND_NONBLOCK, buf, count);
- if (!crng_ready()) {
+ if (!(flags & GRND_INSECURE) && !crng_ready()) {
if (flags & GRND_NONBLOCK)
return -EAGAIN;
ret = wait_for_random_bytes();
--- a/include/uapi/linux/random.h
+++ b/include/uapi/linux/random.h
@@ -48,8 +48,10 @@ struct rand_pool_info {
*
* GRND_NONBLOCK Don't block and return EAGAIN instead
* GRND_RANDOM Use the /dev/random pool instead of /dev/urandom
+ * GRND_INSECURE Return non-cryptographic random bytes
*/
#define GRND_NONBLOCK 0x0001
#define GRND_RANDOM 0x0002
+#define GRND_INSECURE 0x0004
#endif /* _UAPI_LINUX_RANDOM_H */
From: "Jason A. Donenfeld" <[email protected]>
commit d0efdf35a6a71d307a250199af6fce122a7c7e11 upstream.
This isn't used by anything or anywhere, but we can't delete it due to
compatibility. So at least give it the correct value of what it's
supposed to be instead of a garbage one.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1613,7 +1613,7 @@ const struct file_operations urandom_fop
* to avoid breaking old userspaces, but writing to it does not
* change any behavior of the RNG.
*
- * - urandom_min_reseed_secs - fixed to the meaningless value "60".
+ * - urandom_min_reseed_secs - fixed to the value CRNG_RESEED_INTERVAL.
* It is writable to avoid breaking old userspaces, but writing
* to it does not change any behavior of the RNG.
*
@@ -1623,7 +1623,7 @@ const struct file_operations urandom_fop
#include <linux/sysctl.h>
-static int sysctl_random_min_urandom_seed = 60;
+static int sysctl_random_min_urandom_seed = CRNG_RESEED_INTERVAL / HZ;
static int sysctl_random_write_wakeup_bits = POOL_MIN_BITS;
static int sysctl_poolsize = POOL_BITS;
static u8 sysctl_bootid[UUID_SIZE];
From: "Jason A. Donenfeld" <[email protected]>
commit a96cfe2d427064325ecbf56df8816c6b871ec285 upstream.
Rather than sometimes checking `crng_init < 2`, we should always use the
crng_ready() macro, so that should we change anything later, it's
consistent. Additionally, that macro already has a likely() around it,
which means we don't need to open code our own likely() and unlikely()
annotations.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -126,18 +126,13 @@ static void try_to_generate_entropy(void
*/
int wait_for_random_bytes(void)
{
- if (likely(crng_ready()))
- return 0;
-
- do {
+ while (!crng_ready()) {
int ret;
ret = wait_event_interruptible_timeout(crng_init_wait, crng_ready(), HZ);
if (ret)
return ret > 0 ? 0 : ret;
-
try_to_generate_entropy();
- } while (!crng_ready());
-
+ }
return 0;
}
EXPORT_SYMBOL(wait_for_random_bytes);
@@ -292,7 +287,7 @@ static void crng_reseed(void)
++next_gen;
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
- if (crng_init < 2) {
+ if (!crng_ready()) {
crng_init = 2;
finalize_init = true;
}
@@ -360,7 +355,7 @@ static void crng_make_state(u32 chacha_s
* ready, we do fast key erasure with the base_crng directly, because
* this is what crng_pre_init_inject() mutates during early init.
*/
- if (unlikely(!crng_ready())) {
+ if (!crng_ready()) {
bool ready;
spin_lock_irqsave(&base_crng.lock, flags);
@@ -800,7 +795,7 @@ static void credit_entropy_bits(size_t n
entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
} while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
- if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
+ if (!crng_ready() && entropy_count >= POOL_MIN_BITS)
crng_reseed();
}
@@ -957,7 +952,7 @@ int __init rand_initialize(void)
extract_entropy(base_crng.key, sizeof(base_crng.key));
++base_crng.generation;
- if (arch_init && trust_cpu && crng_init < 2) {
+ if (arch_init && trust_cpu && !crng_ready()) {
crng_init = 2;
pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
@@ -1546,7 +1541,7 @@ static long random_ioctl(struct file *f,
case RNDRESEEDCRNG:
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- if (crng_init < 2)
+ if (!crng_ready())
return -ENODATA;
crng_reseed();
return 0;
From: "Jason A. Donenfeld" <[email protected]>
commit 87e7d5abad0cbc9312dea7f889a57d294c1a5fcc upstream.
This is purely cosmetic. Future work involves figuring out which of
these headers we need and which we don't.
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 1 -
1 file changed, 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -196,7 +196,6 @@
#include <linux/uuid.h>
#include <crypto/chacha20.h>
#include <crypto/blake2s.h>
-
#include <asm/processor.h>
#include <asm/uaccess.h>
#include <asm/irq.h>
From: "Jason A. Donenfeld" <[email protected]>
commit b3d51c1f542113342ddfbf6007e38a684b9dbec9 upstream.
The other pool constants are prepended with POOL_, but not these last
ones. Rename them. This will then let us move them into the enum in the
following commit.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 40 ++++++++++++++++++++--------------------
1 file changed, 20 insertions(+), 20 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -363,11 +363,11 @@
* To allow fractional bits to be tracked, the entropy_count field is
* denominated in units of 1/8th bits.
*
- * 2*(ENTROPY_SHIFT + poolbitshift) must <= 31, or the multiply in
+ * 2*(POOL_ENTROPY_SHIFT + poolbitshift) must <= 31, or the multiply in
* credit_entropy_bits() needs to be 64 bits wide.
*/
-#define ENTROPY_SHIFT 3
-#define ENTROPY_BITS() (input_pool.entropy_count >> ENTROPY_SHIFT)
+#define POOL_ENTROPY_SHIFT 3
+#define POOL_ENTROPY_BITS() (input_pool.entropy_count >> POOL_ENTROPY_SHIFT)
/*
* If the entropy count falls under this number of bits, then we
@@ -427,7 +427,7 @@ enum poolinfo {
POOL_BYTES = POOL_WORDS * sizeof(u32),
POOL_BITS = POOL_BYTES * 8,
POOL_BITSHIFT = ilog2(POOL_WORDS) + 5,
- POOL_FRACBITS = POOL_WORDS << (ENTROPY_SHIFT + 5),
+ POOL_FRACBITS = POOL_WORDS << (POOL_ENTROPY_SHIFT + 5),
/* x^128 + x^104 + x^76 + x^51 +x^25 + x + 1 */
POOL_TAP1 = 104,
@@ -651,7 +651,7 @@ static void process_random_ready_list(vo
static void credit_entropy_bits(int nbits)
{
int entropy_count, entropy_bits, orig;
- int nfrac = nbits << ENTROPY_SHIFT;
+ int nfrac = nbits << POOL_ENTROPY_SHIFT;
if (!nbits)
return;
@@ -684,7 +684,7 @@ retry:
* turns no matter how large nbits is.
*/
int pnfrac = nfrac;
- const int s = POOL_BITSHIFT + ENTROPY_SHIFT + 2;
+ const int s = POOL_BITSHIFT + POOL_ENTROPY_SHIFT + 2;
/* The +2 corresponds to the /4 in the denominator */
do {
@@ -705,9 +705,9 @@ retry:
if (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig)
goto retry;
- trace_credit_entropy_bits(nbits, entropy_count >> ENTROPY_SHIFT, _RET_IP_);
+ trace_credit_entropy_bits(nbits, entropy_count >> POOL_ENTROPY_SHIFT, _RET_IP_);
- entropy_bits = entropy_count >> ENTROPY_SHIFT;
+ entropy_bits = entropy_count >> POOL_ENTROPY_SHIFT;
if (crng_init < 2 && entropy_bits >= 128)
crng_reseed(&primary_crng, true);
}
@@ -1238,7 +1238,7 @@ void add_input_randomness(unsigned int t
last_value = value;
add_timer_randomness(&input_timer_state,
(type << 4) ^ code ^ (code >> 4) ^ value);
- trace_add_input_randomness(ENTROPY_BITS());
+ trace_add_input_randomness(POOL_ENTROPY_BITS());
}
EXPORT_SYMBOL_GPL(add_input_randomness);
@@ -1337,7 +1337,7 @@ void add_disk_randomness(struct gendisk
return;
/* first major is 1, so we get >= 0x200 here */
add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
- trace_add_disk_randomness(disk_devt(disk), ENTROPY_BITS());
+ trace_add_disk_randomness(disk_devt(disk), POOL_ENTROPY_BITS());
}
EXPORT_SYMBOL_GPL(add_disk_randomness);
#endif
@@ -1364,7 +1364,7 @@ retry:
entropy_count = orig = READ_ONCE(input_pool.entropy_count);
ibytes = nbytes;
/* never pull more than available */
- have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
+ have_bytes = entropy_count >> (POOL_ENTROPY_SHIFT + 3);
if (have_bytes < 0)
have_bytes = 0;
@@ -1376,7 +1376,7 @@ retry:
pr_warn("negative entropy count: count %d\n", entropy_count);
entropy_count = 0;
}
- nfrac = ibytes << (ENTROPY_SHIFT + 3);
+ nfrac = ibytes << (POOL_ENTROPY_SHIFT + 3);
if ((size_t) entropy_count > nfrac)
entropy_count -= nfrac;
else
@@ -1386,7 +1386,7 @@ retry:
goto retry;
trace_debit_entropy(8 * ibytes);
- if (ibytes && ENTROPY_BITS() < random_write_wakeup_bits) {
+ if (ibytes && POOL_ENTROPY_BITS() < random_write_wakeup_bits) {
wake_up_interruptible(&random_write_wait);
kill_fasync(&fasync, SIGIO, POLL_OUT);
}
@@ -1474,7 +1474,7 @@ static ssize_t _extract_entropy(void *bu
*/
static ssize_t extract_entropy(void *buf, size_t nbytes, int min)
{
- trace_extract_entropy(nbytes, ENTROPY_BITS(), _RET_IP_);
+ trace_extract_entropy(nbytes, POOL_ENTROPY_BITS(), _RET_IP_);
nbytes = account(nbytes, min);
return _extract_entropy(buf, nbytes);
}
@@ -1799,9 +1799,9 @@ urandom_read_nowarn(struct file *file, c
{
int ret;
- nbytes = min_t(size_t, nbytes, INT_MAX >> (ENTROPY_SHIFT + 3));
+ nbytes = min_t(size_t, nbytes, INT_MAX >> (POOL_ENTROPY_SHIFT + 3));
ret = extract_crng_user(buf, nbytes);
- trace_urandom_read(8 * nbytes, 0, ENTROPY_BITS());
+ trace_urandom_read(8 * nbytes, 0, POOL_ENTROPY_BITS());
return ret;
}
@@ -1841,7 +1841,7 @@ random_poll(struct file *file, poll_tabl
mask = 0;
if (crng_ready())
mask |= POLLIN | POLLRDNORM;
- if (ENTROPY_BITS() < random_write_wakeup_bits)
+ if (POOL_ENTROPY_BITS() < random_write_wakeup_bits)
mask |= POLLOUT | POLLWRNORM;
return mask;
}
@@ -1897,7 +1897,7 @@ static long random_ioctl(struct file *f,
switch (cmd) {
case RNDGETENTCNT:
/* inherently racy, no point locking */
- ent_count = ENTROPY_BITS();
+ ent_count = POOL_ENTROPY_BITS();
if (put_user(ent_count, p))
return -EFAULT;
return 0;
@@ -2053,7 +2053,7 @@ static int proc_do_entropy(struct ctl_ta
struct ctl_table fake_table;
int entropy_count;
- entropy_count = *(int *)table->data >> ENTROPY_SHIFT;
+ entropy_count = *(int *)table->data >> POOL_ENTROPY_SHIFT;
fake_table.data = &entropy_count;
fake_table.maxlen = sizeof(entropy_count);
@@ -2272,7 +2272,7 @@ void add_hwgenerator_randomness(const ch
*/
wait_event_interruptible(random_write_wait,
!system_wq || kthread_should_stop() ||
- ENTROPY_BITS() <= random_write_wakeup_bits);
+ POOL_ENTROPY_BITS() <= random_write_wakeup_bits);
mix_pool_bytes(buffer, count);
credit_entropy_bits(entropy);
}
From: "Jason A. Donenfeld" <[email protected]>
commit 6c0eace6e1499712583b6ee62d95161e8b3449f5 upstream.
This gets rid of another abstraction we no longer need. It would be nice
if we could instead make pool an array rather than a pointer, but the
latent entropy plugin won't be able to do its magic in that case. So
instead we put all accesses to the input pool's actual data through the
input_pool_data array directly.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 217 +++++++++++++++++++++++---------------------------
1 file changed, 101 insertions(+), 116 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -124,7 +124,7 @@
*
* The primary kernel interface is
*
- * void get_random_bytes(void *buf, int nbytes);
+ * void get_random_bytes(void *buf, int nbytes);
*
* This interface will return the requested number of random bytes,
* and place it in the requested buffer. This is equivalent to a
@@ -132,10 +132,10 @@
*
* For less critical applications, there are the functions:
*
- * u32 get_random_u32()
- * u64 get_random_u64()
- * unsigned int get_random_int()
- * unsigned long get_random_long()
+ * u32 get_random_u32()
+ * u64 get_random_u64()
+ * unsigned int get_random_int()
+ * unsigned long get_random_long()
*
* These are produced by a cryptographic RNG seeded from get_random_bytes,
* and so do not deplete the entropy pool as much. These are recommended
@@ -197,10 +197,10 @@
* from the devices are:
*
* void add_device_randomness(const void *buf, unsigned int size);
- * void add_input_randomness(unsigned int type, unsigned int code,
+ * void add_input_randomness(unsigned int type, unsigned int code,
* unsigned int value);
* void add_interrupt_randomness(int irq);
- * void add_disk_randomness(struct gendisk *disk);
+ * void add_disk_randomness(struct gendisk *disk);
* void add_hwgenerator_randomness(const char *buffer, size_t count,
* size_t entropy);
* void add_bootloader_randomness(const void *buf, unsigned int size);
@@ -296,8 +296,8 @@
* /dev/random and /dev/urandom created already, they can be created
* by using the commands:
*
- * mknod /dev/random c 1 8
- * mknod /dev/urandom c 1 9
+ * mknod /dev/random c 1 8
+ * mknod /dev/urandom c 1 9
*
* Acknowledgements:
* =================
@@ -444,9 +444,9 @@ static DEFINE_SPINLOCK(random_ready_list
static LIST_HEAD(random_ready_list);
struct crng_state {
- u32 state[16];
- unsigned long init_time;
- spinlock_t lock;
+ u32 state[16];
+ unsigned long init_time;
+ spinlock_t lock;
};
static struct crng_state primary_crng = {
@@ -470,7 +470,7 @@ static bool crng_need_final_init = false
#define crng_ready() (likely(crng_init > 1))
static int crng_init_cnt = 0;
static unsigned long crng_global_init_time = 0;
-#define CRNG_INIT_CNT_THRESH (2*CHACHA20_KEY_SIZE)
+#define CRNG_INIT_CNT_THRESH (2 * CHACHA20_KEY_SIZE)
static void _extract_crng(struct crng_state *crng, u8 out[CHACHA20_BLOCK_SIZE]);
static void _crng_backtrack_protect(struct crng_state *crng,
u8 tmp[CHACHA20_BLOCK_SIZE], int used);
@@ -497,17 +497,12 @@ MODULE_PARM_DESC(ratelimit_disable, "Dis
static u32 input_pool_data[POOL_WORDS] __latent_entropy;
static struct {
- /* read-only data: */
- u32 *pool;
-
- /* read-write data: */
spinlock_t lock;
u16 add_ptr;
u16 input_rotate;
int entropy_count;
} input_pool = {
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
- .pool = input_pool_data
};
static ssize_t extract_entropy(void *buf, size_t nbytes, int min);
@@ -515,7 +510,7 @@ static ssize_t _extract_entropy(void *bu
static void crng_reseed(struct crng_state *crng, bool use_input_pool);
-static u32 const twist_table[8] = {
+static const u32 twist_table[8] = {
0x00000000, 0x3b6e20c8, 0x76dc4190, 0x4db26158,
0xedb88320, 0xd6d6a3e8, 0x9b64c2b0, 0xa00ae278 };
@@ -545,15 +540,15 @@ static void _mix_pool_bytes(const void *
i = (i - 1) & POOL_WORDMASK;
/* XOR in the various taps */
- w ^= input_pool.pool[i];
- w ^= input_pool.pool[(i + POOL_TAP1) & POOL_WORDMASK];
- w ^= input_pool.pool[(i + POOL_TAP2) & POOL_WORDMASK];
- w ^= input_pool.pool[(i + POOL_TAP3) & POOL_WORDMASK];
- w ^= input_pool.pool[(i + POOL_TAP4) & POOL_WORDMASK];
- w ^= input_pool.pool[(i + POOL_TAP5) & POOL_WORDMASK];
+ w ^= input_pool_data[i];
+ w ^= input_pool_data[(i + POOL_TAP1) & POOL_WORDMASK];
+ w ^= input_pool_data[(i + POOL_TAP2) & POOL_WORDMASK];
+ w ^= input_pool_data[(i + POOL_TAP3) & POOL_WORDMASK];
+ w ^= input_pool_data[(i + POOL_TAP4) & POOL_WORDMASK];
+ w ^= input_pool_data[(i + POOL_TAP5) & POOL_WORDMASK];
/* Mix the result back in with a twist */
- input_pool.pool[i] = (w >> 3) ^ twist_table[w & 7];
+ input_pool_data[i] = (w >> 3) ^ twist_table[w & 7];
/*
* Normally, we add 7 bits of rotation to the pool.
@@ -585,10 +580,10 @@ static void mix_pool_bytes(const void *i
}
struct fast_pool {
- u32 pool[4];
- unsigned long last;
- u16 reg_idx;
- u8 count;
+ u32 pool[4];
+ unsigned long last;
+ u16 reg_idx;
+ u8 count;
};
/*
@@ -716,7 +711,7 @@ static int credit_entropy_bits_safe(int
return -EINVAL;
/* Cap the value to avoid overflows */
- nbits = min(nbits, POOL_BITS);
+ nbits = min(nbits, POOL_BITS);
credit_entropy_bits(nbits);
return 0;
@@ -728,7 +723,7 @@ static int credit_entropy_bits_safe(int
*
*********************************************************************/
-#define CRNG_RESEED_INTERVAL (300*HZ)
+#define CRNG_RESEED_INTERVAL (300 * HZ)
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
@@ -751,9 +746,9 @@ early_param("random.trust_cpu", parse_tr
static bool crng_init_try_arch(struct crng_state *crng)
{
- int i;
- bool arch_init = true;
- unsigned long rv;
+ int i;
+ bool arch_init = true;
+ unsigned long rv;
for (i = 4; i < 16; i++) {
if (!arch_get_random_seed_long(&rv) &&
@@ -769,9 +764,9 @@ static bool crng_init_try_arch(struct cr
static bool __init crng_init_try_arch_early(struct crng_state *crng)
{
- int i;
- bool arch_init = true;
- unsigned long rv;
+ int i;
+ bool arch_init = true;
+ unsigned long rv;
for (i = 4; i < 16; i++) {
if (!arch_get_random_seed_long_early(&rv) &&
@@ -841,7 +836,7 @@ static void do_numa_crng_init(struct wor
struct crng_state *crng;
struct crng_state **pool;
- pool = kcalloc(nr_node_ids, sizeof(*pool), GFP_KERNEL|__GFP_NOFAIL);
+ pool = kcalloc(nr_node_ids, sizeof(*pool), GFP_KERNEL | __GFP_NOFAIL);
for_each_online_node(i) {
crng = kmalloc_node(sizeof(struct crng_state),
GFP_KERNEL | __GFP_NOFAIL, i);
@@ -897,7 +892,7 @@ static size_t crng_fast_load(const u8 *c
spin_unlock_irqrestore(&primary_crng.lock, flags);
return 0;
}
- p = (u8 *) &primary_crng.state[4];
+ p = (u8 *)&primary_crng.state[4];
while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
p[crng_init_cnt % CHACHA20_KEY_SIZE] ^= *cp;
cp++; crng_init_cnt++; len--; ret++;
@@ -978,12 +973,12 @@ static struct crng_state *select_crng(vo
*/
static int crng_slow_load(const u8 *cp, size_t len)
{
- unsigned long flags;
- static u8 lfsr = 1;
- u8 tmp;
- unsigned int i, max = CHACHA20_KEY_SIZE;
- const u8 * src_buf = cp;
- u8 * dest_buf = (u8 *) &primary_crng.state[4];
+ unsigned long flags;
+ static u8 lfsr = 1;
+ u8 tmp;
+ unsigned int i, max = CHACHA20_KEY_SIZE;
+ const u8 *src_buf = cp;
+ u8 *dest_buf = (u8 *)&primary_crng.state[4];
if (!spin_trylock_irqsave(&primary_crng.lock, flags))
return 0;
@@ -994,7 +989,7 @@ static int crng_slow_load(const u8 *cp,
if (len > max)
max = len;
- for (i = 0; i < max ; i++) {
+ for (i = 0; i < max; i++) {
tmp = lfsr;
lfsr >>= 1;
if (tmp & 1)
@@ -1009,11 +1004,11 @@ static int crng_slow_load(const u8 *cp,
static void crng_reseed(struct crng_state *crng, bool use_input_pool)
{
- unsigned long flags;
- int i, num;
+ unsigned long flags;
+ int i, num;
union {
- u8 block[CHACHA20_BLOCK_SIZE];
- u32 key[8];
+ u8 block[CHACHA20_BLOCK_SIZE];
+ u32 key[8];
} buf;
if (use_input_pool) {
@@ -1027,11 +1022,11 @@ static void crng_reseed(struct crng_stat
}
spin_lock_irqsave(&crng->lock, flags);
for (i = 0; i < 8; i++) {
- unsigned long rv;
+ unsigned long rv;
if (!arch_get_random_seed_long(&rv) &&
!arch_get_random_long(&rv))
rv = random_get_entropy();
- crng->state[i+4] ^= buf.key[i] ^ rv;
+ crng->state[i + 4] ^= buf.key[i] ^ rv;
}
memzero_explicit(&buf, sizeof(buf));
WRITE_ONCE(crng->init_time, jiffies);
@@ -1039,8 +1034,7 @@ static void crng_reseed(struct crng_stat
crng_finalize_init(crng);
}
-static void _extract_crng(struct crng_state *crng,
- u8 out[CHACHA20_BLOCK_SIZE])
+static void _extract_crng(struct crng_state *crng, u8 out[CHACHA20_BLOCK_SIZE])
{
unsigned long flags, init_time;
@@ -1069,9 +1063,9 @@ static void extract_crng(u8 out[CHACHA20
static void _crng_backtrack_protect(struct crng_state *crng,
u8 tmp[CHACHA20_BLOCK_SIZE], int used)
{
- unsigned long flags;
- u32 *s, *d;
- int i;
+ unsigned long flags;
+ u32 *s, *d;
+ int i;
used = round_up(used, sizeof(u32));
if (used + CHACHA20_KEY_SIZE > CHACHA20_BLOCK_SIZE) {
@@ -1079,9 +1073,9 @@ static void _crng_backtrack_protect(stru
used = 0;
}
spin_lock_irqsave(&crng->lock, flags);
- s = (u32 *) &tmp[used];
+ s = (u32 *)&tmp[used];
d = &crng->state[4];
- for (i=0; i < 8; i++)
+ for (i = 0; i < 8; i++)
*d++ ^= *s++;
spin_unlock_irqrestore(&crng->lock, flags);
}
@@ -1126,7 +1120,6 @@ static ssize_t extract_crng_user(void __
return ret;
}
-
/*********************************************************************
*
* Entropy input management
@@ -1221,11 +1214,11 @@ static void add_timer_randomness(struct
* Round down by 1 bit on general principles,
* and limit entropy estimate to 12 bits.
*/
- credit_entropy_bits(min_t(int, fls(delta>>1), 11));
+ credit_entropy_bits(min_t(int, fls(delta >> 1), 11));
}
void add_input_randomness(unsigned int type, unsigned int code,
- unsigned int value)
+ unsigned int value)
{
static unsigned char last_value;
@@ -1245,19 +1238,19 @@ static DEFINE_PER_CPU(struct fast_pool,
#ifdef ADD_INTERRUPT_BENCH
static unsigned long avg_cycles, avg_deviation;
-#define AVG_SHIFT 8 /* Exponential average factor k=1/256 */
-#define FIXED_1_2 (1 << (AVG_SHIFT-1))
+#define AVG_SHIFT 8 /* Exponential average factor k=1/256 */
+#define FIXED_1_2 (1 << (AVG_SHIFT - 1))
static void add_interrupt_bench(cycles_t start)
{
- long delta = random_get_entropy() - start;
+ long delta = random_get_entropy() - start;
- /* Use a weighted moving average */
- delta = delta - ((avg_cycles + FIXED_1_2) >> AVG_SHIFT);
- avg_cycles += delta;
- /* And average deviation */
- delta = abs(delta) - ((avg_deviation + FIXED_1_2) >> AVG_SHIFT);
- avg_deviation += delta;
+ /* Use a weighted moving average */
+ delta = delta - ((avg_cycles + FIXED_1_2) >> AVG_SHIFT);
+ avg_cycles += delta;
+ /* And average deviation */
+ delta = abs(delta) - ((avg_deviation + FIXED_1_2) >> AVG_SHIFT);
+ avg_deviation += delta;
}
#else
#define add_interrupt_bench(x)
@@ -1265,7 +1258,7 @@ static void add_interrupt_bench(cycles_t
static u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
{
- u32 *ptr = (u32 *) regs;
+ u32 *ptr = (u32 *)regs;
unsigned int idx;
if (regs == NULL)
@@ -1280,12 +1273,12 @@ static u32 get_reg(struct fast_pool *f,
void add_interrupt_randomness(int irq)
{
- struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
- struct pt_regs *regs = get_irq_regs();
- unsigned long now = jiffies;
- cycles_t cycles = random_get_entropy();
- u32 c_high, j_high;
- u64 ip;
+ struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
+ struct pt_regs *regs = get_irq_regs();
+ unsigned long now = jiffies;
+ cycles_t cycles = random_get_entropy();
+ u32 c_high, j_high;
+ u64 ip;
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
@@ -1295,8 +1288,8 @@ void add_interrupt_randomness(int irq)
fast_pool->pool[1] ^= now ^ c_high;
ip = regs ? instruction_pointer(regs) : _RET_IP_;
fast_pool->pool[2] ^= ip;
- fast_pool->pool[3] ^= (sizeof(ip) > 4) ? ip >> 32 :
- get_reg(fast_pool, regs);
+ fast_pool->pool[3] ^=
+ (sizeof(ip) > 4) ? ip >> 32 : get_reg(fast_pool, regs);
fast_mix(fast_pool);
add_interrupt_bench(cycles);
@@ -1310,8 +1303,7 @@ void add_interrupt_randomness(int irq)
return;
}
- if ((fast_pool->count < 64) &&
- !time_after(now, fast_pool->last + HZ))
+ if ((fast_pool->count < 64) && !time_after(now, fast_pool->last + HZ))
return;
if (!spin_trylock(&input_pool.lock))
@@ -1375,7 +1367,7 @@ retry:
entropy_count = 0;
}
nfrac = ibytes << (POOL_ENTROPY_SHIFT + 3);
- if ((size_t) entropy_count > nfrac)
+ if ((size_t)entropy_count > nfrac)
entropy_count -= nfrac;
else
entropy_count = 0;
@@ -1420,7 +1412,7 @@ static void extract_buf(u8 *out)
/* Generate a hash across the pool */
spin_lock_irqsave(&input_pool.lock, flags);
- blake2s_update(&state, (const u8 *)input_pool.pool, POOL_BYTES);
+ blake2s_update(&state, (const u8 *)input_pool_data, POOL_BYTES);
blake2s_final(&state, hash); /* final zeros out state */
/*
@@ -1478,10 +1470,9 @@ static ssize_t extract_entropy(void *buf
}
#define warn_unseeded_randomness(previous) \
- _warn_unseeded_randomness(__func__, (void *) _RET_IP_, (previous))
+ _warn_unseeded_randomness(__func__, (void *)_RET_IP_, (previous))
-static void _warn_unseeded_randomness(const char *func_name, void *caller,
- void **previous)
+static void _warn_unseeded_randomness(const char *func_name, void *caller, void **previous)
{
#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM
const bool print_once = false;
@@ -1489,8 +1480,7 @@ static void _warn_unseeded_randomness(co
static bool print_once __read_mostly;
#endif
- if (print_once ||
- crng_ready() ||
+ if (print_once || crng_ready() ||
(previous && (caller == READ_ONCE(*previous))))
return;
WRITE_ONCE(*previous, caller);
@@ -1542,7 +1532,6 @@ void get_random_bytes(void *buf, int nby
}
EXPORT_SYMBOL(get_random_bytes);
-
/*
* Each time the timer fires, we expect that we got an unpredictable
* jump in the cycle counter. Even if the timer is running on another
@@ -1581,7 +1570,7 @@ static void try_to_generate_entropy(void
__setup_timer_on_stack(&stack.timer, entropy_timer, 0, 0);
while (!crng_ready()) {
if (!timer_pending(&stack.timer))
- mod_timer(&stack.timer, jiffies+1);
+ mod_timer(&stack.timer, jiffies + 1);
mix_pool_bytes(&stack.now, sizeof(stack.now));
schedule();
stack.now = random_get_entropy();
@@ -1791,9 +1780,8 @@ void rand_initialize_disk(struct gendisk
}
#endif
-static ssize_t
-urandom_read_nowarn(struct file *file, char __user *buf, size_t nbytes,
- loff_t *ppos)
+static ssize_t urandom_read_nowarn(struct file *file, char __user *buf,
+ size_t nbytes, loff_t *ppos)
{
int ret;
@@ -1803,8 +1791,8 @@ urandom_read_nowarn(struct file *file, c
return ret;
}
-static ssize_t
-urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
+static ssize_t urandom_read(struct file *file, char __user *buf, size_t nbytes,
+ loff_t *ppos)
{
static int maxwarn = 10;
@@ -1818,8 +1806,8 @@ urandom_read(struct file *file, char __u
return urandom_read_nowarn(file, buf, nbytes, ppos);
}
-static ssize_t
-random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
+static ssize_t random_read(struct file *file, char __user *buf, size_t nbytes,
+ loff_t *ppos)
{
int ret;
@@ -1829,8 +1817,7 @@ random_read(struct file *file, char __us
return urandom_read_nowarn(file, buf, nbytes, ppos);
}
-static unsigned int
-random_poll(struct file *file, poll_table * wait)
+static unsigned int random_poll(struct file *file, poll_table *wait)
{
unsigned int mask;
@@ -1844,8 +1831,7 @@ random_poll(struct file *file, poll_tabl
return mask;
}
-static int
-write_pool(const char __user *buffer, size_t count)
+static int write_pool(const char __user *buffer, size_t count)
{
size_t bytes;
u32 t, buf[16];
@@ -1947,35 +1933,35 @@ static int random_fasync(int fd, struct
}
const struct file_operations random_fops = {
- .read = random_read,
+ .read = random_read,
.write = random_write,
- .poll = random_poll,
+ .poll = random_poll,
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
.llseek = noop_llseek,
};
const struct file_operations urandom_fops = {
- .read = urandom_read,
+ .read = urandom_read,
.write = random_write,
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
.llseek = noop_llseek,
};
-SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count,
- unsigned int, flags)
+SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count, unsigned int,
+ flags)
{
int ret;
- if (flags & ~(GRND_NONBLOCK|GRND_RANDOM|GRND_INSECURE))
+ if (flags & ~(GRND_NONBLOCK | GRND_RANDOM | GRND_INSECURE))
return -EINVAL;
/*
* Requesting insecure and blocking randomness at the same time makes
* no sense.
*/
- if ((flags & (GRND_INSECURE|GRND_RANDOM)) == (GRND_INSECURE|GRND_RANDOM))
+ if ((flags & (GRND_INSECURE | GRND_RANDOM)) == (GRND_INSECURE | GRND_RANDOM))
return -EINVAL;
if (count > INT_MAX)
@@ -2123,7 +2109,7 @@ struct ctl_table random_table[] = {
#endif
{ }
};
-#endif /* CONFIG_SYSCTL */
+#endif /* CONFIG_SYSCTL */
struct batched_entropy {
union {
@@ -2143,7 +2129,7 @@ struct batched_entropy {
* point prior.
*/
static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
- .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
+ .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
};
u64 get_random_u64(void)
@@ -2168,7 +2154,7 @@ u64 get_random_u64(void)
EXPORT_SYMBOL(get_random_u64);
static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
- .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock),
+ .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock),
};
u32 get_random_u32(void)
{
@@ -2200,7 +2186,7 @@ static void invalidate_batched_entropy(v
int cpu;
unsigned long flags;
- for_each_possible_cpu (cpu) {
+ for_each_possible_cpu(cpu) {
struct batched_entropy *batched_entropy;
batched_entropy = per_cpu_ptr(&batched_entropy_u32, cpu);
@@ -2229,8 +2215,7 @@ static void invalidate_batched_entropy(v
* Return: A page aligned address within [start, start + range). On error,
* @start is returned.
*/
-unsigned long
-randomize_page(unsigned long start, unsigned long range)
+unsigned long randomize_page(unsigned long start, unsigned long range)
{
if (!PAGE_ALIGNED(start)) {
range -= PAGE_ALIGN(start) - start;
From: "Jason A. Donenfeld" <[email protected]>
commit dd7aa36e535797926d8eb311da7151919130139d upstream.
The comment about get_random_{u32,u64}() not invoking reseeding got
added in an unrelated commit, that then was recently reverted by
0313bc278dac ("Revert "random: block in /dev/urandom""). So this adds
that little comment snippet back, and improves the wording a bit too.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -227,9 +227,10 @@ static void _warn_unseeded_randomness(co
*
* These interfaces will return the requested number of random bytes
* into the given buffer or as a return value. This is equivalent to
- * a read from /dev/urandom. The integer family of functions may be
- * higher performance for one-off random integers, because they do a
- * bit of buffering.
+ * a read from /dev/urandom. The u32, u64, int, and long family of
+ * functions may be higher performance for one-off random integers,
+ * because they do a bit of buffering and do not invoke reseeding
+ * until the buffer is emptied.
*
*********************************************************************/
From: Chen Lin <[email protected]>
[ Upstream commit 2f2c0d2919a14002760f89f4e02960c735a316d2 ]
When rx_flag == MTK_RX_FLAGS_HWLRO,
rx_data_len = MTK_MAX_LRO_RX_LENGTH(4096 * 3) > PAGE_SIZE.
netdev_alloc_frag is for alloction of page fragment only.
Reference to other drivers and Documentation/vm/page_frags.rst
Branch to use __get_free_pages when ring->frag_size > PAGE_SIZE.
Signed-off-by: Chen Lin <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/net/ethernet/mediatek/mtk_eth_soc.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
index 84d667957221..61a9b60ab022 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
@@ -527,6 +527,17 @@ static inline void mtk_rx_get_desc(struct mtk_rx_dma *rxd,
rxd->rxd4 = READ_ONCE(dma_rxd->rxd4);
}
+static void *mtk_max_lro_buf_alloc(gfp_t gfp_mask)
+{
+ unsigned int size = mtk_max_frag_size(MTK_MAX_LRO_RX_LENGTH);
+ unsigned long data;
+
+ data = __get_free_pages(gfp_mask | __GFP_COMP | __GFP_NOWARN,
+ get_order(size));
+
+ return (void *)data;
+}
+
/* the qdma core needs scratch memory to be setup */
static int mtk_init_fq_dma(struct mtk_eth *eth)
{
@@ -928,7 +939,10 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget,
goto release_desc;
/* alloc new buffer */
- new_data = napi_alloc_frag(ring->frag_size);
+ if (ring->frag_size <= PAGE_SIZE)
+ new_data = napi_alloc_frag(ring->frag_size);
+ else
+ new_data = mtk_max_lro_buf_alloc(GFP_ATOMIC);
if (unlikely(!new_data)) {
netdev->stats.rx_dropped++;
goto release_desc;
@@ -1231,7 +1245,10 @@ static int mtk_rx_alloc(struct mtk_eth *eth, int ring_no, int rx_flag)
return -ENOMEM;
for (i = 0; i < rx_dma_size; i++) {
- ring->data[i] = netdev_alloc_frag(ring->frag_size);
+ if (ring->frag_size <= PAGE_SIZE)
+ ring->data[i] = netdev_alloc_frag(ring->frag_size);
+ else
+ ring->data[i] = mtk_max_lro_buf_alloc(GFP_KERNEL);
if (!ring->data[i])
return -ENOMEM;
}
--
2.35.1
From: Willy Tarreau <[email protected]>
commit b2d057560b8107c633b39aabe517ff9d93f285e3 upstream.
SipHash replaced MD5 in secure_ipv{4,6}_port_ephemeral() via commit
7cd23e5300c1 ("secure_seq: use SipHash in place of MD5"), but the output
remained truncated to 32-bit only. In order to exploit more bits from the
hash, let's make the functions return the full 64-bit of siphash_3u32().
We also make sure the port offset calculation in __inet_hash_connect()
remains done on 32-bit to avoid the need for div_u64_rem() and an extra
cost on 32-bit systems.
Cc: Jason A. Donenfeld <[email protected]>
Cc: Moshe Kol <[email protected]>
Cc: Yossi Gilad <[email protected]>
Cc: Amit Klein <[email protected]>
Reviewed-by: Eric Dumazet <[email protected]>
Signed-off-by: Willy Tarreau <[email protected]>
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/net/inet_hashtables.h | 2 +-
include/net/secure_seq.h | 4 ++--
net/core/secure_seq.c | 4 ++--
net/ipv4/inet_hashtables.c | 10 ++++++----
net/ipv6/inet6_hashtables.c | 4 ++--
5 files changed, 13 insertions(+), 11 deletions(-)
--- a/include/net/inet_hashtables.h
+++ b/include/net/inet_hashtables.h
@@ -382,7 +382,7 @@ static inline void sk_rcv_saddr_set(stru
}
int __inet_hash_connect(struct inet_timewait_death_row *death_row,
- struct sock *sk, u32 port_offset,
+ struct sock *sk, u64 port_offset,
int (*check_established)(struct inet_timewait_death_row *,
struct sock *, __u16,
struct inet_timewait_sock **));
--- a/include/net/secure_seq.h
+++ b/include/net/secure_seq.h
@@ -3,8 +3,8 @@
#include <linux/types.h>
-u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport);
-u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
+u64 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport);
+u64 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
__be16 dport);
__u32 secure_tcp_sequence_number(__be32 saddr, __be32 daddr,
__be16 sport, __be16 dport);
--- a/net/core/secure_seq.c
+++ b/net/core/secure_seq.c
@@ -62,7 +62,7 @@ __u32 secure_tcpv6_sequence_number(const
}
EXPORT_SYMBOL(secure_tcpv6_sequence_number);
-u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
+u64 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
__be16 dport)
{
u32 secret[MD5_MESSAGE_BYTES / 4];
@@ -102,7 +102,7 @@ __u32 secure_tcp_sequence_number(__be32
return seq_scale(hash[0]);
}
-u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport)
+u64 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport)
{
u32 hash[MD5_DIGEST_WORDS];
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -382,7 +382,7 @@ not_unique:
return -EADDRNOTAVAIL;
}
-static u32 inet_sk_port_offset(const struct sock *sk)
+static u64 inet_sk_port_offset(const struct sock *sk)
{
const struct inet_sock *inet = inet_sk(sk);
@@ -549,7 +549,7 @@ EXPORT_SYMBOL_GPL(inet_unhash);
static u32 table_perturb[1 << INET_TABLE_PERTURB_SHIFT];
int __inet_hash_connect(struct inet_timewait_death_row *death_row,
- struct sock *sk, u32 port_offset,
+ struct sock *sk, u64 port_offset,
int (*check_established)(struct inet_timewait_death_row *,
struct sock *, __u16, struct inet_timewait_sock **))
{
@@ -589,7 +589,9 @@ int __inet_hash_connect(struct inet_time
net_get_random_once(table_perturb, sizeof(table_perturb));
index = hash_32(port_offset, INET_TABLE_PERTURB_SHIFT);
- offset = (READ_ONCE(table_perturb[index]) + port_offset) % remaining;
+ offset = READ_ONCE(table_perturb[index]) + port_offset;
+ offset %= remaining;
+
/* In first pass we try ports of @low parity.
* inet_csk_get_port() does the opposite choice.
*/
@@ -670,7 +672,7 @@ ok:
int inet_hash_connect(struct inet_timewait_death_row *death_row,
struct sock *sk)
{
- u32 port_offset = 0;
+ u64 port_offset = 0;
if (!inet_sk(sk)->inet_num)
port_offset = inet_sk_port_offset(sk);
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -242,7 +242,7 @@ not_unique:
return -EADDRNOTAVAIL;
}
-static u32 inet6_sk_port_offset(const struct sock *sk)
+static u64 inet6_sk_port_offset(const struct sock *sk)
{
const struct inet_sock *inet = inet_sk(sk);
@@ -254,7 +254,7 @@ static u32 inet6_sk_port_offset(const st
int inet6_hash_connect(struct inet_timewait_death_row *death_row,
struct sock *sk)
{
- u32 port_offset = 0;
+ u64 port_offset = 0;
if (!inet_sk(sk)->inet_num)
port_offset = inet6_sk_port_offset(sk);
From: "Jason A. Donenfeld" <[email protected]>
commit 58340f8e952b613e0ead0bed58b97b05bf4743c5 upstream.
On PREEMPT_RT, it's problematic to take spinlocks from hard irq
handlers. We can fix this by deferring to a workqueue the dumping of
the fast pool into the input pool.
We accomplish this with some careful rules on fast_pool->count:
- When it's incremented to >= 64, we schedule the work.
- If the top bit is set, we never schedule the work, even if >= 64.
- The worker is responsible for setting it back to 0 when it's done.
There are two small issues around using workqueues for this purpose that
we work around.
The first issue is that mix_interrupt_randomness() might be migrated to
another CPU during CPU hotplug. This issue is rectified by checking that
it hasn't been migrated (after disabling irqs). If it has been migrated,
then we set the count to zero, so that when the CPU comes online again,
it can requeue the work. As part of this, we switch to using an
atomic_t, so that the increment in the irq handler doesn't wipe out the
zeroing if the CPU comes back online while this worker is running.
The second issue is that, though relatively minor in effect, we probably
want to make sure we get a consistent view of the pool onto the stack,
in case it's interrupted by an irq while reading. To do this, we don't
reenable irqs until after the copy. There are only 18 instructions
between the cli and sti, so this is a pretty tiny window.
Cc: Thomas Gleixner <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Cc: Jonathan Neuschäfer <[email protected]>
Acked-by: Sebastian Andrzej Siewior <[email protected]>
Reviewed-by: Sultan Alsawaf <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 63 ++++++++++++++++++++++++++++++++++++++------------
1 file changed, 49 insertions(+), 14 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1174,9 +1174,10 @@ struct fast_pool {
u32 pool32[4];
u64 pool64[2];
};
+ struct work_struct mix;
unsigned long last;
+ atomic_t count;
u16 reg_idx;
- u8 count;
};
/*
@@ -1226,12 +1227,49 @@ static u32 get_reg(struct fast_pool *f,
return *ptr;
}
+static void mix_interrupt_randomness(struct work_struct *work)
+{
+ struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix);
+ u32 pool[4];
+
+ /* Check to see if we're running on the wrong CPU due to hotplug. */
+ local_irq_disable();
+ if (fast_pool != this_cpu_ptr(&irq_randomness)) {
+ local_irq_enable();
+ /*
+ * If we are unlucky enough to have been moved to another CPU,
+ * during CPU hotplug while the CPU was shutdown then we set
+ * our count to zero atomically so that when the CPU comes
+ * back online, it can enqueue work again. The _release here
+ * pairs with the atomic_inc_return_acquire in
+ * add_interrupt_randomness().
+ */
+ atomic_set_release(&fast_pool->count, 0);
+ return;
+ }
+
+ /*
+ * Copy the pool to the stack so that the mixer always has a
+ * consistent view, before we reenable irqs again.
+ */
+ memcpy(pool, fast_pool->pool32, sizeof(pool));
+ atomic_set(&fast_pool->count, 0);
+ fast_pool->last = jiffies;
+ local_irq_enable();
+
+ mix_pool_bytes(pool, sizeof(pool));
+ credit_entropy_bits(1);
+ memzero_explicit(pool, sizeof(pool));
+}
+
void add_interrupt_randomness(int irq)
{
+ enum { MIX_INFLIGHT = 1U << 31 };
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
struct pt_regs *regs = get_irq_regs();
unsigned long now = jiffies;
cycles_t cycles = random_get_entropy();
+ unsigned int new_count;
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
@@ -1251,12 +1289,13 @@ void add_interrupt_randomness(int irq)
}
fast_mix(fast_pool->pool32);
- ++fast_pool->count;
+ /* The _acquire here pairs with the atomic_set_release in mix_interrupt_randomness(). */
+ new_count = (unsigned int)atomic_inc_return_acquire(&fast_pool->count);
if (unlikely(crng_init == 0)) {
- if (fast_pool->count >= 64 &&
+ if (new_count >= 64 &&
crng_fast_load(fast_pool->pool32, sizeof(fast_pool->pool32)) > 0) {
- fast_pool->count = 0;
+ atomic_set(&fast_pool->count, 0);
fast_pool->last = now;
if (spin_trylock(&input_pool.lock)) {
_mix_pool_bytes(&fast_pool->pool32, sizeof(fast_pool->pool32));
@@ -1266,20 +1305,16 @@ void add_interrupt_randomness(int irq)
return;
}
- if ((fast_pool->count < 64) && !time_after(now, fast_pool->last + HZ))
+ if (new_count & MIX_INFLIGHT)
return;
- if (!spin_trylock(&input_pool.lock))
+ if (new_count < 64 && !time_after(now, fast_pool->last + HZ))
return;
- fast_pool->last = now;
- _mix_pool_bytes(&fast_pool->pool32, sizeof(fast_pool->pool32));
- spin_unlock(&input_pool.lock);
-
- fast_pool->count = 0;
-
- /* Award one bit for the contents of the fast pool. */
- credit_entropy_bits(1);
+ if (unlikely(!fast_pool->mix.func))
+ INIT_WORK(&fast_pool->mix, mix_interrupt_randomness);
+ atomic_or(MIX_INFLIGHT, &fast_pool->count);
+ queue_work_on(raw_smp_processor_id(), system_highpri_wq, &fast_pool->mix);
}
EXPORT_SYMBOL_GPL(add_interrupt_randomness);
From: "Jason A. Donenfeld" <[email protected]>
commit b0c3e796f24b588b862b61ce235d3c9417dc8983 upstream.
Some implementations were returning type `unsigned long`, while others
that fell back to get_cycles() were implicitly returning a `cycles_t` or
an untyped constant int literal. That makes for weird and confusing
code, and basically all code in the kernel already handled it like it
was an `unsigned long`. I recently tried to handle it as the largest
type it could be, a `cycles_t`, but doing so doesn't really help with
much.
Instead let's just make random_get_entropy() return an unsigned long all
the time. This also matches the commonly used `arch_get_random_long()`
function, so now RDRAND and RDTSC return the same sized integer, which
means one can fallback to the other more gracefully.
Cc: Dominik Brodowski <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Acked-by: Thomas Gleixner <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 20 +++++++-------------
include/linux/timex.h | 2 +-
2 files changed, 8 insertions(+), 14 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1011,7 +1011,7 @@ int __init rand_initialize(void)
*/
void add_device_randomness(const void *buf, size_t size)
{
- cycles_t cycles = random_get_entropy();
+ unsigned long cycles = random_get_entropy();
unsigned long flags, now = jiffies;
if (crng_init == 0 && size)
@@ -1042,8 +1042,7 @@ struct timer_rand_state {
*/
static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
{
- cycles_t cycles = random_get_entropy();
- unsigned long flags, now = jiffies;
+ unsigned long cycles = random_get_entropy(), now = jiffies, flags;
long delta, delta2, delta3;
spin_lock_irqsave(&input_pool.lock, flags);
@@ -1298,8 +1297,7 @@ static void mix_interrupt_randomness(str
void add_interrupt_randomness(int irq)
{
enum { MIX_INFLIGHT = 1U << 31 };
- cycles_t cycles = random_get_entropy();
- unsigned long now = jiffies;
+ unsigned long cycles = random_get_entropy(), now = jiffies;
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
struct pt_regs *regs = get_irq_regs();
unsigned int new_count;
@@ -1312,16 +1310,12 @@ void add_interrupt_randomness(int irq)
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
- if (sizeof(cycles) == 8)
+ if (sizeof(unsigned long) == 8) {
irq_data.u64[0] = cycles ^ rol64(now, 32) ^ irq;
- else {
+ irq_data.u64[1] = regs ? instruction_pointer(regs) : _RET_IP_;
+ } else {
irq_data.u32[0] = cycles ^ irq;
irq_data.u32[1] = now;
- }
-
- if (sizeof(unsigned long) == 8)
- irq_data.u64[1] = regs ? instruction_pointer(regs) : _RET_IP_;
- else {
irq_data.u32[2] = regs ? instruction_pointer(regs) : _RET_IP_;
irq_data.u32[3] = get_reg(fast_pool, regs);
}
@@ -1368,7 +1362,7 @@ static void entropy_timer(unsigned long
static void try_to_generate_entropy(void)
{
struct {
- cycles_t cycles;
+ unsigned long cycles;
struct timer_list timer;
} stack;
--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -75,7 +75,7 @@
* By default we use get_cycles() for this purpose, but individual
* architectures may override this in their asm/timex.h header file.
*/
-#define random_get_entropy() get_cycles()
+#define random_get_entropy() ((unsigned long)get_cycles())
#endif
/*
From: "Jason A. Donenfeld" <[email protected]>
commit a5ed7cb1a7732ef11959332d507889fbc39ebbb4 upstream.
This pulls all of the entropy extraction-focused functions into the
third labeled section.
No functional changes.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 216 +++++++++++++++++++++++++-------------------------
1 file changed, 109 insertions(+), 107 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -891,23 +891,36 @@ size_t __must_check get_random_bytes_arc
}
EXPORT_SYMBOL(get_random_bytes_arch);
+
+/**********************************************************************
+ *
+ * Entropy accumulation and extraction routines.
+ *
+ * Callers may add entropy via:
+ *
+ * static void mix_pool_bytes(const void *in, size_t nbytes)
+ *
+ * After which, if added entropy should be credited:
+ *
+ * static void credit_entropy_bits(size_t nbits)
+ *
+ * Finally, extract entropy via these two, with the latter one
+ * setting the entropy count to zero and extracting only if there
+ * is POOL_MIN_BITS entropy credited prior:
+ *
+ * static void extract_entropy(void *buf, size_t nbytes)
+ * static bool drain_entropy(void *buf, size_t nbytes)
+ *
+ **********************************************************************/
+
enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
POOL_MIN_BITS = POOL_BITS /* No point in settling for less. */
};
-/*
- * Static global variables
- */
+/* For notifying userspace should write into /dev/random. */
static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
-/**********************************************************************
- *
- * OS independent entropy store. Here are the functions which handle
- * storing entropy in an entropy pool.
- *
- **********************************************************************/
-
static struct {
struct blake2s_state hash;
spinlock_t lock;
@@ -920,28 +933,106 @@ static struct {
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
};
-static void extract_entropy(void *buf, size_t nbytes);
-static bool drain_entropy(void *buf, size_t nbytes);
-
-static void crng_reseed(void);
+static void _mix_pool_bytes(const void *in, size_t nbytes)
+{
+ blake2s_update(&input_pool.hash, in, nbytes);
+}
/*
* This function adds bytes into the entropy "pool". It does not
* update the entropy estimate. The caller should call
* credit_entropy_bits if this is appropriate.
*/
-static void _mix_pool_bytes(const void *in, size_t nbytes)
+static void mix_pool_bytes(const void *in, size_t nbytes)
{
- blake2s_update(&input_pool.hash, in, nbytes);
+ unsigned long flags;
+
+ spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(in, nbytes);
+ spin_unlock_irqrestore(&input_pool.lock, flags);
}
-static void mix_pool_bytes(const void *in, size_t nbytes)
+static void credit_entropy_bits(size_t nbits)
+{
+ unsigned int entropy_count, orig, add;
+
+ if (!nbits)
+ return;
+
+ add = min_t(size_t, nbits, POOL_BITS);
+
+ do {
+ orig = READ_ONCE(input_pool.entropy_count);
+ entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
+ } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
+
+ if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
+ crng_reseed();
+}
+
+/*
+ * This is an HKDF-like construction for using the hashed collected entropy
+ * as a PRF key, that's then expanded block-by-block.
+ */
+static void extract_entropy(void *buf, size_t nbytes)
{
unsigned long flags;
+ u8 seed[BLAKE2S_HASH_SIZE], next_key[BLAKE2S_HASH_SIZE];
+ struct {
+ unsigned long rdseed[32 / sizeof(long)];
+ size_t counter;
+ } block;
+ size_t i;
+
+ for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) {
+ if (!arch_get_random_seed_long(&block.rdseed[i]) &&
+ !arch_get_random_long(&block.rdseed[i]))
+ block.rdseed[i] = random_get_entropy();
+ }
spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(in, nbytes);
+
+ /* seed = HASHPRF(last_key, entropy_input) */
+ blake2s_final(&input_pool.hash, seed);
+
+ /* next_key = HASHPRF(seed, RDSEED || 0) */
+ block.counter = 0;
+ blake2s(next_key, (u8 *)&block, seed, sizeof(next_key), sizeof(block), sizeof(seed));
+ blake2s_init_key(&input_pool.hash, BLAKE2S_HASH_SIZE, next_key, sizeof(next_key));
+
spin_unlock_irqrestore(&input_pool.lock, flags);
+ memzero_explicit(next_key, sizeof(next_key));
+
+ while (nbytes) {
+ i = min_t(size_t, nbytes, BLAKE2S_HASH_SIZE);
+ /* output = HASHPRF(seed, RDSEED || ++counter) */
+ ++block.counter;
+ blake2s(buf, (u8 *)&block, seed, i, sizeof(block), sizeof(seed));
+ nbytes -= i;
+ buf += i;
+ }
+
+ memzero_explicit(seed, sizeof(seed));
+ memzero_explicit(&block, sizeof(block));
+}
+
+/*
+ * First we make sure we have POOL_MIN_BITS of entropy in the pool, and then we
+ * set the entropy count to zero (but don't actually touch any data). Only then
+ * can we extract a new key with extract_entropy().
+ */
+static bool drain_entropy(void *buf, size_t nbytes)
+{
+ unsigned int entropy_count;
+ do {
+ entropy_count = READ_ONCE(input_pool.entropy_count);
+ if (entropy_count < POOL_MIN_BITS)
+ return false;
+ } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
+ extract_entropy(buf, nbytes);
+ wake_up_interruptible(&random_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
+ return true;
}
struct fast_pool {
@@ -984,24 +1075,6 @@ static void fast_mix(u32 pool[4])
pool[2] = c; pool[3] = d;
}
-static void credit_entropy_bits(size_t nbits)
-{
- unsigned int entropy_count, orig, add;
-
- if (!nbits)
- return;
-
- add = min_t(size_t, nbits, POOL_BITS);
-
- do {
- orig = READ_ONCE(input_pool.entropy_count);
- entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
- } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
-
- if (crng_init < 2 && entropy_count >= POOL_MIN_BITS)
- crng_reseed();
-}
-
/*********************************************************************
*
* Entropy input management
@@ -1198,77 +1271,6 @@ void add_disk_randomness(struct gendisk
EXPORT_SYMBOL_GPL(add_disk_randomness);
#endif
-/*********************************************************************
- *
- * Entropy extraction routines
- *
- *********************************************************************/
-
-/*
- * This is an HKDF-like construction for using the hashed collected entropy
- * as a PRF key, that's then expanded block-by-block.
- */
-static void extract_entropy(void *buf, size_t nbytes)
-{
- unsigned long flags;
- u8 seed[BLAKE2S_HASH_SIZE], next_key[BLAKE2S_HASH_SIZE];
- struct {
- unsigned long rdseed[32 / sizeof(long)];
- size_t counter;
- } block;
- size_t i;
-
- for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) {
- if (!arch_get_random_seed_long(&block.rdseed[i]) &&
- !arch_get_random_long(&block.rdseed[i]))
- block.rdseed[i] = random_get_entropy();
- }
-
- spin_lock_irqsave(&input_pool.lock, flags);
-
- /* seed = HASHPRF(last_key, entropy_input) */
- blake2s_final(&input_pool.hash, seed);
-
- /* next_key = HASHPRF(seed, RDSEED || 0) */
- block.counter = 0;
- blake2s(next_key, (u8 *)&block, seed, sizeof(next_key), sizeof(block), sizeof(seed));
- blake2s_init_key(&input_pool.hash, BLAKE2S_HASH_SIZE, next_key, sizeof(next_key));
-
- spin_unlock_irqrestore(&input_pool.lock, flags);
- memzero_explicit(next_key, sizeof(next_key));
-
- while (nbytes) {
- i = min_t(size_t, nbytes, BLAKE2S_HASH_SIZE);
- /* output = HASHPRF(seed, RDSEED || ++counter) */
- ++block.counter;
- blake2s(buf, (u8 *)&block, seed, i, sizeof(block), sizeof(seed));
- nbytes -= i;
- buf += i;
- }
-
- memzero_explicit(seed, sizeof(seed));
- memzero_explicit(&block, sizeof(block));
-}
-
-/*
- * First we make sure we have POOL_MIN_BITS of entropy in the pool, and then we
- * set the entropy count to zero (but don't actually touch any data). Only then
- * can we extract a new key with extract_entropy().
- */
-static bool drain_entropy(void *buf, size_t nbytes)
-{
- unsigned int entropy_count;
- do {
- entropy_count = READ_ONCE(input_pool.entropy_count);
- if (entropy_count < POOL_MIN_BITS)
- return false;
- } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
- extract_entropy(buf, nbytes);
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
- return true;
-}
-
/*
* Each time the timer fires, we expect that we got an unpredictable
* jump in the cycle counter. Even if the timer is running on another
From: "Jason A. Donenfeld" <[email protected]>
commit 8a5b8a4a4ceb353b4dd5bafd09e2b15751bcdb51 upstream.
This expands to exactly the same code that it replaces, but makes things
consistent by using the same macro for jiffy comparisons throughout.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -325,7 +325,7 @@ static bool crng_has_old_seed(void)
interval = max_t(unsigned int, CRNG_RESEED_START_INTERVAL,
(unsigned int)uptime / 2 * HZ);
}
- return time_after(jiffies, READ_ONCE(base_crng.birth) + interval);
+ return time_is_before_jiffies(READ_ONCE(base_crng.birth) + interval);
}
/*
From: Andy Lutomirski <[email protected]>
commit 48446f198f9adcb499b30332488dfd5bc3f176f6 upstream.
The separate blocking pool is going away. Start by ignoring
GRND_RANDOM in getentropy(2).
This should not materially break any API. Any code that worked
without this change should work at least as well with this change.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/705c5a091b63cc5da70c99304bb97e0109be0a26.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 3 ---
include/uapi/linux/random.h | 2 +-
2 files changed, 1 insertion(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2200,9 +2200,6 @@ SYSCALL_DEFINE3(getrandom, char __user *
if (count > INT_MAX)
count = INT_MAX;
- if (flags & GRND_RANDOM)
- return _random_read(flags & GRND_NONBLOCK, buf, count);
-
if (!(flags & GRND_INSECURE) && !crng_ready()) {
if (flags & GRND_NONBLOCK)
return -EAGAIN;
--- a/include/uapi/linux/random.h
+++ b/include/uapi/linux/random.h
@@ -47,7 +47,7 @@ struct rand_pool_info {
* Flags for getrandom(2)
*
* GRND_NONBLOCK Don't block and return EAGAIN instead
- * GRND_RANDOM Use the /dev/random pool instead of /dev/urandom
+ * GRND_RANDOM No effect
* GRND_INSECURE Return non-cryptographic random bytes
*/
#define GRND_NONBLOCK 0x0001
From: Jens Axboe <[email protected]>
commit 79025e727a846be6fd215ae9cdb654368ac3f9a6 upstream.
Now that random/urandom is using {read,write}_iter, we can wire it up to
using the generic splice handlers.
Fixes: 36e2c7421f02 ("fs: don't allow splice read/write without explicit ops")
Signed-off-by: Jens Axboe <[email protected]>
[Jason: added the splice_write path. Note that sendfile() and such still
does not work for read, though it does for write, because of a file
type restriction in splice_direct_to_actor(), which I'll address
separately.]
Cc: Al Viro <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 ++++
1 file changed, 4 insertions(+)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1382,6 +1382,8 @@ const struct file_operations random_fops
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
.llseek = noop_llseek,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
};
const struct file_operations urandom_fops = {
@@ -1390,6 +1392,8 @@ const struct file_operations urandom_fop
.unlocked_ioctl = random_ioctl,
.fasync = random_fasync,
.llseek = noop_llseek,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
};
From: "Jason A. Donenfeld" <[email protected]>
commit 5f75d9f3babea8ae0a2d06724656874f41d317f5 upstream.
Now that we've re-documented the various sections, we can remove the
outdated text here and replace it with a high-level overview.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 179 +++++---------------------------------------------
1 file changed, 19 insertions(+), 160 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2,168 +2,27 @@
/*
* Copyright (C) 2017-2022 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
* Copyright Matt Mackall <[email protected]>, 2003, 2004, 2005
- * Copyright Theodore Ts'o, 1994, 1995, 1996, 1997, 1998, 1999. All
- * rights reserved.
- */
-
-/*
- * Exported interfaces ---- output
- * ===============================
- *
- * There are four exported interfaces; two for use within the kernel,
- * and two for use from userspace.
- *
- * Exported interfaces ---- userspace output
- * -----------------------------------------
- *
- * The userspace interfaces are two character devices /dev/random and
- * /dev/urandom. /dev/random is suitable for use when very high
- * quality randomness is desired (for example, for key generation or
- * one-time pads), as it will only return a maximum of the number of
- * bits of randomness (as estimated by the random number generator)
- * contained in the entropy pool.
- *
- * The /dev/urandom device does not have this limit, and will return
- * as many bytes as are requested. As more and more random bytes are
- * requested without giving time for the entropy pool to recharge,
- * this will result in random numbers that are merely cryptographically
- * strong. For many applications, however, this is acceptable.
- *
- * Exported interfaces ---- kernel output
- * --------------------------------------
- *
- * The primary kernel interfaces are:
- *
- * void get_random_bytes(void *buf, size_t nbytes);
- * u32 get_random_u32()
- * u64 get_random_u64()
- * unsigned int get_random_int()
- * unsigned long get_random_long()
- *
- * These interfaces will return the requested number of random bytes
- * into the given buffer or as a return value. This is equivalent to a
- * read from /dev/urandom. The get_random_{u32,u64,int,long}() family
- * of functions may be higher performance for one-off random integers,
- * because they do a bit of buffering.
- *
- * prandom_u32()
- * -------------
- *
- * For even weaker applications, see the pseudorandom generator
- * prandom_u32(), prandom_max(), and prandom_bytes(). If the random
- * numbers aren't security-critical at all, these are *far* cheaper.
- * Useful for self-tests, random error simulation, randomized backoffs,
- * and any other application where you trust that nobody is trying to
- * maliciously mess with you by guessing the "random" numbers.
- *
- * Exported interfaces ---- input
- * ==============================
- *
- * The current exported interfaces for gathering environmental noise
- * from the devices are:
- *
- * void add_device_randomness(const void *buf, size_t size);
- * void add_input_randomness(unsigned int type, unsigned int code,
- * unsigned int value);
- * void add_interrupt_randomness(int irq);
- * void add_disk_randomness(struct gendisk *disk);
- * void add_hwgenerator_randomness(const void *buffer, size_t count,
- * size_t entropy);
- * void add_bootloader_randomness(const void *buf, size_t size);
- *
- * add_device_randomness() is for adding data to the random pool that
- * is likely to differ between two devices (or possibly even per boot).
- * This would be things like MAC addresses or serial numbers, or the
- * read-out of the RTC. This does *not* add any actual entropy to the
- * pool, but it initializes the pool to different values for devices
- * that might otherwise be identical and have very little entropy
- * available to them (particularly common in the embedded world).
- *
- * add_input_randomness() uses the input layer interrupt timing, as well as
- * the event type information from the hardware.
- *
- * add_interrupt_randomness() uses the interrupt timing as random
- * inputs to the entropy pool. Using the cycle counters and the irq source
- * as inputs, it feeds the randomness roughly once a second.
- *
- * add_disk_randomness() uses what amounts to the seek time of block
- * layer request events, on a per-disk_devt basis, as input to the
- * entropy pool. Note that high-speed solid state drives with very low
- * seek times do not make for good sources of entropy, as their seek
- * times are usually fairly consistent.
- *
- * All of these routines try to estimate how many bits of randomness a
- * particular randomness source. They do this by keeping track of the
- * first and second order deltas of the event timings.
- *
- * add_hwgenerator_randomness() is for true hardware RNGs, and will credit
- * entropy as specified by the caller. If the entropy pool is full it will
- * block until more entropy is needed.
- *
- * add_bootloader_randomness() is the same as add_hwgenerator_randomness() or
- * add_device_randomness(), depending on whether or not the configuration
- * option CONFIG_RANDOM_TRUST_BOOTLOADER is set.
- *
- * Ensuring unpredictability at system startup
- * ============================================
- *
- * When any operating system starts up, it will go through a sequence
- * of actions that are fairly predictable by an adversary, especially
- * if the start-up does not involve interaction with a human operator.
- * This reduces the actual number of bits of unpredictability in the
- * entropy pool below the value in entropy_count. In order to
- * counteract this effect, it helps to carry information in the
- * entropy pool across shut-downs and start-ups. To do this, put the
- * following lines an appropriate script which is run during the boot
- * sequence:
- *
- * echo "Initializing random number generator..."
- * random_seed=/var/run/random-seed
- * # Carry a random seed from start-up to start-up
- * # Load and then save the whole entropy pool
- * if [ -f $random_seed ]; then
- * cat $random_seed >/dev/urandom
- * else
- * touch $random_seed
- * fi
- * chmod 600 $random_seed
- * dd if=/dev/urandom of=$random_seed count=1 bs=512
- *
- * and the following lines in an appropriate script which is run as
- * the system is shutdown:
- *
- * # Carry a random seed from shut-down to start-up
- * # Save the whole entropy pool
- * echo "Saving random seed..."
- * random_seed=/var/run/random-seed
- * touch $random_seed
- * chmod 600 $random_seed
- * dd if=/dev/urandom of=$random_seed count=1 bs=512
- *
- * For example, on most modern systems using the System V init
- * scripts, such code fragments would be found in
- * /etc/rc.d/init.d/random. On older Linux systems, the correct script
- * location might be in /etc/rcb.d/rc.local or /etc/rc.d/rc.0.
- *
- * Effectively, these commands cause the contents of the entropy pool
- * to be saved at shut-down time and reloaded into the entropy pool at
- * start-up. (The 'dd' in the addition to the bootup script is to
- * make sure that /etc/random-seed is different for every start-up,
- * even if the system crashes without executing rc.0.) Even with
- * complete knowledge of the start-up activities, predicting the state
- * of the entropy pool requires knowledge of the previous history of
- * the system.
- *
- * Configuring the /dev/random driver under Linux
- * ==============================================
+ * Copyright Theodore Ts'o, 1994, 1995, 1996, 1997, 1998, 1999. All rights reserved.
*
- * The /dev/random driver under Linux uses minor numbers 8 and 9 of
- * the /dev/mem major number (#1). So if your system does not have
- * /dev/random and /dev/urandom created already, they can be created
- * by using the commands:
+ * This driver produces cryptographically secure pseudorandom data. It is divided
+ * into roughly six sections, each with a section header:
*
- * mknod /dev/random c 1 8
- * mknod /dev/urandom c 1 9
+ * - Initialization and readiness waiting.
+ * - Fast key erasure RNG, the "crng".
+ * - Entropy accumulation and extraction routines.
+ * - Entropy collection routines.
+ * - Userspace reader/writer interfaces.
+ * - Sysctl interface.
+ *
+ * The high level overview is that there is one input pool, into which
+ * various pieces of data are hashed. Some of that data is then "credited" as
+ * having a certain number of bits of entropy. When enough bits of entropy are
+ * available, the hash is finalized and handed as a key to a stream cipher that
+ * expands it indefinitely for various consumers. This key is periodically
+ * refreshed as the various entropy collectors, described below, add data to the
+ * input pool and credit it. There is currently no Fortuna-like scheduler
+ * involved, which can lead to malicious entropy sources causing a premature
+ * reseed, and the entropy estimates are, at best, conservative guesses.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
From: Mark Brown <[email protected]>
commit 2b6c6e3d9ce3aa0e547ac25d60e06fe035cd9f79 upstream.
The section at the top of random.c which documents the input functions
available does not document add_hwgenerator_randomness() which might lead
a reader to overlook it. Add a brief note about it.
Signed-off-by: Mark Brown <[email protected]>
[Jason: reorganize position of function in doc comment and also document
add_bootloader_randomness() while we're at it.]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 11 +++++++++++
1 file changed, 11 insertions(+)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -202,6 +202,9 @@
* unsigned int value);
* void add_interrupt_randomness(int irq, int irq_flags);
* void add_disk_randomness(struct gendisk *disk);
+ * void add_hwgenerator_randomness(const char *buffer, size_t count,
+ * size_t entropy);
+ * void add_bootloader_randomness(const void *buf, unsigned int size);
*
* add_device_randomness() is for adding data to the random pool that
* is likely to differ between two devices (or possibly even per boot).
@@ -228,6 +231,14 @@
* particular randomness source. They do this by keeping track of the
* first and second order deltas of the event timings.
*
+ * add_hwgenerator_randomness() is for true hardware RNGs, and will credit
+ * entropy as specified by the caller. If the entropy pool is full it will
+ * block until more entropy is needed.
+ *
+ * add_bootloader_randomness() is the same as add_hwgenerator_randomness() or
+ * add_device_randomness(), depending on whether or not the configuration
+ * option CONFIG_RANDOM_TRUST_BOOTLOADER is set.
+ *
* Ensuring unpredictability at system startup
* ============================================
*
From: "Jason A. Donenfeld" <[email protected]>
commit fe222a6ca2d53c38433cba5d3be62a39099e708e upstream.
Currently time_init() is called after rand_initialize(), but
rand_initialize() makes use of the timer on various platforms, and
sometimes this timer needs to be initialized by time_init() first. In
order for random_get_entropy() to not return zero during early boot when
it's potentially used as an entropy source, reverse the order of these
two calls. The block doing random initialization was right before
time_init() before, so changing the order shouldn't have any complicated
effects.
Cc: Andrew Morton <[email protected]>
Reviewed-by: Stafford Horne <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
init/main.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--- a/init/main.c
+++ b/init/main.c
@@ -578,11 +578,13 @@ asmlinkage __visible void __init start_k
hrtimers_init();
softirq_init();
timekeeping_init();
+ time_init();
/*
* For best initial stack canary entropy, prepare it after:
* - setup_arch() for any UEFI RNG entropy and boot cmdline access
* - timekeeping_init() for ktime entropy used in rand_initialize()
+ * - time_init() for making random_get_entropy() work on some platforms
* - rand_initialize() to get any arch-specific entropy like RDRAND
* - add_latent_entropy() to get any latent entropy
* - adding command line entropy
@@ -592,7 +594,6 @@ asmlinkage __visible void __init start_k
add_device_randomness(command_line, strlen(command_line));
boot_init_stack_canary();
- time_init();
sched_clock_postinit();
printk_nmi_init();
perf_event_init();
From: "Jason A. Donenfeld" <[email protected]>
commit 91c2afca290ed3034841c8c8532e69ed9e16cf34 upstream.
Continuing the reasoning of "random: ensure early RDSEED goes through
mixer on init", we don't want RDRAND interacting with anything without
going through the mixer function, as a backdoored CPU could presumably
cancel out data during an xor, which it'd have a harder time doing when
being forced through a cryptographic hash function. There's actually no
need at all to be calling RDRAND in write_pool(), because before we
extract from the pool, we always do so with 32 bytes of RDSEED hashed in
at that stage. Xoring at this stage is needless and introduces a minor
liability.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 14 ++------------
1 file changed, 2 insertions(+), 12 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1357,25 +1357,15 @@ static unsigned int random_poll(struct f
static int write_pool(const char __user *buffer, size_t count)
{
size_t bytes;
- u32 t, buf[16];
+ u8 buf[BLAKE2S_BLOCK_SIZE];
const char __user *p = buffer;
while (count > 0) {
- int b, i = 0;
-
bytes = min(count, sizeof(buf));
- if (copy_from_user(&buf, p, bytes))
+ if (copy_from_user(buf, p, bytes))
return -EFAULT;
-
- for (b = bytes; b > 0; b -= sizeof(u32), i++) {
- if (!arch_get_random_int(&t))
- break;
- buf[i] ^= t;
- }
-
count -= bytes;
p += bytes;
-
mix_pool_bytes(buf, bytes);
cond_resched();
}
From: "Jason A. Donenfeld" <[email protected]>
commit 9f13fb0cd11ed2327abff69f6501a2c124c88b5a upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is really not the best we can do.
Instead, at least calling random_get_entropy_fallback() would be
preferable, because that always needs to return _something_, even
falling back to jiffies eventually. It's not as though
random_get_entropy_fallback() is super high precision or guaranteed to
be entropic, but basically anything that's not zero all the time is
better than returning zero all the time.
This is accomplished by just including the asm-generic code like on
other architectures, which means we can get rid of the empty stub
function here.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Richard Weinberger <[email protected]>
Cc: Anton Ivanov <[email protected]>
Acked-by: Johannes Berg <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/um/include/asm/timex.h | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
--- a/arch/um/include/asm/timex.h
+++ b/arch/um/include/asm/timex.h
@@ -1,13 +1,8 @@
#ifndef __UM_TIMEX_H
#define __UM_TIMEX_H
-typedef unsigned long cycles_t;
-
-static inline cycles_t get_cycles (void)
-{
- return 0;
-}
-
#define CLOCK_TICK_RATE (HZ)
+#include <asm-generic/timex.h>
+
#endif
From: James Smart <[email protected]>
[ Upstream commit 336d63615466b4c06b9401c987813fd19bdde39b ]
After issuing a LIP, a specific target vendor does not ACC the FLOGI that
lpfc sends. However, it does send its own FLOGI that lpfc ACCs. The
target then establishes the port IDs by sending a PLOGI. lpfc PLOGI_ACCs
and starts the RPI registration for DID 0x000001. The target then sends a
LOGO to the fabric DID. lpfc is currently treating the LOGO from the
fabric DID as a link down and cleans up all the ndlps. The ndlp for DID
0x000001 is put back into NPR and discovery stops, leaving the port in
stuck in bypassed mode.
Change lpfc behavior such that if a LOGO is received for the fabric DID in
PT2PT topology skip the lpfc_linkdown_port() routine and just move the
fabric DID back to NPR.
Link: https://lore.kernel.org/r/[email protected]
Co-developed-by: Justin Tee <[email protected]>
Signed-off-by: Justin Tee <[email protected]>
Signed-off-by: James Smart <[email protected]>
Signed-off-by: Martin K. Petersen <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/scsi/lpfc/lpfc_nportdisc.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/scsi/lpfc/lpfc_nportdisc.c b/drivers/scsi/lpfc/lpfc_nportdisc.c
index 30b5f65b29d1..7f230d0b2fd6 100644
--- a/drivers/scsi/lpfc/lpfc_nportdisc.c
+++ b/drivers/scsi/lpfc/lpfc_nportdisc.c
@@ -633,7 +633,8 @@ lpfc_rcv_logo(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
else
lpfc_els_rsp_acc(vport, ELS_CMD_ACC, cmdiocb, ndlp, NULL);
if (ndlp->nlp_DID == Fabric_DID) {
- if (vport->port_state <= LPFC_FDISC)
+ if (vport->port_state <= LPFC_FDISC ||
+ vport->fc_flag & FC_PT2PT)
goto out;
lpfc_linkdown_port(vport);
spin_lock_irq(shost->host_lock);
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit e85c0fc1d94c52483a603651748d4c76d6aa1c6b upstream.
Per the thread linked below, "premature next" is not considered to be a
realistic threat model, and leads to more serious security problems.
"Premature next" is the scenario in which:
- Attacker compromises the current state of a fully initialized RNG via
some kind of infoleak.
- New bits of entropy are added directly to the key used to generate the
/dev/urandom stream, without any buffering or pooling.
- Attacker then, somehow having read access to /dev/urandom, samples RNG
output and brute forces the individual new bits that were added.
- Result: the RNG never "recovers" from the initial compromise, a
so-called violation of what academics term "post-compromise security".
The usual solutions to this involve some form of delaying when entropy
gets mixed into the crng. With Fortuna, this involves multiple input
buckets. With what the Linux RNG was trying to do prior, this involves
entropy estimation.
However, by delaying when entropy gets mixed in, it also means that RNG
compromises are extremely dangerous during the window of time before
the RNG has gathered enough entropy, during which time nonces may become
predictable (or repeated), ephemeral keys may not be secret, and so
forth. Moreover, it's unclear how realistic "premature next" is from an
attack perspective, if these attacks even make sense in practice.
Put together -- and discussed in more detail in the thread below --
these constitute grounds for just doing away with the current code that
pretends to handle premature next. I say "pretends" because it wasn't
doing an especially great job at it either; should we change our mind
about this direction, we would probably implement Fortuna to "fix" the
"problem", in which case, removing the pretend solution still makes
sense.
This also reduces the crng reseed period from 5 minutes down to 1
minute. The rationale from the thread might lead us toward reducing that
even further in the future (or even eliminating it), but that remains a
topic of a future commit.
At a high level, this patch changes semantics from:
Before: Seed for the first time after 256 "bits" of estimated
entropy have been accumulated since the system booted. Thereafter,
reseed once every five minutes, but only if 256 new "bits" have been
accumulated since the last reseeding.
After: Seed for the first time after 256 "bits" of estimated entropy
have been accumulated since the system booted. Thereafter, reseed
once every minute.
Most of this patch is renaming and removing: POOL_MIN_BITS becomes
POOL_INIT_BITS, credit_entropy_bits() becomes credit_init_bits(),
crng_reseed() loses its "force" parameter since it's now always true,
the drain_entropy() function no longer has any use so it's removed,
entropy estimation is skipped if we've already init'd, the various
notifiers for "low on entropy" are now only active prior to init, and
finally, some documentation comments are cleaned up here and there.
Link: https://lore.kernel.org/lkml/[email protected]/
Cc: Theodore Ts'o <[email protected]>
Cc: Nadia Heninger <[email protected]>
Cc: Tom Ristenpart <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 174 +++++++++++++++++---------------------------------
1 file changed, 62 insertions(+), 112 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -15,14 +15,12 @@
* - Sysctl interface.
*
* The high level overview is that there is one input pool, into which
- * various pieces of data are hashed. Some of that data is then "credited" as
- * having a certain number of bits of entropy. When enough bits of entropy are
- * available, the hash is finalized and handed as a key to a stream cipher that
- * expands it indefinitely for various consumers. This key is periodically
- * refreshed as the various entropy collectors, described below, add data to the
- * input pool and credit it. There is currently no Fortuna-like scheduler
- * involved, which can lead to malicious entropy sources causing a premature
- * reseed, and the entropy estimates are, at best, conservative guesses.
+ * various pieces of data are hashed. Prior to initialization, some of that
+ * data is then "credited" as having a certain number of bits of entropy.
+ * When enough bits of entropy are available, the hash is finalized and
+ * handed as a key to a stream cipher that expands it indefinitely for
+ * various consumers. This key is periodically refreshed as the various
+ * entropy collectors, described below, add data to the input pool.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
@@ -234,7 +232,10 @@ static void _warn_unseeded_randomness(co
*
*********************************************************************/
-enum { CRNG_RESEED_INTERVAL = 300 * HZ };
+enum {
+ CRNG_RESEED_START_INTERVAL = HZ,
+ CRNG_RESEED_INTERVAL = 60 * HZ
+};
static struct {
u8 key[CHACHA20_KEY_SIZE] __aligned(__alignof__(long));
@@ -254,16 +255,10 @@ static DEFINE_PER_CPU(struct crng, crngs
.generation = ULONG_MAX
};
-/* Used by crng_reseed() to extract a new seed from the input pool. */
-static bool drain_entropy(void *buf, size_t nbytes);
-/* Used by crng_make_state() to extract a new seed when crng_init==0. */
+/* Used by crng_reseed() and crng_make_state() to extract a new seed from the input pool. */
static void extract_entropy(void *buf, size_t nbytes);
-/*
- * This extracts a new crng key from the input pool, but only if there is a
- * sufficient amount of entropy available, in order to mitigate bruteforcing
- * of newly added bits.
- */
+/* This extracts a new crng key from the input pool. */
static void crng_reseed(void)
{
unsigned long flags;
@@ -271,9 +266,7 @@ static void crng_reseed(void)
u8 key[CHACHA20_KEY_SIZE];
bool finalize_init = false;
- /* Only reseed if we can, to prevent brute forcing a small amount of new bits. */
- if (!drain_entropy(key, sizeof(key)))
- return;
+ extract_entropy(key, sizeof(key));
/*
* We copy the new key into the base_crng, overwriting the old one,
@@ -345,10 +338,10 @@ static void crng_fast_key_erasure(u8 key
}
/*
- * Return whether the crng seed is considered to be sufficiently
- * old that a reseeding might be attempted. This happens if the last
- * reseeding was CRNG_RESEED_INTERVAL ago, or during early boot, at
- * an interval proportional to the uptime.
+ * Return whether the crng seed is considered to be sufficiently old
+ * that a reseeding is needed. This happens if the last reseeding
+ * was CRNG_RESEED_INTERVAL ago, or during early boot, at an interval
+ * proportional to the uptime.
*/
static bool crng_has_old_seed(void)
{
@@ -360,7 +353,7 @@ static bool crng_has_old_seed(void)
if (uptime >= CRNG_RESEED_INTERVAL / HZ * 2)
WRITE_ONCE(early_boot, false);
else
- interval = max_t(unsigned int, 5 * HZ,
+ interval = max_t(unsigned int, CRNG_RESEED_START_INTERVAL,
(unsigned int)uptime / 2 * HZ);
}
return time_after(jiffies, READ_ONCE(base_crng.birth) + interval);
@@ -402,8 +395,8 @@ static void crng_make_state(u32 chacha_s
}
/*
- * If the base_crng is old enough, we try to reseed, which in turn
- * bumps the generation counter that we check below.
+ * If the base_crng is old enough, we reseed, which in turn bumps the
+ * generation counter that we check below.
*/
if (unlikely(crng_has_old_seed()))
crng_reseed();
@@ -732,30 +725,24 @@ EXPORT_SYMBOL(get_random_bytes_arch);
*
* After which, if added entropy should be credited:
*
- * static void credit_entropy_bits(size_t nbits)
+ * static void credit_init_bits(size_t nbits)
*
- * Finally, extract entropy via these two, with the latter one
- * setting the entropy count to zero and extracting only if there
- * is POOL_MIN_BITS entropy credited prior:
+ * Finally, extract entropy via:
*
* static void extract_entropy(void *buf, size_t nbytes)
- * static bool drain_entropy(void *buf, size_t nbytes)
*
**********************************************************************/
enum {
POOL_BITS = BLAKE2S_HASH_SIZE * 8,
- POOL_MIN_BITS = POOL_BITS, /* No point in settling for less. */
- POOL_FAST_INIT_BITS = POOL_MIN_BITS / 2
+ POOL_INIT_BITS = POOL_BITS, /* No point in settling for less. */
+ POOL_FAST_INIT_BITS = POOL_INIT_BITS / 2
};
-/* For notifying userspace should write into /dev/random. */
-static DECLARE_WAIT_QUEUE_HEAD(random_write_wait);
-
static struct {
struct blake2s_state hash;
spinlock_t lock;
- unsigned int entropy_count;
+ unsigned int init_bits;
} input_pool = {
.hash.h = { BLAKE2S_IV0 ^ (0x01010000 | BLAKE2S_HASH_SIZE),
BLAKE2S_IV1, BLAKE2S_IV2, BLAKE2S_IV3, BLAKE2S_IV4,
@@ -770,9 +757,9 @@ static void _mix_pool_bytes(const void *
}
/*
- * This function adds bytes into the entropy "pool". It does not
- * update the entropy estimate. The caller should call
- * credit_entropy_bits if this is appropriate.
+ * This function adds bytes into the input pool. It does not
+ * update the initialization bit counter; the caller should call
+ * credit_init_bits if this is appropriate.
*/
static void mix_pool_bytes(const void *in, size_t nbytes)
{
@@ -829,43 +816,24 @@ static void extract_entropy(void *buf, s
memzero_explicit(&block, sizeof(block));
}
-/*
- * First we make sure we have POOL_MIN_BITS of entropy in the pool, and then we
- * set the entropy count to zero (but don't actually touch any data). Only then
- * can we extract a new key with extract_entropy().
- */
-static bool drain_entropy(void *buf, size_t nbytes)
-{
- unsigned int entropy_count;
- do {
- entropy_count = READ_ONCE(input_pool.entropy_count);
- if (entropy_count < POOL_MIN_BITS)
- return false;
- } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
- extract_entropy(buf, nbytes);
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
- return true;
-}
-
-static void credit_entropy_bits(size_t nbits)
+static void credit_init_bits(size_t nbits)
{
- unsigned int entropy_count, orig, add;
+ unsigned int init_bits, orig, add;
unsigned long flags;
- if (!nbits)
+ if (crng_ready() || !nbits)
return;
add = min_t(size_t, nbits, POOL_BITS);
do {
- orig = READ_ONCE(input_pool.entropy_count);
- entropy_count = min_t(unsigned int, POOL_BITS, orig + add);
- } while (cmpxchg(&input_pool.entropy_count, orig, entropy_count) != orig);
+ orig = READ_ONCE(input_pool.init_bits);
+ init_bits = min_t(unsigned int, POOL_BITS, orig + add);
+ } while (cmpxchg(&input_pool.init_bits, orig, init_bits) != orig);
- if (!crng_ready() && entropy_count >= POOL_MIN_BITS)
+ if (!crng_ready() && init_bits >= POOL_INIT_BITS)
crng_reseed();
- else if (unlikely(crng_init == 0 && entropy_count >= POOL_FAST_INIT_BITS)) {
+ else if (unlikely(crng_init == 0 && init_bits >= POOL_FAST_INIT_BITS)) {
spin_lock_irqsave(&base_crng.lock, flags);
if (crng_init == 0) {
extract_entropy(base_crng.key, sizeof(base_crng.key));
@@ -971,13 +939,10 @@ int __init rand_initialize(void)
_mix_pool_bytes(&now, sizeof(now));
_mix_pool_bytes(utsname(), sizeof(*(utsname())));
- extract_entropy(base_crng.key, sizeof(base_crng.key));
- ++base_crng.generation;
-
- if (arch_init && trust_cpu && !crng_ready()) {
- crng_init = 2;
- pr_notice("crng init done (trusting CPU's manufacturer)\n");
- }
+ if (crng_ready())
+ crng_reseed();
+ else if (arch_init && trust_cpu)
+ credit_init_bits(BLAKE2S_BLOCK_SIZE * 8);
if (ratelimit_disable) {
urandom_warning.interval = 0;
@@ -1031,6 +996,9 @@ static void add_timer_randomness(struct
_mix_pool_bytes(&num, sizeof(num));
spin_unlock_irqrestore(&input_pool.lock, flags);
+ if (crng_ready())
+ return;
+
/*
* Calculate number of bits of randomness we probably added.
* We take into account the first, second and third-order deltas
@@ -1061,7 +1029,7 @@ static void add_timer_randomness(struct
* Round down by 1 bit on general principles,
* and limit entropy estimate to 12 bits.
*/
- credit_entropy_bits(min_t(unsigned int, fls(delta >> 1), 11));
+ credit_init_bits(min_t(unsigned int, fls(delta >> 1), 11));
}
void add_input_randomness(unsigned int type, unsigned int code,
@@ -1114,18 +1082,15 @@ void rand_initialize_disk(struct gendisk
void add_hwgenerator_randomness(const void *buffer, size_t count,
size_t entropy)
{
+ mix_pool_bytes(buffer, count);
+ credit_init_bits(entropy);
+
/*
- * Throttle writing if we're above the trickle threshold.
- * We'll be woken up again once below POOL_MIN_BITS, when
- * the calling thread is about to terminate, or once
- * CRNG_RESEED_INTERVAL has elapsed.
+ * Throttle writing to once every CRNG_RESEED_INTERVAL, unless
+ * we're not yet initialized.
*/
- wait_event_interruptible_timeout(random_write_wait,
- kthread_should_stop() ||
- input_pool.entropy_count < POOL_MIN_BITS,
- CRNG_RESEED_INTERVAL);
- mix_pool_bytes(buffer, count);
- credit_entropy_bits(entropy);
+ if (!kthread_should_stop() && crng_ready())
+ schedule_timeout_interruptible(CRNG_RESEED_INTERVAL);
}
EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
@@ -1137,7 +1102,7 @@ void add_bootloader_randomness(const voi
{
mix_pool_bytes(buf, size);
if (trust_bootloader)
- credit_entropy_bits(size * 8);
+ credit_init_bits(size * 8);
}
EXPORT_SYMBOL_GPL(add_bootloader_randomness);
@@ -1238,7 +1203,7 @@ static void mix_interrupt_randomness(str
local_irq_enable();
mix_pool_bytes(pool, sizeof(pool));
- credit_entropy_bits(1);
+ credit_init_bits(1);
memzero_explicit(pool, sizeof(pool));
}
@@ -1285,7 +1250,7 @@ EXPORT_SYMBOL_GPL(add_interrupt_randomne
*/
static void entropy_timer(unsigned long data)
{
- credit_entropy_bits(1);
+ credit_init_bits(1);
}
/*
@@ -1378,16 +1343,8 @@ SYSCALL_DEFINE3(getrandom, char __user *
static unsigned int random_poll(struct file *file, poll_table *wait)
{
- unsigned int mask;
-
poll_wait(file, &crng_init_wait, wait);
- poll_wait(file, &random_write_wait, wait);
- mask = 0;
- if (crng_ready())
- mask |= POLLIN | POLLRDNORM;
- if (input_pool.entropy_count < POOL_MIN_BITS)
- mask |= POLLOUT | POLLWRNORM;
- return mask;
+ return crng_ready() ? POLLIN | POLLRDNORM : POLLOUT | POLLWRNORM;
}
static int write_pool(const char __user *ubuf, size_t count)
@@ -1460,7 +1417,7 @@ static long random_ioctl(struct file *f,
switch (cmd) {
case RNDGETENTCNT:
/* Inherently racy, no point locking. */
- if (put_user(input_pool.entropy_count, p))
+ if (put_user(input_pool.init_bits, p))
return -EFAULT;
return 0;
case RNDADDTOENTCNT:
@@ -1470,7 +1427,7 @@ static long random_ioctl(struct file *f,
return -EFAULT;
if (ent_count < 0)
return -EINVAL;
- credit_entropy_bits(ent_count);
+ credit_init_bits(ent_count);
return 0;
case RNDADDENTROPY:
if (!capable(CAP_SYS_ADMIN))
@@ -1484,20 +1441,13 @@ static long random_ioctl(struct file *f,
retval = write_pool((const char __user *)p, size);
if (retval < 0)
return retval;
- credit_entropy_bits(ent_count);
+ credit_init_bits(ent_count);
return 0;
case RNDZAPENTCNT:
case RNDCLEARPOOL:
- /*
- * Clear the entropy pool counters. We no longer clear
- * the entropy pool, as that's silly.
- */
+ /* No longer has any effect. */
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- if (xchg(&input_pool.entropy_count, 0) >= POOL_MIN_BITS) {
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
- }
return 0;
case RNDRESEEDCRNG:
if (!capable(CAP_SYS_ADMIN))
@@ -1554,7 +1504,7 @@ const struct file_operations urandom_fop
*
* - write_wakeup_threshold - the amount of entropy in the input pool
* below which write polls to /dev/random will unblock, requesting
- * more entropy, tied to the POOL_MIN_BITS constant. It is writable
+ * more entropy, tied to the POOL_INIT_BITS constant. It is writable
* to avoid breaking old userspaces, but writing to it does not
* change any behavior of the RNG.
*
@@ -1569,7 +1519,7 @@ const struct file_operations urandom_fop
#include <linux/sysctl.h>
static int sysctl_random_min_urandom_seed = CRNG_RESEED_INTERVAL / HZ;
-static int sysctl_random_write_wakeup_bits = POOL_MIN_BITS;
+static int sysctl_random_write_wakeup_bits = POOL_INIT_BITS;
static int sysctl_poolsize = POOL_BITS;
static u8 sysctl_bootid[UUID_SIZE];
@@ -1626,7 +1576,7 @@ struct ctl_table random_table[] = {
},
{
.procname = "entropy_avail",
- .data = &input_pool.entropy_count,
+ .data = &input_pool.init_bits,
.maxlen = sizeof(int),
.mode = 0444,
.proc_handler = proc_dointvec,
From: "Jason A. Donenfeld" <[email protected]>
commit fed7ef061686cc813b1f3d8d0edc6c35b4d3537b upstream.
Since all changes of crng_init now go through credit_init_bits(), we can
fix a long standing race in which two concurrent callers of
credit_init_bits() have the new bit count >= some threshold, but are
doing so with crng_init as a lower threshold, checked outside of a lock,
resulting in crng_reseed() or similar being called twice.
In order to fix this, we can use the original cmpxchg value of the bit
count, and only change crng_init when the bit count transitions from
below a threshold to meeting the threshold.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 48 ++++++++++++++++++++++--------------------------
1 file changed, 22 insertions(+), 26 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -265,7 +265,6 @@ static void crng_reseed(void)
unsigned long flags;
unsigned long next_gen;
u8 key[CHACHA20_KEY_SIZE];
- bool finalize_init = false;
extract_entropy(key, sizeof(key));
@@ -282,28 +281,10 @@ static void crng_reseed(void)
++next_gen;
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
- if (!crng_ready()) {
+ if (!crng_ready())
crng_init = CRNG_READY;
- finalize_init = true;
- }
spin_unlock_irqrestore(&base_crng.lock, flags);
memzero_explicit(key, sizeof(key));
- if (finalize_init) {
- process_random_ready_list();
- wake_up_interruptible(&crng_init_wait);
- kill_fasync(&fasync, SIGIO, POLL_IN);
- pr_notice("crng init done\n");
- if (unseeded_warning.missed) {
- pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
- unseeded_warning.missed);
- unseeded_warning.missed = 0;
- }
- if (urandom_warning.missed) {
- pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
- urandom_warning.missed);
- urandom_warning.missed = 0;
- }
- }
}
/*
@@ -819,7 +800,7 @@ static void extract_entropy(void *buf, s
static void credit_init_bits(size_t nbits)
{
- unsigned int init_bits, orig, add;
+ unsigned int new, orig, add;
unsigned long flags;
if (crng_ready() || !nbits)
@@ -829,13 +810,28 @@ static void credit_init_bits(size_t nbit
do {
orig = READ_ONCE(input_pool.init_bits);
- init_bits = min_t(unsigned int, POOL_BITS, orig + add);
- } while (cmpxchg(&input_pool.init_bits, orig, init_bits) != orig);
+ new = min_t(unsigned int, POOL_BITS, orig + add);
+ } while (cmpxchg(&input_pool.init_bits, orig, new) != orig);
- if (!crng_ready() && init_bits >= POOL_READY_BITS)
- crng_reseed();
- else if (unlikely(crng_init == CRNG_EMPTY && init_bits >= POOL_EARLY_BITS)) {
+ if (orig < POOL_READY_BITS && new >= POOL_READY_BITS) {
+ crng_reseed(); /* Sets crng_init to CRNG_READY under base_crng.lock. */
+ process_random_ready_list();
+ wake_up_interruptible(&crng_init_wait);
+ kill_fasync(&fasync, SIGIO, POLL_IN);
+ pr_notice("crng init done\n");
+ if (unseeded_warning.missed) {
+ pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
+ unseeded_warning.missed);
+ unseeded_warning.missed = 0;
+ }
+ if (urandom_warning.missed) {
+ pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
+ urandom_warning.missed);
+ urandom_warning.missed = 0;
+ }
+ } else if (orig < POOL_EARLY_BITS && new >= POOL_EARLY_BITS) {
spin_lock_irqsave(&base_crng.lock, flags);
+ /* Check if crng_init is CRNG_EMPTY, to avoid race with crng_reseed(). */
if (crng_init == CRNG_EMPTY) {
extract_entropy(base_crng.key, sizeof(base_crng.key));
crng_init = CRNG_EARLY;
From: Wang Yufen <[email protected]>
[ Upstream commit f638a84afef3dfe10554c51820c16e39a278c915 ]
When len >= INT_MAX - transhdrlen, ulen = len + transhdrlen will be
overflow. To fix, we can follow what udpv6 does and subtract the
transhdrlen from the max.
Signed-off-by: Wang Yufen <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
net/l2tp/l2tp_ip6.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/net/l2tp/l2tp_ip6.c b/net/l2tp/l2tp_ip6.c
index 76ef758db112..e412020029df 100644
--- a/net/l2tp/l2tp_ip6.c
+++ b/net/l2tp/l2tp_ip6.c
@@ -518,14 +518,15 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
struct ipcm6_cookie ipc6;
int addr_len = msg->msg_namelen;
int transhdrlen = 4; /* zero session-id */
- int ulen = len + transhdrlen;
+ int ulen;
int err;
/* Rough check on arithmetic overflow,
better check is made in ip6_append_data().
*/
- if (len > INT_MAX)
+ if (len > INT_MAX - transhdrlen)
return -EMSGSIZE;
+ ulen = len + transhdrlen;
/* Mirror BSD error message compatibility */
if (msg->msg_flags & MSG_OOB)
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit a49c010e61e1938be851f5e49ac219d49b704103 upstream.
This is called from various hwgenerator drivers, so rather than having
one "safe" version for userspace and one "unsafe" version for the
kernel, just make everything safe; the checks are cheap and sensible to
have anyway.
Reported-by: Sultan Alsawaf <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 29 +++++++++--------------------
1 file changed, 9 insertions(+), 20 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -448,18 +448,15 @@ static void process_random_ready_list(vo
spin_unlock_irqrestore(&random_ready_list_lock, flags);
}
-/*
- * Credit (or debit) the entropy store with n bits of entropy.
- * Use credit_entropy_bits_safe() if the value comes from userspace
- * or otherwise should be checked for extreme values.
- */
static void credit_entropy_bits(int nbits)
{
int entropy_count, orig;
- if (!nbits)
+ if (nbits <= 0)
return;
+ nbits = min(nbits, POOL_BITS);
+
do {
orig = READ_ONCE(input_pool.entropy_count);
entropy_count = min(POOL_BITS, orig + nbits);
@@ -471,18 +468,6 @@ static void credit_entropy_bits(int nbit
crng_reseed(&primary_crng, true);
}
-static int credit_entropy_bits_safe(int nbits)
-{
- if (nbits < 0)
- return -EINVAL;
-
- /* Cap the value to avoid overflows */
- nbits = min(nbits, POOL_BITS);
-
- credit_entropy_bits(nbits);
- return 0;
-}
-
/*********************************************************************
*
* CRNG using CHACHA20
@@ -1577,7 +1562,10 @@ static long random_ioctl(struct file *f,
return -EPERM;
if (get_user(ent_count, p))
return -EFAULT;
- return credit_entropy_bits_safe(ent_count);
+ if (ent_count < 0)
+ return -EINVAL;
+ credit_entropy_bits(ent_count);
+ return 0;
case RNDADDENTROPY:
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -1590,7 +1578,8 @@ static long random_ioctl(struct file *f,
retval = write_pool((const char __user *)p, size);
if (retval < 0)
return retval;
- return credit_entropy_bits_safe(ent_count);
+ credit_entropy_bits(ent_count);
+ return 0;
case RNDZAPENTCNT:
case RNDCLEARPOOL:
/*
From: Andy Lutomirski <[email protected]>
commit 84df7cdfbb215a34657b39f4257dab739efa2df9 upstream.
There is no pool that pulls, so it was just dead code.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/4a05fe0c7a5c831389ef4aea51d24528ac8682c7.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 40 ----------------------------------------
1 file changed, 40 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -531,10 +531,8 @@ struct entropy_store {
const struct poolinfo *poolinfo;
__u32 *pool;
const char *name;
- struct entropy_store *pull;
/* read-write data: */
- unsigned long last_pulled;
spinlock_t lock;
unsigned short add_ptr;
unsigned short input_rotate;
@@ -1416,41 +1414,6 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
*********************************************************************/
/*
- * This utility inline function is responsible for transferring entropy
- * from the primary pool to the secondary extraction pool. We make
- * sure we pull enough for a 'catastrophic reseed'.
- */
-static void _xfer_secondary_pool(struct entropy_store *r, size_t nbytes);
-static void xfer_secondary_pool(struct entropy_store *r, size_t nbytes)
-{
- if (!r->pull ||
- r->entropy_count >= (nbytes << (ENTROPY_SHIFT + 3)) ||
- r->entropy_count > r->poolinfo->poolfracbits)
- return;
-
- _xfer_secondary_pool(r, nbytes);
-}
-
-static void _xfer_secondary_pool(struct entropy_store *r, size_t nbytes)
-{
- __u32 tmp[OUTPUT_POOL_WORDS];
-
- int bytes = nbytes;
-
- /* pull at least as much as a wakeup */
- bytes = max_t(int, bytes, random_read_wakeup_bits / 8);
- /* but never more than the buffer size */
- bytes = min_t(int, bytes, sizeof(tmp));
-
- trace_xfer_secondary_pool(r->name, bytes * 8, nbytes * 8,
- ENTROPY_BITS(r), ENTROPY_BITS(r->pull));
- bytes = extract_entropy(r->pull, tmp, bytes,
- random_read_wakeup_bits / 8, 0);
- mix_pool_bytes(r, tmp, bytes);
- credit_entropy_bits(r, bytes*8);
-}
-
-/*
* This function decides how many bytes to actually take from the
* given pool, and also debits the entropy count accordingly.
*/
@@ -1613,7 +1576,6 @@ static ssize_t extract_entropy(struct en
spin_unlock_irqrestore(&r->lock, flags);
trace_extract_entropy(r->name, EXTRACT_SIZE,
ENTROPY_BITS(r), _RET_IP_);
- xfer_secondary_pool(r, EXTRACT_SIZE);
extract_buf(r, tmp);
spin_lock_irqsave(&r->lock, flags);
memcpy(r->last_data, tmp, EXTRACT_SIZE);
@@ -1622,7 +1584,6 @@ static ssize_t extract_entropy(struct en
}
trace_extract_entropy(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
- xfer_secondary_pool(r, nbytes);
nbytes = account(r, nbytes, min, reserved);
return _extract_entropy(r, buf, nbytes, fips_enabled);
@@ -1833,7 +1794,6 @@ static void __init init_std_data(struct
ktime_t now = ktime_get_real();
unsigned long rv;
- r->last_pulled = jiffies;
mix_pool_bytes(r, &now, sizeof(now));
for (i = r->poolinfo->poolbytes; i > 0; i -= sizeof(rv)) {
if (!arch_get_random_seed_long(&rv) &&
From: Linus Torvalds <[email protected]>
commit 901c7280ca0d5e2b4a8929fbe0bfb007ac2a6544 upstream.
Halil Pasic points out [1] that the full revert of that commit (revert
in bddac7c1e02b), and that a partial revert that only reverts the
problematic case, but still keeps some of the cleanups is probably
better. 
And that partial revert [2] had already been verified by Oleksandr
Natalenko to also fix the issue, I had just missed that in the long
discussion.
So let's reinstate the cleanups from commit aa6f8dcbab47 ("swiotlb:
rework "fix info leak with DMA_FROM_DEVICE""), and effectively only
revert the part that caused problems.
Link: https://lore.kernel.org/all/[email protected]/ [1]
Link: https://lore.kernel.org/all/[email protected]/ [2]
Link: https://lore.kernel.org/all/[email protected]/ [3]
Suggested-by: Halil Pasic <[email protected]>
Tested-by: Oleksandr Natalenko <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
[OP: backport to 4.14: apply swiotlb_tbl_map_single() changes in lib/swiotlb.c]
Signed-off-by: Ovidiu Panait <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
[bwh: Backported to 4.9: adjust context]
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/DMA-attributes.txt | 10 ----------
include/linux/dma-mapping.h | 7 -------
lib/swiotlb.c | 12 ++++++++----
3 files changed, 8 insertions(+), 21 deletions(-)
--- a/Documentation/DMA-attributes.txt
+++ b/Documentation/DMA-attributes.txt
@@ -143,13 +143,3 @@ So, this provides a way for drivers to a
where allocation failures are not a problem, and shouldn't bother the logs.
NOTE: At the moment DMA_ATTR_NO_WARN is only implemented on PowerPC.
-
-DMA_ATTR_PRIVILEGED
--------------------
-
-Some advanced peripherals such as remote processors and GPUs perform
-accesses to DMA buffers in both privileged "supervisor" and unprivileged
-"user" modes. This attribute is used to indicate to the DMA-mapping
-subsystem that the buffer is fully accessible at the elevated privilege
-level (and ideally inaccessible or at least read-only at the
-lesser-privileged levels).
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -61,13 +61,6 @@
* allocation failure reports (similarly to __GFP_NOWARN).
*/
#define DMA_ATTR_NO_WARN (1UL << 8)
-/*
- * This is a hint to the DMA-mapping subsystem that the device is expected
- * to overwrite the entire mapped size, thus the caller does not require any
- * of the previous buffer contents to be preserved. This allows
- * bounce-buffering implementations to optimise DMA_FROM_DEVICE transfers.
- */
-#define DMA_ATTR_OVERWRITE (1UL << 10)
/*
* A dma_addr_t can hold any valid DMA or bus address for the platform.
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -532,10 +532,14 @@ found:
*/
for (i = 0; i < nslots; i++)
io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
- if (!(attrs & DMA_ATTR_OVERWRITE) || dir == DMA_TO_DEVICE ||
- dir == DMA_BIDIRECTIONAL)
- swiotlb_bounce(orig_addr, tlb_addr, size, DMA_TO_DEVICE);
-
+ /*
+ * When dir == DMA_FROM_DEVICE we could omit the copy from the orig
+ * to the tlb buffer, if we knew for sure the device will
+ * overwirte the entire current content. But we don't. Thus
+ * unconditional bounce may prevent leaking swiotlb content (i.e.
+ * kernel memory) to user-space.
+ */
+ swiotlb_bounce(orig_addr, tlb_addr, size, DMA_TO_DEVICE);
return tlb_addr;
}
EXPORT_SYMBOL_GPL(swiotlb_tbl_map_single);
From: Andy Shevchenko <[email protected]>
commit 9def051018c08e65c532822749e857eb4b2e12e7 upstream.
Deduplicate le32_to_cpu_array() and cpu_to_le32_array() by moving them
to the generic header.
No functional change implied.
Signed-off-by: Andy Shevchenko <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/md4.c | 17 -----------------
crypto/md5.c | 17 -----------------
include/linux/byteorder/generic.h | 17 +++++++++++++++++
3 files changed, 17 insertions(+), 34 deletions(-)
--- a/crypto/md4.c
+++ b/crypto/md4.c
@@ -64,23 +64,6 @@ static inline u32 H(u32 x, u32 y, u32 z)
#define ROUND2(a,b,c,d,k,s) (a = lshift(a + G(b,c,d) + k + (u32)0x5A827999,s))
#define ROUND3(a,b,c,d,k,s) (a = lshift(a + H(b,c,d) + k + (u32)0x6ED9EBA1,s))
-/* XXX: this stuff can be optimized */
-static inline void le32_to_cpu_array(u32 *buf, unsigned int words)
-{
- while (words--) {
- __le32_to_cpus(buf);
- buf++;
- }
-}
-
-static inline void cpu_to_le32_array(u32 *buf, unsigned int words)
-{
- while (words--) {
- __cpu_to_le32s(buf);
- buf++;
- }
-}
-
static void md4_transform(u32 *hash, u32 const *in)
{
u32 a, b, c, d;
--- a/crypto/md5.c
+++ b/crypto/md5.c
@@ -30,23 +30,6 @@ const u8 md5_zero_message_hash[MD5_DIGES
};
EXPORT_SYMBOL_GPL(md5_zero_message_hash);
-/* XXX: this stuff can be optimized */
-static inline void le32_to_cpu_array(u32 *buf, unsigned int words)
-{
- while (words--) {
- __le32_to_cpus(buf);
- buf++;
- }
-}
-
-static inline void cpu_to_le32_array(u32 *buf, unsigned int words)
-{
- while (words--) {
- __cpu_to_le32s(buf);
- buf++;
- }
-}
-
static inline void md5_transform_helper(struct md5_state *ctx)
{
le32_to_cpu_array(ctx->block, sizeof(ctx->block) / sizeof(u32));
--- a/include/linux/byteorder/generic.h
+++ b/include/linux/byteorder/generic.h
@@ -155,6 +155,23 @@ static inline void le64_add_cpu(__le64 *
*var = cpu_to_le64(le64_to_cpu(*var) + val);
}
+/* XXX: this stuff can be optimized */
+static inline void le32_to_cpu_array(u32 *buf, unsigned int words)
+{
+ while (words--) {
+ __le32_to_cpus(buf);
+ buf++;
+ }
+}
+
+static inline void cpu_to_le32_array(u32 *buf, unsigned int words)
+{
+ while (words--) {
+ __cpu_to_le32s(buf);
+ buf++;
+ }
+}
+
static inline void be16_add_cpu(__be16 *var, u16 val)
{
*var = cpu_to_be16(be16_to_cpu(*var) + val);
From: Stephan Mueller <[email protected]>
commit db07cd26ac6a418dc2823187958edcfdb415fa83 upstream.
FIPS 140-2 section 4.9.2 requires a continuous self test of the noise
source. Up to kernel 4.8 drivers/char/random.c provided this continuous
self test. Afterwards it was moved to a location that is inconsistent
with the FIPS 140-2 requirements. The relevant patch was
e192be9d9a30555aae2ca1dc3aad37cba484cd4a .
Thus, the FIPS 140-2 CTRNG is added to the DRBG when it obtains the
seed. This patch resurrects the function drbg_fips_continous_test that
existed some time ago and applies it to the noise sources. The patch
that removed the drbg_fips_continous_test was
b3614763059b82c26bdd02ffcb1c016c1132aad0 .
The Jitter RNG implements its own FIPS 140-2 self test and thus does not
need to be subjected to the test in the DRBG.
The patch contains a tiny fix to ensure proper zeroization in case of an
error during the Jitter RNG data gathering.
Signed-off-by: Stephan Mueller <[email protected]>
Reviewed-by: Yann Droneaud <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++--
include/crypto/drbg.h | 2 +
2 files changed, 93 insertions(+), 3 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -220,6 +220,57 @@ static inline unsigned short drbg_sec_st
}
/*
+ * FIPS 140-2 continuous self test for the noise source
+ * The test is performed on the noise source input data. Thus, the function
+ * implicitly knows the size of the buffer to be equal to the security
+ * strength.
+ *
+ * Note, this function disregards the nonce trailing the entropy data during
+ * initial seeding.
+ *
+ * drbg->drbg_mutex must have been taken.
+ *
+ * @drbg DRBG handle
+ * @entropy buffer of seed data to be checked
+ *
+ * return:
+ * 0 on success
+ * -EAGAIN on when the CTRNG is not yet primed
+ * < 0 on error
+ */
+static int drbg_fips_continuous_test(struct drbg_state *drbg,
+ const unsigned char *entropy)
+{
+ unsigned short entropylen = drbg_sec_strength(drbg->core->flags);
+ int ret = 0;
+
+ if (!IS_ENABLED(CONFIG_CRYPTO_FIPS))
+ return 0;
+
+ /* skip test if we test the overall system */
+ if (list_empty(&drbg->test_data.list))
+ return 0;
+ /* only perform test in FIPS mode */
+ if (!fips_enabled)
+ return 0;
+
+ if (!drbg->fips_primed) {
+ /* Priming of FIPS test */
+ memcpy(drbg->prev, entropy, entropylen);
+ drbg->fips_primed = true;
+ /* priming: another round is needed */
+ return -EAGAIN;
+ }
+ ret = memcmp(drbg->prev, entropy, entropylen);
+ if (!ret)
+ panic("DRBG continuous self test failed\n");
+ memcpy(drbg->prev, entropy, entropylen);
+
+ /* the test shall pass when the two values are not equal */
+ return 0;
+}
+
+/*
* Convert an integer into a byte representation of this integer.
* The byte representation is big-endian
*
@@ -1000,6 +1051,22 @@ static inline int __drbg_seed(struct drb
return ret;
}
+static inline int drbg_get_random_bytes(struct drbg_state *drbg,
+ unsigned char *entropy,
+ unsigned int entropylen)
+{
+ int ret;
+
+ do {
+ get_random_bytes(entropy, entropylen);
+ ret = drbg_fips_continuous_test(drbg, entropy);
+ if (ret && ret != -EAGAIN)
+ return ret;
+ } while (ret);
+
+ return 0;
+}
+
static void drbg_async_seed(struct work_struct *work)
{
struct drbg_string data;
@@ -1008,16 +1075,20 @@ static void drbg_async_seed(struct work_
seed_work);
unsigned int entropylen = drbg_sec_strength(drbg->core->flags);
unsigned char entropy[32];
+ int ret;
BUG_ON(!entropylen);
BUG_ON(entropylen > sizeof(entropy));
- get_random_bytes(entropy, entropylen);
drbg_string_fill(&data, entropy, entropylen);
list_add_tail(&data.list, &seedlist);
mutex_lock(&drbg->drbg_mutex);
+ ret = drbg_get_random_bytes(drbg, entropy, entropylen);
+ if (ret)
+ goto unlock;
+
/* If nonblocking pool is initialized, deactivate Jitter RNG */
crypto_free_rng(drbg->jent);
drbg->jent = NULL;
@@ -1032,6 +1103,7 @@ static void drbg_async_seed(struct work_
if (drbg->seeded)
drbg->reseed_threshold = drbg_max_requests(drbg);
+unlock:
mutex_unlock(&drbg->drbg_mutex);
memzero_explicit(entropy, entropylen);
@@ -1083,7 +1155,9 @@ static int drbg_seed(struct drbg_state *
BUG_ON((entropylen * 2) > sizeof(entropy));
/* Get seed from in-kernel /dev/urandom */
- get_random_bytes(entropy, entropylen);
+ ret = drbg_get_random_bytes(drbg, entropy, entropylen);
+ if (ret)
+ goto out;
if (!drbg->jent) {
drbg_string_fill(&data1, entropy, entropylen);
@@ -1096,7 +1170,7 @@ static int drbg_seed(struct drbg_state *
entropylen);
if (ret) {
pr_devel("DRBG: jent failed with %d\n", ret);
- return ret;
+ goto out;
}
drbg_string_fill(&data1, entropy, entropylen * 2);
@@ -1123,6 +1197,7 @@ static int drbg_seed(struct drbg_state *
ret = __drbg_seed(drbg, &seedlist, reseed);
+out:
memzero_explicit(entropy, entropylen * 2);
return ret;
@@ -1144,6 +1219,11 @@ static inline void drbg_dealloc_state(st
drbg->reseed_ctr = 0;
drbg->d_ops = NULL;
drbg->core = NULL;
+ if (IS_ENABLED(CONFIG_CRYPTO_FIPS)) {
+ kzfree(drbg->prev);
+ drbg->prev = NULL;
+ drbg->fips_primed = false;
+ }
}
/*
@@ -1213,6 +1293,14 @@ static inline int drbg_alloc_state(struc
drbg->scratchpad = PTR_ALIGN(drbg->scratchpadbuf, ret + 1);
}
+ if (IS_ENABLED(CONFIG_CRYPTO_FIPS)) {
+ drbg->prev = kzalloc(drbg_sec_strength(drbg->core->flags),
+ GFP_KERNEL);
+ if (!drbg->prev)
+ goto fini;
+ drbg->fips_primed = false;
+ }
+
return 0;
fini:
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -131,6 +131,8 @@ struct drbg_state {
bool seeded; /* DRBG fully seeded? */
bool pr; /* Prediction resistance enabled? */
+ bool fips_primed; /* Continuous test primed? */
+ unsigned char *prev; /* FIPS 140-2 continuous test value */
struct work_struct seed_work; /* asynchronous seeding support */
struct crypto_rng *jent;
const struct drbg_state_ops *d_ops;
From: Eric Dumazet <[email protected]>
commit 190cc82489f46f9d88e73c81a47e14f80a791e1a upstream.
RFC 6056 (Recommendations for Transport-Protocol Port Randomization)
provides good summary of why source selection needs extra care.
David Dworken reminded us that linux implements Algorithm 3
as described in RFC 6056 3.3.3
Quoting David :
In the context of the web, this creates an interesting info leak where
websites can count how many TCP connections a user's computer is
establishing over time. For example, this allows a website to count
exactly how many subresources a third party website loaded.
This also allows:
- Distinguishing between different users behind a VPN based on
distinct source port ranges.
- Tracking users over time across multiple networks.
- Covert communication channels between different browsers/browser
profiles running on the same computer
- Tracking what applications are running on a computer based on
the pattern of how fast source ports are getting incremented.
Section 3.3.4 describes an enhancement, that reduces
attackers ability to use the basic information currently
stored into the shared 'u32 hint'.
This change also decreases collision rate when
multiple applications need to connect() to
different destinations.
Signed-off-by: Eric Dumazet <[email protected]>
Reported-by: David Dworken <[email protected]>
Cc: Willem de Bruijn <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
[bwh: Backported to 4.9: adjust context]
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 20 +++++++++++++++++---
1 file changed, 17 insertions(+), 3 deletions(-)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -537,6 +537,17 @@ void inet_unhash(struct sock *sk)
}
EXPORT_SYMBOL_GPL(inet_unhash);
+/* RFC 6056 3.3.4. Algorithm 4: Double-Hash Port Selection Algorithm
+ * Note that we use 32bit integers (vs RFC 'short integers')
+ * because 2^16 is not a multiple of num_ephemeral and this
+ * property might be used by clever attacker.
+ * RFC claims using TABLE_LENGTH=10 buckets gives an improvement,
+ * we use 256 instead to really give more isolation and
+ * privacy, this only consumes 1 KB of kernel memory.
+ */
+#define INET_TABLE_PERTURB_SHIFT 8
+static u32 table_perturb[1 << INET_TABLE_PERTURB_SHIFT];
+
int __inet_hash_connect(struct inet_timewait_death_row *death_row,
struct sock *sk, u32 port_offset,
int (*check_established)(struct inet_timewait_death_row *,
@@ -550,7 +561,7 @@ int __inet_hash_connect(struct inet_time
struct inet_bind_bucket *tb;
u32 remaining, offset;
int ret, i, low, high;
- static u32 hint;
+ u32 index;
if (port) {
head = &hinfo->bhash[inet_bhashfn(net, port,
@@ -575,7 +586,10 @@ int __inet_hash_connect(struct inet_time
if (likely(remaining > 1))
remaining &= ~1U;
- offset = (hint + port_offset) % remaining;
+ net_get_random_once(table_perturb, sizeof(table_perturb));
+ index = hash_32(port_offset, INET_TABLE_PERTURB_SHIFT);
+
+ offset = (READ_ONCE(table_perturb[index]) + port_offset) % remaining;
/* In first pass we try ports of @low parity.
* inet_csk_get_port() does the opposite choice.
*/
@@ -628,7 +642,7 @@ next_port:
return -EADDRNOTAVAIL;
ok:
- hint += i + 2;
+ WRITE_ONCE(table_perturb[index], READ_ONCE(table_perturb[index]) + i + 2);
/* Head lock still held and bh's disabled */
inet_bind_hash(sk, tb, port);
From: "Jason A. Donenfeld" <[email protected]>
commit 12e45a2a6308105469968951e6d563e8f4fea187 upstream.
RDRAND and RDSEED can fail sometimes, which is fine. We currently
initialize the RNG with 512 bits of RDRAND/RDSEED. We only need 256 bits
of those to succeed in order to initialize the RNG. Instead of the
current "all or nothing" approach, actually credit these contributions
the amount that is actually contributed.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -892,9 +892,8 @@ early_param("random.trust_bootloader", p
*/
int __init random_init(const char *command_line)
{
- size_t i;
ktime_t now = ktime_get_real();
- bool arch_init = true;
+ unsigned int i, arch_bytes;
unsigned long rv;
#if defined(LATENT_ENTROPY_PLUGIN)
@@ -902,11 +901,12 @@ int __init random_init(const char *comma
_mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed));
#endif
- for (i = 0; i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
+ for (i = 0, arch_bytes = BLAKE2S_BLOCK_SIZE;
+ i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
if (!arch_get_random_seed_long_early(&rv) &&
!arch_get_random_long_early(&rv)) {
rv = random_get_entropy();
- arch_init = false;
+ arch_bytes -= sizeof(rv);
}
_mix_pool_bytes(&rv, sizeof(rv));
}
@@ -917,8 +917,8 @@ int __init random_init(const char *comma
if (crng_ready())
crng_reseed();
- else if (arch_init && trust_cpu)
- credit_init_bits(BLAKE2S_BLOCK_SIZE * 8);
+ else if (trust_cpu)
+ credit_init_bits(arch_bytes * 8);
return 0;
}
From: "Jason A. Donenfeld" <[email protected]>
commit d8d83d8ab0a453e17e68b3a3bed1f940c34b8646 upstream.
Basically nobody should use blake2s in an HMAC construction; it already
has a keyed variant. But unfortunately for historical reasons, Noise,
used by WireGuard, uses HKDF quite strictly, which means we have to use
this. Because this really shouldn't be used by others, this commit moves
it into wireguard's noise.c locally, so that kernels that aren't using
WireGuard don't get this superfluous code baked in. On m68k systems,
this shaves off ~314 bytes.
Cc: Herbert Xu <[email protected]>
Tested-by: Geert Uytterhoeven <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
[Jason: for stable, skip the wireguard changes, since this kernel
doesn't have wireguard.]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/crypto/blake2s.h | 3 ---
lib/crypto/blake2s-selftest.c | 31 -------------------------------
lib/crypto/blake2s.c | 37 -------------------------------------
3 files changed, 71 deletions(-)
--- a/include/crypto/blake2s.h
+++ b/include/crypto/blake2s.h
@@ -100,7 +100,4 @@ static inline void blake2s(u8 *out, cons
blake2s_final(&state, out);
}
-void blake2s256_hmac(u8 *out, const u8 *in, const u8 *key, const size_t inlen,
- const size_t keylen);
-
#endif /* BLAKE2S_H */
--- a/lib/crypto/blake2s-selftest.c
+++ b/lib/crypto/blake2s-selftest.c
@@ -15,7 +15,6 @@
* #include <stdio.h>
*
* #include <openssl/evp.h>
- * #include <openssl/hmac.h>
*
* #define BLAKE2S_TESTVEC_COUNT 256
*
@@ -58,16 +57,6 @@
* }
* printf("};\n\n");
*
- * printf("static const u8 blake2s_hmac_testvecs[][BLAKE2S_HASH_SIZE] __initconst = {\n");
- *
- * HMAC(EVP_blake2s256(), key, sizeof(key), buf, sizeof(buf), hash, NULL);
- * print_vec(hash, BLAKE2S_OUTBYTES);
- *
- * HMAC(EVP_blake2s256(), buf, sizeof(buf), key, sizeof(key), hash, NULL);
- * print_vec(hash, BLAKE2S_OUTBYTES);
- *
- * printf("};\n");
- *
* return 0;
*}
*/
@@ -554,15 +543,6 @@ static const u8 blake2s_testvecs[][BLAKE
0xd6, 0x98, 0x6b, 0x07, 0x10, 0x65, 0x52, 0x65, },
};
-static const u8 blake2s_hmac_testvecs[][BLAKE2S_HASH_SIZE] __initconst = {
- { 0xce, 0xe1, 0x57, 0x69, 0x82, 0xdc, 0xbf, 0x43, 0xad, 0x56, 0x4c, 0x70,
- 0xed, 0x68, 0x16, 0x96, 0xcf, 0xa4, 0x73, 0xe8, 0xe8, 0xfc, 0x32, 0x79,
- 0x08, 0x0a, 0x75, 0x82, 0xda, 0x3f, 0x05, 0x11, },
- { 0x77, 0x2f, 0x0c, 0x71, 0x41, 0xf4, 0x4b, 0x2b, 0xb3, 0xc6, 0xb6, 0xf9,
- 0x60, 0xde, 0xe4, 0x52, 0x38, 0x66, 0xe8, 0xbf, 0x9b, 0x96, 0xc4, 0x9f,
- 0x60, 0xd9, 0x24, 0x37, 0x99, 0xd6, 0xec, 0x31, },
-};
-
bool __init blake2s_selftest(void)
{
u8 key[BLAKE2S_KEY_SIZE];
@@ -607,16 +587,5 @@ bool __init blake2s_selftest(void)
}
}
- if (success) {
- blake2s256_hmac(hash, buf, key, sizeof(buf), sizeof(key));
- success &= !memcmp(hash, blake2s_hmac_testvecs[0], BLAKE2S_HASH_SIZE);
-
- blake2s256_hmac(hash, key, buf, sizeof(key), sizeof(buf));
- success &= !memcmp(hash, blake2s_hmac_testvecs[1], BLAKE2S_HASH_SIZE);
-
- if (!success)
- pr_err("blake2s256_hmac self-test: FAIL\n");
- }
-
return success;
}
--- a/lib/crypto/blake2s.c
+++ b/lib/crypto/blake2s.c
@@ -59,43 +59,6 @@ void blake2s_final(struct blake2s_state
}
EXPORT_SYMBOL(blake2s_final);
-void blake2s256_hmac(u8 *out, const u8 *in, const u8 *key, const size_t inlen,
- const size_t keylen)
-{
- struct blake2s_state state;
- u8 x_key[BLAKE2S_BLOCK_SIZE] __aligned(__alignof__(u32)) = { 0 };
- u8 i_hash[BLAKE2S_HASH_SIZE] __aligned(__alignof__(u32));
- int i;
-
- if (keylen > BLAKE2S_BLOCK_SIZE) {
- blake2s_init(&state, BLAKE2S_HASH_SIZE);
- blake2s_update(&state, key, keylen);
- blake2s_final(&state, x_key);
- } else
- memcpy(x_key, key, keylen);
-
- for (i = 0; i < BLAKE2S_BLOCK_SIZE; ++i)
- x_key[i] ^= 0x36;
-
- blake2s_init(&state, BLAKE2S_HASH_SIZE);
- blake2s_update(&state, x_key, BLAKE2S_BLOCK_SIZE);
- blake2s_update(&state, in, inlen);
- blake2s_final(&state, i_hash);
-
- for (i = 0; i < BLAKE2S_BLOCK_SIZE; ++i)
- x_key[i] ^= 0x5c ^ 0x36;
-
- blake2s_init(&state, BLAKE2S_HASH_SIZE);
- blake2s_update(&state, x_key, BLAKE2S_BLOCK_SIZE);
- blake2s_update(&state, i_hash, BLAKE2S_HASH_SIZE);
- blake2s_final(&state, i_hash);
-
- memcpy(out, i_hash, BLAKE2S_HASH_SIZE);
- memzero_explicit(x_key, BLAKE2S_BLOCK_SIZE);
- memzero_explicit(i_hash, BLAKE2S_HASH_SIZE);
-}
-EXPORT_SYMBOL(blake2s256_hmac);
-
static int __init mod_init(void)
{
if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
From: Trond Myklebust <[email protected]>
[ Upstream commit fe44fb23d6ccde4c914c44ef74ab8d9d9ba02bea ]
If the server tells us that a pNFS layout is not available for a
specific file, then we should not keep pounding it with further
layoutget requests.
Fixes: 183d9e7b112a ("pnfs: rework LAYOUTGET retry handling")
Signed-off-by: Trond Myklebust <[email protected]>
Signed-off-by: Anna Schumaker <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
fs/nfs/pnfs.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index f19cded49b29..317d22f84492 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1753,6 +1753,12 @@ pnfs_update_layout(struct inode *ino,
/* Fallthrough */
case -EAGAIN:
break;
+ case -ENODATA:
+ /* The server returned NFS4ERR_LAYOUTUNAVAILABLE */
+ pnfs_layout_set_fail_bit(
+ lo, pnfs_iomode_to_fail_bit(iomode));
+ lseg = NULL;
+ goto out_put_layout_hdr;
default:
if (!nfs_error_is_fatal(PTR_ERR(lseg))) {
pnfs_layout_clear_fail_bit(lo, pnfs_iomode_to_fail_bit(iomode));
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit a4b5c26b79ffdfcfb816c198f2fc2b1e7b5b580f upstream.
There are no code changes here; this is just a reordering of functions,
so that in subsequent commits, the timer entropy functions can call into
the interrupt ones.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 238 +++++++++++++++++++++++++-------------------------
1 file changed, 119 insertions(+), 119 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -852,13 +852,13 @@ static void credit_init_bits(size_t nbit
* the above entropy accumulation routines:
*
* void add_device_randomness(const void *buf, size_t size);
- * void add_input_randomness(unsigned int type, unsigned int code,
- * unsigned int value);
- * void add_disk_randomness(struct gendisk *disk);
* void add_hwgenerator_randomness(const void *buffer, size_t count,
* size_t entropy);
* void add_bootloader_randomness(const void *buf, size_t size);
* void add_interrupt_randomness(int irq);
+ * void add_input_randomness(unsigned int type, unsigned int code,
+ * unsigned int value);
+ * void add_disk_randomness(struct gendisk *disk);
*
* add_device_randomness() adds data to the input pool that
* is likely to differ between two devices (or possibly even per boot).
@@ -868,19 +868,6 @@ static void credit_init_bits(size_t nbit
* that might otherwise be identical and have very little entropy
* available to them (particularly common in the embedded world).
*
- * add_input_randomness() uses the input layer interrupt timing, as well
- * as the event type information from the hardware.
- *
- * add_disk_randomness() uses what amounts to the seek time of block
- * layer request events, on a per-disk_devt basis, as input to the
- * entropy pool. Note that high-speed solid state drives with very low
- * seek times do not make for good sources of entropy, as their seek
- * times are usually fairly consistent.
- *
- * The above two routines try to estimate how many bits of entropy
- * to credit. They do this by keeping track of the first and second
- * order deltas of the event timings.
- *
* add_hwgenerator_randomness() is for true hardware RNGs, and will credit
* entropy as specified by the caller. If the entropy pool is full it will
* block until more entropy is needed.
@@ -894,6 +881,19 @@ static void credit_init_bits(size_t nbit
* as inputs, it feeds the input pool roughly once a second or after 64
* interrupts, crediting 1 bit of entropy for whichever comes first.
*
+ * add_input_randomness() uses the input layer interrupt timing, as well
+ * as the event type information from the hardware.
+ *
+ * add_disk_randomness() uses what amounts to the seek time of block
+ * layer request events, on a per-disk_devt basis, as input to the
+ * entropy pool. Note that high-speed solid state drives with very low
+ * seek times do not make for good sources of entropy, as their seek
+ * times are usually fairly consistent.
+ *
+ * The last two routines try to estimate how many bits of entropy
+ * to credit. They do this by keeping track of the first and second
+ * order deltas of the event timings.
+ *
**********************************************************************/
static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
@@ -971,109 +971,6 @@ void add_device_randomness(const void *b
}
EXPORT_SYMBOL(add_device_randomness);
-/* There is one of these per entropy source */
-struct timer_rand_state {
- unsigned long last_time;
- long last_delta, last_delta2;
-};
-
-/*
- * This function adds entropy to the entropy "pool" by using timing
- * delays. It uses the timer_rand_state structure to make an estimate
- * of how many bits of entropy this call has added to the pool.
- *
- * The number "num" is also added to the pool - it should somehow describe
- * the type of event which just happened. This is currently 0-255 for
- * keyboard scan codes, and 256 upwards for interrupts.
- */
-static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
-{
- unsigned long entropy = random_get_entropy(), now = jiffies, flags;
- long delta, delta2, delta3;
-
- spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(&entropy, sizeof(entropy));
- _mix_pool_bytes(&num, sizeof(num));
- spin_unlock_irqrestore(&input_pool.lock, flags);
-
- if (crng_ready())
- return;
-
- /*
- * Calculate number of bits of randomness we probably added.
- * We take into account the first, second and third-order deltas
- * in order to make our estimate.
- */
- delta = now - READ_ONCE(state->last_time);
- WRITE_ONCE(state->last_time, now);
-
- delta2 = delta - state->last_delta;
- state->last_delta = delta;
-
- delta3 = delta2 - state->last_delta2;
- state->last_delta2 = delta2;
-
- if (delta < 0)
- delta = -delta;
- if (delta2 < 0)
- delta2 = -delta2;
- if (delta3 < 0)
- delta3 = -delta3;
- if (delta > delta2)
- delta = delta2;
- if (delta > delta3)
- delta = delta3;
-
- /*
- * delta is now minimum absolute delta.
- * Round down by 1 bit on general principles,
- * and limit entropy estimate to 12 bits.
- */
- credit_init_bits(min_t(unsigned int, fls(delta >> 1), 11));
-}
-
-void add_input_randomness(unsigned int type, unsigned int code,
- unsigned int value)
-{
- static unsigned char last_value;
- static struct timer_rand_state input_timer_state = { INITIAL_JIFFIES };
-
- /* Ignore autorepeat and the like. */
- if (value == last_value)
- return;
-
- last_value = value;
- add_timer_randomness(&input_timer_state,
- (type << 4) ^ code ^ (code >> 4) ^ value);
-}
-EXPORT_SYMBOL_GPL(add_input_randomness);
-
-#ifdef CONFIG_BLOCK
-void add_disk_randomness(struct gendisk *disk)
-{
- if (!disk || !disk->random)
- return;
- /* First major is 1, so we get >= 0x200 here. */
- add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
-}
-EXPORT_SYMBOL_GPL(add_disk_randomness);
-
-void rand_initialize_disk(struct gendisk *disk)
-{
- struct timer_rand_state *state;
-
- /*
- * If kzalloc returns null, we just won't use that entropy
- * source.
- */
- state = kzalloc(sizeof(struct timer_rand_state), GFP_KERNEL);
- if (state) {
- state->last_time = INITIAL_JIFFIES;
- disk->random = state;
- }
-}
-#endif
-
/*
* Interface for in-kernel drivers of true hardware RNGs.
* Those devices may produce endless random bits and will be throttled
@@ -1235,6 +1132,109 @@ void add_interrupt_randomness(int irq)
}
EXPORT_SYMBOL_GPL(add_interrupt_randomness);
+/* There is one of these per entropy source */
+struct timer_rand_state {
+ unsigned long last_time;
+ long last_delta, last_delta2;
+};
+
+/*
+ * This function adds entropy to the entropy "pool" by using timing
+ * delays. It uses the timer_rand_state structure to make an estimate
+ * of how many bits of entropy this call has added to the pool.
+ *
+ * The number "num" is also added to the pool - it should somehow describe
+ * the type of event which just happened. This is currently 0-255 for
+ * keyboard scan codes, and 256 upwards for interrupts.
+ */
+static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
+{
+ unsigned long entropy = random_get_entropy(), now = jiffies, flags;
+ long delta, delta2, delta3;
+
+ spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(&entropy, sizeof(entropy));
+ _mix_pool_bytes(&num, sizeof(num));
+ spin_unlock_irqrestore(&input_pool.lock, flags);
+
+ if (crng_ready())
+ return;
+
+ /*
+ * Calculate number of bits of randomness we probably added.
+ * We take into account the first, second and third-order deltas
+ * in order to make our estimate.
+ */
+ delta = now - READ_ONCE(state->last_time);
+ WRITE_ONCE(state->last_time, now);
+
+ delta2 = delta - READ_ONCE(state->last_delta);
+ WRITE_ONCE(state->last_delta, delta);
+
+ delta3 = delta2 - READ_ONCE(state->last_delta2);
+ WRITE_ONCE(state->last_delta2, delta2);
+
+ if (delta < 0)
+ delta = -delta;
+ if (delta2 < 0)
+ delta2 = -delta2;
+ if (delta3 < 0)
+ delta3 = -delta3;
+ if (delta > delta2)
+ delta = delta2;
+ if (delta > delta3)
+ delta = delta3;
+
+ /*
+ * delta is now minimum absolute delta.
+ * Round down by 1 bit on general principles,
+ * and limit entropy estimate to 12 bits.
+ */
+ credit_init_bits(min_t(unsigned int, fls(delta >> 1), 11));
+}
+
+void add_input_randomness(unsigned int type, unsigned int code,
+ unsigned int value)
+{
+ static unsigned char last_value;
+ static struct timer_rand_state input_timer_state = { INITIAL_JIFFIES };
+
+ /* Ignore autorepeat and the like. */
+ if (value == last_value)
+ return;
+
+ last_value = value;
+ add_timer_randomness(&input_timer_state,
+ (type << 4) ^ code ^ (code >> 4) ^ value);
+}
+EXPORT_SYMBOL_GPL(add_input_randomness);
+
+#ifdef CONFIG_BLOCK
+void add_disk_randomness(struct gendisk *disk)
+{
+ if (!disk || !disk->random)
+ return;
+ /* First major is 1, so we get >= 0x200 here. */
+ add_timer_randomness(disk->random, 0x100 + disk_devt(disk));
+}
+EXPORT_SYMBOL_GPL(add_disk_randomness);
+
+void rand_initialize_disk(struct gendisk *disk)
+{
+ struct timer_rand_state *state;
+
+ /*
+ * If kzalloc returns null, we just won't use that entropy
+ * source.
+ */
+ state = kzalloc(sizeof(struct timer_rand_state), GFP_KERNEL);
+ if (state) {
+ state->last_time = INITIAL_JIFFIES;
+ disk->random = state;
+ }
+}
+#endif
+
/*
* Each time the timer fires, we expect that we got an unpredictable
* jump in the cycle counter. Even if the timer is running on another
From: Willy Tarreau <[email protected]>
commit 4c2c8f03a5ab7cb04ec64724d7d176d00bcc91e5 upstream.
Moshe Kol, Amit Klein, and Yossi Gilad reported being able to accurately
identify a client by forcing it to emit only 40 times more connections
than there are entries in the table_perturb[] table. The previous two
improvements consisting in resalting the secret every 10s and adding
randomness to each port selection only slightly improved the situation,
and the current value of 2^8 was too small as it's not very difficult
to make a client emit 10k connections in less than 10 seconds.
Thus we're increasing the perturb table from 2^8 to 2^16 so that the
same precision now requires 2.6M connections, which is more difficult in
this time frame and harder to hide as a background activity. The impact
is that the table now uses 256 kB instead of 1 kB, which could mostly
affect devices making frequent outgoing connections. However such
components usually target a small set of destinations (load balancers,
database clients, perf assessment tools), and in practice only a few
entries will be visited, like before.
A live test at 1 million connections per second showed no performance
difference from the previous value.
Reported-by: Moshe Kol <[email protected]>
Reported-by: Yossi Gilad <[email protected]>
Reported-by: Amit Klein <[email protected]>
Reviewed-by: Eric Dumazet <[email protected]>
Signed-off-by: Willy Tarreau <[email protected]>
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -541,11 +541,12 @@ EXPORT_SYMBOL_GPL(inet_unhash);
* Note that we use 32bit integers (vs RFC 'short integers')
* because 2^16 is not a multiple of num_ephemeral and this
* property might be used by clever attacker.
- * RFC claims using TABLE_LENGTH=10 buckets gives an improvement,
- * we use 256 instead to really give more isolation and
- * privacy, this only consumes 1 KB of kernel memory.
+ * RFC claims using TABLE_LENGTH=10 buckets gives an improvement, though
+ * attacks were since demonstrated, thus we use 65536 instead to really
+ * give more isolation and privacy, at the expense of 256kB of kernel
+ * memory.
*/
-#define INET_TABLE_PERTURB_SHIFT 8
+#define INET_TABLE_PERTURB_SHIFT 16
#define INET_TABLE_PERTURB_SIZE (1 << INET_TABLE_PERTURB_SHIFT)
static u32 *table_perturb;
From: Jason A. Donenfeld <[email protected]>
[ Upstream commit 846bb97e131d7938847963cca00657c995b1fce1 ]
This commit changes the default Kconfig values of RANDOM_TRUST_CPU and
RANDOM_TRUST_BOOTLOADER to be Y by default. It does not change any
existing configs or change any kernel behavior. The reason for this is
several fold.
As background, I recently had an email thread with the kernel
maintainers of Fedora/RHEL, Debian, Ubuntu, Gentoo, Arch, NixOS, Alpine,
SUSE, and Void as recipients. I noted that some distros trust RDRAND,
some trust EFI, and some trust both, and I asked why or why not. There
wasn't really much of a "debate" but rather an interesting discussion of
what the historical reasons have been for this, and it came up that some
distros just missed the introduction of the bootloader Kconfig knob,
while another didn't want to enable it until there was a boot time
switch to turn it off for more concerned users (which has since been
added). The result of the rather uneventful discussion is that every
major Linux distro enables these two options by default.
While I didn't have really too strong of an opinion going into this
thread -- and I mostly wanted to learn what the distros' thinking was
one way or another -- ultimately I think their choice was a decent
enough one for a default option (which can be disabled at boot time).
I'll try to summarize the pros and cons:
Pros:
- The RNG machinery gets initialized super quickly, and there's no
messing around with subsequent blocking behavior.
- The bootloader mechanism is used by kexec in order for the prior
kernel to initialize the RNG of the next kernel, which increases
the entropy available to early boot daemons of the next kernel.
- Previous objections related to backdoors centered around
Dual_EC_DRBG-like kleptographic systems, in which observing some
amount of the output stream enables an adversary holding the right key
to determine the entire output stream.
This used to be a partially justified concern, because RDRAND output
was mixed into the output stream in varying ways, some of which may
have lacked pre-image resistance (e.g. XOR or an LFSR).
But this is no longer the case. Now, all usage of RDRAND and
bootloader seeds go through a cryptographic hash function. This means
that the CPU would have to compute a hash pre-image, which is not
considered to be feasible (otherwise the hash function would be
terribly broken).
- More generally, if the CPU is backdoored, the RNG is probably not the
realistic vector of choice for an attacker.
- These CPU or bootloader seeds are far from being the only source of
entropy. Rather, there is generally a pretty huge amount of entropy,
not all of which is credited, especially on CPUs that support
instructions like RDRAND. In other words, assuming RDRAND outputs all
zeros, an attacker would *still* have to accurately model every single
other entropy source also in use.
- The RNG now reseeds itself quite rapidly during boot, starting at 2
seconds, then 4, then 8, then 16, and so forth, so that other sources
of entropy get used without much delay.
- Paranoid users can set random.trust_{cpu,bootloader}=no in the kernel
command line, and paranoid system builders can set the Kconfig options
to N, so there's no reduction or restriction of optionality.
- It's a practical default.
- All the distros have it set this way. Microsoft and Apple trust it
too. Bandwagon.
Cons:
- RDRAND *could* still be backdoored with something like a fixed key or
limited space serial number seed or another indexable scheme like
that. (However, it's hard to imagine threat models where the CPU is
backdoored like this, yet people are still okay making *any*
computations with it or connecting it to networks, etc.)
- RDRAND *could* be defective, rather than backdoored, and produce
garbage that is in one way or another insufficient for crypto.
- Suggesting a *reduction* in paranoia, as this commit effectively does,
may cause some to question my personal integrity as a "security
person".
- Bootloader seeds and RDRAND are generally very difficult if not all
together impossible to audit.
Keep in mind that this doesn't actually change any behavior. This
is just a change in the default Kconfig value. The distros already are
shipping kernels that set things this way.
Ard made an additional argument in [1]:
We're at the mercy of firmware and micro-architecture anyway, given
that we are also relying on it to ensure that every instruction in
the kernel's executable image has been faithfully copied to memory,
and that the CPU implements those instructions as documented. So I
don't think firmware or ISA bugs related to RNGs deserve special
treatment - if they are broken, we should quirk around them like we
usually do. So enabling these by default is a step in the right
direction IMHO.
In [2], Phil pointed out that having this disabled masked a bug that CI
otherwise would have caught:
A clean 5.15.45 boots cleanly, whereas a downstream kernel shows the
static key warning (but it does go on to boot). The significant
difference is that our defconfigs set CONFIG_RANDOM_TRUST_BOOTLOADER=y
defining that on top of multi_v7_defconfig demonstrates the issue on
a clean 5.15.45. Conversely, not setting that option in a
downstream kernel build avoids the warning
[1] https://lore.kernel.org/lkml/CAMj1kXGi+ieviFjXv9zQBSaGyyzeGW_VpMpTLJK8PJb2QHEQ-w@mail.gmail.com/
[2] https://lore.kernel.org/lkml/[email protected]/
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/char/Kconfig | 54 +++++++++++++++++++++++++++-----------------
1 file changed, 33 insertions(+), 21 deletions(-)
diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 1de665ae0255..1d3813d4f5c8 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -593,29 +593,41 @@ config TILE_SROM
source "drivers/char/xillybus/Kconfig"
-endmenu
-
config RANDOM_TRUST_CPU
- bool "Trust the CPU manufacturer to initialize Linux's CRNG"
- depends on X86 || S390 || PPC
- default n
+ bool "Initialize RNG using CPU RNG instructions"
+ default y
+ depends on ARCH_RANDOM
help
- Assume that CPU manufacturer (e.g., Intel or AMD for RDSEED or
- RDRAND, IBM for the S390 and Power PC architectures) is trustworthy
- for the purposes of initializing Linux's CRNG. Since this is not
- something that can be independently audited, this amounts to trusting
- that CPU manufacturer (perhaps with the insistence or mandate
- of a Nation State's intelligence or law enforcement agencies)
- has not installed a hidden back door to compromise the CPU's
- random number generation facilities. This can also be configured
- at boot with "random.trust_cpu=on/off".
+ Initialize the RNG using random numbers supplied by the CPU's
+ RNG instructions (e.g. RDRAND), if supported and available. These
+ random numbers are never used directly, but are rather hashed into
+ the main input pool, and this happens regardless of whether or not
+ this option is enabled. Instead, this option controls whether the
+ they are credited and hence can initialize the RNG. Additionally,
+ other sources of randomness are always used, regardless of this
+ setting. Enabling this implies trusting that the CPU can supply high
+ quality and non-backdoored random numbers.
+
+ Say Y here unless you have reason to mistrust your CPU or believe
+ its RNG facilities may be faulty. This may also be configured at
+ boot time with "random.trust_cpu=on/off".
config RANDOM_TRUST_BOOTLOADER
- bool "Trust the bootloader to initialize Linux's CRNG"
+ bool "Initialize RNG using bootloader-supplied seed"
+ default y
help
- Some bootloaders can provide entropy to increase the kernel's initial
- device randomness. Say Y here to assume the entropy provided by the
- booloader is trustworthy so it will be added to the kernel's entropy
- pool. Otherwise, say N here so it will be regarded as device input that
- only mixes the entropy pool. This can also be configured at boot with
- "random.trust_bootloader=on/off".
+ Initialize the RNG using a seed supplied by the bootloader or boot
+ environment (e.g. EFI or a bootloader-generated device tree). This
+ seed is not used directly, but is rather hashed into the main input
+ pool, and this happens regardless of whether or not this option is
+ enabled. Instead, this option controls whether the seed is credited
+ and hence can initialize the RNG. Additionally, other sources of
+ randomness are always used, regardless of this setting. Enabling
+ this implies trusting that the bootloader can supply high quality and
+ non-backdoored seeds.
+
+ Say Y here unless you have reason to mistrust your bootloader or
+ believe its RNG facilities may be faulty. This may also be configured
+ at boot time with "random.trust_bootloader=on/off".
+
+endmenu
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit a3f9e8910e1584d7725ef7d5ac870920d42d0bb4 upstream.
The only time that we need to wake up /dev/random writers on
RNDCLEARPOOL/RNDZAPPOOL is when we're changing from a value that is
greater than or equal to POOL_MIN_BITS to zero, because if we're
changing from below POOL_MIN_BITS to zero, the writers are already
unblocked.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1578,7 +1578,7 @@ static long random_ioctl(struct file *f,
*/
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- if (xchg(&input_pool.entropy_count, 0)) {
+ if (xchg(&input_pool.entropy_count, 0) >= POOL_MIN_BITS) {
wake_up_interruptible(&random_write_wait);
kill_fasync(&fasync, SIGIO, POLL_OUT);
}
From: "Jason A. Donenfeld" <[email protected]>
commit 4a072c71f49b0a0e495ea13423bdb850da73c58c upstream.
Odd versions of gcc for the sh4 architecture will actually warn about
flags being used while uninitialized, so we set them to zero. Non crazy
gccs will optimize that out again, so it doesn't make a difference.
Next, over aggressive gccs could inline the expression that defines
use_lock, which could then introduce a race resulting in a lock
imbalance. By using READ_ONCE, we prevent that fate. Finally, we make
that assignment const, so that gcc can still optimize a nice amount.
Finally, we fix a potential deadlock between primary_crng.lock and
batched_entropy_reset_lock, where they could be called in opposite
order. Moving the call to invalidate_batched_entropy to outside the lock
rectifies this issue.
Fixes: b169c13de473a85b3c859bb36216a4cb5f00a54a
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Cc: [email protected]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -815,13 +815,13 @@ static int crng_fast_load(const char *cp
p[crng_init_cnt % CHACHA20_KEY_SIZE] ^= *cp;
cp++; crng_init_cnt++; len--;
}
+ spin_unlock_irqrestore(&primary_crng.lock, flags);
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
invalidate_batched_entropy();
crng_init = 1;
wake_up_interruptible(&crng_init_wait);
pr_notice("random: fast init done\n");
}
- spin_unlock_irqrestore(&primary_crng.lock, flags);
return 1;
}
@@ -904,6 +904,7 @@ static void crng_reseed(struct crng_stat
}
memzero_explicit(&buf, sizeof(buf));
WRITE_ONCE(crng->init_time, jiffies);
+ spin_unlock_irqrestore(&crng->lock, flags);
if (crng == &primary_crng && crng_init < 2) {
numa_crng_init();
invalidate_batched_entropy();
@@ -924,7 +925,6 @@ static void crng_reseed(struct crng_stat
urandom_warning.missed = 0;
}
}
- spin_unlock_irqrestore(&crng->lock, flags);
}
static inline void crng_wait_ready(void)
@@ -2108,8 +2108,8 @@ static DEFINE_PER_CPU(struct batched_ent
u64 get_random_u64(void)
{
u64 ret;
- bool use_lock = crng_init < 2;
- unsigned long flags;
+ bool use_lock = READ_ONCE(crng_init) < 2;
+ unsigned long flags = 0;
struct batched_entropy *batch;
#if BITS_PER_LONG == 64
@@ -2140,8 +2140,8 @@ static DEFINE_PER_CPU(struct batched_ent
u32 get_random_u32(void)
{
u32 ret;
- bool use_lock = crng_init < 2;
- unsigned long flags;
+ bool use_lock = READ_ONCE(crng_init) < 2;
+ unsigned long flags = 0;
struct batched_entropy *batch;
if (arch_get_random_int(&ret))
From: Miaoqian Lin <[email protected]>
commit 4757c9ade34178b351580133771f510b5ffcf9c8 upstream.
of_parse_phandle() returns a node pointer with refcount
incremented, we should use of_node_put() on it when not need anymore.
Add missing of_node_put() to avoid refcount leak.
of_node_put() will check NULL pointer.
Fixes: 24a28e428351 ("USB: gadget driver for LPC32xx")
Cc: stable <[email protected]>
Signed-off-by: Miaoqian Lin <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/usb/gadget/udc/lpc32xx_udc.c | 1 +
1 file changed, 1 insertion(+)
--- a/drivers/usb/gadget/udc/lpc32xx_udc.c
+++ b/drivers/usb/gadget/udc/lpc32xx_udc.c
@@ -3034,6 +3034,7 @@ static int lpc32xx_udc_probe(struct plat
}
udc->isp1301_i2c_client = isp1301_get_client(isp1301_node);
+ of_node_put(isp1301_node);
if (!udc->isp1301_i2c_client) {
retval = -EPROBE_DEFER;
goto phy_fail;
From: "Jason A. Donenfeld" <[email protected]>
commit 9a1536b093bb5bf60689021275fd24d513bb8db0 upstream.
With SHA-1 no longer being used for anything performance oriented, and
also soon to be phased out entirely, we can make up for the space added
by unrolled BLAKE2s by simply re-rolling SHA-1. Since SHA-1 is so much
more complex, re-rolling it more or less takes care of the code size
added by BLAKE2s. And eventually, hopefully we'll see SHA-1 removed
entirely from most small kernel builds.
Cc: Herbert Xu <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Tested-by: Geert Uytterhoeven <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
lib/sha1.c | 95 ++++++++-----------------------------------------------------
1 file changed, 14 insertions(+), 81 deletions(-)
--- a/lib/sha1.c
+++ b/lib/sha1.c
@@ -9,6 +9,7 @@
#include <linux/export.h>
#include <linux/bitops.h>
#include <linux/cryptohash.h>
+#include <linux/string.h>
#include <asm/unaligned.h>
/*
@@ -54,7 +55,8 @@
#define SHA_ROUND(t, input, fn, constant, A, B, C, D, E) do { \
__u32 TEMP = input(t); setW(t, TEMP); \
E += TEMP + rol32(A,5) + (fn) + (constant); \
- B = ror32(B, 2); } while (0)
+ B = ror32(B, 2); \
+ TEMP = E; E = D; D = C; C = B; B = A; A = TEMP; } while (0)
#define T_0_15(t, A, B, C, D, E) SHA_ROUND(t, SHA_SRC, (((C^D)&B)^D) , 0x5a827999, A, B, C, D, E )
#define T_16_19(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (((C^D)&B)^D) , 0x5a827999, A, B, C, D, E )
@@ -81,6 +83,7 @@
void sha_transform(__u32 *digest, const char *data, __u32 *array)
{
__u32 A, B, C, D, E;
+ unsigned int i = 0;
A = digest[0];
B = digest[1];
@@ -89,94 +92,24 @@ void sha_transform(__u32 *digest, const
E = digest[4];
/* Round 1 - iterations 0-16 take their input from 'data' */
- T_0_15( 0, A, B, C, D, E);
- T_0_15( 1, E, A, B, C, D);
- T_0_15( 2, D, E, A, B, C);
- T_0_15( 3, C, D, E, A, B);
- T_0_15( 4, B, C, D, E, A);
- T_0_15( 5, A, B, C, D, E);
- T_0_15( 6, E, A, B, C, D);
- T_0_15( 7, D, E, A, B, C);
- T_0_15( 8, C, D, E, A, B);
- T_0_15( 9, B, C, D, E, A);
- T_0_15(10, A, B, C, D, E);
- T_0_15(11, E, A, B, C, D);
- T_0_15(12, D, E, A, B, C);
- T_0_15(13, C, D, E, A, B);
- T_0_15(14, B, C, D, E, A);
- T_0_15(15, A, B, C, D, E);
+ for (; i < 16; ++i)
+ T_0_15(i, A, B, C, D, E);
/* Round 1 - tail. Input from 512-bit mixing array */
- T_16_19(16, E, A, B, C, D);
- T_16_19(17, D, E, A, B, C);
- T_16_19(18, C, D, E, A, B);
- T_16_19(19, B, C, D, E, A);
+ for (; i < 20; ++i)
+ T_16_19(i, A, B, C, D, E);
/* Round 2 */
- T_20_39(20, A, B, C, D, E);
- T_20_39(21, E, A, B, C, D);
- T_20_39(22, D, E, A, B, C);
- T_20_39(23, C, D, E, A, B);
- T_20_39(24, B, C, D, E, A);
- T_20_39(25, A, B, C, D, E);
- T_20_39(26, E, A, B, C, D);
- T_20_39(27, D, E, A, B, C);
- T_20_39(28, C, D, E, A, B);
- T_20_39(29, B, C, D, E, A);
- T_20_39(30, A, B, C, D, E);
- T_20_39(31, E, A, B, C, D);
- T_20_39(32, D, E, A, B, C);
- T_20_39(33, C, D, E, A, B);
- T_20_39(34, B, C, D, E, A);
- T_20_39(35, A, B, C, D, E);
- T_20_39(36, E, A, B, C, D);
- T_20_39(37, D, E, A, B, C);
- T_20_39(38, C, D, E, A, B);
- T_20_39(39, B, C, D, E, A);
+ for (; i < 40; ++i)
+ T_20_39(i, A, B, C, D, E);
/* Round 3 */
- T_40_59(40, A, B, C, D, E);
- T_40_59(41, E, A, B, C, D);
- T_40_59(42, D, E, A, B, C);
- T_40_59(43, C, D, E, A, B);
- T_40_59(44, B, C, D, E, A);
- T_40_59(45, A, B, C, D, E);
- T_40_59(46, E, A, B, C, D);
- T_40_59(47, D, E, A, B, C);
- T_40_59(48, C, D, E, A, B);
- T_40_59(49, B, C, D, E, A);
- T_40_59(50, A, B, C, D, E);
- T_40_59(51, E, A, B, C, D);
- T_40_59(52, D, E, A, B, C);
- T_40_59(53, C, D, E, A, B);
- T_40_59(54, B, C, D, E, A);
- T_40_59(55, A, B, C, D, E);
- T_40_59(56, E, A, B, C, D);
- T_40_59(57, D, E, A, B, C);
- T_40_59(58, C, D, E, A, B);
- T_40_59(59, B, C, D, E, A);
+ for (; i < 60; ++i)
+ T_40_59(i, A, B, C, D, E);
/* Round 4 */
- T_60_79(60, A, B, C, D, E);
- T_60_79(61, E, A, B, C, D);
- T_60_79(62, D, E, A, B, C);
- T_60_79(63, C, D, E, A, B);
- T_60_79(64, B, C, D, E, A);
- T_60_79(65, A, B, C, D, E);
- T_60_79(66, E, A, B, C, D);
- T_60_79(67, D, E, A, B, C);
- T_60_79(68, C, D, E, A, B);
- T_60_79(69, B, C, D, E, A);
- T_60_79(70, A, B, C, D, E);
- T_60_79(71, E, A, B, C, D);
- T_60_79(72, D, E, A, B, C);
- T_60_79(73, C, D, E, A, B);
- T_60_79(74, B, C, D, E, A);
- T_60_79(75, A, B, C, D, E);
- T_60_79(76, E, A, B, C, D);
- T_60_79(77, D, E, A, B, C);
- T_60_79(78, C, D, E, A, B);
- T_60_79(79, B, C, D, E, A);
+ for (; i < 80; ++i)
+ T_60_79(i, A, B, C, D, E);
digest[0] += A;
digest[1] += B;
From: Charles Keepax <[email protected]>
[ Upstream commit 91e90c712fade0b69cdff7cc6512f6099bd18ae5 ]
The Bypass Volume is accidentally using a -6dB minimum TLV rather than
the correct -60dB minimum. Add a new TLV to correct this.
Signed-off-by: Charles Keepax <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Mark Brown <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
sound/soc/codecs/cs42l52.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/sound/soc/codecs/cs42l52.c b/sound/soc/codecs/cs42l52.c
index f733f6b42b53..47f2439fd7b0 100644
--- a/sound/soc/codecs/cs42l52.c
+++ b/sound/soc/codecs/cs42l52.c
@@ -141,6 +141,8 @@ static DECLARE_TLV_DB_SCALE(mic_tlv, 1600, 100, 0);
static DECLARE_TLV_DB_SCALE(pga_tlv, -600, 50, 0);
+static DECLARE_TLV_DB_SCALE(pass_tlv, -6000, 50, 0);
+
static DECLARE_TLV_DB_SCALE(mix_tlv, -5150, 50, 0);
static DECLARE_TLV_DB_SCALE(beep_tlv, -56, 200, 0);
@@ -355,7 +357,7 @@ static const struct snd_kcontrol_new cs42l52_snd_controls[] = {
CS42L52_SPKB_VOL, 0, 0x40, 0xC0, hl_tlv),
SOC_DOUBLE_R_SX_TLV("Bypass Volume", CS42L52_PASSTHRUA_VOL,
- CS42L52_PASSTHRUB_VOL, 0, 0x88, 0x90, pga_tlv),
+ CS42L52_PASSTHRUB_VOL, 0, 0x88, 0x90, pass_tlv),
SOC_DOUBLE("Bypass Mute", CS42L52_MISC_CTL, 4, 5, 1, 0),
--
2.35.1
From: Richard Henderson <[email protected]>
commit 647f50d5d9d933b644b29c54f13ac52af1b1774d upstream.
The arm64 version of archrandom.h will need to be able to test for
support and read the random number without preemption, so a separate
query predicate is not practical.
Since this part of the generic interface is unused, remove it.
Signed-off-by: Richard Henderson <[email protected]>
Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/random.h | 8 --------
1 file changed, 8 deletions(-)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -104,10 +104,6 @@ static inline bool arch_get_random_int(u
{
return 0;
}
-static inline bool arch_has_random(void)
-{
- return 0;
-}
static inline bool arch_get_random_seed_long(unsigned long *v)
{
return 0;
@@ -116,10 +112,6 @@ static inline bool arch_get_random_seed_
{
return 0;
}
-static inline bool arch_has_random_seed(void)
-{
- return 0;
-}
#endif
#endif /* _LINUX_RANDOM_H */
From: Richard Henderson <[email protected]>
commit 98dcfce69729f9ce0fb14f96a39bbdba21429597 upstream.
The generic interface uses bool not int; match that.
Reviewed-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Richard Henderson <[email protected]>
Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/powerpc/include/asm/archrandom.h | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
--- a/arch/powerpc/include/asm/archrandom.h
+++ b/arch/powerpc/include/asm/archrandom.h
@@ -5,27 +5,28 @@
#include <asm/machdep.h>
-static inline int arch_get_random_long(unsigned long *v)
+static inline bool arch_get_random_long(unsigned long *v)
{
- return 0;
+ return false;
}
-static inline int arch_get_random_int(unsigned int *v)
+static inline bool arch_get_random_int(unsigned int *v)
{
- return 0;
+ return false;
}
-static inline int arch_get_random_seed_long(unsigned long *v)
+static inline bool arch_get_random_seed_long(unsigned long *v)
{
if (ppc_md.get_random_seed)
return ppc_md.get_random_seed(v);
- return 0;
+ return false;
}
-static inline int arch_get_random_seed_int(unsigned int *v)
+
+static inline bool arch_get_random_seed_int(unsigned int *v)
{
unsigned long val;
- int rc;
+ bool rc;
rc = arch_get_random_seed_long(&val);
if (rc)
From: Tejun Heo <[email protected]>
commit 3347fa0928210d96aaa2bd6cd5a8391d5e630873 upstream.
Workqueue is currently initialized in an early init call; however,
there are cases where early boot code has to be split and reordered to
come after workqueue initialization or the same code path which makes
use of workqueues is used both before workqueue initailization and
after. The latter cases have to gate workqueue usages with
keventd_up() tests, which is nasty and easy to get wrong.
Workqueue usages have become widespread and it'd be a lot more
convenient if it can be used very early from boot. This patch splits
workqueue initialization into two steps. workqueue_init_early() which
sets up the basic data structures so that workqueues can be created
and work items queued, and workqueue_init() which actually brings up
workqueues online and starts executing queued work items. The former
step can be done very early during boot once memory allocation,
cpumasks and idr are initialized. The latter right after kthreads
become available.
This allows work item queueing and canceling from very early boot
which is what most of these use cases want.
* As systemd_wq being initialized doesn't indicate that workqueue is
fully online anymore, update keventd_up() to test wq_online instead.
The follow-up patches will get rid of all its usages and the
function itself.
* Flushing doesn't make sense before workqueue is fully initialized.
The flush functions trigger WARN and return immediately before fully
online.
* Work items are never in-flight before fully online. Canceling can
always succeed by skipping the flush step.
* Some code paths can no longer assume to be called with irq enabled
as irq is disabled during early boot. Use irqsave/restore
operations instead.
v2: Watchdog init, which requires timer to be running, moved from
workqueue_init_early() to workqueue_init().
Signed-off-by: Tejun Heo <[email protected]>
Suggested-by: Linus Torvalds <[email protected]>
Link: http://lkml.kernel.org/r/CA+55aFx0vPuMuxn00rBSM192n-Du5uxy+4AvKa0SBSOVJeuCGg@mail.gmail.com
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/workqueue.h | 7 +++-
init/main.c | 10 ++++++
kernel/workqueue.c | 76 ++++++++++++++++++++++++++++++++++++----------
3 files changed, 76 insertions(+), 17 deletions(-)
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -359,6 +359,8 @@ extern struct workqueue_struct *system_f
extern struct workqueue_struct *system_power_efficient_wq;
extern struct workqueue_struct *system_freezable_power_efficient_wq;
+extern bool wq_online;
+
extern struct workqueue_struct *
__alloc_workqueue_key(const char *fmt, unsigned int flags, int max_active,
struct lock_class_key *key, const char *lock_name, ...) __printf(1, 6);
@@ -598,7 +600,7 @@ static inline bool schedule_delayed_work
*/
static inline bool keventd_up(void)
{
- return system_wq != NULL;
+ return wq_online;
}
#ifndef CONFIG_SMP
@@ -635,4 +637,7 @@ int workqueue_online_cpu(unsigned int cp
int workqueue_offline_cpu(unsigned int cpu);
#endif
+int __init workqueue_init_early(void);
+int __init workqueue_init(void);
+
#endif
--- a/init/main.c
+++ b/init/main.c
@@ -554,6 +554,14 @@ asmlinkage __visible void __init start_k
"Interrupts were enabled *very* early, fixing it\n"))
local_irq_disable();
idr_init_cache();
+
+ /*
+ * Allow workqueue creation and work item queueing/cancelling
+ * early. Work item execution depends on kthreads and starts after
+ * workqueue_init().
+ */
+ workqueue_init_early();
+
rcu_init();
/* trace_printk() and trace points may be used after this */
@@ -1026,6 +1034,8 @@ static noinline void __init kernel_init_
smp_prepare_cpus(setup_max_cpus);
+ workqueue_init();
+
do_pre_smp_initcalls();
lockup_detector_init();
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -292,6 +292,8 @@ module_param_named(disable_numa, wq_disa
static bool wq_power_efficient = IS_ENABLED(CONFIG_WQ_POWER_EFFICIENT_DEFAULT);
module_param_named(power_efficient, wq_power_efficient, bool, 0444);
+bool wq_online; /* can kworkers be created yet? */
+
static bool wq_numa_enabled; /* unbound NUMA affinity enabled */
/* buf for wq_update_unbound_numa_attrs(), protected by CPU hotplug exclusion */
@@ -2588,6 +2590,9 @@ void flush_workqueue(struct workqueue_st
};
int next_color;
+ if (WARN_ON(!wq_online))
+ return;
+
lock_map_acquire(&wq->lockdep_map);
lock_map_release(&wq->lockdep_map);
@@ -2848,6 +2853,9 @@ bool flush_work(struct work_struct *work
{
struct wq_barrier barr;
+ if (WARN_ON(!wq_online))
+ return false;
+
lock_map_acquire(&work->lockdep_map);
lock_map_release(&work->lockdep_map);
@@ -2918,7 +2926,13 @@ static bool __cancel_work_timer(struct w
mark_work_canceling(work);
local_irq_restore(flags);
- flush_work(work);
+ /*
+ * This allows canceling during early boot. We know that @work
+ * isn't executing.
+ */
+ if (wq_online)
+ flush_work(work);
+
clear_work_data(work);
/*
@@ -3368,7 +3382,7 @@ static struct worker_pool *get_unbound_p
goto fail;
/* create and start the initial worker */
- if (!create_worker(pool))
+ if (wq_online && !create_worker(pool))
goto fail;
/* install */
@@ -3439,6 +3453,7 @@ static void pwq_adjust_max_active(struct
{
struct workqueue_struct *wq = pwq->wq;
bool freezable = wq->flags & WQ_FREEZABLE;
+ unsigned long flags;
/* for @wq->saved_max_active */
lockdep_assert_held(&wq->mutex);
@@ -3447,7 +3462,8 @@ static void pwq_adjust_max_active(struct
if (!freezable && pwq->max_active == wq->saved_max_active)
return;
- spin_lock_irq(&pwq->pool->lock);
+ /* this function can be called during early boot w/ irq disabled */
+ spin_lock_irqsave(&pwq->pool->lock, flags);
/*
* During [un]freezing, the caller is responsible for ensuring that
@@ -3477,7 +3493,7 @@ static void pwq_adjust_max_active(struct
pwq->max_active = 0;
}
- spin_unlock_irq(&pwq->pool->lock);
+ spin_unlock_irqrestore(&pwq->pool->lock, flags);
}
/* initialize newly alloced @pwq which is associated with @wq and @pool */
@@ -5550,7 +5566,17 @@ static void __init wq_numa_init(void)
wq_numa_enabled = true;
}
-static int __init init_workqueues(void)
+/**
+ * workqueue_init_early - early init for workqueue subsystem
+ *
+ * This is the first half of two-staged workqueue subsystem initialization
+ * and invoked as soon as the bare basics - memory allocation, cpumasks and
+ * idr are up. It sets up all the data structures and system workqueues
+ * and allows early boot code to create workqueues and queue/cancel work
+ * items. Actual work item execution starts only after kthreads can be
+ * created and scheduled right before early initcalls.
+ */
+int __init workqueue_init_early(void)
{
int std_nice[NR_STD_WORKER_POOLS] = { 0, HIGHPRI_NICE_LEVEL };
int i, cpu;
@@ -5583,16 +5609,6 @@ static int __init init_workqueues(void)
}
}
- /* create the initial worker */
- for_each_online_cpu(cpu) {
- struct worker_pool *pool;
-
- for_each_cpu_worker_pool(pool, cpu) {
- pool->flags &= ~POOL_DISASSOCIATED;
- BUG_ON(!create_worker(pool));
- }
- }
-
/* create default unbound and ordered wq attrs */
for (i = 0; i < NR_STD_WORKER_POOLS; i++) {
struct workqueue_attrs *attrs;
@@ -5629,8 +5645,36 @@ static int __init init_workqueues(void)
!system_power_efficient_wq ||
!system_freezable_power_efficient_wq);
+ return 0;
+}
+
+/**
+ * workqueue_init - bring workqueue subsystem fully online
+ *
+ * This is the latter half of two-staged workqueue subsystem initialization
+ * and invoked as soon as kthreads can be created and scheduled.
+ * Workqueues have been created and work items queued on them, but there
+ * are no kworkers executing the work items yet. Populate the worker pools
+ * with the initial workers and enable future kworker creations.
+ */
+int __init workqueue_init(void)
+{
+ struct worker_pool *pool;
+ int cpu, bkt;
+
+ /* create the initial workers */
+ for_each_online_cpu(cpu) {
+ for_each_cpu_worker_pool(pool, cpu) {
+ pool->flags &= ~POOL_DISASSOCIATED;
+ BUG_ON(!create_worker(pool));
+ }
+ }
+
+ hash_for_each(unbound_pool_hash, bkt, pool, hash_node)
+ BUG_ON(!create_worker(pool));
+
+ wq_online = true;
wq_watchdog_init();
return 0;
}
-early_initcall(init_workqueues);
From: "Jason A. Donenfeld" <[email protected]>
commit 8566417221fcec51346ec164e920dacb979c6b5f upstream.
This is a preparatory commit for the following one. We simply inline the
various functions that rand_initialize() calls that have no other
callers. The compiler was doing this anyway before. Doing this will
allow us to reorganize this after. We can then move the trust_cpu and
parse_trust_cpu definitions a bit closer to where they're actually used,
which makes the code easier to read.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 90 ++++++++++++++++++--------------------------------
1 file changed, 33 insertions(+), 57 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -477,42 +477,6 @@ static DECLARE_WAIT_QUEUE_HEAD(crng_init
static void invalidate_batched_entropy(void);
-static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
-static int __init parse_trust_cpu(char *arg)
-{
- return kstrtobool(arg, &trust_cpu);
-}
-early_param("random.trust_cpu", parse_trust_cpu);
-
-static bool __init crng_init_try_arch_early(void)
-{
- int i;
- bool arch_init = true;
- unsigned long rv;
-
- for (i = 4; i < 16; i++) {
- if (!arch_get_random_seed_long_early(&rv) &&
- !arch_get_random_long_early(&rv)) {
- rv = random_get_entropy();
- arch_init = false;
- }
- primary_crng.state[i] ^= rv;
- }
-
- return arch_init;
-}
-
-static void __init crng_initialize(void)
-{
- extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
- if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) {
- invalidate_batched_entropy();
- crng_init = 2;
- pr_notice("crng init done (trusting CPU's manufacturer)\n");
- }
- primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
-}
-
/*
* crng_fast_load() can be called by code in the interrupt service
* path. So we can't afford to dilly-dally. Returns the number of
@@ -1272,17 +1236,28 @@ int __must_check get_random_bytes_arch(v
}
EXPORT_SYMBOL(get_random_bytes_arch);
+static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU);
+static int __init parse_trust_cpu(char *arg)
+{
+ return kstrtobool(arg, &trust_cpu);
+}
+early_param("random.trust_cpu", parse_trust_cpu);
+
/*
- * init_std_data - initialize pool with system data
- *
- * This function clears the pool's entropy count and mixes some system
- * data into the pool to prepare it for use. The pool is not cleared
- * as that can only decrease the entropy in the pool.
+ * Note that setup_arch() may call add_device_randomness()
+ * long before we get here. This allows seeding of the pools
+ * with some platform dependent data very early in the boot
+ * process. But it limits our options here. We must use
+ * statically allocated structures that already have all
+ * initializations complete at compile time. We should also
+ * take care not to overwrite the precious per platform data
+ * we were given.
*/
-static void __init init_std_data(void)
+int __init rand_initialize(void)
{
int i;
ktime_t now = ktime_get_real();
+ bool arch_init = true;
unsigned long rv;
mix_pool_bytes(&now, sizeof(now));
@@ -1293,22 +1268,23 @@ static void __init init_std_data(void)
mix_pool_bytes(&rv, sizeof(rv));
}
mix_pool_bytes(utsname(), sizeof(*(utsname())));
-}
-/*
- * Note that setup_arch() may call add_device_randomness()
- * long before we get here. This allows seeding of the pools
- * with some platform dependent data very early in the boot
- * process. But it limits our options here. We must use
- * statically allocated structures that already have all
- * initializations complete at compile time. We should also
- * take care not to overwrite the precious per platform data
- * we were given.
- */
-int __init rand_initialize(void)
-{
- init_std_data();
- crng_initialize();
+ extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
+ for (i = 4; i < 16; i++) {
+ if (!arch_get_random_seed_long_early(&rv) &&
+ !arch_get_random_long_early(&rv)) {
+ rv = random_get_entropy();
+ arch_init = false;
+ }
+ primary_crng.state[i] ^= rv;
+ }
+ if (arch_init && trust_cpu && crng_init < 2) {
+ invalidate_batched_entropy();
+ crng_init = 2;
+ pr_notice("crng init done (trusting CPU's manufacturer)\n");
+ }
+ primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
+
if (ratelimit_disable) {
urandom_warning.interval = 0;
unseeded_warning.interval = 0;
From: Nicolai Stange <[email protected]>
commit 2bcd25443868aa8863779a6ebc6c9319633025d2 upstream.
Currently, the DRBG implementation schedules asynchronous works from
random_ready_callbacks for reseeding the DRBG instances with output from
get_random_bytes() once the latter has sufficient entropy available.
However, as the get_random_bytes() initialization state can get queried by
means of rng_is_initialized() now, there is no real need for this
asynchronous reseeding logic anymore and it's better to keep things simple
by doing it synchronously when needed instead, i.e. from drbg_generate()
once rng_is_initialized() has flipped to true.
Of course, for this to work, drbg_generate() would need some means by which
it can tell whether or not rng_is_initialized() has flipped to true since
the last seeding from get_random_bytes(). Or equivalently, whether or not
the last seed from get_random_bytes() has happened when
rng_is_initialized() was still evaluating to false.
As it currently stands, enum drbg_seed_state allows for the representation
of two different DRBG seeding states: DRBG_SEED_STATE_UNSEEDED and
DRBG_SEED_STATE_FULL. The former makes drbg_generate() to invoke a full
reseeding operation involving both, the rather expensive jitterentropy as
well as the get_random_bytes() randomness sources. The DRBG_SEED_STATE_FULL
state on the other hand implies that no reseeding at all is required for a
!->pr DRBG variant.
Introduce the new DRBG_SEED_STATE_PARTIAL state to enum drbg_seed_state for
representing the condition that a DRBG was being seeded when
rng_is_initialized() had still been false. In particular, this new state
implies that
- the given DRBG instance has been fully seeded from the jitterentropy
source (if enabled)
- and drbg_generate() is supposed to reseed from get_random_bytes()
*only* once rng_is_initialized() turns to true.
Up to now, the __drbg_seed() helper used to set the given DRBG instance's
->seeded state to constant DRBG_SEED_STATE_FULL. Introduce a new argument
allowing for the specification of the to be written ->seeded value instead.
Make the first of its two callers, drbg_seed(), determine the appropriate
value based on rng_is_initialized(). The remaining caller,
drbg_async_seed(), is known to get invoked only once rng_is_initialized()
is true, hence let it pass constant DRBG_SEED_STATE_FULL for the new
argument to __drbg_seed().
There is no change in behaviour, except for that the pr_devel() in
drbg_generate() would now report "unseeded" for ->pr DRBG instances which
had last been seeded when rng_is_initialized() was still evaluating to
false.
Signed-off-by: Nicolai Stange <[email protected]>
Reviewed-by: Stephan Müller <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 12 ++++++++----
include/crypto/drbg.h | 1 +
2 files changed, 9 insertions(+), 4 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1037,14 +1037,14 @@ static const struct drbg_state_ops drbg_
******************************************************************/
static inline int __drbg_seed(struct drbg_state *drbg, struct list_head *seed,
- int reseed)
+ int reseed, enum drbg_seed_state new_seed_state)
{
int ret = drbg->d_ops->update(drbg, seed, reseed);
if (ret)
return ret;
- drbg->seeded = DRBG_SEED_STATE_FULL;
+ drbg->seeded = new_seed_state;
/* 10.1.1.2 / 10.1.1.3 step 5 */
drbg->reseed_ctr = 1;
@@ -1094,7 +1094,7 @@ static void drbg_async_seed(struct work_
*/
drbg->seeded = DRBG_SEED_STATE_UNSEEDED;
- __drbg_seed(drbg, &seedlist, true);
+ __drbg_seed(drbg, &seedlist, true, DRBG_SEED_STATE_FULL);
if (drbg->seeded == DRBG_SEED_STATE_FULL)
drbg->reseed_threshold = drbg_max_requests(drbg);
@@ -1124,6 +1124,7 @@ static int drbg_seed(struct drbg_state *
unsigned int entropylen = drbg_sec_strength(drbg->core->flags);
struct drbg_string data1;
LIST_HEAD(seedlist);
+ enum drbg_seed_state new_seed_state = DRBG_SEED_STATE_FULL;
/* 9.1 / 9.2 / 9.3.1 step 3 */
if (pers && pers->len > (drbg_max_addtl(drbg))) {
@@ -1151,6 +1152,9 @@ static int drbg_seed(struct drbg_state *
BUG_ON((entropylen * 2) > sizeof(entropy));
/* Get seed from in-kernel /dev/urandom */
+ if (!rng_is_initialized())
+ new_seed_state = DRBG_SEED_STATE_PARTIAL;
+
ret = drbg_get_random_bytes(drbg, entropy, entropylen);
if (ret)
goto out;
@@ -1207,7 +1211,7 @@ static int drbg_seed(struct drbg_state *
memset(drbg->C, 0, drbg_statelen(drbg));
}
- ret = __drbg_seed(drbg, &seedlist, reseed);
+ ret = __drbg_seed(drbg, &seedlist, reseed, new_seed_state);
out:
memzero_explicit(entropy, entropylen * 2);
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -107,6 +107,7 @@ struct drbg_test_data {
enum drbg_seed_state {
DRBG_SEED_STATE_UNSEEDED,
+ DRBG_SEED_STATE_PARTIAL, /* Seeded with !rng_is_initialized() */
DRBG_SEED_STATE_FULL,
};
From: Mark Rutland <[email protected]>
commit 5cbe0f13b51ac2fb2fd55902cff8d0077fc084c0 upstream.
Currently crng_initialize() is used for both the primary CRNG and
secondary CRNGs. While we wish to share common logic, we need to do a
number of additional things for the primary CRNG, and this would be
easier to deal with were these handled in separate functions.
This patch splits crng_initialize() into crng_initialize_primary() and
crng_initialize_secondary(), with common logic factored out into a
crng_init_try_arch() helper.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <[email protected]>
Cc: Mark Brown <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 36 ++++++++++++++++++++++++------------
1 file changed, 24 insertions(+), 12 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -783,27 +783,39 @@ static int __init parse_trust_cpu(char *
}
early_param("random.trust_cpu", parse_trust_cpu);
-static void crng_initialize(struct crng_state *crng)
+static bool crng_init_try_arch(struct crng_state *crng)
{
int i;
- int arch_init = 1;
+ bool arch_init = true;
unsigned long rv;
- memcpy(&crng->state[0], "expand 32-byte k", 16);
- if (crng == &primary_crng)
- _extract_entropy(&input_pool, &crng->state[4],
- sizeof(__u32) * 12, 0);
- else
- _get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
for (i = 4; i < 16; i++) {
if (!arch_get_random_seed_long(&rv) &&
!arch_get_random_long(&rv)) {
rv = random_get_entropy();
- arch_init = 0;
+ arch_init = false;
}
crng->state[i] ^= rv;
}
- if (trust_cpu && arch_init) {
+
+ return arch_init;
+}
+
+static void crng_initialize_secondary(struct crng_state *crng)
+{
+ memcpy(&crng->state[0], "expand 32-byte k", 16);
+ _get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
+ crng_init_try_arch(crng);
+ crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
+}
+
+static void __init crng_initialize_primary(struct crng_state *crng)
+{
+ memcpy(&crng->state[0], "expand 32-byte k", 16);
+ _extract_entropy(&input_pool, &crng->state[4], sizeof(__u32) * 12, 0);
+ if (crng_init_try_arch(crng) && trust_cpu) {
+ invalidate_batched_entropy();
+ numa_crng_init();
crng_init = 2;
pr_notice("crng done (trusting CPU's manufacturer)\n");
}
@@ -852,7 +864,7 @@ static void do_numa_crng_init(struct wor
crng = kmalloc_node(sizeof(struct crng_state),
GFP_KERNEL | __GFP_NOFAIL, i);
spin_lock_init(&crng->lock);
- crng_initialize(crng);
+ crng_initialize_secondary(crng);
pool[i] = crng;
}
/* pairs with READ_ONCE() in select_crng() */
@@ -1780,7 +1792,7 @@ int __init rand_initialize(void)
init_std_data(&input_pool);
if (crng_need_final_init)
crng_finalize_init(&primary_crng);
- crng_initialize(&primary_crng);
+ crng_initialize_primary(&primary_crng);
crng_global_init_time = jiffies;
if (ratelimit_disable) {
urandom_warning.interval = 0;
From: "Jason A. Donenfeld" <[email protected]>
commit 1c99c6a7c3c599a68321b01b9ec243215ede5a68 upstream.
For situations in which we don't have a c0 counter register available,
we've been falling back to reading the c0 "random" register, which is
usually bounded by the amount of TLB entries and changes every other
cycle or so. This means it wraps extremely often. We can do better by
combining this fast-changing counter with a potentially slower-changing
counter from random_get_entropy_fallback() in the more significant bits.
This commit combines the two, taking into account that the changing bits
are in a different bit position depending on the CPU model. In addition,
we previously were falling back to 0 for ancient CPUs that Linux does
not support anyway; remove that dead path entirely.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Tested-by: Maciej W. Rozycki <[email protected]>
Acked-by: Thomas Bogendoerfer <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/mips/include/asm/timex.h | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
--- a/arch/mips/include/asm/timex.h
+++ b/arch/mips/include/asm/timex.h
@@ -76,25 +76,24 @@ static inline cycles_t get_cycles(void)
else
return 0; /* no usable counter */
}
+#define get_cycles get_cycles
/*
* Like get_cycles - but where c0_count is not available we desperately
* use c0_random in an attempt to get at least a little bit of entropy.
- *
- * R6000 and R6000A neither have a count register nor a random register.
- * That leaves no entropy source in the CPU itself.
*/
static inline unsigned long random_get_entropy(void)
{
- unsigned int prid = read_c0_prid();
- unsigned int imp = prid & PRID_IMP_MASK;
+ unsigned int c0_random;
- if (can_use_mips_counter(prid))
+ if (can_use_mips_counter(read_c0_prid()))
return read_c0_count();
- else if (likely(imp != PRID_IMP_R6000 && imp != PRID_IMP_R6000A))
- return read_c0_random();
+
+ if (cpu_has_3kex)
+ c0_random = (read_c0_random() >> 8) & 0x3f;
else
- return 0; /* no usable register */
+ c0_random = read_c0_random() & 0x3f;
+ return (random_get_entropy_fallback() << 6) | (0x3f - c0_random);
}
#define random_get_entropy random_get_entropy
From: Xiaohui Zhang <[email protected]>
[ Upstream commit 8a4d480702b71184fabcf379b80bf7539716752e ]
Similar to the handling of play_deferred in commit 19cfe912c37b
("Bluetooth: btusb: Fix memory leak in play_deferred"), we thought
a patch might be needed here as well.
Currently usb_submit_urb is called directly to submit deferred tx
urbs after unanchor them.
So the usb_giveback_urb_bh would failed to unref it in usb_unanchor_urb
and cause memory leak.
Put those urbs in tx_anchor to avoid the leak, and also fix the error
handling.
Signed-off-by: Xiaohui Zhang <[email protected]>
Acked-by: Krzysztof Kozlowski <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/nfc/nfcmrvl/usb.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/nfc/nfcmrvl/usb.c b/drivers/nfc/nfcmrvl/usb.c
index 585a0f20835b..3263e2a2bdfd 100644
--- a/drivers/nfc/nfcmrvl/usb.c
+++ b/drivers/nfc/nfcmrvl/usb.c
@@ -401,13 +401,25 @@ static void nfcmrvl_play_deferred(struct nfcmrvl_usb_drv_data *drv_data)
int err;
while ((urb = usb_get_from_anchor(&drv_data->deferred))) {
+ usb_anchor_urb(urb, &drv_data->tx_anchor);
+
err = usb_submit_urb(urb, GFP_ATOMIC);
- if (err)
+ if (err) {
+ kfree(urb->setup_packet);
+ usb_unanchor_urb(urb);
+ usb_free_urb(urb);
break;
+ }
drv_data->tx_in_flight++;
+ usb_free_urb(urb);
+ }
+
+ /* Cleanup the rest deferred urbs. */
+ while ((urb = usb_get_from_anchor(&drv_data->deferred))) {
+ kfree(urb->setup_packet);
+ usb_free_urb(urb);
}
- usb_scuttle_anchored_urbs(&drv_data->deferred);
}
static int nfcmrvl_resume(struct usb_interface *intf)
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit f5eab0e2db4f881fb2b62b3fdad5b9be673dd7ae upstream.
The current fast_mix() function is a piece of classic mailing list
crypto, where it just sort of sprung up by an anonymous author without a
lot of real analysis of what precisely it was accomplishing. As an ARX
permutation alone, there are some easily searchable differential trails
in it, and as a means of preventing malicious interrupts, it completely
fails, since it xors new data into the entire state every time. It can't
really be analyzed as a random permutation, because it clearly isn't,
and it can't be analyzed as an interesting linear algebraic structure
either, because it's also not that. There really is very little one can
say about it in terms of entropy accumulation. It might diffuse bits,
some of the time, maybe, we hope, I guess. But for the most part, it
fails to accomplish anything concrete.
As a reminder, the simple goal of add_interrupt_randomness() is to
simply accumulate entropy until ~64 interrupts have elapsed, and then
dump it into the main input pool, which uses a cryptographic hash.
It would be nice to have something cryptographically strong in the
interrupt handler itself, in case a malicious interrupt compromises a
per-cpu fast pool within the 64 interrupts / 1 second window, and then
inside of that same window somehow can control its return address and
cycle counter, even if that's a bit far fetched. However, with a very
CPU-limited budget, actually doing that remains an active research
project (and perhaps there'll be something useful for Linux to come out
of it). And while the abundance of caution would be nice, this isn't
*currently* the security model, and we don't yet have a fast enough
solution to make it our security model. Plus there's not exactly a
pressing need to do that. (And for the avoidance of doubt, the actual
cluster of 64 accumulated interrupts still gets dumped into our
cryptographically secure input pool.)
So, for now we are going to stick with the existing interrupt security
model, which assumes that each cluster of 64 interrupt data samples is
mostly non-malicious and not colluding with an infoleaker. With this as
our goal, we have a few more choices, simply aiming to accumulate
entropy, while discarding the least amount of it.
We know from <https://eprint.iacr.org/2019/198> that random oracles,
instantiated as computational hash functions, make good entropy
accumulators and extractors, which is the justification for using
BLAKE2s in the main input pool. As mentioned, we don't have that luxury
here, but we also don't have the same security model requirements,
because we're assuming that there aren't malicious inputs. A
pseudorandom function instance can approximately behave like a random
oracle, provided that the key is uniformly random. But since we're not
concerned with malicious inputs, we can pick a fixed key, which is not
secret, knowing that "nature" won't interact with a sufficiently chosen
fixed key by accident. So we pick a PRF with a fixed initial key, and
accumulate into it continuously, dumping the result every 64 interrupts
into our cryptographically secure input pool.
For this, we make use of SipHash-1-x on 64-bit and HalfSipHash-1-x on
32-bit, which are already in use in the kernel's hsiphash family of
functions and achieve the same performance as the function they replace.
It would be nice to do two rounds, but we don't exactly have the CPU
budget handy for that, and one round alone is already sufficient.
As mentioned, we start with a fixed initial key (zeros is fine), and
allow SipHash's symmetry breaking constants to turn that into a useful
starting point. Also, since we're dumping the result (or half of it on
64-bit so as to tax our hash function the same amount on all platforms)
into the cryptographically secure input pool, there's no point in
finalizing SipHash's output, since it'll wind up being finalized by
something much stronger. This means that all we need to do is use the
ordinary round function word-by-word, as normal SipHash does.
Simplified, the flow is as follows:
Initialize:
siphash_state_t state;
siphash_init(&state, key={0, 0, 0, 0});
Update (accumulate) on interrupt:
siphash_update(&state, interrupt_data_and_timing);
Dump into input pool after 64 interrupts:
blake2s_update(&input_pool, &state, sizeof(state) / 2);
The result of all of this is that the security model is unchanged from
before -- we assume non-malicious inputs -- yet we now implement that
model with a stronger argument. I would like to emphasize, again, that
the purpose of this commit is to improve the existing design, by making
it analyzable, without changing any fundamental assumptions. There may
well be value down the road in changing up the existing design, using
something cryptographically strong, or simply using a ring buffer of
samples rather than having a fast_mix() at all, or changing which and
how much data we collect each interrupt so that we can use something
linear, or a variety of other ideas. This commit does not invalidate the
potential for those in the future.
For example, in the future, if we're able to characterize the data we're
collecting on each interrupt, we may be able to inch toward information
theoretic accumulators. <https://eprint.iacr.org/2021/523> shows that `s
= ror32(s, 7) ^ x` and `s = ror64(s, 19) ^ x` make very good
accumulators for 2-monotone distributions, which would apply to
timestamp counters, like random_get_entropy() or jiffies, but would not
apply to our current combination of the two values, or to the various
function addresses and register values we mix in. Alternatively,
<https://eprint.iacr.org/2021/1002> shows that max-period linear
functions with no non-trivial invariant subspace make good extractors,
used in the form `s = f(s) ^ x`. However, this only works if the input
data is both identical and independent, and obviously a collection of
address values and counters fails; so it goes with theoretical papers.
Future directions here may involve trying to characterize more precisely
what we actually need to collect in the interrupt handler, and building
something specific around that.
However, as mentioned, the morass of data we're gathering at the
interrupt handler presently defies characterization, and so we use
SipHash for now, which works well and performs well.
Cc: Theodore Ts'o <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Reviewed-by: Jean-Philippe Aumasson <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 94 +++++++++++++++++++++++++++++---------------------
1 file changed, 55 insertions(+), 39 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1143,48 +1143,51 @@ void add_bootloader_randomness(const voi
EXPORT_SYMBOL_GPL(add_bootloader_randomness);
struct fast_pool {
- union {
- u32 pool32[4];
- u64 pool64[2];
- };
struct work_struct mix;
+ unsigned long pool[4];
unsigned long last;
unsigned int count;
u16 reg_idx;
};
+static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = {
+#ifdef CONFIG_64BIT
+ /* SipHash constants */
+ .pool = { 0x736f6d6570736575UL, 0x646f72616e646f6dUL,
+ 0x6c7967656e657261UL, 0x7465646279746573UL }
+#else
+ /* HalfSipHash constants */
+ .pool = { 0, 0, 0x6c796765U, 0x74656462U }
+#endif
+};
+
/*
- * This is a fast mixing routine used by the interrupt randomness
- * collector. It's hardcoded for an 128 bit pool and assumes that any
- * locks that might be needed are taken by the caller.
+ * This is [Half]SipHash-1-x, starting from an empty key. Because
+ * the key is fixed, it assumes that its inputs are non-malicious,
+ * and therefore this has no security on its own. s represents the
+ * 128 or 256-bit SipHash state, while v represents a 128-bit input.
*/
-static void fast_mix(u32 pool[4])
+static void fast_mix(unsigned long s[4], const unsigned long *v)
{
- u32 a = pool[0], b = pool[1];
- u32 c = pool[2], d = pool[3];
-
- a += b; c += d;
- b = rol32(b, 6); d = rol32(d, 27);
- d ^= a; b ^= c;
-
- a += b; c += d;
- b = rol32(b, 16); d = rol32(d, 14);
- d ^= a; b ^= c;
-
- a += b; c += d;
- b = rol32(b, 6); d = rol32(d, 27);
- d ^= a; b ^= c;
-
- a += b; c += d;
- b = rol32(b, 16); d = rol32(d, 14);
- d ^= a; b ^= c;
+ size_t i;
- pool[0] = a; pool[1] = b;
- pool[2] = c; pool[3] = d;
+ for (i = 0; i < 16 / sizeof(long); ++i) {
+ s[3] ^= v[i];
+#ifdef CONFIG_64BIT
+ s[0] += s[1]; s[1] = rol64(s[1], 13); s[1] ^= s[0]; s[0] = rol64(s[0], 32);
+ s[2] += s[3]; s[3] = rol64(s[3], 16); s[3] ^= s[2];
+ s[0] += s[3]; s[3] = rol64(s[3], 21); s[3] ^= s[0];
+ s[2] += s[1]; s[1] = rol64(s[1], 17); s[1] ^= s[2]; s[2] = rol64(s[2], 32);
+#else
+ s[0] += s[1]; s[1] = rol32(s[1], 5); s[1] ^= s[0]; s[0] = rol32(s[0], 16);
+ s[2] += s[3]; s[3] = rol32(s[3], 8); s[3] ^= s[2];
+ s[0] += s[3]; s[3] = rol32(s[3], 7); s[3] ^= s[0];
+ s[2] += s[1]; s[1] = rol32(s[1], 13); s[1] ^= s[2]; s[2] = rol32(s[2], 16);
+#endif
+ s[0] ^= v[i];
+ }
}
-static DEFINE_PER_CPU(struct fast_pool, irq_randomness);
-
#ifdef CONFIG_SMP
/*
* This function is called when the CPU has just come online, with
@@ -1226,7 +1229,15 @@ static unsigned long get_reg(struct fast
static void mix_interrupt_randomness(struct work_struct *work)
{
struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix);
- u32 pool[4];
+ /*
+ * The size of the copied stack pool is explicitly 16 bytes so that we
+ * tax mix_pool_byte()'s compression function the same amount on all
+ * platforms. This means on 64-bit we copy half the pool into this,
+ * while on 32-bit we copy all of it. The entropy is supposed to be
+ * sufficiently dispersed between bits that in the sponge-like
+ * half case, on average we don't wind up "losing" some.
+ */
+ u8 pool[16];
/* Check to see if we're running on the wrong CPU due to hotplug. */
local_irq_disable();
@@ -1239,7 +1250,7 @@ static void mix_interrupt_randomness(str
* Copy the pool to the stack so that the mixer always has a
* consistent view, before we reenable irqs again.
*/
- memcpy(pool, fast_pool->pool32, sizeof(pool));
+ memcpy(pool, fast_pool->pool, sizeof(pool));
fast_pool->count = 0;
fast_pool->last = jiffies;
local_irq_enable();
@@ -1263,25 +1274,30 @@ void add_interrupt_randomness(int irq)
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
struct pt_regs *regs = get_irq_regs();
unsigned int new_count;
+ union {
+ u32 u32[4];
+ u64 u64[2];
+ unsigned long longs[16 / sizeof(long)];
+ } irq_data;
if (cycles == 0)
cycles = get_reg(fast_pool, regs);
if (sizeof(cycles) == 8)
- fast_pool->pool64[0] ^= cycles ^ rol64(now, 32) ^ irq;
+ irq_data.u64[0] = cycles ^ rol64(now, 32) ^ irq;
else {
- fast_pool->pool32[0] ^= cycles ^ irq;
- fast_pool->pool32[1] ^= now;
+ irq_data.u32[0] = cycles ^ irq;
+ irq_data.u32[1] = now;
}
if (sizeof(unsigned long) == 8)
- fast_pool->pool64[1] ^= regs ? instruction_pointer(regs) : _RET_IP_;
+ irq_data.u64[1] = regs ? instruction_pointer(regs) : _RET_IP_;
else {
- fast_pool->pool32[2] ^= regs ? instruction_pointer(regs) : _RET_IP_;
- fast_pool->pool32[3] ^= get_reg(fast_pool, regs);
+ irq_data.u32[2] = regs ? instruction_pointer(regs) : _RET_IP_;
+ irq_data.u32[3] = get_reg(fast_pool, regs);
}
- fast_mix(fast_pool->pool32);
+ fast_mix(fast_pool->pool, irq_data.longs);
new_count = ++fast_pool->count;
if (new_count & MIX_INFLIGHT)
From: Nicolai Stange <[email protected]>
commit 262d83a4290c331cd4f617a457408bdb82fbb738 upstream.
Since commit 42ea507fae1a ("crypto: drbg - reseed often if seedsource is
degraded"), the maximum seed lifetime represented by ->reseed_threshold
gets temporarily lowered if the get_random_bytes() source cannot provide
sufficient entropy yet, as is common during boot, and restored back to
the original value again once that has changed.
More specifically, if the add_random_ready_callback() invoked from
drbg_prepare_hrng() in the course of DRBG instantiation does not return
-EALREADY, that is, if get_random_bytes() has not been fully initialized
at this point yet, drbg_prepare_hrng() will lower ->reseed_threshold
to a value of 50. The drbg_async_seed() scheduled from said
random_ready_callback will eventually restore the original value.
A future patch will replace the random_ready_callback based notification
mechanism and thus, there will be no add_random_ready_callback() return
value anymore which could get compared to -EALREADY.
However, there's __drbg_seed() which gets invoked in the course of both,
the DRBG instantiation as well as the eventual reseeding from
get_random_bytes() in aforementioned drbg_async_seed(), if any. Moreover,
it knows about the get_random_bytes() initialization state by the time the
seed data had been obtained from it: the new_seed_state argument introduced
with the previous patch would get set to DRBG_SEED_STATE_PARTIAL in case
get_random_bytes() had not been fully initialized yet and to
DRBG_SEED_STATE_FULL otherwise. Thus, __drbg_seed() provides a convenient
alternative for managing that ->reseed_threshold lowering and restoring at
a central place.
Move all ->reseed_threshold adjustment code from drbg_prepare_hrng() and
drbg_async_seed() respectively to __drbg_seed(). Make __drbg_seed()
lower the ->reseed_threshold to 50 in case its new_seed_state argument
equals DRBG_SEED_STATE_PARTIAL and let it restore the original value
otherwise.
There is no change in behaviour.
Signed-off-by: Nicolai Stange <[email protected]>
Reviewed-by: Stephan Müller <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 29 ++++++++++++++++++++---------
1 file changed, 20 insertions(+), 9 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1048,6 +1048,26 @@ static inline int __drbg_seed(struct drb
/* 10.1.1.2 / 10.1.1.3 step 5 */
drbg->reseed_ctr = 1;
+ switch (drbg->seeded) {
+ case DRBG_SEED_STATE_UNSEEDED:
+ /* Impossible, but handle it to silence compiler warnings. */
+ case DRBG_SEED_STATE_PARTIAL:
+ /*
+ * Require frequent reseeds until the seed source is
+ * fully initialized.
+ */
+ drbg->reseed_threshold = 50;
+ break;
+
+ case DRBG_SEED_STATE_FULL:
+ /*
+ * Seed source has become fully initialized, frequent
+ * reseeds no longer required.
+ */
+ drbg->reseed_threshold = drbg_max_requests(drbg);
+ break;
+ }
+
return ret;
}
@@ -1096,9 +1116,6 @@ static void drbg_async_seed(struct work_
__drbg_seed(drbg, &seedlist, true, DRBG_SEED_STATE_FULL);
- if (drbg->seeded == DRBG_SEED_STATE_FULL)
- drbg->reseed_threshold = drbg_max_requests(drbg);
-
unlock:
mutex_unlock(&drbg->drbg_mutex);
@@ -1532,12 +1549,6 @@ static int drbg_prepare_hrng(struct drbg
return err;
}
- /*
- * Require frequent reseeds until the seed source is fully
- * initialized.
- */
- drbg->reseed_threshold = 50;
-
return err;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 0f392c95391f2d708b12971a07edaa7973f9eece upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is really not the best we can do.
Instead, at least calling random_get_entropy_fallback() would be
preferable, because that always needs to return _something_, even
falling back to jiffies eventually. It's not as though
random_get_entropy_fallback() is super high precision or guaranteed to
be entropic, but basically anything that's not zero all the time is
better than returning zero all the time.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Acked-by: Geert Uytterhoeven <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/m68k/include/asm/timex.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/arch/m68k/include/asm/timex.h
+++ b/arch/m68k/include/asm/timex.h
@@ -34,7 +34,7 @@ static inline unsigned long random_get_e
{
if (mach_random_get_entropy)
return mach_random_get_entropy();
- return 0;
+ return random_get_entropy_fallback();
}
#define random_get_entropy random_get_entropy
From: Richard Henderson <[email protected]>
commit 904caa6413c87aacbf7d0682da617c39ca18cf1a upstream.
We must not use the pointer output without validating the
success of the random read.
Reviewed-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Richard Henderson <[email protected]>
Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/random.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -96,19 +96,19 @@ unsigned long randomize_page(unsigned lo
#ifdef CONFIG_ARCH_RANDOM
# include <asm/archrandom.h>
#else
-static inline bool arch_get_random_long(unsigned long *v)
+static inline bool __must_check arch_get_random_long(unsigned long *v)
{
return false;
}
-static inline bool arch_get_random_int(unsigned int *v)
+static inline bool __must_check arch_get_random_int(unsigned int *v)
{
return false;
}
-static inline bool arch_get_random_seed_long(unsigned long *v)
+static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
{
return false;
}
-static inline bool arch_get_random_seed_int(unsigned int *v)
+static inline bool __must_check arch_get_random_seed_int(unsigned int *v)
{
return false;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 069c4ea6871c18bd368f27756e0f91ffb524a788 upstream.
A semicolon was missing, and the almost-alphabetical-but-not ordering
was confusing, so regroup these by category instead.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/sysctl/kernel.txt | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -791,6 +791,9 @@ This is a directory, with the following
* ``boot_id``: a UUID generated the first time this is retrieved, and
unvarying after that;
+* ``uuid``: a UUID generated every time this is retrieved (this can
+ thus be used to generate UUIDs at will);
+
* ``entropy_avail``: the pool's entropy count, in bits;
* ``poolsize``: the entropy pool size, in bits;
@@ -798,10 +801,7 @@ This is a directory, with the following
* ``urandom_min_reseed_secs``: obsolete (used to determine the minimum
number of seconds between urandom pool reseeding). This file is
writable for compatibility purposes, but writing to it has no effect
- on any RNG behavior.
-
-* ``uuid``: a UUID generated every time this is retrieved (this can
- thus be used to generate UUIDs at will);
+ on any RNG behavior;
* ``write_wakeup_threshold``: when the entropy count drops below this
(as a number of bits), processes waiting to write to ``/dev/random``
From: Yangtao Li <[email protected]>
commit 870e05b1b18814911cb2703a977f447cb974f0f9 upstream.
WARN_ON() already contains an unlikely(), so it's not necessary to use
unlikely.
Signed-off-by: Yangtao Li <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -739,10 +739,9 @@ retry:
} while (unlikely(entropy_count < pool_size-2 && pnfrac));
}
- if (unlikely(entropy_count < 0)) {
+ if (WARN_ON(entropy_count < 0)) {
pr_warn("random: negative entropy/overflow: pool %s count %d\n",
r->name, entropy_count);
- WARN_ON(1);
entropy_count = 0;
} else if (entropy_count > pool_size)
entropy_count = pool_size;
@@ -1432,10 +1431,9 @@ retry:
if (ibytes < min)
ibytes = 0;
- if (unlikely(entropy_count < 0)) {
+ if (WARN_ON(entropy_count < 0)) {
pr_warn("random: negative entropy count: pool %s count %d\n",
r->name, entropy_count);
- WARN_ON(1);
entropy_count = 0;
}
nfrac = ibytes << (ENTROPY_SHIFT + 3);
From: Dominik Brodowski <[email protected]>
commit f7e67b8e803185d0aabe7f29d25a35c8be724a78 upstream.
Currently, if CONFIG_RANDOM_TRUST_BOOTLOADER is enabled, multiple calls
to add_bootloader_randomness() are broken and can cause a NULL pointer
dereference, as noted by Ivan T. Ivanov. This is not only a hypothetical
problem, as qemu on arm64 may provide bootloader entropy via EFI and via
devicetree.
On the first call to add_hwgenerator_randomness(), crng_fast_load() is
executed, and if the seed is long enough, crng_init will be set to 1.
On subsequent calls to add_bootloader_randomness() and then to
add_hwgenerator_randomness(), crng_fast_load() will be skipped. Instead,
wait_event_interruptible() and then credit_entropy_bits() will be called.
If the entropy count for that second seed is large enough, that proceeds
to crng_reseed().
However, both wait_event_interruptible() and crng_reseed() depends
(at least in numa_crng_init()) on workqueues. Therefore, test whether
system_wq is already initialized, which is a sufficient indicator that
workqueue_init_early() has progressed far enough.
If we wind up hitting the !system_wq case, we later want to do what
would have been done there when wqs are up, so set a flag, and do that
work later from the rand_initialize() call.
Reported-by: Ivan T. Ivanov <[email protected]>
Fixes: 18b915ac6b0a ("efi/random: Treat EFI_RNG_PROTOCOL output as bootloader randomness")
Cc: [email protected]
Signed-off-by: Dominik Brodowski <[email protected]>
[Jason: added crng_need_done state and related logic.]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 60 +++++++++++++++++++++++++++++++-------------------
1 file changed, 38 insertions(+), 22 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -497,6 +497,7 @@ static struct crng_state primary_crng =
* its value (from 0->1->2).
*/
static int crng_init = 0;
+static bool crng_need_final_init = false;
#define crng_ready() (likely(crng_init > 1))
static int crng_init_cnt = 0;
static unsigned long crng_global_init_time = 0;
@@ -886,6 +887,38 @@ static void crng_initialize(struct crng_
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
+static void crng_finalize_init(struct crng_state *crng)
+{
+ if (crng != &primary_crng || crng_init >= 2)
+ return;
+ if (!system_wq) {
+ /* We can't call numa_crng_init until we have workqueues,
+ * so mark this for processing later. */
+ crng_need_final_init = true;
+ return;
+ }
+
+ invalidate_batched_entropy();
+ numa_crng_init();
+ crng_init = 2;
+ process_random_ready_list();
+ wake_up_interruptible(&crng_init_wait);
+ kill_fasync(&fasync, SIGIO, POLL_IN);
+ pr_notice("crng init done\n");
+ if (unseeded_warning.missed) {
+ pr_notice("random: %d get_random_xx warning(s) missed "
+ "due to ratelimiting\n",
+ unseeded_warning.missed);
+ unseeded_warning.missed = 0;
+ }
+ if (urandom_warning.missed) {
+ pr_notice("random: %d urandom warning(s) missed "
+ "due to ratelimiting\n",
+ urandom_warning.missed);
+ urandom_warning.missed = 0;
+ }
+}
+
#ifdef CONFIG_NUMA
static void do_numa_crng_init(struct work_struct *work)
{
@@ -1091,27 +1124,7 @@ static void crng_reseed(struct crng_stat
memzero_explicit(&buf, sizeof(buf));
WRITE_ONCE(crng->init_time, jiffies);
spin_unlock_irqrestore(&crng->lock, flags);
- if (crng == &primary_crng && crng_init < 2) {
- numa_crng_init();
- invalidate_batched_entropy();
- numa_crng_init();
- crng_init = 2;
- process_random_ready_list();
- wake_up_interruptible(&crng_init_wait);
- pr_notice("random: crng init done\n");
- if (unseeded_warning.missed) {
- pr_notice("random: %d get_random_xx warning(s) missed "
- "due to ratelimiting\n",
- unseeded_warning.missed);
- unseeded_warning.missed = 0;
- }
- if (urandom_warning.missed) {
- pr_notice("random: %d urandom warning(s) missed "
- "due to ratelimiting\n",
- urandom_warning.missed);
- urandom_warning.missed = 0;
- }
- }
+ crng_finalize_init(crng);
}
static void _extract_crng(struct crng_state *crng,
@@ -1949,6 +1962,8 @@ int __init rand_initialize(void)
{
init_std_data(&input_pool);
init_std_data(&blocking_pool);
+ if (crng_need_final_init)
+ crng_finalize_init(&primary_crng);
crng_initialize(&primary_crng);
crng_global_init_time = jiffies;
if (ratelimit_disable) {
@@ -2467,7 +2482,8 @@ void add_hwgenerator_randomness(const ch
* We'll be woken up again once below random_write_wakeup_thresh,
* or when the calling thread is about to terminate.
*/
- wait_event_interruptible(random_write_wait, kthread_should_stop() ||
+ wait_event_interruptible(random_write_wait,
+ !system_wq || kthread_should_stop() ||
ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
mix_pool_bytes(poolp, buffer, count);
credit_entropy_bits(poolp, entropy);
From: Willy Tarreau <[email protected]>
commit e8161345ddbb66e449abde10d2fdce93f867eba9 upstream.
In commit 190cc82489f4 ("tcp: change source port randomizarion at
connect() time"), the table_perturb[] array was introduced and an
index was taken from the port_offset via hash_32(). But it turns
out that hash_32() performs a multiplication while the input here
comes from the output of SipHash in secure_seq, that is well
distributed enough to avoid the need for yet another hash.
Suggested-by: Amit Klein <[email protected]>
Reviewed-by: Eric Dumazet <[email protected]>
Signed-off-by: Willy Tarreau <[email protected]>
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -590,7 +590,7 @@ int __inet_hash_connect(struct inet_time
net_get_random_once(table_perturb,
INET_TABLE_PERTURB_SIZE * sizeof(*table_perturb));
- index = hash_32(port_offset, INET_TABLE_PERTURB_SHIFT);
+ index = port_offset & (INET_TABLE_PERTURB_SIZE - 1);
offset = READ_ONCE(table_perturb[index]) + (port_offset >> 32);
offset %= remaining;
From: "Jason A. Donenfeld" <[email protected]>
commit a4bfa9b31802c14ff5847123c12b98d5e36b3985 upstream.
There were a few things added under the "if (fips_enabled)" banner,
which never really got completed, and the FIPS people anyway are
choosing a different direction. Rather than keep around this halfbaked
code, get rid of it so that we can focus on a single design of the RNG
rather than two designs.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 40 ++++------------------------------------
1 file changed, 4 insertions(+), 36 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -337,8 +337,6 @@
#include <linux/spinlock.h>
#include <linux/kthread.h>
#include <linux/percpu.h>
-#include <linux/cryptohash.h>
-#include <linux/fips.h>
#include <linux/ptrace.h>
#include <linux/kmemcheck.h>
#include <linux/workqueue.h>
@@ -519,14 +517,12 @@ struct entropy_store {
u16 add_ptr;
u16 input_rotate;
int entropy_count;
- unsigned int last_data_init:1;
- u8 last_data[EXTRACT_SIZE];
};
static ssize_t extract_entropy(struct entropy_store *r, void *buf,
size_t nbytes, int min, int rsvd);
static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes, int fips);
+ size_t nbytes);
static void crng_reseed(struct crng_state *crng, struct entropy_store *r);
static u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
@@ -822,7 +818,7 @@ static void crng_initialize_secondary(st
static void __init crng_initialize_primary(struct crng_state *crng)
{
- _extract_entropy(&input_pool, &crng->state[4], sizeof(u32) * 12, 0);
+ _extract_entropy(&input_pool, &crng->state[4], sizeof(u32) * 12);
if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
@@ -1478,22 +1474,13 @@ static void extract_buf(struct entropy_s
}
static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
- size_t nbytes, int fips)
+ size_t nbytes)
{
ssize_t ret = 0, i;
u8 tmp[EXTRACT_SIZE];
- unsigned long flags;
while (nbytes) {
extract_buf(r, tmp);
-
- if (fips) {
- spin_lock_irqsave(&r->lock, flags);
- if (!memcmp(tmp, r->last_data, EXTRACT_SIZE))
- panic("Hardware RNG duplicated output!\n");
- memcpy(r->last_data, tmp, EXTRACT_SIZE);
- spin_unlock_irqrestore(&r->lock, flags);
- }
i = min_t(int, nbytes, EXTRACT_SIZE);
memcpy(buf, tmp, i);
nbytes -= i;
@@ -1519,28 +1506,9 @@ static ssize_t _extract_entropy(struct e
static ssize_t extract_entropy(struct entropy_store *r, void *buf,
size_t nbytes, int min, int reserved)
{
- u8 tmp[EXTRACT_SIZE];
- unsigned long flags;
-
- /* if last_data isn't primed, we need EXTRACT_SIZE extra bytes */
- if (fips_enabled) {
- spin_lock_irqsave(&r->lock, flags);
- if (!r->last_data_init) {
- r->last_data_init = 1;
- spin_unlock_irqrestore(&r->lock, flags);
- trace_extract_entropy(r->name, EXTRACT_SIZE,
- ENTROPY_BITS(r), _RET_IP_);
- extract_buf(r, tmp);
- spin_lock_irqsave(&r->lock, flags);
- memcpy(r->last_data, tmp, EXTRACT_SIZE);
- }
- spin_unlock_irqrestore(&r->lock, flags);
- }
-
trace_extract_entropy(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
nbytes = account(r, nbytes, min, reserved);
-
- return _extract_entropy(r, buf, nbytes, fips_enabled);
+ return _extract_entropy(r, buf, nbytes);
}
#define warn_unseeded_randomness(previous) \
From: Ian Abbott <[email protected]>
commit 242439f7e279d86b3f73b5de724bc67b2f8aeb07 upstream.
The expression for setting the size of the allocated bulk TX buffer
(`devpriv->usb_tx_buf`) is calling `usb_endpoint_maxp(devpriv->ep_rx)`,
which is using the wrong endpoint (should be `devpriv->ep_tx`). Fix it.
Fixes: a23461c47482 ("comedi: vmk80xx: fix transfer-buffer overflow")
Cc: Johan Hovold <[email protected]>
Cc: [email protected] # 4.9+
Reviewed-by: Johan Hovold <[email protected]>
Signed-off-by: Ian Abbott <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/staging/comedi/drivers/vmk80xx.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/staging/comedi/drivers/vmk80xx.c
+++ b/drivers/staging/comedi/drivers/vmk80xx.c
@@ -694,7 +694,7 @@ static int vmk80xx_alloc_usb_buffers(str
if (!devpriv->usb_rx_buf)
return -ENOMEM;
- size = max(usb_endpoint_maxp(devpriv->ep_rx), MIN_BUF_SIZE);
+ size = max(usb_endpoint_maxp(devpriv->ep_tx), MIN_BUF_SIZE);
devpriv->usb_tx_buf = kzalloc(size, GFP_KERNEL);
if (!devpriv->usb_tx_buf)
return -ENOMEM;
From: "Jason A. Donenfeld" <[email protected]>
commit e10e2f58030c5c211d49042a8c2a1b93d40b2ffb upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is really not the best we can do.
Instead, at least calling random_get_entropy_fallback() would be
preferable, because that always needs to return _something_, even
falling back to jiffies eventually. It's not as though
random_get_entropy_fallback() is super high precision or guaranteed to
be entropic, but basically anything that's not zero all the time is
better than returning zero all the time.
This is accomplished by just including the asm-generic code like on
other architectures, which means we can get rid of the empty stub
function here.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Acked-by: Max Filippov <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/xtensa/include/asm/timex.h | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
--- a/arch/xtensa/include/asm/timex.h
+++ b/arch/xtensa/include/asm/timex.h
@@ -30,10 +30,6 @@
extern unsigned long ccount_freq;
-typedef unsigned long long cycles_t;
-
-#define get_cycles() (0)
-
void local_timer_setup(unsigned cpu);
/*
@@ -69,4 +65,6 @@ static inline void set_linux_timer (unsi
WSR_CCOMPARE(LINUX_TIMER, ccompare);
}
+#include <asm-generic/timex.h>
+
#endif /* _XTENSA_TIMEX_H */
From: Stephen Boyd <[email protected]>
commit ff296293b3538d19278a7f7cd1f3aa600ad9164c upstream.
The kthread calling this function is freezable after commit 03a3bb7ae631
("hwrng: core - Freeze khwrng thread during suspend") is applied.
Unfortunately, this function uses wait_event_interruptible() but doesn't
check for the kthread being woken up by the fake freezer signal. When a
user suspends the system, this kthread will wake up and if it fails the
entropy size check it will immediately go back to sleep and not go into
the freezer. Eventually, suspend will fail because the task never froze
and a warning message like this may appear:
PM: suspend entry (deep)
Filesystems sync: 0.000 seconds
Freezing user space processes ... (elapsed 0.001 seconds) done.
OOM killer disabled.
Freezing remaining freezable tasks ...
Freezing of tasks failed after 20.003 seconds (1 tasks refusing to freeze, wq_busy=0):
hwrng R running task 0 289 2 0x00000020
[<c08c64c4>] (__schedule) from [<c08c6a10>] (schedule+0x3c/0xc0)
[<c08c6a10>] (schedule) from [<c05dbd8c>] (add_hwgenerator_randomness+0xb0/0x100)
[<c05dbd8c>] (add_hwgenerator_randomness) from [<bf1803c8>] (hwrng_fillfn+0xc0/0x14c [rng_core])
[<bf1803c8>] (hwrng_fillfn [rng_core]) from [<c015abec>] (kthread+0x134/0x148)
[<c015abec>] (kthread) from [<c01010e8>] (ret_from_fork+0x14/0x2c)
Check for a freezer signal here and skip adding any randomness if the
task wakes up because it was frozen. This should make the kthread freeze
properly and suspend work again.
Fixes: 03a3bb7ae631 ("hwrng: core - Freeze khwrng thread during suspend")
Reported-by: Keerthy <[email protected]>
Tested-by: Keerthy <[email protected]>
Signed-off-by: Stephen Boyd <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2473,6 +2473,7 @@ void add_hwgenerator_randomness(const ch
size_t entropy)
{
struct entropy_store *poolp = &input_pool;
+ bool frozen = false;
if (unlikely(crng_init == 0)) {
crng_fast_load(buffer, count);
@@ -2483,9 +2484,12 @@ void add_hwgenerator_randomness(const ch
* We'll be woken up again once below random_write_wakeup_thresh,
* or when the calling thread is about to terminate.
*/
- wait_event_interruptible(random_write_wait, kthread_should_stop() ||
+ wait_event_interruptible(random_write_wait,
+ kthread_freezable_should_stop(&frozen) ||
ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
- mix_pool_bytes(poolp, buffer, count);
- credit_entropy_bits(poolp, entropy);
+ if (!frozen) {
+ mix_pool_bytes(poolp, buffer, count);
+ credit_entropy_bits(poolp, entropy);
+ }
}
EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
From: Charles Keepax <[email protected]>
[ Upstream commit 7fbd6dd68127927e844912a16741016d432a0737 ]
This driver specified the maximum value rather than the number of volume
levels on the SX controls, this is incorrect, so correct them.
Reported-by: David Rhodes <[email protected]>
Signed-off-by: Charles Keepax <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Mark Brown <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
sound/soc/codecs/cs53l30.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/sound/soc/codecs/cs53l30.c b/sound/soc/codecs/cs53l30.c
index cb47fb595ff4..5a16020423fe 100644
--- a/sound/soc/codecs/cs53l30.c
+++ b/sound/soc/codecs/cs53l30.c
@@ -351,22 +351,22 @@ static const struct snd_kcontrol_new cs53l30_snd_controls[] = {
SOC_ENUM("ADC2 NG Delay", adc2_ng_delay_enum),
SOC_SINGLE_SX_TLV("ADC1A PGA Volume",
- CS53L30_ADC1A_AFE_CTL, 0, 0x34, 0x18, pga_tlv),
+ CS53L30_ADC1A_AFE_CTL, 0, 0x34, 0x24, pga_tlv),
SOC_SINGLE_SX_TLV("ADC1B PGA Volume",
- CS53L30_ADC1B_AFE_CTL, 0, 0x34, 0x18, pga_tlv),
+ CS53L30_ADC1B_AFE_CTL, 0, 0x34, 0x24, pga_tlv),
SOC_SINGLE_SX_TLV("ADC2A PGA Volume",
- CS53L30_ADC2A_AFE_CTL, 0, 0x34, 0x18, pga_tlv),
+ CS53L30_ADC2A_AFE_CTL, 0, 0x34, 0x24, pga_tlv),
SOC_SINGLE_SX_TLV("ADC2B PGA Volume",
- CS53L30_ADC2B_AFE_CTL, 0, 0x34, 0x18, pga_tlv),
+ CS53L30_ADC2B_AFE_CTL, 0, 0x34, 0x24, pga_tlv),
SOC_SINGLE_SX_TLV("ADC1A Digital Volume",
- CS53L30_ADC1A_DIG_VOL, 0, 0xA0, 0x0C, dig_tlv),
+ CS53L30_ADC1A_DIG_VOL, 0, 0xA0, 0x6C, dig_tlv),
SOC_SINGLE_SX_TLV("ADC1B Digital Volume",
- CS53L30_ADC1B_DIG_VOL, 0, 0xA0, 0x0C, dig_tlv),
+ CS53L30_ADC1B_DIG_VOL, 0, 0xA0, 0x6C, dig_tlv),
SOC_SINGLE_SX_TLV("ADC2A Digital Volume",
- CS53L30_ADC2A_DIG_VOL, 0, 0xA0, 0x0C, dig_tlv),
+ CS53L30_ADC2A_DIG_VOL, 0, 0xA0, 0x6C, dig_tlv),
SOC_SINGLE_SX_TLV("ADC2B Digital Volume",
- CS53L30_ADC2B_DIG_VOL, 0, 0xA0, 0x0C, dig_tlv),
+ CS53L30_ADC2B_DIG_VOL, 0, 0xA0, 0x6C, dig_tlv),
};
static const struct snd_soc_dapm_widget cs53l30_dapm_widgets[] = {
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit a02cf3d0dd77244fd5333ac48d78871de459ae6d upstream.
Continuing the reasoning of "random: use RDSEED instead of RDRAND in
entropy extraction" from this series, at init time we also don't want to
be xoring RDSEED directly into the crng. Instead it's safer to put it
into our entropy collector and then re-extract it, so that it goes
through a hash function with preimage resistance. As a matter of hygiene,
we also order these now so that the RDSEED byte are hashed in first,
followed by the bytes that are likely more predictable (e.g. utsname()).
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 16 +++++-----------
1 file changed, 5 insertions(+), 11 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1260,24 +1260,18 @@ int __init rand_initialize(void)
bool arch_init = true;
unsigned long rv;
- mix_pool_bytes(&now, sizeof(now));
for (i = BLAKE2S_BLOCK_SIZE; i > 0; i -= sizeof(rv)) {
- if (!arch_get_random_seed_long(&rv) &&
- !arch_get_random_long(&rv))
- rv = random_get_entropy();
- mix_pool_bytes(&rv, sizeof(rv));
- }
- mix_pool_bytes(utsname(), sizeof(*(utsname())));
-
- extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
- for (i = 4; i < 16; i++) {
if (!arch_get_random_seed_long_early(&rv) &&
!arch_get_random_long_early(&rv)) {
rv = random_get_entropy();
arch_init = false;
}
- primary_crng.state[i] ^= rv;
+ mix_pool_bytes(&rv, sizeof(rv));
}
+ mix_pool_bytes(&now, sizeof(now));
+ mix_pool_bytes(utsname(), sizeof(*(utsname())));
+
+ extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
if (arch_init && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
crng_init = 2;
From: "Jason A. Donenfeld" <[email protected]>
commit cc1e127bfa95b5fb2f9307e7168bf8b2b45b4c5e upstream.
The CONFIG_WARN_ALL_UNSEEDED_RANDOM debug option controls whether the
kernel warns about all unseeded randomness or just the first instance.
There's some complicated rate limiting and comparison to the previous
caller, such that even with CONFIG_WARN_ALL_UNSEEDED_RANDOM enabled,
developers still don't see all the messages or even an accurate count of
how many were missed. This is the result of basically parallel
mechanisms aimed at accomplishing more or less the same thing, added at
different points in random.c history, which sort of compete with the
first-instance-only limiting we have now.
It turns out, however, that nobody cares about the first unseeded
randomness instance of in-kernel users. The same first user has been
there for ages now, and nobody is doing anything about it. It isn't even
clear that anybody _can_ do anything about it. Most places that can do
something about it have switched over to using get_random_bytes_wait()
or wait_for_random_bytes(), which is the right thing to do, but there is
still much code that needs randomness sometimes during init, and as a
geeneral rule, if you're not using one of the _wait functions or the
readiness notifier callback, you're bound to be doing it wrong just
based on that fact alone.
So warning about this same first user that can't easily change is simply
not an effective mechanism for anything at all. Users can't do anything
about it, as the Kconfig text points out -- the problem isn't in
userspace code -- and kernel developers don't or more often can't react
to it.
Instead, show the warning for all instances when CONFIG_WARN_ALL_UNSEEDED_RANDOM
is set, so that developers can debug things need be, or if it isn't set,
don't show a warning at all.
At the same time, CONFIG_WARN_ALL_UNSEEDED_RANDOM now implies setting
random.ratelimit_disable=1 on by default, since if you care about one
you probably care about the other too. And we can clean up usage around
the related urandom_warning ratelimiter as well (whose behavior isn't
changing), so that it properly counts missed messages after the 10
message threshold is reached.
Cc: Theodore Ts'o <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 61 ++++++++++++++------------------------------------
lib/Kconfig.debug | 5 +---
2 files changed, 20 insertions(+), 46 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -87,11 +87,10 @@ static DEFINE_SPINLOCK(random_ready_chai
static RAW_NOTIFIER_HEAD(random_ready_chain);
/* Control how we warn userspace. */
-static struct ratelimit_state unseeded_warning =
- RATELIMIT_STATE_INIT("warn_unseeded_randomness", HZ, 3);
static struct ratelimit_state urandom_warning =
RATELIMIT_STATE_INIT("warn_urandom_randomness", HZ, 3);
-static int ratelimit_disable __read_mostly;
+static int ratelimit_disable __read_mostly =
+ IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM);
module_param_named(ratelimit_disable, ratelimit_disable, int, 0644);
MODULE_PARM_DESC(ratelimit_disable, "Disable random ratelimit suppression");
@@ -184,27 +183,15 @@ static void process_random_ready_list(vo
spin_unlock_irqrestore(&random_ready_chain_lock, flags);
}
-#define warn_unseeded_randomness(previous) \
- _warn_unseeded_randomness(__func__, (void *)_RET_IP_, (previous))
+#define warn_unseeded_randomness() \
+ _warn_unseeded_randomness(__func__, (void *)_RET_IP_)
-static void _warn_unseeded_randomness(const char *func_name, void *caller, void **previous)
+static void _warn_unseeded_randomness(const char *func_name, void *caller)
{
-#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM
- const bool print_once = false;
-#else
- static bool print_once __read_mostly;
-#endif
-
- if (print_once || crng_ready() ||
- (previous && (caller == READ_ONCE(*previous))))
+ if (!IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM) || crng_ready())
return;
- WRITE_ONCE(*previous, caller);
-#ifndef CONFIG_WARN_ALL_UNSEEDED_RANDOM
- print_once = true;
-#endif
- if (__ratelimit(&unseeded_warning))
- printk_deferred(KERN_NOTICE "random: %s called from %pS with crng_init=%d\n",
- func_name, caller, crng_init);
+ printk_deferred(KERN_NOTICE "random: %s called from %pS with crng_init=%d\n",
+ func_name, caller, crng_init);
}
@@ -455,9 +442,7 @@ static void _get_random_bytes(void *buf,
*/
void get_random_bytes(void *buf, size_t nbytes)
{
- static void *previous;
-
- warn_unseeded_randomness(&previous);
+ warn_unseeded_randomness();
_get_random_bytes(buf, nbytes);
}
EXPORT_SYMBOL(get_random_bytes);
@@ -551,10 +536,9 @@ u64 get_random_u64(void)
u64 ret;
unsigned long flags;
struct batched_entropy *batch;
- static void *previous;
unsigned long next_gen;
- warn_unseeded_randomness(&previous);
+ warn_unseeded_randomness();
if (!crng_ready()) {
_get_random_bytes(&ret, sizeof(ret));
@@ -589,10 +573,9 @@ u32 get_random_u32(void)
u32 ret;
unsigned long flags;
struct batched_entropy *batch;
- static void *previous;
unsigned long next_gen;
- warn_unseeded_randomness(&previous);
+ warn_unseeded_randomness();
if (!crng_ready()) {
_get_random_bytes(&ret, sizeof(ret));
@@ -819,16 +802,9 @@ static void credit_init_bits(size_t nbit
wake_up_interruptible(&crng_init_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
pr_notice("crng init done\n");
- if (unseeded_warning.missed) {
- pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
- unseeded_warning.missed);
- unseeded_warning.missed = 0;
- }
- if (urandom_warning.missed) {
+ if (urandom_warning.missed)
pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
urandom_warning.missed);
- urandom_warning.missed = 0;
- }
} else if (orig < POOL_EARLY_BITS && new >= POOL_EARLY_BITS) {
spin_lock_irqsave(&base_crng.lock, flags);
/* Check if crng_init is CRNG_EMPTY, to avoid race with crng_reseed(). */
@@ -941,10 +917,6 @@ int __init rand_initialize(void)
else if (arch_init && trust_cpu)
credit_init_bits(BLAKE2S_BLOCK_SIZE * 8);
- if (ratelimit_disable) {
- urandom_warning.interval = 0;
- unseeded_warning.interval = 0;
- }
return 0;
}
@@ -1390,11 +1362,14 @@ static ssize_t urandom_read(struct file
{
static int maxwarn = 10;
- if (!crng_ready() && maxwarn > 0) {
- maxwarn--;
- if (__ratelimit(&urandom_warning))
+ if (!crng_ready()) {
+ if (!ratelimit_disable && maxwarn <= 0)
+ ++urandom_warning.missed;
+ else if (ratelimit_disable || __ratelimit(&urandom_warning)) {
+ --maxwarn;
pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
current->comm, nbytes);
+ }
}
return get_random_bytes_user(buf, nbytes);
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1195,9 +1195,8 @@ config WARN_ALL_UNSEEDED_RANDOM
time. This is really bad from a security perspective, and
so architecture maintainers really need to do what they can
to get the CRNG seeded sooner after the system is booted.
- However, since users can not do anything actionble to
- address this, by default the kernel will issue only a single
- warning for the first use of unseeded randomness.
+ However, since users cannot do anything actionable to
+ address this, by default this option is disabled.
Say Y here if you want to receive warnings for all uses of
unseeded randomness. This will be of use primarily for
From: "Jason A. Donenfeld" <[email protected]>
commit ac9756c79797bb98972736b13cfb239fd2cffb79 upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is really not the best we can do.
Instead, at least calling random_get_entropy_fallback() would be
preferable, because that always needs to return _something_, even
falling back to jiffies eventually. It's not as though
random_get_entropy_fallback() is super high precision or guaranteed to
be entropic, but basically anything that's not zero all the time is
better than returning zero all the time.
This is accomplished by just including the asm-generic code like on
other architectures, which means we can get rid of the empty stub
function here.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: David S. Miller <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/sparc/include/asm/timex_32.h | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--- a/arch/sparc/include/asm/timex_32.h
+++ b/arch/sparc/include/asm/timex_32.h
@@ -8,8 +8,6 @@
#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */
-/* XXX Maybe do something better at some point... -DaveM */
-typedef unsigned long cycles_t;
-#define get_cycles() (0)
+#include <asm-generic/timex.h>
#endif
From: Yangtao Li <[email protected]>
commit 09a6d00a42ce0e63e2a15be3d070974bcc656ec7 upstream.
Since it is not being used, so delete it.
Signed-off-by: Yangtao Li <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 30 ------------------------------
1 file changed, 30 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -432,36 +432,6 @@ static const struct poolinfo {
/* was: x^128 + x^103 + x^76 + x^51 +x^25 + x + 1 */
/* x^128 + x^104 + x^76 + x^51 +x^25 + x + 1 */
{ S(128), 104, 76, 51, 25, 1 },
- /* was: x^32 + x^26 + x^20 + x^14 + x^7 + x + 1 */
- /* x^32 + x^26 + x^19 + x^14 + x^7 + x + 1 */
- { S(32), 26, 19, 14, 7, 1 },
-#if 0
- /* x^2048 + x^1638 + x^1231 + x^819 + x^411 + x + 1 -- 115 */
- { S(2048), 1638, 1231, 819, 411, 1 },
-
- /* x^1024 + x^817 + x^615 + x^412 + x^204 + x + 1 -- 290 */
- { S(1024), 817, 615, 412, 204, 1 },
-
- /* x^1024 + x^819 + x^616 + x^410 + x^207 + x^2 + 1 -- 115 */
- { S(1024), 819, 616, 410, 207, 2 },
-
- /* x^512 + x^411 + x^308 + x^208 + x^104 + x + 1 -- 225 */
- { S(512), 411, 308, 208, 104, 1 },
-
- /* x^512 + x^409 + x^307 + x^206 + x^102 + x^2 + 1 -- 95 */
- { S(512), 409, 307, 206, 102, 2 },
- /* x^512 + x^409 + x^309 + x^205 + x^103 + x^2 + 1 -- 95 */
- { S(512), 409, 309, 205, 103, 2 },
-
- /* x^256 + x^205 + x^155 + x^101 + x^52 + x + 1 -- 125 */
- { S(256), 205, 155, 101, 52, 1 },
-
- /* x^128 + x^103 + x^78 + x^51 + x^27 + x^2 + 1 -- 70 */
- { S(128), 103, 78, 51, 27, 2 },
-
- /* x^64 + x^52 + x^39 + x^26 + x^14 + x + 1 -- 15 */
- { S(64), 52, 39, 26, 14, 1 },
-#endif
};
/*
From: zijun_hu <[email protected]>
[ Upstream commit 3fad4cdac235c5b13227d0c09854c689ae62c70b ]
get_cpu_number() doesn't use existing helper to iterate over possible
CPUs, It will cause an error in case of discontinuous @cpu_possible_mask
such as 0b11110001, which can result from a core having failed to come
up on a SMP machine.
Fixed by using existing helper for_each_possible_cpu().
Signed-off-by: zijun_hu <[email protected]>
Signed-off-by: Marc Zyngier <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/irqchip/irq-gic-v3.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 2ab6060031a4..9ae24ffb9b09 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -982,7 +982,7 @@ static int get_cpu_number(struct device_node *dn)
{
const __be32 *cell;
u64 hwid;
- int i;
+ int cpu;
cell = of_get_property(dn, "reg", NULL);
if (!cell)
@@ -996,9 +996,9 @@ static int get_cpu_number(struct device_node *dn)
if (hwid & ~MPIDR_HWID_BITMASK)
return -1;
- for (i = 0; i < num_possible_cpus(); i++)
- if (cpu_logical_map(i) == hwid)
- return i;
+ for_each_possible_cpu(cpu)
+ if (cpu_logical_map(cpu) == hwid)
+ return cpu;
return -1;
}
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 0f63702718c91d89c922081ac1e6baeddc2d8b1a upstream.
We no longer have an output pool. Rather, we have just a wakeup bits
threshold for /dev/random reads, presumably so that processes don't
hang. This value, random_write_wakeup_bits, is configurable anyway. So
all the no longer usefully named OUTPUT_POOL constants were doing was
setting a reasonable default for random_write_wakeup_bits. This commit
gets rid of the constants and just puts it all in the default value of
random_write_wakeup_bits.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -364,8 +364,6 @@
*/
#define INPUT_POOL_SHIFT 12
#define INPUT_POOL_WORDS (1 << (INPUT_POOL_SHIFT-5))
-#define OUTPUT_POOL_SHIFT 10
-#define OUTPUT_POOL_WORDS (1 << (OUTPUT_POOL_SHIFT-5))
#define EXTRACT_SIZE (BLAKE2S_HASH_SIZE / 2)
/*
@@ -383,7 +381,7 @@
* should wake up processes which are selecting or polling on write
* access to /dev/random.
*/
-static int random_write_wakeup_bits = 28 * OUTPUT_POOL_WORDS;
+static int random_write_wakeup_bits = 28 * (1 << 5);
/*
* Originally, we used a primitive polynomial of degree .poolwords
From: Slark Xiao <[email protected]>
commit 158f7585bfcea4aae0ad4128d032a80fec550df1 upstream.
Adding support for Cinterion device MV31 with Qualcomm
new baseline. Use different PIDs to separate it from
previous base line products.
All interfaces settings keep same as previous.
Below is test evidence:
T: Bus=03 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 6 Spd=480 MxCh= 0
D: Ver= 2.10 Cls=ef(misc ) Sub=02 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=1e2d ProdID=00b8 Rev=04.14
S: Manufacturer=Cinterion
S: Product=Cinterion PID 0x00B8 USB Mobile Broadband
S: SerialNumber=90418e79
C: #Ifs= 6 Cfg#= 1 Atr=a0 MxPwr=500mA
I: If#=0x0 Alt= 0 #EPs= 1 Cls=02(commc) Sub=0e Prot=00 Driver=cdc_mbim
I: If#=0x1 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
I: If#=0x2 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=40 Driver=option
I: If#=0x3 Alt= 0 #EPs= 1 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
I: If#=0x4 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=60 Driver=option
I: If#=0x5 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=30 Driver=option
T: Bus=03 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 7 Spd=480 MxCh= 0
D: Ver= 2.10 Cls=ef(misc ) Sub=02 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=1e2d ProdID=00b9 Rev=04.14
S: Manufacturer=Cinterion
S: Product=Cinterion PID 0x00B9 USB Mobile Broadband
S: SerialNumber=90418e79
C: #Ifs= 4 Cfg#= 1 Atr=a0 MxPwr=500mA
I: If#=0x0 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=50 Driver=qmi_wwan
I: If#=0x1 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=40 Driver=option
I: If#=0x2 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=60 Driver=option
I: If#=0x3 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=30 Driver=option
For PID 00b8, interface 3 is GNSS port which don't use serial driver.
Signed-off-by: Slark Xiao <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
[ johan: rename defines using a "2" infix ]
Cc: [email protected]
Signed-off-by: Johan Hovold <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/usb/serial/option.c | 6 ++++++
1 file changed, 6 insertions(+)
--- a/drivers/usb/serial/option.c
+++ b/drivers/usb/serial/option.c
@@ -430,6 +430,8 @@ static void option_instat_callback(struc
#define CINTERION_PRODUCT_CLS8 0x00b0
#define CINTERION_PRODUCT_MV31_MBIM 0x00b3
#define CINTERION_PRODUCT_MV31_RMNET 0x00b7
+#define CINTERION_PRODUCT_MV31_2_MBIM 0x00b8
+#define CINTERION_PRODUCT_MV31_2_RMNET 0x00b9
#define CINTERION_PRODUCT_MV32_WA 0x00f1
#define CINTERION_PRODUCT_MV32_WB 0x00f2
@@ -1953,6 +1955,10 @@ static const struct usb_device_id option
.driver_info = RSVD(3)},
{ USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_MV31_RMNET, 0xff),
.driver_info = RSVD(0)},
+ { USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_MV31_2_MBIM, 0xff),
+ .driver_info = RSVD(3)},
+ { USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_MV31_2_RMNET, 0xff),
+ .driver_info = RSVD(0)},
{ USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_MV32_WA, 0xff),
.driver_info = RSVD(3)},
{ USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_MV32_WB, 0xff),
From: Willy Tarreau <[email protected]>
commit e9261476184be1abd486c9434164b2acbe0ed6c2 upstream.
We'll need to further increase the size of this table and it's likely
that at some point its size will not be suitable anymore for a static
table. Let's allocate it on boot from inet_hashinfo2_init(), which is
called from tcp_init().
Cc: Moshe Kol <[email protected]>
Cc: Yossi Gilad <[email protected]>
Cc: Amit Klein <[email protected]>
Reviewed-by: Eric Dumazet <[email protected]>
Signed-off-by: Willy Tarreau <[email protected]>
Signed-off-by: Jakub Kicinski <[email protected]>
[bwh: Backported to 4.9:
- There is no inet_hashinfo2_init(), so allocate the table in
inet_hashinfo_init() when called by TCP
- Adjust context]
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -546,7 +546,8 @@ EXPORT_SYMBOL_GPL(inet_unhash);
* privacy, this only consumes 1 KB of kernel memory.
*/
#define INET_TABLE_PERTURB_SHIFT 8
-static u32 table_perturb[1 << INET_TABLE_PERTURB_SHIFT];
+#define INET_TABLE_PERTURB_SIZE (1 << INET_TABLE_PERTURB_SHIFT)
+static u32 *table_perturb;
int __inet_hash_connect(struct inet_timewait_death_row *death_row,
struct sock *sk, u64 port_offset,
@@ -586,7 +587,8 @@ int __inet_hash_connect(struct inet_time
if (likely(remaining > 1))
remaining &= ~1U;
- net_get_random_once(table_perturb, sizeof(table_perturb));
+ net_get_random_once(table_perturb,
+ INET_TABLE_PERTURB_SIZE * sizeof(*table_perturb));
index = hash_32(port_offset, INET_TABLE_PERTURB_SHIFT);
offset = READ_ONCE(table_perturb[index]) + (port_offset >> 32);
@@ -691,6 +693,15 @@ void inet_hashinfo_init(struct inet_hash
INIT_HLIST_NULLS_HEAD(&h->listening_hash[i].nulls_head,
i + LISTENING_NULLS_BASE);
}
+
+ if (h != &tcp_hashinfo)
+ return;
+
+ /* this one is used for source ports of outgoing connections */
+ table_perturb = kmalloc_array(INET_TABLE_PERTURB_SIZE,
+ sizeof(*table_perturb), GFP_KERNEL);
+ if (!table_perturb)
+ panic("TCP: failed to alloc table_perturb");
}
EXPORT_SYMBOL_GPL(inet_hashinfo_init);
From: Ilpo Järvinen <[email protected]>
commit be03b0651ffd8bab69dfd574c6818b446c0753ce upstream.
Not all LSR register flags are preserved across reads. Therefore, LSR
readers must store the non-preserved bits into lsr_save_flags.
This fix was initially mixed into feature commit f6f586102add ("serial:
8250: Handle UART without interrupt on TEMT using em485"). However,
that feature change had a flaw and it was reverted to make room for
simpler approach providing the same feature. The embedded fix got
reverted with the feature change.
Re-add the lsr_save_flags fix and properly mark it's a fix.
Link: https://lore.kernel.org/all/[email protected]/T/#m1737eef986bd20cf19593e344cebd7b0244945fc
Fixes: e490c9144cfa ("tty: Add software emulated RS485 support for 8250")
Cc: stable <[email protected]>
Acked-by: Uwe Kleine-König <[email protected]>
Signed-off-by: Uwe Kleine-König <[email protected]>
Signed-off-by: Ilpo Järvinen <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/tty/serial/8250/8250_port.c | 2 ++
1 file changed, 2 insertions(+)
--- a/drivers/tty/serial/8250/8250_port.c
+++ b/drivers/tty/serial/8250/8250_port.c
@@ -1473,6 +1473,8 @@ static inline void __stop_tx(struct uart
if (em485) {
unsigned char lsr = serial_in(p, UART_LSR);
+ p->lsr_saved_flags |= lsr & LSR_SAVE_FLAGS;
+
/*
* To provide required timeing and allow FIFO transfer,
* __stop_tx_rs485() must be called only when both FIFO and
From: "Jason A. Donenfeld" <[email protected]>
commit 1097710bc9660e1e588cf2186a35db3d95c4d258 upstream.
Alpha defines a get_cycles() function, but it does not do the usual
`#define get_cycles get_cycles` dance, making it impossible for generic
code to see if an arch-specific function was defined. While the
get_cycles() ifdef is not currently used, the following timekeeping
patch in this series will depend on the macro existing (or not existing)
when defining random_get_entropy().
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Richard Henderson <[email protected]>
Cc: Ivan Kokshaysky <[email protected]>
Acked-by: Matt Turner <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/alpha/include/asm/timex.h | 1 +
1 file changed, 1 insertion(+)
--- a/arch/alpha/include/asm/timex.h
+++ b/arch/alpha/include/asm/timex.h
@@ -27,5 +27,6 @@ static inline cycles_t get_cycles (void)
__asm__ __volatile__ ("rpcc %0" : "=r"(ret));
return ret;
}
+#define get_cycles get_cycles
#endif
From: Jan Varho <[email protected]>
commit 527a9867af29ff89f278d037db704e0ed50fb666 upstream.
add_hwgenerator_randomness() tries to only use the required amount of input
for fast init, but credits all the entropy, rather than a fraction of
it. Since it's hard to determine how much entropy is left over out of a
non-unformly random sample, either give it all to fast init or credit
it, but don't attempt to do both. In the process, we can clean up the
injection code to no longer need to return a value.
Signed-off-by: Jan Varho <[email protected]>
[Jason: expanded commit message]
Fixes: 73c7733f122e ("random: do not throw away excess input to crng_fast_load")
Cc: [email protected] # 5.17+, requires af704c856e88
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 23 ++++++-----------------
1 file changed, 6 insertions(+), 17 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -438,11 +438,8 @@ static void crng_make_state(u32 chacha_s
* This shouldn't be set by functions like add_device_randomness(),
* where we can't trust the buffer passed to it is guaranteed to be
* unpredictable (so it might not have any entropy at all).
- *
- * Returns the number of bytes processed from input, which is bounded
- * by CRNG_INIT_CNT_THRESH if account is true.
*/
-static size_t crng_pre_init_inject(const void *input, size_t len, bool account)
+static void crng_pre_init_inject(const void *input, size_t len, bool account)
{
static int crng_init_cnt = 0;
struct blake2s_state hash;
@@ -453,18 +450,15 @@ static size_t crng_pre_init_inject(const
spin_lock_irqsave(&base_crng.lock, flags);
if (crng_init != 0) {
spin_unlock_irqrestore(&base_crng.lock, flags);
- return 0;
+ return;
}
- if (account)
- len = min_t(size_t, len, CRNG_INIT_CNT_THRESH - crng_init_cnt);
-
blake2s_update(&hash, base_crng.key, sizeof(base_crng.key));
blake2s_update(&hash, input, len);
blake2s_final(&hash, base_crng.key);
if (account) {
- crng_init_cnt += len;
+ crng_init_cnt += min_t(size_t, len, CRNG_INIT_CNT_THRESH - crng_init_cnt);
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
++base_crng.generation;
crng_init = 1;
@@ -475,8 +469,6 @@ static size_t crng_pre_init_inject(const
if (crng_init == 1)
pr_notice("fast init done\n");
-
- return len;
}
static void _get_random_bytes(void *buf, size_t nbytes)
@@ -1134,12 +1126,9 @@ void add_hwgenerator_randomness(const vo
size_t entropy)
{
if (unlikely(crng_init == 0 && entropy < POOL_MIN_BITS)) {
- size_t ret = crng_pre_init_inject(buffer, count, true);
- mix_pool_bytes(buffer, ret);
- count -= ret;
- buffer += ret;
- if (!count || crng_init == 0)
- return;
+ crng_pre_init_inject(buffer, count, true);
+ mix_pool_bytes(buffer, count);
+ return;
}
/*
From: "Jason A. Donenfeld" <[email protected]>
commit a6adf8e7a605250b911e94793fd077933709ff9e upstream.
This pulls all of the userspace read/write-focused functions into the
fifth labeled section.
No functional changes.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 125 ++++++++++++++++++++++++++++++--------------------
1 file changed, 77 insertions(+), 48 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1473,30 +1473,61 @@ static void try_to_generate_entropy(void
mix_pool_bytes(&stack.now, sizeof(stack.now));
}
-static ssize_t urandom_read(struct file *file, char __user *buf, size_t nbytes,
- loff_t *ppos)
+
+/**********************************************************************
+ *
+ * Userspace reader/writer interfaces.
+ *
+ * getrandom(2) is the primary modern interface into the RNG and should
+ * be used in preference to anything else.
+ *
+ * Reading from /dev/random has the same functionality as calling
+ * getrandom(2) with flags=0. In earlier versions, however, it had
+ * vastly different semantics and should therefore be avoided, to
+ * prevent backwards compatibility issues.
+ *
+ * Reading from /dev/urandom has the same functionality as calling
+ * getrandom(2) with flags=GRND_INSECURE. Because it does not block
+ * waiting for the RNG to be ready, it should not be used.
+ *
+ * Writing to either /dev/random or /dev/urandom adds entropy to
+ * the input pool but does not credit it.
+ *
+ * Polling on /dev/random indicates when the RNG is initialized, on
+ * the read side, and when it wants new entropy, on the write side.
+ *
+ * Both /dev/random and /dev/urandom have the same set of ioctls for
+ * adding entropy, getting the entropy count, zeroing the count, and
+ * reseeding the crng.
+ *
+ **********************************************************************/
+
+SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count, unsigned int,
+ flags)
{
- static int maxwarn = 10;
+ if (flags & ~(GRND_NONBLOCK | GRND_RANDOM | GRND_INSECURE))
+ return -EINVAL;
- if (!crng_ready() && maxwarn > 0) {
- maxwarn--;
- if (__ratelimit(&urandom_warning))
- pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
- current->comm, nbytes);
- }
+ /*
+ * Requesting insecure and blocking randomness at the same time makes
+ * no sense.
+ */
+ if ((flags & (GRND_INSECURE | GRND_RANDOM)) == (GRND_INSECURE | GRND_RANDOM))
+ return -EINVAL;
- return get_random_bytes_user(buf, nbytes);
-}
+ if (count > INT_MAX)
+ count = INT_MAX;
-static ssize_t random_read(struct file *file, char __user *buf, size_t nbytes,
- loff_t *ppos)
-{
- int ret;
+ if (!(flags & GRND_INSECURE) && !crng_ready()) {
+ int ret;
- ret = wait_for_random_bytes();
- if (ret != 0)
- return ret;
- return get_random_bytes_user(buf, nbytes);
+ if (flags & GRND_NONBLOCK)
+ return -EAGAIN;
+ ret = wait_for_random_bytes();
+ if (unlikely(ret))
+ return ret;
+ }
+ return get_random_bytes_user(buf, count);
}
static unsigned int random_poll(struct file *file, poll_table *wait)
@@ -1548,6 +1579,32 @@ static ssize_t random_write(struct file
return (ssize_t)count;
}
+static ssize_t urandom_read(struct file *file, char __user *buf, size_t nbytes,
+ loff_t *ppos)
+{
+ static int maxwarn = 10;
+
+ if (!crng_ready() && maxwarn > 0) {
+ maxwarn--;
+ if (__ratelimit(&urandom_warning))
+ pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
+ current->comm, nbytes);
+ }
+
+ return get_random_bytes_user(buf, nbytes);
+}
+
+static ssize_t random_read(struct file *file, char __user *buf, size_t nbytes,
+ loff_t *ppos)
+{
+ int ret;
+
+ ret = wait_for_random_bytes();
+ if (ret != 0)
+ return ret;
+ return get_random_bytes_user(buf, nbytes);
+}
+
static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
{
int size, ent_count;
@@ -1556,7 +1613,7 @@ static long random_ioctl(struct file *f,
switch (cmd) {
case RNDGETENTCNT:
- /* inherently racy, no point locking */
+ /* Inherently racy, no point locking. */
if (put_user(input_pool.entropy_count, p))
return -EFAULT;
return 0;
@@ -1630,34 +1687,6 @@ const struct file_operations urandom_fop
.llseek = noop_llseek,
};
-SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count, unsigned int,
- flags)
-{
- if (flags & ~(GRND_NONBLOCK | GRND_RANDOM | GRND_INSECURE))
- return -EINVAL;
-
- /*
- * Requesting insecure and blocking randomness at the same time makes
- * no sense.
- */
- if ((flags & (GRND_INSECURE | GRND_RANDOM)) == (GRND_INSECURE | GRND_RANDOM))
- return -EINVAL;
-
- if (count > INT_MAX)
- count = INT_MAX;
-
- if (!(flags & GRND_INSECURE) && !crng_ready()) {
- int ret;
-
- if (flags & GRND_NONBLOCK)
- return -EAGAIN;
- ret = wait_for_random_bytes();
- if (unlikely(ret))
- return ret;
- }
- return get_random_bytes_user(buf, count);
-}
-
/********************************************************************
*
* Sysctl interface
From: Willy Tarreau <[email protected]>
commit 9e9b70ae923baf2b5e8a0ea4fd0c8451801ac526 upstream.
Amit Klein suggests that we use different parts of port_offset for the
table's index and the port offset so that there is no direct relation
between them.
Cc: Jason A. Donenfeld <[email protected]>
Cc: Moshe Kol <[email protected]>
Cc: Yossi Gilad <[email protected]>
Cc: Amit Klein <[email protected]>
Reviewed-by: Eric Dumazet <[email protected]>
Signed-off-by: Willy Tarreau <[email protected]>
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -589,7 +589,7 @@ int __inet_hash_connect(struct inet_time
net_get_random_once(table_perturb, sizeof(table_perturb));
index = hash_32(port_offset, INET_TABLE_PERTURB_SHIFT);
- offset = READ_ONCE(table_perturb[index]) + port_offset;
+ offset = READ_ONCE(table_perturb[index]) + (port_offset >> 32);
offset %= remaining;
/* In first pass we try ports of @low parity.
From: Andy Lutomirski <[email protected]>
commit c6f1deb158789abba02a7eba600747843eeb3a57 upstream.
/dev/random and getrandom() never warn. Split the meat of
urandom_read() into urandom_read_nowarn() and leave the warning code
in urandom_read().
This has no effect on kernel behavior, but it makes subsequent
patches more straightforward. It also makes the fact that
getrandom() never warns more obvious.
Signed-off-by: Andy Lutomirski <[email protected]>
Link: https://lore.kernel.org/r/c87ab200588de746431d9f916501ef11e5242b13.1577088521.git.luto@kernel.org
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2015,11 +2015,22 @@ random_read(struct file *file, char __us
}
static ssize_t
+urandom_read_nowarn(struct file *file, char __user *buf, size_t nbytes,
+ loff_t *ppos)
+{
+ int ret;
+
+ nbytes = min_t(size_t, nbytes, INT_MAX >> (ENTROPY_SHIFT + 3));
+ ret = extract_crng_user(buf, nbytes);
+ trace_urandom_read(8 * nbytes, 0, ENTROPY_BITS(&input_pool));
+ return ret;
+}
+
+static ssize_t
urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
{
unsigned long flags;
static int maxwarn = 10;
- int ret;
if (!crng_ready() && maxwarn > 0) {
maxwarn--;
@@ -2031,10 +2042,8 @@ urandom_read(struct file *file, char __u
crng_init_cnt = 0;
spin_unlock_irqrestore(&primary_crng.lock, flags);
}
- nbytes = min_t(size_t, nbytes, INT_MAX >> (ENTROPY_SHIFT + 3));
- ret = extract_crng_user(buf, nbytes);
- trace_urandom_read(8 * nbytes, 0, ENTROPY_BITS(&input_pool));
- return ret;
+
+ return urandom_read_nowarn(file, buf, nbytes, ppos);
}
static unsigned int
@@ -2194,7 +2203,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
if (unlikely(ret))
return ret;
}
- return urandom_read(NULL, buf, count, NULL);
+ return urandom_read_nowarn(NULL, buf, count, NULL);
}
/********************************************************************
From: Charles Keepax <[email protected]>
[ Upstream commit 8bf5aabf524eec61013e506f764a0b2652dc5665 ]
The datasheet specifies the range of the mixer volumes as between
-51.5dB and 12dB with a 0.5dB step. Update the TLVs for this.
Signed-off-by: Charles Keepax <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Mark Brown <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
sound/soc/codecs/cs42l52.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sound/soc/codecs/cs42l52.c b/sound/soc/codecs/cs42l52.c
index 0d9c4a57301b..f733f6b42b53 100644
--- a/sound/soc/codecs/cs42l52.c
+++ b/sound/soc/codecs/cs42l52.c
@@ -141,7 +141,7 @@ static DECLARE_TLV_DB_SCALE(mic_tlv, 1600, 100, 0);
static DECLARE_TLV_DB_SCALE(pga_tlv, -600, 50, 0);
-static DECLARE_TLV_DB_SCALE(mix_tlv, -50, 50, 0);
+static DECLARE_TLV_DB_SCALE(mix_tlv, -5150, 50, 0);
static DECLARE_TLV_DB_SCALE(beep_tlv, -56, 200, 0);
@@ -368,7 +368,7 @@ static const struct snd_kcontrol_new cs42l52_snd_controls[] = {
CS42L52_ADCB_VOL, 0, 0xA0, 0x78, ipd_tlv),
SOC_DOUBLE_R_SX_TLV("ADC Mixer Volume",
CS42L52_ADCA_MIXER_VOL, CS42L52_ADCB_MIXER_VOL,
- 0, 0x19, 0x7F, ipd_tlv),
+ 0, 0x19, 0x7F, mix_tlv),
SOC_DOUBLE("ADC Switch", CS42L52_ADC_MISC_CTL, 0, 1, 1, 0),
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 77553cf8f44863b31da242cf24671d76ddb61597 upstream.
We leave around these old sysctls for compatibility, and we keep them
"writable" for compatibility, but even after writing, we should keep
reporting the same value. This is consistent with how userspaces tend to
use sysctl_random_write_wakeup_bits, writing to it, and then later
reading from it and using the value.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1663,6 +1663,13 @@ static int proc_do_uuid(struct ctl_table
return proc_dostring(&fake_table, 0, buffer, lenp, ppos);
}
+/* The same as proc_dointvec, but writes don't change anything. */
+static int proc_do_rointvec(struct ctl_table *table, int write, void __user *buffer,
+ size_t *lenp, loff_t *ppos)
+{
+ return write ? 0 : proc_dointvec(table, 0, buffer, lenp, ppos);
+}
+
extern struct ctl_table random_table[];
struct ctl_table random_table[] = {
{
@@ -1684,14 +1691,14 @@ struct ctl_table random_table[] = {
.data = &sysctl_random_write_wakeup_bits,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = proc_dointvec,
+ .proc_handler = proc_do_rointvec,
},
{
.procname = "urandom_min_reseed_secs",
.data = &sysctl_random_min_urandom_seed,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = proc_dointvec,
+ .proc_handler = proc_do_rointvec,
},
{
.procname = "boot_id",
From: Rasmus Villemoes <[email protected]>
commit 764ed189c82090c1d85f0e30636156736d8f09a8 upstream.
Since the definition of struct crng_state is private to random.c, and
primary_crng is neither declared or used elsewhere, there's no reason
for that symbol to have external linkage.
Signed-off-by: Rasmus Villemoes <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -416,7 +416,7 @@ struct crng_state {
spinlock_t lock;
};
-struct crng_state primary_crng = {
+static struct crng_state primary_crng = {
.lock = __SPIN_LOCK_UNLOCKED(primary_crng.lock),
};
From: Eric Biggers <[email protected]>
commit 8786841bc2020f7f2513a6c74e64912f07b9c0dc upstream.
Use the full path in the include guards for the BLAKE2s headers to avoid
ambiguity and to match the convention for most files in include/crypto/.
Signed-off-by: Eric Biggers <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/crypto/blake2s.h | 6 +++---
include/crypto/internal/blake2s.h | 6 +++---
2 files changed, 6 insertions(+), 6 deletions(-)
--- a/include/crypto/blake2s.h
+++ b/include/crypto/blake2s.h
@@ -3,8 +3,8 @@
* Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
*/
-#ifndef BLAKE2S_H
-#define BLAKE2S_H
+#ifndef _CRYPTO_BLAKE2S_H
+#define _CRYPTO_BLAKE2S_H
#include <linux/bug.h>
#include <linux/types.h>
@@ -99,4 +99,4 @@ static inline void blake2s(u8 *out, cons
blake2s_final(&state, out);
}
-#endif /* BLAKE2S_H */
+#endif /* _CRYPTO_BLAKE2S_H */
--- a/include/crypto/internal/blake2s.h
+++ b/include/crypto/internal/blake2s.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 OR MIT */
-#ifndef BLAKE2S_INTERNAL_H
-#define BLAKE2S_INTERNAL_H
+#ifndef _CRYPTO_INTERNAL_BLAKE2S_H
+#define _CRYPTO_INTERNAL_BLAKE2S_H
#include <crypto/blake2s.h>
@@ -16,4 +16,4 @@ static inline void blake2s_set_lastblock
state->f[0] = -1;
}
-#endif /* BLAKE2S_INTERNAL_H */
+#endif /* _CRYPTO_INTERNAL_BLAKE2S_H */
From: Christian Borntraeger <[email protected]>
commit 3ae11dbcfac906a8c3a480e98660a823130dc16a upstream.
The switch to a keyed guest does not require a classic sske as the other
guest CPUs are not accessing the key before the switch is complete.
By using the NQ SSKE things are faster especially with multiple guests.
Signed-off-by: Christian Borntraeger <[email protected]>
Suggested-by: Janis Schoetterl-Glausch <[email protected]>
Reviewed-by: Claudio Imbrenda <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Christian Borntraeger <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/s390/mm/pgtable.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/arch/s390/mm/pgtable.c
+++ b/arch/s390/mm/pgtable.c
@@ -595,7 +595,7 @@ void ptep_zap_key(struct mm_struct *mm,
PGSTE_GR_BIT | PGSTE_GC_BIT);
ptev = pte_val(*ptep);
if (!(ptev & _PAGE_INVALID) && (ptev & _PAGE_WRITE))
- page_set_storage_key(ptev & PAGE_MASK, PAGE_DEFAULT_KEY, 1);
+ page_set_storage_key(ptev & PAGE_MASK, PAGE_DEFAULT_KEY, 0);
pgste_set_unlock(ptep, pgste);
preempt_enable();
}
From: "Jason A. Donenfeld" <[email protected]>
commit 4b758eda851eb9336ca86a0041a4d3da55f66511 upstream.
All platforms are now guaranteed to provide some value for
random_get_entropy(). In case some bug leads to this not being so, we
print a warning, because that indicates that something is really very
wrong (and likely other things are impacted too). This should never be
hit, but it's a good and cheap way of finding out if something ever is
problematic.
Since we now have viable fallback code for random_get_entropy() on all
platforms, which is, in the worst case, not worse than jiffies, we can
count on getting the best possible value out of it. That means there's
no longer a use for using jiffies as entropy input. It also means we no
longer have a reason for doing the round-robin register flow in the IRQ
handler, which was always of fairly dubious value.
Instead we can greatly simplify the IRQ handler inputs and also unify
the construction between 64-bits and 32-bits. We now collect the cycle
counter and the return address, since those are the two things that
matter. Because the return address and the irq number are likely
related, to the extent we mix in the irq number, we can just xor it into
the top unchanging bytes of the return address, rather than the bottom
changing bytes of the cycle counter as before. Then, we can do a fixed 2
rounds of SipHash/HSipHash. Finally, we use the same construction of
hashing only half of the [H]SipHash state on 32-bit and 64-bit. We're
not actually discarding any entropy, since that entropy is carried
through until the next time. And more importantly, it lets us do the
same sponge-like construction everywhere.
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 86 +++++++++++++++-----------------------------------
1 file changed, 26 insertions(+), 60 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1018,15 +1018,14 @@ int __init rand_initialize(void)
*/
void add_device_randomness(const void *buf, size_t size)
{
- unsigned long cycles = random_get_entropy();
- unsigned long flags, now = jiffies;
+ unsigned long entropy = random_get_entropy();
+ unsigned long flags;
if (crng_init == 0 && size)
crng_pre_init_inject(buf, size, false);
spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(&cycles, sizeof(cycles));
- _mix_pool_bytes(&now, sizeof(now));
+ _mix_pool_bytes(&entropy, sizeof(entropy));
_mix_pool_bytes(buf, size);
spin_unlock_irqrestore(&input_pool.lock, flags);
}
@@ -1049,12 +1048,11 @@ struct timer_rand_state {
*/
static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
{
- unsigned long cycles = random_get_entropy(), now = jiffies, flags;
+ unsigned long entropy = random_get_entropy(), now = jiffies, flags;
long delta, delta2, delta3;
spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(&cycles, sizeof(cycles));
- _mix_pool_bytes(&now, sizeof(now));
+ _mix_pool_bytes(&entropy, sizeof(entropy));
_mix_pool_bytes(&num, sizeof(num));
spin_unlock_irqrestore(&input_pool.lock, flags);
@@ -1182,7 +1180,6 @@ struct fast_pool {
unsigned long pool[4];
unsigned long last;
unsigned int count;
- u16 reg_idx;
};
static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = {
@@ -1200,13 +1197,13 @@ static DEFINE_PER_CPU(struct fast_pool,
* This is [Half]SipHash-1-x, starting from an empty key. Because
* the key is fixed, it assumes that its inputs are non-malicious,
* and therefore this has no security on its own. s represents the
- * 128 or 256-bit SipHash state, while v represents a 128-bit input.
+ * four-word SipHash state, while v represents a two-word input.
*/
-static void fast_mix(unsigned long s[4], const unsigned long *v)
+static void fast_mix(unsigned long s[4], const unsigned long v[2])
{
size_t i;
- for (i = 0; i < 16 / sizeof(long); ++i) {
+ for (i = 0; i < 2; ++i) {
s[3] ^= v[i];
#ifdef CONFIG_64BIT
s[0] += s[1]; s[1] = rol64(s[1], 13); s[1] ^= s[0]; s[0] = rol64(s[0], 32);
@@ -1246,33 +1243,17 @@ int random_online_cpu(unsigned int cpu)
}
#endif
-static unsigned long get_reg(struct fast_pool *f, struct pt_regs *regs)
-{
- unsigned long *ptr = (unsigned long *)regs;
- unsigned int idx;
-
- if (regs == NULL)
- return 0;
- idx = READ_ONCE(f->reg_idx);
- if (idx >= sizeof(struct pt_regs) / sizeof(unsigned long))
- idx = 0;
- ptr += idx++;
- WRITE_ONCE(f->reg_idx, idx);
- return *ptr;
-}
-
static void mix_interrupt_randomness(struct work_struct *work)
{
struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix);
/*
- * The size of the copied stack pool is explicitly 16 bytes so that we
- * tax mix_pool_byte()'s compression function the same amount on all
- * platforms. This means on 64-bit we copy half the pool into this,
- * while on 32-bit we copy all of it. The entropy is supposed to be
- * sufficiently dispersed between bits that in the sponge-like
- * half case, on average we don't wind up "losing" some.
+ * The size of the copied stack pool is explicitly 2 longs so that we
+ * only ever ingest half of the siphash output each time, retaining
+ * the other half as the next "key" that carries over. The entropy is
+ * supposed to be sufficiently dispersed between bits so on average
+ * we don't wind up "losing" some.
*/
- u8 pool[16];
+ unsigned long pool[2];
/* Check to see if we're running on the wrong CPU due to hotplug. */
local_irq_disable();
@@ -1304,36 +1285,21 @@ static void mix_interrupt_randomness(str
void add_interrupt_randomness(int irq)
{
enum { MIX_INFLIGHT = 1U << 31 };
- unsigned long cycles = random_get_entropy(), now = jiffies;
+ unsigned long entropy = random_get_entropy();
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
struct pt_regs *regs = get_irq_regs();
unsigned int new_count;
- union {
- u32 u32[4];
- u64 u64[2];
- unsigned long longs[16 / sizeof(long)];
- } irq_data;
-
- if (cycles == 0)
- cycles = get_reg(fast_pool, regs);
-
- if (sizeof(unsigned long) == 8) {
- irq_data.u64[0] = cycles ^ rol64(now, 32) ^ irq;
- irq_data.u64[1] = regs ? instruction_pointer(regs) : _RET_IP_;
- } else {
- irq_data.u32[0] = cycles ^ irq;
- irq_data.u32[1] = now;
- irq_data.u32[2] = regs ? instruction_pointer(regs) : _RET_IP_;
- irq_data.u32[3] = get_reg(fast_pool, regs);
- }
- fast_mix(fast_pool->pool, irq_data.longs);
+ fast_mix(fast_pool->pool, (unsigned long[2]){
+ entropy,
+ (regs ? instruction_pointer(regs) : _RET_IP_) ^ swab(irq)
+ });
new_count = ++fast_pool->count;
if (new_count & MIX_INFLIGHT)
return;
- if (new_count < 64 && (!time_after(now, fast_pool->last + HZ) ||
+ if (new_count < 64 && (!time_is_before_jiffies(fast_pool->last + HZ) ||
unlikely(crng_init == 0)))
return;
@@ -1369,28 +1335,28 @@ static void entropy_timer(unsigned long
static void try_to_generate_entropy(void)
{
struct {
- unsigned long cycles;
+ unsigned long entropy;
struct timer_list timer;
} stack;
- stack.cycles = random_get_entropy();
+ stack.entropy = random_get_entropy();
/* Slow counter - or none. Don't even bother */
- if (stack.cycles == random_get_entropy())
+ if (stack.entropy == random_get_entropy())
return;
__setup_timer_on_stack(&stack.timer, entropy_timer, 0, 0);
while (!crng_ready() && !signal_pending(current)) {
if (!timer_pending(&stack.timer))
mod_timer(&stack.timer, jiffies + 1);
- mix_pool_bytes(&stack.cycles, sizeof(stack.cycles));
+ mix_pool_bytes(&stack.entropy, sizeof(stack.entropy));
schedule();
- stack.cycles = random_get_entropy();
+ stack.entropy = random_get_entropy();
}
del_timer_sync(&stack.timer);
destroy_timer_on_stack(&stack.timer);
- mix_pool_bytes(&stack.cycles, sizeof(stack.cycles));
+ mix_pool_bytes(&stack.entropy, sizeof(stack.entropy));
}
From: "Jason A. Donenfeld" <[email protected]>
commit c04e72700f2293013dab40208e809369378f224c upstream.
In the event that random_get_entropy() can't access a cycle counter or
similar, falling back to returning 0 is really not the best we can do.
Instead, at least calling random_get_entropy_fallback() would be
preferable, because that always needs to return _something_, even
falling back to jiffies eventually. It's not as though
random_get_entropy_fallback() is super high precision or guaranteed to
be entropic, but basically anything that's not zero all the time is
better than returning zero all the time.
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Acked-by: Dinh Nguyen <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/nios2/include/asm/timex.h | 3 +++
1 file changed, 3 insertions(+)
--- a/arch/nios2/include/asm/timex.h
+++ b/arch/nios2/include/asm/timex.h
@@ -20,5 +20,8 @@
typedef unsigned long cycles_t;
extern cycles_t get_cycles(void);
+#define get_cycles get_cycles
+
+#define random_get_entropy() (((unsigned long)get_cycles()) ?: random_get_entropy_fallback())
#endif
From: Robert Eckelmann <[email protected]>
commit 908e698f2149c3d6a67d9ae15c75545a3f392559 upstream.
Add support for Agilent E5805A (rebranded ION Edgeport/4) to io_ti.
Signed-off-by: Robert Eckelmann <[email protected]>
Link: https://lore.kernel.org/r/20220521230808.30931eca@octoberrain
Cc: [email protected]
Signed-off-by: Johan Hovold <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/usb/serial/io_ti.c | 2 ++
drivers/usb/serial/io_usbvend.h | 1 +
2 files changed, 3 insertions(+)
--- a/drivers/usb/serial/io_ti.c
+++ b/drivers/usb/serial/io_ti.c
@@ -172,6 +172,7 @@ static const struct usb_device_id edgepo
{ USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_TI_EDGEPORT_8S) },
{ USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_TI_EDGEPORT_416) },
{ USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_TI_EDGEPORT_416B) },
+ { USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_E5805A) },
{ }
};
@@ -210,6 +211,7 @@ static const struct usb_device_id id_tab
{ USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_TI_EDGEPORT_8S) },
{ USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_TI_EDGEPORT_416) },
{ USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_TI_EDGEPORT_416B) },
+ { USB_DEVICE(USB_VENDOR_ID_ION, ION_DEVICE_ID_E5805A) },
{ }
};
--- a/drivers/usb/serial/io_usbvend.h
+++ b/drivers/usb/serial/io_usbvend.h
@@ -215,6 +215,7 @@
//
// Definitions for other product IDs
#define ION_DEVICE_ID_MT4X56USB 0x1403 // OEM device
+#define ION_DEVICE_ID_E5805A 0x1A01 // OEM device (rebranded Edgeport/4)
#define GENERATION_ID_FROM_USB_PRODUCT_ID(ProductId) \
From: Sergey Shtylyov <[email protected]>
[ Upstream commit bf476fe22aa1851bab4728e0c49025a6a0bea307 ]
In an unlikely (and probably wrong?) case that the 'ppi' parameter of
ata_host_alloc_pinfo() points to an array starting with a NULL pointer,
there's going to be a kernel oops as the 'pi' local variable won't get
reassigned from the initial value of NULL. Initialize 'pi' instead to
'&ata_dummy_port_info' to fix the possible kernel oops for good...
Found by Linux Verification Center (linuxtesting.org) with the SVACE static
analysis tool.
Signed-off-by: Sergey Shtylyov <[email protected]>
Signed-off-by: Damien Le Moal <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/ata/libata-core.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 35db918a1de5..42f0a592b5ab 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -6051,7 +6051,7 @@ struct ata_host *ata_host_alloc_pinfo(struct device *dev,
const struct ata_port_info * const * ppi,
int n_ports)
{
- const struct ata_port_info *pi;
+ const struct ata_port_info *pi = &ata_dummy_port_info;
struct ata_host *host;
int i, j;
@@ -6059,7 +6059,7 @@ struct ata_host *ata_host_alloc_pinfo(struct device *dev,
if (!host)
return NULL;
- for (i = 0, j = 0, pi = NULL; i < host->n_ports; i++) {
+ for (i = 0, j = 0; i < host->n_ports; i++) {
struct ata_port *ap = host->ports[i];
if (ppi[j])
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 246c03dd899164d0186b6d685d6387f228c28d93 upstream.
In preparation for separating responsibilities, break out the entropy
count management part of crng_reseed() into its own function.
No functional changes.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 36 +++++++++++++++++++++++-------------
1 file changed, 23 insertions(+), 13 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -261,6 +261,7 @@ static struct {
};
static void extract_entropy(void *buf, size_t nbytes);
+static bool drain_entropy(void *buf, size_t nbytes);
static void crng_reseed(void);
@@ -506,23 +507,13 @@ static void crng_slow_load(const void *c
static void crng_reseed(void)
{
unsigned long flags;
- int entropy_count;
unsigned long next_gen;
u8 key[CHACHA20_KEY_SIZE];
bool finalize_init = false;
- /*
- * First we make sure we have POOL_MIN_BITS of entropy in the pool,
- * and then we drain all of it. Only then can we extract a new key.
- */
- do {
- entropy_count = READ_ONCE(input_pool.entropy_count);
- if (entropy_count < POOL_MIN_BITS)
- return;
- } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
- extract_entropy(key, sizeof(key));
- wake_up_interruptible(&random_write_wait);
- kill_fasync(&fasync, SIGIO, POLL_OUT);
+ /* Only reseed if we can, to prevent brute forcing a small amount of new bits. */
+ if (!drain_entropy(key, sizeof(key)))
+ return;
/*
* We copy the new key into the base_crng, overwriting the old one,
@@ -950,6 +941,25 @@ static void extract_entropy(void *buf, s
memzero_explicit(&block, sizeof(block));
}
+/*
+ * First we make sure we have POOL_MIN_BITS of entropy in the pool, and then we
+ * set the entropy count to zero (but don't actually touch any data). Only then
+ * can we extract a new key with extract_entropy().
+ */
+static bool drain_entropy(void *buf, size_t nbytes)
+{
+ unsigned int entropy_count;
+ do {
+ entropy_count = READ_ONCE(input_pool.entropy_count);
+ if (entropy_count < POOL_MIN_BITS)
+ return false;
+ } while (cmpxchg(&input_pool.entropy_count, entropy_count, 0) != entropy_count);
+ extract_entropy(buf, nbytes);
+ wake_up_interruptible(&random_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
+ return true;
+}
+
#define warn_unseeded_randomness(previous) \
_warn_unseeded_randomness(__func__, (void *)_RET_IP_, (previous))
From: Miaoqian Lin <[email protected]>
[ Upstream commit 1c245358ce0b13669f6d1625f7a4e05c41f28980 ]
platform_get_irq() returns negative error number instead 0 on failure.
And the doc of platform_get_irq() provides a usage example:
int irq = platform_get_irq(pdev, 0);
if (irq < 0)
return irq;
Fix the check of return value to catch errors correctly.
Fixes: eb1f2930609b ("Driver for the Atmel on-chip SSC on AT32AP and AT91")
Reviewed-by: Claudiu Beznea <[email protected]>
Signed-off-by: Miaoqian Lin <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/misc/atmel-ssc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/atmel-ssc.c b/drivers/misc/atmel-ssc.c
index 8c9a444d61d3..65bc573d6ab4 100644
--- a/drivers/misc/atmel-ssc.c
+++ b/drivers/misc/atmel-ssc.c
@@ -190,9 +190,9 @@ static int ssc_probe(struct platform_device *pdev)
clk_disable_unprepare(ssc->clk);
ssc->irq = platform_get_irq(pdev, 0);
- if (!ssc->irq) {
+ if (ssc->irq < 0) {
dev_dbg(&pdev->dev, "could not get irq\n");
- return -ENXIO;
+ return ssc->irq;
}
mutex_lock(&user_lock);
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 434537ae54ad37e93555de21b6ac8133d6d773a9 upstream.
In 79a8468747c5 ("random: check for increase of entropy_count because of
signed conversion"), a number of checks were added around what values
were passed to account(), because account() was doing fancy fixed point
fractional arithmetic, and a user had some ability to pass large values
directly into it. One of things in that commit was limiting those values
to INT_MAX >> 6. The first >> 3 was for bytes to bits, and the next >> 3
was for bits to 1/8 fractional bits.
However, for several years now, urandom reads no longer touch entropy
accounting, and so this check serves no purpose. The current flow is:
urandom_read_nowarn()-->get_random_bytes_user()-->chacha20_block()
Of course, we don't want that size_t to be truncated when adding it into
the ssize_t. But we arrive at urandom_read_nowarn() in the first place
either via ordinary fops, which limits reads to MAX_RW_COUNT, or via
getrandom() which limits reads to INT_MAX.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Jann Horn <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1336,9 +1336,8 @@ void rand_initialize_disk(struct gendisk
static ssize_t urandom_read_nowarn(struct file *file, char __user *buf,
size_t nbytes, loff_t *ppos)
{
- int ret;
+ ssize_t ret;
- nbytes = min_t(size_t, nbytes, INT_MAX >> 6);
ret = get_random_bytes_user(buf, nbytes);
trace_urandom_read(nbytes, input_pool.entropy_count);
return ret;
From: "Jason A. Donenfeld" <[email protected]>
commit b777c38239fec5a528e59f55b379e31b1a187524 upstream.
add_hwgenerator_randomness() is a function implemented and documented
inside of random.c. It is the way that hardware RNGs push data into it.
Therefore, it should be declared in random.h. Otherwise sparse complains
with:
random.c:1137:6: warning: symbol 'add_hwgenerator_randomness' was not declared. Should it be static?
The alternative would be to include hw_random.h into random.c, but that
wouldn't really be good for anything except slowing down compile time.
Cc: Matt Mackall <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Acked-by: Herbert Xu <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 1 +
include/linux/hw_random.h | 2 --
include/linux/random.h | 2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -12,6 +12,7 @@
#include <linux/device.h>
#include <linux/hw_random.h>
+#include <linux/random.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
--- a/include/linux/hw_random.h
+++ b/include/linux/hw_random.h
@@ -60,7 +60,5 @@ extern int devm_hwrng_register(struct de
/** Unregister a Hardware Random Number Generator driver. */
extern void hwrng_unregister(struct hwrng *rng);
extern void devm_hwrng_unregister(struct device *dve, struct hwrng *rng);
-/** Feed random bits into the pool. */
-extern void add_hwgenerator_randomness(const void *buffer, size_t count, size_t entropy);
#endif /* LINUX_HWRANDOM_H_ */
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -32,6 +32,8 @@ static inline void add_latent_entropy(vo
extern void add_input_randomness(unsigned int type, unsigned int code,
unsigned int value) __latent_entropy;
extern void add_interrupt_randomness(int irq) __latent_entropy;
+extern void add_hwgenerator_randomness(const void *buffer, size_t count,
+ size_t entropy);
extern void get_random_bytes(void *buf, size_t nbytes);
extern int wait_for_random_bytes(void);
From: Theodore Ts'o <[email protected]>
commit 58be0106c5306b939b07b4b8bf00669a20593f4b upstream.
Fixes: eb9d1bf079bb: "random: only read from /dev/random after its pool has received 128 bits"
Reported-by: kernel test robot <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -774,8 +774,11 @@ retry:
if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
goto retry;
- if (has_initialized)
+ if (has_initialized) {
r->initialized = 1;
+ wake_up_interruptible(&random_read_wait);
+ kill_fasync(&fasync, SIGIO, POLL_IN);
+ }
trace_credit_entropy_bits(r->name, nbits,
entropy_count >> ENTROPY_SHIFT, _RET_IP_);
@@ -791,6 +794,13 @@ retry:
entropy_bits = r->entropy_count >> ENTROPY_SHIFT;
}
+ /* initialize the blocking pool if necessary */
+ if (entropy_bits >= random_read_wakeup_bits &&
+ !other->initialized) {
+ schedule_work(&other->push_work);
+ return;
+ }
+
/* should we wake readers? */
if (entropy_bits >= random_read_wakeup_bits &&
wq_has_sleeper(&random_read_wait)) {
@@ -1992,8 +2002,8 @@ _random_read(int nonblock, char __user *
return -EAGAIN;
wait_event_interruptible(random_read_wait,
- ENTROPY_BITS(&input_pool) >=
- random_read_wakeup_bits);
+ blocking_pool.initialized &&
+ (ENTROPY_BITS(&input_pool) >= random_read_wakeup_bits));
if (signal_pending(current))
return -ERESTARTSYS;
}
From: "Jason A. Donenfeld" <[email protected]>
commit 408835832158df0357e18e96da7f2d1ed6b80e7f upstream.
PowerPC defines a get_cycles() function, but it does not do the usual
`#define get_cycles get_cycles` dance, making it impossible for generic
code to see if an arch-specific function was defined. While the
get_cycles() ifdef is not currently used, the following timekeeping
patch in this series will depend on the macro existing (or not existing)
when defining random_get_entropy().
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Paul Mackerras <[email protected]>
Acked-by: Michael Ellerman <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/powerpc/include/asm/timex.h | 1 +
1 file changed, 1 insertion(+)
--- a/arch/powerpc/include/asm/timex.h
+++ b/arch/powerpc/include/asm/timex.h
@@ -53,6 +53,7 @@ static inline cycles_t get_cycles(void)
return ret;
#endif
}
+#define get_cycles get_cycles
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_TIMEX_H */
From: Zhang Yi <[email protected]>
commit b55c3cd102a6f48b90e61c44f7f3dda8c290c694 upstream.
We capture a NULL pointer issue when resizing a corrupt ext4 image which
is freshly clear resize_inode feature (not run e2fsck). It could be
simply reproduced by following steps. The problem is because of the
resize_inode feature was cleared, and it will convert the filesystem to
meta_bg mode in ext4_resize_fs(), but the es->s_reserved_gdt_blocks was
not reduced to zero, so could we mistakenly call reserve_backup_gdb()
and passing an uninitialized resize_inode to it when adding new group
descriptors.
mkfs.ext4 /dev/sda 3G
tune2fs -O ^resize_inode /dev/sda #forget to run requested e2fsck
mount /dev/sda /mnt
resize2fs /dev/sda 8G
========
BUG: kernel NULL pointer dereference, address: 0000000000000028
CPU: 19 PID: 3243 Comm: resize2fs Not tainted 5.18.0-rc7-00001-gfde086c5ebfd #748
...
RIP: 0010:ext4_flex_group_add+0xe08/0x2570
...
Call Trace:
<TASK>
ext4_resize_fs+0xbec/0x1660
__ext4_ioctl+0x1749/0x24e0
ext4_ioctl+0x12/0x20
__x64_sys_ioctl+0xa6/0x110
do_syscall_64+0x3b/0x90
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f2dd739617b
========
The fix is simple, add a check in ext4_resize_begin() to make sure that
the es->s_reserved_gdt_blocks is zero when the resize_inode feature is
disabled.
Cc: [email protected]
Signed-off-by: Zhang Yi <[email protected]>
Reviewed-by: Ritesh Harjani <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
fs/ext4/resize.c | 10 ++++++++++
1 file changed, 10 insertions(+)
--- a/fs/ext4/resize.c
+++ b/fs/ext4/resize.c
@@ -52,6 +52,16 @@ int ext4_resize_begin(struct super_block
return -EPERM;
/*
+ * If the reserved GDT blocks is non-zero, the resize_inode feature
+ * should always be set.
+ */
+ if (EXT4_SB(sb)->s_es->s_reserved_gdt_blocks &&
+ !ext4_has_feature_resize_inode(sb)) {
+ ext4_error(sb, "resize_inode disabled but reserved GDT blocks non-zero");
+ return -EFSCORRUPTED;
+ }
+
+ /*
* If we are not using the primary superblock/GDT copy don't resize,
* because the user tools have no way of handling this. Probably a
* bad time to do it anyways.
From: chengkaitao <[email protected]>
[ Upstream commit a58a7f97ba11391d2d0d408e0b24f38d86ae748e ]
The reference must be released when device_register(&vm_cmdline_parent)
failed. Add the corresponding 'put_device()' in the error handling path.
Signed-off-by: chengkaitao <[email protected]>
Message-Id: <[email protected]>
Signed-off-by: Michael S. Tsirkin <[email protected]>
Acked-by: Jason Wang <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
drivers/virtio/virtio_mmio.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 50840984fbfa..f62da3b7c27b 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -630,6 +630,7 @@ static int vm_cmdline_set(const char *device,
if (!vm_cmdline_parent_registered) {
err = device_register(&vm_cmdline_parent);
if (err) {
+ put_device(&vm_cmdline_parent);
pr_err("Failed to register parent device!\n");
return err;
}
--
2.35.1
From: "Jason A. Donenfeld" <[email protected]>
commit 5acd35487dc911541672b3ffc322851769c32a56 upstream.
We previously rolled our own randomness readiness notifier, which only
has two users in the whole kernel. Replace this with a more standard
atomic notifier block that serves the same purpose with less code. Also
unexport the symbols, because no modules use it, only unconditional
builtins. The only drawback is that it's possible for a notification
handler returning the "stop" code to prevent further processing, but
given that there are only two users, and that we're unexporting this
anyway, that doesn't seem like a significant drawback for the
simplification we receive here.
Cc: Greg Kroah-Hartman <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
[Jason: for stable, also backported to crypto/drbg.c, not unexporting.]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 17 +++++-------
drivers/char/random.c | 69 ++++++++++++++-----------------------------------
include/crypto/drbg.h | 2 -
include/linux/random.h | 10 ++-----
lib/random32.c | 13 +++++----
5 files changed, 41 insertions(+), 70 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1390,12 +1390,13 @@ static int drbg_generate_long(struct drb
return 0;
}
-static void drbg_schedule_async_seed(struct random_ready_callback *rdy)
+static int drbg_schedule_async_seed(struct notifier_block *nb, unsigned long action, void *data)
{
- struct drbg_state *drbg = container_of(rdy, struct drbg_state,
+ struct drbg_state *drbg = container_of(nb, struct drbg_state,
random_ready);
schedule_work(&drbg->seed_work);
+ return 0;
}
static int drbg_prepare_hrng(struct drbg_state *drbg)
@@ -1408,10 +1409,8 @@ static int drbg_prepare_hrng(struct drbg
INIT_WORK(&drbg->seed_work, drbg_async_seed);
- drbg->random_ready.owner = THIS_MODULE;
- drbg->random_ready.func = drbg_schedule_async_seed;
-
- err = add_random_ready_callback(&drbg->random_ready);
+ drbg->random_ready.notifier_call = drbg_schedule_async_seed;
+ err = register_random_ready_notifier(&drbg->random_ready);
switch (err) {
case 0:
@@ -1422,7 +1421,7 @@ static int drbg_prepare_hrng(struct drbg
/* fall through */
default:
- drbg->random_ready.func = NULL;
+ drbg->random_ready.notifier_call = NULL;
return err;
}
@@ -1528,8 +1527,8 @@ free_everything:
*/
static int drbg_uninstantiate(struct drbg_state *drbg)
{
- if (drbg->random_ready.func) {
- del_random_ready_callback(&drbg->random_ready);
+ if (drbg->random_ready.notifier_call) {
+ unregister_random_ready_notifier(&drbg->random_ready);
cancel_work_sync(&drbg->seed_work);
crypto_free_rng(drbg->jent);
drbg->jent = NULL;
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -84,8 +84,8 @@ static int crng_init = 0;
/* Various types of waiters for crng_init->2 transition. */
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
static struct fasync_struct *fasync;
-static DEFINE_SPINLOCK(random_ready_list_lock);
-static LIST_HEAD(random_ready_list);
+static DEFINE_SPINLOCK(random_ready_chain_lock);
+static RAW_NOTIFIER_HEAD(random_ready_chain);
/* Control how we warn userspace. */
static struct ratelimit_state unseeded_warning =
@@ -148,72 +148,45 @@ EXPORT_SYMBOL(wait_for_random_bytes);
*
* returns: 0 if callback is successfully added
* -EALREADY if pool is already initialised (callback not called)
- * -ENOENT if module for callback is not alive
*/
-int add_random_ready_callback(struct random_ready_callback *rdy)
+int register_random_ready_notifier(struct notifier_block *nb)
{
- struct module *owner;
unsigned long flags;
- int err = -EALREADY;
+ int ret = -EALREADY;
if (crng_ready())
- return err;
+ return ret;
- owner = rdy->owner;
- if (!try_module_get(owner))
- return -ENOENT;
-
- spin_lock_irqsave(&random_ready_list_lock, flags);
- if (crng_ready())
- goto out;
-
- owner = NULL;
-
- list_add(&rdy->list, &random_ready_list);
- err = 0;
-
-out:
- spin_unlock_irqrestore(&random_ready_list_lock, flags);
-
- module_put(owner);
-
- return err;
+ spin_lock_irqsave(&random_ready_chain_lock, flags);
+ if (!crng_ready())
+ ret = raw_notifier_chain_register(&random_ready_chain, nb);
+ spin_unlock_irqrestore(&random_ready_chain_lock, flags);
+ return ret;
}
-EXPORT_SYMBOL(add_random_ready_callback);
+EXPORT_SYMBOL(register_random_ready_notifier);
/*
* Delete a previously registered readiness callback function.
*/
-void del_random_ready_callback(struct random_ready_callback *rdy)
+int unregister_random_ready_notifier(struct notifier_block *nb)
{
unsigned long flags;
- struct module *owner = NULL;
-
- spin_lock_irqsave(&random_ready_list_lock, flags);
- if (!list_empty(&rdy->list)) {
- list_del_init(&rdy->list);
- owner = rdy->owner;
- }
- spin_unlock_irqrestore(&random_ready_list_lock, flags);
+ int ret;
- module_put(owner);
+ spin_lock_irqsave(&random_ready_chain_lock, flags);
+ ret = raw_notifier_chain_unregister(&random_ready_chain, nb);
+ spin_unlock_irqrestore(&random_ready_chain_lock, flags);
+ return ret;
}
-EXPORT_SYMBOL(del_random_ready_callback);
+EXPORT_SYMBOL(unregister_random_ready_notifier);
static void process_random_ready_list(void)
{
unsigned long flags;
- struct random_ready_callback *rdy, *tmp;
- spin_lock_irqsave(&random_ready_list_lock, flags);
- list_for_each_entry_safe(rdy, tmp, &random_ready_list, list) {
- struct module *owner = rdy->owner;
-
- list_del_init(&rdy->list);
- rdy->func(rdy);
- module_put(owner);
- }
- spin_unlock_irqrestore(&random_ready_list_lock, flags);
+ spin_lock_irqsave(&random_ready_chain_lock, flags);
+ raw_notifier_call_chain(&random_ready_chain, 0, NULL);
+ spin_unlock_irqrestore(&random_ready_chain_lock, flags);
}
#define warn_unseeded_randomness(previous) \
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -136,7 +136,7 @@ struct drbg_state {
const struct drbg_state_ops *d_ops;
const struct drbg_core *core;
struct drbg_string test_data;
- struct random_ready_callback random_ready;
+ struct notifier_block random_ready;
};
static inline __u8 drbg_statelen(struct drbg_state *drbg)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -10,11 +10,7 @@
#include <uapi/linux/random.h>
-struct random_ready_callback {
- struct list_head list;
- void (*func)(struct random_ready_callback *rdy);
- struct module *owner;
-};
+struct notifier_block;
extern void add_device_randomness(const void *, size_t);
extern void add_bootloader_randomness(const void *, size_t);
@@ -39,8 +35,8 @@ extern void get_random_bytes(void *buf,
extern int wait_for_random_bytes(void);
extern int __init rand_initialize(void);
extern bool rng_is_initialized(void);
-extern int add_random_ready_callback(struct random_ready_callback *rdy);
-extern void del_random_ready_callback(struct random_ready_callback *rdy);
+extern int register_random_ready_notifier(struct notifier_block *nb);
+extern int unregister_random_ready_notifier(struct notifier_block *nb);
extern size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes);
#ifndef MODULE
--- a/lib/random32.c
+++ b/lib/random32.c
@@ -39,6 +39,7 @@
#include <linux/sched.h>
#include <linux/bitops.h>
#include <linux/slab.h>
+#include <linux/notifier.h>
#include <asm/unaligned.h>
/**
@@ -545,9 +546,11 @@ static void prandom_reseed(unsigned long
* To avoid worrying about whether it's safe to delay that interrupt
* long enough to seed all CPUs, just schedule an immediate timer event.
*/
-static void prandom_timer_start(struct random_ready_callback *unused)
+static int prandom_timer_start(struct notifier_block *nb,
+ unsigned long action, void *data)
{
mod_timer(&seed_timer, jiffies);
+ return 0;
}
/*
@@ -556,13 +559,13 @@ static void prandom_timer_start(struct r
*/
static int __init prandom_init_late(void)
{
- static struct random_ready_callback random_ready = {
- .func = prandom_timer_start
+ static struct notifier_block random_ready = {
+ .notifier_call = prandom_timer_start
};
- int ret = add_random_ready_callback(&random_ready);
+ int ret = register_random_ready_notifier(&random_ready);
if (ret == -EALREADY) {
- prandom_timer_start(&random_ready);
+ prandom_timer_start(&random_ready, 0, NULL);
ret = 0;
}
return ret;
From: "Jason A. Donenfeld" <[email protected]>
commit e3e33fc2ea7fcefd0d761db9d6219f83b4248f5c upstream.
Years ago, a separate fast pool was added for interrupts, so that the
cost associated with taking the input pool spinlocks and mixing into it
would be avoided in places where latency is critical. However, one
oversight was that add_input_randomness() and add_disk_randomness()
still sometimes are called directly from the interrupt handler, rather
than being deferred to a thread. This means that some unlucky interrupts
will be caught doing a blake2s_compress() call and potentially spinning
on input_pool.lock, which can also be taken by unprivileged users by
writing into /dev/urandom.
In order to fix this, add_timer_randomness() now checks whether it is
being called from a hard IRQ and if so, just mixes into the per-cpu IRQ
fast pool using fast_mix(), which is much faster and can be done
lock-free. A nice consequence of this, as well, is that it means hard
IRQ context FPU support is likely no longer useful.
The entropy estimation algorithm used by add_timer_randomness() is also
somewhat different than the one used for add_interrupt_randomness(). The
former looks at deltas of deltas of deltas, while the latter just waits
for 64 interrupts for one bit or for one second since the last bit. In
order to bridge these, and since add_interrupt_randomness() runs after
an add_timer_randomness() that's called from hard IRQ, we add to the
fast pool credit the related amount, and then subtract one to account
for add_interrupt_randomness()'s contribution.
A downside of this, however, is that the num argument is potentially
attacker controlled, which puts a bit more pressure on the fast_mix()
sponge to do more than it's really intended to do. As a mitigating
factor, the first 96 bits of input aren't attacker controlled (a cycle
counter followed by zeros), which means it's essentially two rounds of
siphash rather than one, which is somewhat better. It's also not that
much different from add_interrupt_randomness()'s use of the irq stack
instruction pointer register.
Cc: Thomas Gleixner <[email protected]>
Cc: Filipe Manana <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 51 +++++++++++++++++++++++++++++++++++---------------
1 file changed, 36 insertions(+), 15 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1082,6 +1082,7 @@ static void mix_interrupt_randomness(str
* we don't wind up "losing" some.
*/
unsigned long pool[2];
+ unsigned int count;
/* Check to see if we're running on the wrong CPU due to hotplug. */
local_irq_disable();
@@ -1095,12 +1096,13 @@ static void mix_interrupt_randomness(str
* consistent view, before we reenable irqs again.
*/
memcpy(pool, fast_pool->pool, sizeof(pool));
+ count = fast_pool->count;
fast_pool->count = 0;
fast_pool->last = jiffies;
local_irq_enable();
mix_pool_bytes(pool, sizeof(pool));
- credit_init_bits(1);
+ credit_init_bits(max(1u, (count & U16_MAX) / 64));
memzero_explicit(pool, sizeof(pool));
}
@@ -1140,22 +1142,30 @@ struct timer_rand_state {
/*
* This function adds entropy to the entropy "pool" by using timing
- * delays. It uses the timer_rand_state structure to make an estimate
- * of how many bits of entropy this call has added to the pool.
- *
- * The number "num" is also added to the pool - it should somehow describe
- * the type of event which just happened. This is currently 0-255 for
- * keyboard scan codes, and 256 upwards for interrupts.
+ * delays. It uses the timer_rand_state structure to make an estimate
+ * of how many bits of entropy this call has added to the pool. The
+ * value "num" is also added to the pool; it should somehow describe
+ * the type of event that just happened.
*/
static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
{
unsigned long entropy = random_get_entropy(), now = jiffies, flags;
long delta, delta2, delta3;
+ unsigned int bits;
- spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(&entropy, sizeof(entropy));
- _mix_pool_bytes(&num, sizeof(num));
- spin_unlock_irqrestore(&input_pool.lock, flags);
+ /*
+ * If we're in a hard IRQ, add_interrupt_randomness() will be called
+ * sometime after, so mix into the fast pool.
+ */
+ if (in_irq()) {
+ fast_mix(this_cpu_ptr(&irq_randomness)->pool,
+ (unsigned long[2]){ entropy, num });
+ } else {
+ spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(&entropy, sizeof(entropy));
+ _mix_pool_bytes(&num, sizeof(num));
+ spin_unlock_irqrestore(&input_pool.lock, flags);
+ }
if (crng_ready())
return;
@@ -1186,11 +1196,22 @@ static void add_timer_randomness(struct
delta = delta3;
/*
- * delta is now minimum absolute delta.
- * Round down by 1 bit on general principles,
- * and limit entropy estimate to 12 bits.
+ * delta is now minimum absolute delta. Round down by 1 bit
+ * on general principles, and limit entropy estimate to 11 bits.
+ */
+ bits = min(fls(delta >> 1), 11);
+
+ /*
+ * As mentioned above, if we're in a hard IRQ, add_interrupt_randomness()
+ * will run after this, which uses a different crediting scheme of 1 bit
+ * per every 64 interrupts. In order to let that function do accounting
+ * close to the one in this function, we credit a full 64/64 bit per bit,
+ * and then subtract one to account for the extra one added.
*/
- credit_init_bits(min_t(unsigned int, fls(delta >> 1), 11));
+ if (in_irq())
+ this_cpu_ptr(&irq_randomness)->count += max(1u, bits * 64) - 1;
+ else
+ credit_init_bits(bits);
}
void add_input_randomness(unsigned int type, unsigned int code,
From: "Jason A. Donenfeld" <[email protected]>
commit e73aaae2fa9024832e1f42e30c787c7baf61d014 upstream.
The SipHash family of permutations is currently used in three places:
- siphash.c itself, used in the ordinary way it was intended.
- random32.c, in a construction from an anonymous contributor.
- random.c, as part of its fast_mix function.
Each one of these places reinvents the wheel with the same C code, same
rotation constants, and same symmetry-breaking constants.
This commit tidies things up a bit by placing macros for the
permutations and constants into siphash.h, where each of the three .c
users can access them. It also leaves a note dissuading more users of
them from emerging.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 30 +++++++-----------------------
include/linux/prandom.h | 23 +++++++----------------
include/linux/siphash.h | 28 ++++++++++++++++++++++++++++
lib/siphash.c | 32 ++++++++++----------------------
4 files changed, 52 insertions(+), 61 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -51,6 +51,7 @@
#include <linux/syscalls.h>
#include <linux/completion.h>
#include <linux/uuid.h>
+#include <linux/siphash.h>
#include <crypto/chacha20.h>
#include <crypto/blake2s.h>
#include <asm/processor.h>
@@ -1012,12 +1013,11 @@ struct fast_pool {
static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = {
#ifdef CONFIG_64BIT
- /* SipHash constants */
- .pool = { 0x736f6d6570736575UL, 0x646f72616e646f6dUL,
- 0x6c7967656e657261UL, 0x7465646279746573UL }
+#define FASTMIX_PERM SIPHASH_PERMUTATION
+ .pool = { SIPHASH_CONST_0, SIPHASH_CONST_1, SIPHASH_CONST_2, SIPHASH_CONST_3 }
#else
- /* HalfSipHash constants */
- .pool = { 0, 0, 0x6c796765U, 0x74656462U }
+#define FASTMIX_PERM HSIPHASH_PERMUTATION
+ .pool = { HSIPHASH_CONST_0, HSIPHASH_CONST_1, HSIPHASH_CONST_2, HSIPHASH_CONST_3 }
#endif
};
@@ -1029,27 +1029,11 @@ static DEFINE_PER_CPU(struct fast_pool,
*/
static void fast_mix(unsigned long s[4], unsigned long v1, unsigned long v2)
{
-#ifdef CONFIG_64BIT
-#define PERM() do { \
- s[0] += s[1]; s[1] = rol64(s[1], 13); s[1] ^= s[0]; s[0] = rol64(s[0], 32); \
- s[2] += s[3]; s[3] = rol64(s[3], 16); s[3] ^= s[2]; \
- s[0] += s[3]; s[3] = rol64(s[3], 21); s[3] ^= s[0]; \
- s[2] += s[1]; s[1] = rol64(s[1], 17); s[1] ^= s[2]; s[2] = rol64(s[2], 32); \
-} while (0)
-#else
-#define PERM() do { \
- s[0] += s[1]; s[1] = rol32(s[1], 5); s[1] ^= s[0]; s[0] = rol32(s[0], 16); \
- s[2] += s[3]; s[3] = rol32(s[3], 8); s[3] ^= s[2]; \
- s[0] += s[3]; s[3] = rol32(s[3], 7); s[3] ^= s[0]; \
- s[2] += s[1]; s[1] = rol32(s[1], 13); s[1] ^= s[2]; s[2] = rol32(s[2], 16); \
-} while (0)
-#endif
-
s[3] ^= v1;
- PERM();
+ FASTMIX_PERM(s[0], s[1], s[2], s[3]);
s[0] ^= v1;
s[3] ^= v2;
- PERM();
+ FASTMIX_PERM(s[0], s[1], s[2], s[3]);
s[0] ^= v2;
}
--- a/include/linux/prandom.h
+++ b/include/linux/prandom.h
@@ -10,6 +10,7 @@
#include <linux/types.h>
#include <linux/percpu.h>
+#include <linux/siphash.h>
u32 prandom_u32(void);
void prandom_bytes(void *buf, size_t nbytes);
@@ -21,15 +22,10 @@ void prandom_reseed_late(void);
* The core SipHash round function. Each line can be executed in
* parallel given enough CPU resources.
*/
-#define PRND_SIPROUND(v0, v1, v2, v3) ( \
- v0 += v1, v1 = rol64(v1, 13), v2 += v3, v3 = rol64(v3, 16), \
- v1 ^= v0, v0 = rol64(v0, 32), v3 ^= v2, \
- v0 += v3, v3 = rol64(v3, 21), v2 += v1, v1 = rol64(v1, 17), \
- v3 ^= v0, v1 ^= v2, v2 = rol64(v2, 32) \
-)
+#define PRND_SIPROUND(v0, v1, v2, v3) SIPHASH_PERMUTATION(v0, v1, v2, v3)
-#define PRND_K0 (0x736f6d6570736575 ^ 0x6c7967656e657261)
-#define PRND_K1 (0x646f72616e646f6d ^ 0x7465646279746573)
+#define PRND_K0 (SIPHASH_CONST_0 ^ SIPHASH_CONST_2)
+#define PRND_K1 (SIPHASH_CONST_1 ^ SIPHASH_CONST_3)
#elif BITS_PER_LONG == 32
/*
@@ -37,14 +33,9 @@ void prandom_reseed_late(void);
* This is weaker, but 32-bit machines are not used for high-traffic
* applications, so there is less output for an attacker to analyze.
*/
-#define PRND_SIPROUND(v0, v1, v2, v3) ( \
- v0 += v1, v1 = rol32(v1, 5), v2 += v3, v3 = rol32(v3, 8), \
- v1 ^= v0, v0 = rol32(v0, 16), v3 ^= v2, \
- v0 += v3, v3 = rol32(v3, 7), v2 += v1, v1 = rol32(v1, 13), \
- v3 ^= v0, v1 ^= v2, v2 = rol32(v2, 16) \
-)
-#define PRND_K0 0x6c796765
-#define PRND_K1 0x74656462
+#define PRND_SIPROUND(v0, v1, v2, v3) HSIPHASH_PERMUTATION(v0, v1, v2, v3)
+#define PRND_K0 (HSIPHASH_CONST_0 ^ HSIPHASH_CONST_2)
+#define PRND_K1 (HSIPHASH_CONST_1 ^ HSIPHASH_CONST_3)
#else
#error Unsupported BITS_PER_LONG
--- a/include/linux/siphash.h
+++ b/include/linux/siphash.h
@@ -136,4 +136,32 @@ static inline u32 hsiphash(const void *d
return ___hsiphash_aligned(data, len, key);
}
+/*
+ * These macros expose the raw SipHash and HalfSipHash permutations.
+ * Do not use them directly! If you think you have a use for them,
+ * be sure to CC the maintainer of this file explaining why.
+ */
+
+#define SIPHASH_PERMUTATION(a, b, c, d) ( \
+ (a) += (b), (b) = rol64((b), 13), (b) ^= (a), (a) = rol64((a), 32), \
+ (c) += (d), (d) = rol64((d), 16), (d) ^= (c), \
+ (a) += (d), (d) = rol64((d), 21), (d) ^= (a), \
+ (c) += (b), (b) = rol64((b), 17), (b) ^= (c), (c) = rol64((c), 32))
+
+#define SIPHASH_CONST_0 0x736f6d6570736575ULL
+#define SIPHASH_CONST_1 0x646f72616e646f6dULL
+#define SIPHASH_CONST_2 0x6c7967656e657261ULL
+#define SIPHASH_CONST_3 0x7465646279746573ULL
+
+#define HSIPHASH_PERMUTATION(a, b, c, d) ( \
+ (a) += (b), (b) = rol32((b), 5), (b) ^= (a), (a) = rol32((a), 16), \
+ (c) += (d), (d) = rol32((d), 8), (d) ^= (c), \
+ (a) += (d), (d) = rol32((d), 7), (d) ^= (a), \
+ (c) += (b), (b) = rol32((b), 13), (b) ^= (c), (c) = rol32((c), 16))
+
+#define HSIPHASH_CONST_0 0U
+#define HSIPHASH_CONST_1 0U
+#define HSIPHASH_CONST_2 0x6c796765U
+#define HSIPHASH_CONST_3 0x74656462U
+
#endif /* _LINUX_SIPHASH_H */
--- a/lib/siphash.c
+++ b/lib/siphash.c
@@ -18,19 +18,13 @@
#include <asm/word-at-a-time.h>
#endif
-#define SIPROUND \
- do { \
- v0 += v1; v1 = rol64(v1, 13); v1 ^= v0; v0 = rol64(v0, 32); \
- v2 += v3; v3 = rol64(v3, 16); v3 ^= v2; \
- v0 += v3; v3 = rol64(v3, 21); v3 ^= v0; \
- v2 += v1; v1 = rol64(v1, 17); v1 ^= v2; v2 = rol64(v2, 32); \
- } while (0)
+#define SIPROUND SIPHASH_PERMUTATION(v0, v1, v2, v3)
#define PREAMBLE(len) \
- u64 v0 = 0x736f6d6570736575ULL; \
- u64 v1 = 0x646f72616e646f6dULL; \
- u64 v2 = 0x6c7967656e657261ULL; \
- u64 v3 = 0x7465646279746573ULL; \
+ u64 v0 = SIPHASH_CONST_0; \
+ u64 v1 = SIPHASH_CONST_1; \
+ u64 v2 = SIPHASH_CONST_2; \
+ u64 v3 = SIPHASH_CONST_3; \
u64 b = ((u64)(len)) << 56; \
v3 ^= key->key[1]; \
v2 ^= key->key[0]; \
@@ -389,19 +383,13 @@ u32 hsiphash_4u32(const u32 first, const
}
EXPORT_SYMBOL(hsiphash_4u32);
#else
-#define HSIPROUND \
- do { \
- v0 += v1; v1 = rol32(v1, 5); v1 ^= v0; v0 = rol32(v0, 16); \
- v2 += v3; v3 = rol32(v3, 8); v3 ^= v2; \
- v0 += v3; v3 = rol32(v3, 7); v3 ^= v0; \
- v2 += v1; v1 = rol32(v1, 13); v1 ^= v2; v2 = rol32(v2, 16); \
- } while (0)
+#define HSIPROUND HSIPHASH_PERMUTATION(v0, v1, v2, v3)
#define HPREAMBLE(len) \
- u32 v0 = 0; \
- u32 v1 = 0; \
- u32 v2 = 0x6c796765U; \
- u32 v3 = 0x74656462U; \
+ u32 v0 = HSIPHASH_CONST_0; \
+ u32 v1 = HSIPHASH_CONST_1; \
+ u32 v2 = HSIPHASH_CONST_2; \
+ u32 v3 = HSIPHASH_CONST_3; \
u32 b = ((u32)(len)) << 24; \
v3 ^= key->key[1]; \
v2 ^= key->key[0]; \
From: Willy Tarreau <[email protected]>
commit ca7af0402550f9a0b3316d5f1c30904e42ed257d upstream.
Here we're randomly adding between 0 and 7 random increments to the
selected source port in order to add some noise in the source port
selection that will make the next port less predictable.
With the default port range of 32768-60999 this means a worst case
reuse scenario of 14116/8=1764 connections between two consecutive
uses of the same port, with an average of 14116/4.5=3137. This code
was stressed at more than 800000 connections per second to a fixed
target with all connections closed by the client using RSTs (worst
condition) and only 2 connections failed among 13 billion, despite
the hash being reseeded every 10 seconds, indicating a perfectly
safe situation.
Cc: Moshe Kol <[email protected]>
Cc: Yossi Gilad <[email protected]>
Cc: Amit Klein <[email protected]>
Reviewed-by: Eric Dumazet <[email protected]>
Signed-off-by: Willy Tarreau <[email protected]>
Signed-off-by: Jakub Kicinski <[email protected]>
Signed-off-by: Ben Hutchings <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
net/ipv4/inet_hashtables.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -644,11 +644,12 @@ next_port:
return -EADDRNOTAVAIL;
ok:
- /* If our first attempt found a candidate, skip next candidate
- * in 1/16 of cases to add some noise.
+ /* Here we want to add a little bit of randomness to the next source
+ * port that will be chosen. We use a max() with a random here so that
+ * on low contention the randomness is maximal and on high contention
+ * it may be inexistent.
*/
- if (!i && !(prandom_u32() % 16))
- i = 2;
+ i = max_t(int, i, (prandom_u32() & 7) * 2);
WRITE_ONCE(table_perturb[index], READ_ONCE(table_perturb[index]) + i + 2);
/* Head lock still held and bh's disabled */
From: Yury Norov <[email protected]>
[ Upstream commit d5767057c9a76a29f073dad66b7fa12a90e8c748 ]
ext2_swab() is defined locally in lib/find_bit.c However it is not
specific to ext2, neither to bitmaps.
There are many potential users of it, so rename it to just swab() and
move to include/uapi/linux/swab.h
ABI guarantees that size of unsigned long corresponds to BITS_PER_LONG,
therefore drop unneeded cast.
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Yury Norov <[email protected]>
Cc: Allison Randal <[email protected]>
Cc: Joe Perches <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: William Breathitt Gray <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/swab.h | 1 +
include/uapi/linux/swab.h | 10 ++++++++++
lib/find_bit.c | 16 ++--------------
3 files changed, 13 insertions(+), 14 deletions(-)
--- a/include/linux/swab.h
+++ b/include/linux/swab.h
@@ -6,6 +6,7 @@
# define swab16 __swab16
# define swab32 __swab32
# define swab64 __swab64
+# define swab __swab
# define swahw32 __swahw32
# define swahb32 __swahb32
# define swab16p __swab16p
--- a/include/uapi/linux/swab.h
+++ b/include/uapi/linux/swab.h
@@ -3,6 +3,7 @@
#include <linux/types.h>
#include <linux/compiler.h>
+#include <asm/bitsperlong.h>
#include <asm/swab.h>
/*
@@ -131,6 +132,15 @@ static inline __attribute_const__ __u32
__fswab64(x))
#endif
+static __always_inline unsigned long __swab(const unsigned long y)
+{
+#if BITS_PER_LONG == 64
+ return __swab64(y);
+#else /* BITS_PER_LONG == 32 */
+ return __swab32(y);
+#endif
+}
+
/**
* __swahw32 - return a word-swapped 32-bit value
* @x: value to wordswap
--- a/lib/find_bit.c
+++ b/lib/find_bit.c
@@ -133,18 +133,6 @@ EXPORT_SYMBOL(find_last_bit);
#ifdef __BIG_ENDIAN
-/* include/linux/byteorder does not support "unsigned long" type */
-static inline unsigned long ext2_swab(const unsigned long y)
-{
-#if BITS_PER_LONG == 64
- return (unsigned long) __swab64((u64) y);
-#elif BITS_PER_LONG == 32
- return (unsigned long) __swab32((u32) y);
-#else
-#error BITS_PER_LONG not defined
-#endif
-}
-
#if !defined(find_next_bit_le) || !defined(find_next_zero_bit_le)
static unsigned long _find_next_bit_le(const unsigned long *addr,
unsigned long nbits, unsigned long start, unsigned long invert)
@@ -157,7 +145,7 @@ static unsigned long _find_next_bit_le(c
tmp = addr[start / BITS_PER_LONG] ^ invert;
/* Handle 1st word. */
- tmp &= ext2_swab(BITMAP_FIRST_WORD_MASK(start));
+ tmp &= swab(BITMAP_FIRST_WORD_MASK(start));
start = round_down(start, BITS_PER_LONG);
while (!tmp) {
@@ -168,7 +156,7 @@ static unsigned long _find_next_bit_le(c
tmp = addr[start / BITS_PER_LONG] ^ invert;
}
- return min(start + __ffs(ext2_swab(tmp)), nbits);
+ return min(start + __ffs(swab(tmp)), nbits);
}
#endif
From: "Jason A. Donenfeld" <[email protected]>
commit 77760fd7f7ae3dfd03668204e708d1568d75447d upstream.
Rather than use spinlocks to protect batched entropy, we can instead
disable interrupts locally, since we're dealing with per-cpu data, and
manage resets with a basic generation counter. At the same time, we
can't quite do this on PREEMPT_RT, where we still want spinlocks-as-
mutexes semantics. So we use a local_lock_t, which provides the right
behavior for each. Because this is a per-cpu lock, that generation
counter is still doing the necessary CPU-to-CPU communication.
This should improve performance a bit. It will also fix the linked splat
that Jonathan received with a PROVE_RAW_LOCK_NESTING=y.
Reviewed-by: Sebastian Andrzej Siewior <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Suggested-by: Andy Lutomirski <[email protected]>
Reported-by: Jonathan Neuschäfer <[email protected]>
Tested-by: Jonathan Neuschäfer <[email protected]>
Link: https://lore.kernel.org/lkml/YfMa0QgsjCVdRAvJ@latitude/
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 58 +++++++++++++++++++++++---------------------------
1 file changed, 27 insertions(+), 31 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1770,13 +1770,15 @@ struct ctl_table random_table[] = {
};
#endif /* CONFIG_SYSCTL */
+static atomic_t batch_generation = ATOMIC_INIT(0);
+
struct batched_entropy {
union {
u64 entropy_u64[CHACHA20_BLOCK_SIZE / sizeof(u64)];
u32 entropy_u32[CHACHA20_BLOCK_SIZE / sizeof(u32)];
};
unsigned int position;
- spinlock_t batch_lock;
+ int generation;
};
/*
@@ -1787,9 +1789,7 @@ struct batched_entropy {
* wait_for_random_bytes() should be called and return 0 at least once at any
* point prior.
*/
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
- .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
-};
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64);
u64 get_random_u64(void)
{
@@ -1797,67 +1797,63 @@ u64 get_random_u64(void)
unsigned long flags;
struct batched_entropy *batch;
static void *previous;
+ int next_gen;
warn_unseeded_randomness(&previous);
+ local_irq_save(flags);
batch = raw_cpu_ptr(&batched_entropy_u64);
- spin_lock_irqsave(&batch->batch_lock, flags);
- if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) {
+
+ next_gen = atomic_read(&batch_generation);
+ if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0 ||
+ next_gen != batch->generation) {
extract_crng((u8 *)batch->entropy_u64);
batch->position = 0;
+ batch->generation = next_gen;
}
+
ret = batch->entropy_u64[batch->position++];
- spin_unlock_irqrestore(&batch->batch_lock, flags);
+ local_irq_restore(flags);
return ret;
}
EXPORT_SYMBOL(get_random_u64);
-static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
- .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock),
-};
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32);
+
u32 get_random_u32(void)
{
u32 ret;
unsigned long flags;
struct batched_entropy *batch;
static void *previous;
+ int next_gen;
warn_unseeded_randomness(&previous);
+ local_irq_save(flags);
batch = raw_cpu_ptr(&batched_entropy_u32);
- spin_lock_irqsave(&batch->batch_lock, flags);
- if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) {
+
+ next_gen = atomic_read(&batch_generation);
+ if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0 ||
+ next_gen != batch->generation) {
extract_crng((u8 *)batch->entropy_u32);
batch->position = 0;
+ batch->generation = next_gen;
}
+
ret = batch->entropy_u32[batch->position++];
- spin_unlock_irqrestore(&batch->batch_lock, flags);
+ local_irq_restore(flags);
return ret;
}
EXPORT_SYMBOL(get_random_u32);
/* It's important to invalidate all potential batched entropy that might
* be stored before the crng is initialized, which we can do lazily by
- * simply resetting the counter to zero so that it's re-extracted on the
- * next usage. */
+ * bumping the generation counter.
+ */
static void invalidate_batched_entropy(void)
{
- int cpu;
- unsigned long flags;
-
- for_each_possible_cpu(cpu) {
- struct batched_entropy *batched_entropy;
-
- batched_entropy = per_cpu_ptr(&batched_entropy_u32, cpu);
- spin_lock_irqsave(&batched_entropy->batch_lock, flags);
- batched_entropy->position = 0;
- spin_unlock(&batched_entropy->batch_lock);
-
- batched_entropy = per_cpu_ptr(&batched_entropy_u64, cpu);
- spin_lock(&batched_entropy->batch_lock);
- batched_entropy->position = 0;
- spin_unlock_irqrestore(&batched_entropy->batch_lock, flags);
- }
+ atomic_inc(&batch_generation);
}
/**
From: "Stephan M?ller" <[email protected]>
commit 97f2650e504033376e8813691cb6eccf73151676 upstream.
As the Jitter RNG provides an SP800-90B compliant noise source, use this
noise source always for the (re)seeding of the DRBG.
To make sure the DRBG is always properly seeded, the reseed threshold
is reduced to 1<<20 generate operations.
The Jitter RNG may report health test failures. Such health test
failures are treated as transient as follows. The DRBG will not reseed
from the Jitter RNG (but from get_random_bytes) in case of a health
test failure. Though, it produces the requested random number.
The Jitter RNG has a failure counter where at most 1024 consecutive
resets due to a health test failure are considered as a transient error.
If more consecutive resets are required, the Jitter RNG will return
a permanent error which is returned to the caller by the DRBG. With this
approach, the worst case reseed threshold is significantly lower than
mandated by SP800-90A in order to seed with an SP800-90B noise source:
the DRBG has a reseed threshold of 2^20 * 1024 = 2^30 generate requests.
Yet, in case of a transient Jitter RNG health test failure, the DRBG is
seeded with the data obtained from get_random_bytes.
However, if the Jitter RNG fails during the initial seeding operation
even due to a health test error, the DRBG will send an error to the
caller because at that time, the DRBG has received no seed that is
SP800-90B compliant.
Signed-off-by: Stephan Mueller <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
crypto/drbg.c | 26 +++++++++++++++++++-------
include/crypto/drbg.h | 6 +-----
2 files changed, 20 insertions(+), 12 deletions(-)
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1089,10 +1089,6 @@ static void drbg_async_seed(struct work_
if (ret)
goto unlock;
- /* If nonblocking pool is initialized, deactivate Jitter RNG */
- crypto_free_rng(drbg->jent);
- drbg->jent = NULL;
-
/* Set seeded to false so that if __drbg_seed fails the
* next generate call will trigger a reseed.
*/
@@ -1170,7 +1166,23 @@ static int drbg_seed(struct drbg_state *
entropylen);
if (ret) {
pr_devel("DRBG: jent failed with %d\n", ret);
- goto out;
+
+ /*
+ * Do not treat the transient failure of the
+ * Jitter RNG as an error that needs to be
+ * reported. The combined number of the
+ * maximum reseed threshold times the maximum
+ * number of Jitter RNG transient errors is
+ * less than the reseed threshold required by
+ * SP800-90A allowing us to treat the
+ * transient errors as such.
+ *
+ * However, we mandate that at least the first
+ * seeding operation must succeed with the
+ * Jitter RNG.
+ */
+ if (!reseed || ret != -EAGAIN)
+ goto out;
}
drbg_string_fill(&data1, entropy, entropylen * 2);
@@ -1495,6 +1507,8 @@ static int drbg_prepare_hrng(struct drbg
if (list_empty(&drbg->test_data.list))
return 0;
+ drbg->jent = crypto_alloc_rng("jitterentropy_rng", 0, 0);
+
INIT_WORK(&drbg->seed_work, drbg_async_seed);
drbg->random_ready.notifier_call = drbg_schedule_async_seed;
@@ -1513,8 +1527,6 @@ static int drbg_prepare_hrng(struct drbg
return err;
}
- drbg->jent = crypto_alloc_rng("jitterentropy_rng", 0, 0);
-
/*
* Require frequent reseeds until the seed source is fully
* initialized.
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -186,11 +186,7 @@ static inline size_t drbg_max_addtl(stru
static inline size_t drbg_max_requests(struct drbg_state *drbg)
{
/* SP800-90A requires 2**48 maximum requests before reseeding */
-#if (__BITS_PER_LONG == 32)
- return SIZE_MAX;
-#else
- return (1UL<<48);
-#endif
+ return (1<<20);
}
/*
From: Richard Henderson <[email protected]>
commit 66f5ae899ada79c0e9a3d8d954f93a72344cd350 upstream.
Keep the generic fallback versions in sync with the other architecture
specific implementations and use the proper name for false.
Suggested-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Richard Henderson <[email protected]>
Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
include/linux/random.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -98,19 +98,19 @@ unsigned long randomize_page(unsigned lo
#else
static inline bool arch_get_random_long(unsigned long *v)
{
- return 0;
+ return false;
}
static inline bool arch_get_random_int(unsigned int *v)
{
- return 0;
+ return false;
}
static inline bool arch_get_random_seed_long(unsigned long *v)
{
- return 0;
+ return false;
}
static inline bool arch_get_random_seed_int(unsigned int *v)
{
- return 0;
+ return false;
}
#endif
From: Baokun Li <[email protected]>
commit a08f789d2ab5242c07e716baf9a835725046be89 upstream.
Hulk Robot reported a BUG_ON:
==================================================================
kernel BUG at fs/ext4/mballoc.c:3211!
[...]
RIP: 0010:ext4_mb_mark_diskspace_used.cold+0x85/0x136f
[...]
Call Trace:
ext4_mb_new_blocks+0x9df/0x5d30
ext4_ext_map_blocks+0x1803/0x4d80
ext4_map_blocks+0x3a4/0x1a10
ext4_writepages+0x126d/0x2c30
do_writepages+0x7f/0x1b0
__filemap_fdatawrite_range+0x285/0x3b0
file_write_and_wait_range+0xb1/0x140
ext4_sync_file+0x1aa/0xca0
vfs_fsync_range+0xfb/0x260
do_fsync+0x48/0xa0
[...]
==================================================================
Above issue may happen as follows:
-------------------------------------
do_fsync
vfs_fsync_range
ext4_sync_file
file_write_and_wait_range
__filemap_fdatawrite_range
do_writepages
ext4_writepages
mpage_map_and_submit_extent
mpage_map_one_extent
ext4_map_blocks
ext4_mb_new_blocks
ext4_mb_normalize_request
>>> start + size <= ac->ac_o_ex.fe_logical
ext4_mb_regular_allocator
ext4_mb_simple_scan_group
ext4_mb_use_best_found
ext4_mb_new_preallocation
ext4_mb_new_inode_pa
ext4_mb_use_inode_pa
>>> set ac->ac_b_ex.fe_len <= 0
ext4_mb_mark_diskspace_used
>>> BUG_ON(ac->ac_b_ex.fe_len <= 0);
we can easily reproduce this problem with the following commands:
`fallocate -l100M disk`
`mkfs.ext4 -b 1024 -g 256 disk`
`mount disk /mnt`
`fsstress -d /mnt -l 0 -n 1000 -p 1`
The size must be smaller than or equal to EXT4_BLOCKS_PER_GROUP.
Therefore, "start + size <= ac->ac_o_ex.fe_logical" may occur
when the size is truncated. So start should be the start position of
the group where ac_o_ex.fe_logical is located after alignment.
In addition, when the value of fe_logical or EXT4_BLOCKS_PER_GROUP
is very large, the value calculated by start_off is more accurate.
Cc: [email protected]
Fixes: cd648b8a8fd5 ("ext4: trim allocation requests to group size")
Reported-by: Hulk Robot <[email protected]>
Signed-off-by: Baokun Li <[email protected]>
Reviewed-by: Ritesh Harjani <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
fs/ext4/mballoc.c | 9 +++++++++
1 file changed, 9 insertions(+)
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -3142,6 +3142,15 @@ ext4_mb_normalize_request(struct ext4_al
size = size >> bsbits;
start = start_off >> bsbits;
+ /*
+ * For tiny groups (smaller than 8MB) the chosen allocation
+ * alignment may be larger than group size. Make sure the
+ * alignment does not move allocation to a different group which
+ * makes mballoc fail assertions later.
+ */
+ start = max(start, rounddown(ac->ac_o_ex.fe_logical,
+ (ext4_lblk_t)EXT4_BLOCKS_PER_GROUP(ac->ac_sb)));
+
/* don't cover already allocated blocks in selected range */
if (ar->pleft && start <= ar->lleft) {
size -= ar->lleft + 1 - start;
From: "Jason A. Donenfeld" <[email protected]>
commit 0deff3c43206c24e746b1410f11125707ad3040e upstream.
This pulls all of the sysctl-focused functions into the sixth labeled
section.
No functional changes.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 35 ++++++++++++++++++++++++++++++-----
1 file changed, 30 insertions(+), 5 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1687,9 +1687,34 @@ const struct file_operations urandom_fop
.llseek = noop_llseek,
};
+
/********************************************************************
*
- * Sysctl interface
+ * Sysctl interface.
+ *
+ * These are partly unused legacy knobs with dummy values to not break
+ * userspace and partly still useful things. They are usually accessible
+ * in /proc/sys/kernel/random/ and are as follows:
+ *
+ * - boot_id - a UUID representing the current boot.
+ *
+ * - uuid - a random UUID, different each time the file is read.
+ *
+ * - poolsize - the number of bits of entropy that the input pool can
+ * hold, tied to the POOL_BITS constant.
+ *
+ * - entropy_avail - the number of bits of entropy currently in the
+ * input pool. Always <= poolsize.
+ *
+ * - write_wakeup_threshold - the amount of entropy in the input pool
+ * below which write polls to /dev/random will unblock, requesting
+ * more entropy, tied to the POOL_MIN_BITS constant. It is writable
+ * to avoid breaking old userspaces, but writing to it does not
+ * change any behavior of the RNG.
+ *
+ * - urandom_min_reseed_secs - fixed to the meaningless value "60".
+ * It is writable to avoid breaking old userspaces, but writing
+ * to it does not change any behavior of the RNG.
*
********************************************************************/
@@ -1697,8 +1722,8 @@ const struct file_operations urandom_fop
#include <linux/sysctl.h>
-static int random_min_urandom_seed = 60;
-static int random_write_wakeup_bits = POOL_MIN_BITS;
+static int sysctl_random_min_urandom_seed = 60;
+static int sysctl_random_write_wakeup_bits = POOL_MIN_BITS;
static int sysctl_poolsize = POOL_BITS;
static char sysctl_bootid[16];
@@ -1756,14 +1781,14 @@ struct ctl_table random_table[] = {
},
{
.procname = "write_wakeup_threshold",
- .data = &random_write_wakeup_bits,
+ .data = &sysctl_random_write_wakeup_bits,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "urandom_min_reseed_secs",
- .data = &random_min_urandom_seed,
+ .data = &sysctl_random_min_urandom_seed,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
From: "Jason A. Donenfeld" <[email protected]>
commit afba0b80b977b2a8f16234f2acd982f82710ba33 upstream.
Since rand_initialize() is run while interrupts are still off and
nothing else is running, we don't need to repeatedly take and release
the pool spinlock, especially in the RDSEED loop.
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -974,10 +974,10 @@ int __init rand_initialize(void)
rv = random_get_entropy();
arch_init = false;
}
- mix_pool_bytes(&rv, sizeof(rv));
+ _mix_pool_bytes(&rv, sizeof(rv));
}
- mix_pool_bytes(&now, sizeof(now));
- mix_pool_bytes(utsname(), sizeof(*(utsname())));
+ _mix_pool_bytes(&now, sizeof(now));
+ _mix_pool_bytes(utsname(), sizeof(*(utsname())));
extract_entropy(base_crng.key, sizeof(base_crng.key));
++base_crng.generation;
From: "Jason A. Donenfeld" <[email protected]>
commit 7a7ff644aeaf071d433caffb3b8ea57354b55bd3 upstream.
In order to chip away at the "premature first" problem, we augment our
existing entropy accounting with more frequent reseedings at boot.
The idea is that at boot, we're getting entropy from various places, and
we're not very sure which of early boot entropy is good and which isn't.
Even when we're crediting the entropy, we're still not totally certain
that it's any good. Since boot is the one time (aside from a compromise)
that we have zero entropy, it's important that we shepherd entropy into
the crng fairly often.
At the same time, we don't want a "premature next" problem, whereby an
attacker can brute force individual bits of added entropy. In lieu of
going full-on Fortuna (for now), we can pick a simpler strategy of just
reseeding more often during the first 5 minutes after boot. This is
still bounded by the 256-bit entropy credit requirement, so we'll skip a
reseeding if we haven't reached that, but in case entropy /is/ coming
in, this ensures that it makes its way into the crng rather rapidly
during these early stages.
Ordinarily we reseed if the previous reseeding is 300 seconds old. This
commit changes things so that for the first 600 seconds of boot time, we
reseed if the previous reseeding is uptime / 2 seconds old. That means
that we'll reseed at the very least double the uptime of the previous
reseeding.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -337,6 +337,28 @@ static void crng_fast_key_erasure(u8 key
}
/*
+ * Return whether the crng seed is considered to be sufficiently
+ * old that a reseeding might be attempted. This happens if the last
+ * reseeding was CRNG_RESEED_INTERVAL ago, or during early boot, at
+ * an interval proportional to the uptime.
+ */
+static bool crng_has_old_seed(void)
+{
+ static bool early_boot = true;
+ unsigned long interval = CRNG_RESEED_INTERVAL;
+
+ if (unlikely(READ_ONCE(early_boot))) {
+ time64_t uptime = ktime_get_seconds();
+ if (uptime >= CRNG_RESEED_INTERVAL / HZ * 2)
+ WRITE_ONCE(early_boot, false);
+ else
+ interval = max_t(unsigned int, 5 * HZ,
+ (unsigned int)uptime / 2 * HZ);
+ }
+ return time_after(jiffies, READ_ONCE(base_crng.birth) + interval);
+}
+
+/*
* This function returns a ChaCha state that you may use for generating
* random data. It also returns up to 32 bytes on its own of random data
* that may be used; random_data_len may not be greater than 32.
@@ -369,10 +391,10 @@ static void crng_make_state(u32 chacha_s
}
/*
- * If the base_crng is more than 5 minutes old, we reseed, which
- * in turn bumps the generation counter that we check below.
+ * If the base_crng is old enough, we try to reseed, which in turn
+ * bumps the generation counter that we check below.
*/
- if (unlikely(time_after(jiffies, READ_ONCE(base_crng.birth) + CRNG_RESEED_INTERVAL)))
+ if (unlikely(crng_has_old_seed()))
crng_reseed();
local_irq_save(flags);
From: "Jason A. Donenfeld" <[email protected]>
commit 64276a9939ff414f2f0db38036cf4e1a0a703394 upstream.
Rather than hard coding various lengths, we can use the right constants.
Strings should be `char *` while buffers should be `u8 *`. Rather than
have a nonsensical and unused maxlength, just remove it. Finally, use
snprintf instead of sprintf, just out of good hygiene.
As well, remove the old comment about returning a binary UUID via the
binary sysctl syscall. That syscall was removed from the kernel in 5.5,
and actually, the "uuid_strategy" function and related infrastructure
for even serving it via the binary sysctl syscall was removed with
894d2491153a ("sysctl drivers: Remove dead binary sysctl support") back
in 2.6.33.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 29 +++++++++++++----------------
include/linux/uuid.h | 1 +
2 files changed, 14 insertions(+), 16 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1655,22 +1655,25 @@ const struct file_operations urandom_fop
static int sysctl_random_min_urandom_seed = 60;
static int sysctl_random_write_wakeup_bits = POOL_MIN_BITS;
static int sysctl_poolsize = POOL_BITS;
-static char sysctl_bootid[16];
+static u8 sysctl_bootid[UUID_SIZE];
/*
* This function is used to return both the bootid UUID, and random
- * UUID. The difference is in whether table->data is NULL; if it is,
+ * UUID. The difference is in whether table->data is NULL; if it is,
* then a new UUID is generated and returned to the user.
- *
- * If the user accesses this via the proc interface, the UUID will be
- * returned as an ASCII string in the standard UUID format; if via the
- * sysctl system call, as 16 bytes of binary data.
*/
static int proc_do_uuid(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos)
{
- struct ctl_table fake_table;
- unsigned char buf[64], tmp_uuid[16], *uuid;
+ u8 tmp_uuid[UUID_SIZE], *uuid;
+ char uuid_string[UUID_STRING_LEN + 1];
+ struct ctl_table fake_table = {
+ .data = uuid_string,
+ .maxlen = UUID_STRING_LEN
+ };
+
+ if (write)
+ return -EPERM;
uuid = table->data;
if (!uuid) {
@@ -1685,12 +1688,8 @@ static int proc_do_uuid(struct ctl_table
spin_unlock(&bootid_spinlock);
}
- sprintf(buf, "%pU", uuid);
-
- fake_table.data = buf;
- fake_table.maxlen = sizeof(buf);
-
- return proc_dostring(&fake_table, write, buffer, lenp, ppos);
+ snprintf(uuid_string, sizeof(uuid_string), "%pU", uuid);
+ return proc_dostring(&fake_table, 0, buffer, lenp, ppos);
}
extern struct ctl_table random_table[];
@@ -1726,13 +1725,11 @@ struct ctl_table random_table[] = {
{
.procname = "boot_id",
.data = &sysctl_bootid,
- .maxlen = 16,
.mode = 0444,
.proc_handler = proc_do_uuid,
},
{
.procname = "uuid",
- .maxlen = 16,
.mode = 0444,
.proc_handler = proc_do_uuid,
},
--- a/include/linux/uuid.h
+++ b/include/linux/uuid.h
@@ -23,6 +23,7 @@
* not including trailing NUL.
*/
#define UUID_STRING_LEN 36
+#define UUID_SIZE 16
static inline int uuid_le_cmp(const uuid_le u1, const uuid_le u2)
{
From: "Jason A. Donenfeld" <[email protected]>
commit 2f14062bb14b0fcfcc21e6dc7d5b5c0d25966164 upstream.
Currently, start_kernel() adds latent entropy and the command line to
the entropy bool *after* the RNG has been initialized, deferring when
it's actually used by things like stack canaries until the next time
the pool is seeded. This surely is not intended.
Rather than splitting up which entropy gets added where and when between
start_kernel() and random_init(), just do everything in random_init(),
which should eliminate these kinds of bugs in the future.
While we're at it, rename the awkwardly titled "rand_initialize()" to
the more standard "random_init()" nomenclature.
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 13 ++++++++-----
include/linux/random.h | 16 +++++++---------
init/main.c | 10 +++-------
3 files changed, 18 insertions(+), 21 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -884,12 +884,13 @@ early_param("random.trust_bootloader", p
/*
* The first collection of entropy occurs at system boot while interrupts
- * are still turned off. Here we push in RDSEED, a timestamp, and utsname().
- * Depending on the above configuration knob, RDSEED may be considered
- * sufficient for initialization. Note that much earlier setup may already
- * have pushed entropy into the input pool by the time we get here.
+ * are still turned off. Here we push in latent entropy, RDSEED, a timestamp,
+ * utsname(), and the command line. Depending on the above configuration knob,
+ * RDSEED may be considered sufficient for initialization. Note that much
+ * earlier setup may already have pushed entropy into the input pool by the
+ * time we get here.
*/
-int __init rand_initialize(void)
+int __init random_init(const char *command_line)
{
size_t i;
ktime_t now = ktime_get_real();
@@ -911,6 +912,8 @@ int __init rand_initialize(void)
}
_mix_pool_bytes(&now, sizeof(now));
_mix_pool_bytes(utsname(), sizeof(*(utsname())));
+ _mix_pool_bytes(command_line, strlen(command_line));
+ add_latent_entropy();
if (crng_ready())
crng_reseed();
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -14,26 +14,24 @@ struct notifier_block;
extern void add_device_randomness(const void *, size_t);
extern void add_bootloader_randomness(const void *, size_t);
+extern void add_input_randomness(unsigned int type, unsigned int code,
+ unsigned int value) __latent_entropy;
+extern void add_interrupt_randomness(int irq) __latent_entropy;
+extern void add_hwgenerator_randomness(const void *buffer, size_t count,
+ size_t entropy);
#if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__)
static inline void add_latent_entropy(void)
{
- add_device_randomness((const void *)&latent_entropy,
- sizeof(latent_entropy));
+ add_device_randomness((const void *)&latent_entropy, sizeof(latent_entropy));
}
#else
static inline void add_latent_entropy(void) {}
#endif
-extern void add_input_randomness(unsigned int type, unsigned int code,
- unsigned int value) __latent_entropy;
-extern void add_interrupt_randomness(int irq) __latent_entropy;
-extern void add_hwgenerator_randomness(const void *buffer, size_t count,
- size_t entropy);
-
extern void get_random_bytes(void *buf, size_t nbytes);
extern int wait_for_random_bytes(void);
-extern int __init rand_initialize(void);
+extern int __init random_init(const char *command_line);
extern bool rng_is_initialized(void);
extern int register_random_ready_notifier(struct notifier_block *nb);
extern int unregister_random_ready_notifier(struct notifier_block *nb);
--- a/init/main.c
+++ b/init/main.c
@@ -583,15 +583,11 @@ asmlinkage __visible void __init start_k
/*
* For best initial stack canary entropy, prepare it after:
* - setup_arch() for any UEFI RNG entropy and boot cmdline access
- * - timekeeping_init() for ktime entropy used in rand_initialize()
+ * - timekeeping_init() for ktime entropy used in random_init()
* - time_init() for making random_get_entropy() work on some platforms
- * - rand_initialize() to get any arch-specific entropy like RDRAND
- * - add_latent_entropy() to get any latent entropy
- * - adding command line entropy
+ * - random_init() to initialize the RNG from from early entropy sources
*/
- rand_initialize();
- add_latent_entropy();
- add_device_randomness(command_line, strlen(command_line));
+ random_init(command_line);
boot_init_stack_canary();
sched_clock_postinit();
From: "Jason A. Donenfeld" <[email protected]>
commit 0791e8b655cc373718f0f58800fdc625a3447ac5 upstream.
Now that we have an explicit base_crng generation counter, we don't need
a separate one for batched entropy. Rather, we can just move the
generation forward every time we change crng_init state or update the
base_crng key.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 29 ++++++++---------------------
1 file changed, 8 insertions(+), 21 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -429,8 +429,6 @@ static DEFINE_PER_CPU(struct crng, crngs
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
-static void invalidate_batched_entropy(void);
-
/*
* crng_fast_load() can be called by code in the interrupt service
* path. So we can't afford to dilly-dally. Returns the number of
@@ -453,7 +451,7 @@ static size_t crng_fast_load(const void
src++; crng_init_cnt++; len--; ret++;
}
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
- invalidate_batched_entropy();
+ ++base_crng.generation;
crng_init = 1;
}
spin_unlock_irqrestore(&base_crng.lock, flags);
@@ -581,7 +579,6 @@ static void crng_reseed(void)
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
if (crng_init < 2) {
- invalidate_batched_entropy();
crng_init = 2;
finalize_init = true;
}
@@ -1306,8 +1303,9 @@ int __init rand_initialize(void)
mix_pool_bytes(utsname(), sizeof(*(utsname())));
extract_entropy(base_crng.key, sizeof(base_crng.key));
+ ++base_crng.generation;
+
if (arch_init && trust_cpu && crng_init < 2) {
- invalidate_batched_entropy();
crng_init = 2;
pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
@@ -1645,8 +1643,6 @@ struct ctl_table random_table[] = {
};
#endif /* CONFIG_SYSCTL */
-static atomic_t batch_generation = ATOMIC_INIT(0);
-
struct batched_entropy {
union {
/*
@@ -1659,8 +1655,8 @@ struct batched_entropy {
u64 entropy_u64[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u64))];
u32 entropy_u32[CHACHA20_BLOCK_SIZE * 3 / (2 * sizeof(u32))];
};
+ unsigned long generation;
unsigned int position;
- int generation;
};
/*
@@ -1679,14 +1675,14 @@ u64 get_random_u64(void)
unsigned long flags;
struct batched_entropy *batch;
static void *previous;
- int next_gen;
+ unsigned long next_gen;
warn_unseeded_randomness(&previous);
local_irq_save(flags);
batch = raw_cpu_ptr(&batched_entropy_u64);
- next_gen = atomic_read(&batch_generation);
+ next_gen = READ_ONCE(base_crng.generation);
if (batch->position >= ARRAY_SIZE(batch->entropy_u64) ||
next_gen != batch->generation) {
_get_random_bytes(batch->entropy_u64, sizeof(batch->entropy_u64));
@@ -1712,14 +1708,14 @@ u32 get_random_u32(void)
unsigned long flags;
struct batched_entropy *batch;
static void *previous;
- int next_gen;
+ unsigned long next_gen;
warn_unseeded_randomness(&previous);
local_irq_save(flags);
batch = raw_cpu_ptr(&batched_entropy_u32);
- next_gen = atomic_read(&batch_generation);
+ next_gen = READ_ONCE(base_crng.generation);
if (batch->position >= ARRAY_SIZE(batch->entropy_u32) ||
next_gen != batch->generation) {
_get_random_bytes(batch->entropy_u32, sizeof(batch->entropy_u32));
@@ -1735,15 +1731,6 @@ u32 get_random_u32(void)
}
EXPORT_SYMBOL(get_random_u32);
-/* It's important to invalidate all potential batched entropy that might
- * be stored before the crng is initialized, which we can do lazily by
- * bumping the generation counter.
- */
-static void invalidate_batched_entropy(void)
-{
- atomic_inc(&batch_generation);
-}
-
/**
* randomize_page - Generate a random, page aligned address
* @start: The smallest acceptable address the caller will take.
From: "Jason A. Donenfeld" <[email protected]>
commit 57c0900b91d8891ab43f0e6b464d059fda51d102 upstream.
Itanium defines a get_cycles() function, but it does not do the usual
`#define get_cycles get_cycles` dance, making it impossible for generic
code to see if an arch-specific function was defined. While the
get_cycles() ifdef is not currently used, the following timekeeping
patch in this series will depend on the macro existing (or not existing)
when defining random_get_entropy().
Cc: Thomas Gleixner <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
arch/ia64/include/asm/timex.h | 1 +
1 file changed, 1 insertion(+)
--- a/arch/ia64/include/asm/timex.h
+++ b/arch/ia64/include/asm/timex.h
@@ -38,6 +38,7 @@ get_cycles (void)
ret = ia64_getreg(_IA64_REG_AR_ITC);
return ret;
}
+#define get_cycles get_cycles
extern void ia64_cpu_local_tick (void);
extern unsigned long long ia64_native_sched_clock (void);
From: Charles Keepax <[email protected]>
[ Upstream commit a8928ada9b96944cadd8b65d191e33199fd38782 ]
A couple of the SX volume controls specify 0x84 as the lowest volume
value, however the correct value from the datasheet is 0x44. The
datasheet don't include spaces in the value it displays as binary so
this was almost certainly just a typo reading 1000100.
Signed-off-by: Charles Keepax <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Mark Brown <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
---
sound/soc/codecs/cs42l56.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sound/soc/codecs/cs42l56.c b/sound/soc/codecs/cs42l56.c
index a2535a7eb4bb..f9f8a9112ff8 100644
--- a/sound/soc/codecs/cs42l56.c
+++ b/sound/soc/codecs/cs42l56.c
@@ -405,9 +405,9 @@ static const struct snd_kcontrol_new cs42l56_snd_controls[] = {
SOC_DOUBLE("ADC Boost Switch", CS42L56_GAIN_BIAS_CTL, 3, 2, 1, 1),
SOC_DOUBLE_R_SX_TLV("Headphone Volume", CS42L56_HPA_VOLUME,
- CS42L56_HPB_VOLUME, 0, 0x84, 0x48, hl_tlv),
+ CS42L56_HPB_VOLUME, 0, 0x44, 0x48, hl_tlv),
SOC_DOUBLE_R_SX_TLV("LineOut Volume", CS42L56_LOA_VOLUME,
- CS42L56_LOB_VOLUME, 0, 0x84, 0x48, hl_tlv),
+ CS42L56_LOB_VOLUME, 0, 0x44, 0x48, hl_tlv),
SOC_SINGLE_TLV("Bass Shelving Volume", CS42L56_TONE_CTL,
0, 0x00, 1, tone_tlv),
--
2.35.1
From: Sebastian Andrzej Siewior <[email protected]>
commit 703f7066f40599c290babdb79dd61319264987e9 upstream.
Since commit
ee3e00e9e7101 ("random: use registers from interrupted code for CPU's w/o a cycle counter")
the irq_flags argument is no longer used.
Remove unused irq_flags.
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Dexuan Cui <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Haiyang Zhang <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: K. Y. Srinivasan <[email protected]>
Cc: Stephen Hemminger <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Sebastian Andrzej Siewior <[email protected]>
Acked-by: Wei Liu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 ++--
drivers/hv/vmbus_drv.c | 2 +-
include/linux/random.h | 2 +-
kernel/irq/handle.c | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -200,7 +200,7 @@
* void add_device_randomness(const void *buf, unsigned int size);
* void add_input_randomness(unsigned int type, unsigned int code,
* unsigned int value);
- * void add_interrupt_randomness(int irq, int irq_flags);
+ * void add_interrupt_randomness(int irq);
* void add_disk_randomness(struct gendisk *disk);
* void add_hwgenerator_randomness(const char *buffer, size_t count,
* size_t entropy);
@@ -1323,7 +1323,7 @@ static __u32 get_reg(struct fast_pool *f
return *ptr;
}
-void add_interrupt_randomness(int irq, int irq_flags)
+void add_interrupt_randomness(int irq)
{
struct entropy_store *r;
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -828,7 +828,7 @@ static void vmbus_isr(void)
tasklet_schedule(hv_context.msg_dpc[cpu]);
}
- add_interrupt_randomness(HYPERVISOR_CALLBACK_VECTOR, 0);
+ add_interrupt_randomness(HYPERVISOR_CALLBACK_VECTOR);
}
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -34,7 +34,7 @@ static inline void add_latent_entropy(vo
extern void add_input_randomness(unsigned int type, unsigned int code,
unsigned int value) __latent_entropy;
-extern void add_interrupt_randomness(int irq, int irq_flags) __latent_entropy;
+extern void add_interrupt_randomness(int irq) __latent_entropy;
extern void get_random_bytes(void *buf, int nbytes);
extern int wait_for_random_bytes(void);
--- a/kernel/irq/handle.c
+++ b/kernel/irq/handle.c
@@ -184,7 +184,7 @@ irqreturn_t handle_irq_event_percpu(stru
retval = __handle_irq_event_percpu(desc, &flags);
- add_interrupt_randomness(desc->irq_data.irq, flags);
+ add_interrupt_randomness(desc->irq_data.irq);
if (!noirqdebug)
note_interrupt(desc, retval);
From: Mark Rutland <[email protected]>
commit 253d3194c2b58152fe830fd27c2fd83ebc6fe5ee upstream.
Some architectures (e.g. arm64) can have heterogeneous CPUs, and the
boot CPU may be able to provide entropy while secondary CPUs cannot. On
such systems, arch_get_random_long() and arch_get_random_seed_long()
will fail unless support for RNG instructions has been detected on all
CPUs. This prevents the boot CPU from being able to provide
(potentially) trusted entropy when seeding the primary CRNG.
To make it possible to seed the primary CRNG from the boot CPU without
adversely affecting the runtime versions of arch_get_random_long() and
arch_get_random_seed_long(), this patch adds new early versions of the
functions used when initializing the primary CRNG.
Default implementations are provided atop of the existing
arch_get_random_long() and arch_get_random_seed_long() so that only
architectures with such constraints need to provide the new helpers.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <[email protected]>
Cc: Mark Brown <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 20 +++++++++++++++++++-
include/linux/random.h | 22 ++++++++++++++++++++++
2 files changed, 41 insertions(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -801,6 +801,24 @@ static bool crng_init_try_arch(struct cr
return arch_init;
}
+static bool __init crng_init_try_arch_early(struct crng_state *crng)
+{
+ int i;
+ bool arch_init = true;
+ unsigned long rv;
+
+ for (i = 4; i < 16; i++) {
+ if (!arch_get_random_seed_long_early(&rv) &&
+ !arch_get_random_long_early(&rv)) {
+ rv = random_get_entropy();
+ arch_init = false;
+ }
+ crng->state[i] ^= rv;
+ }
+
+ return arch_init;
+}
+
static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
{
memcpy(&crng->state[0], "expand 32-byte k", 16);
@@ -813,7 +831,7 @@ static void __init crng_initialize_prima
{
memcpy(&crng->state[0], "expand 32-byte k", 16);
_extract_entropy(&input_pool, &crng->state[4], sizeof(__u32) * 12, 0);
- if (crng_init_try_arch(crng) && trust_cpu) {
+ if (crng_init_try_arch_early(crng) && trust_cpu) {
invalidate_batched_entropy();
numa_crng_init();
crng_init = 2;
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -6,6 +6,8 @@
#ifndef _LINUX_RANDOM_H
#define _LINUX_RANDOM_H
+#include <linux/bug.h>
+#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/once.h>
@@ -114,4 +116,24 @@ static inline bool __must_check arch_get
}
#endif
+/*
+ * Called from the boot CPU during startup; not valid to call once
+ * secondary CPUs are up and preemption is possible.
+ */
+#ifndef arch_get_random_seed_long_early
+static inline bool __init arch_get_random_seed_long_early(unsigned long *v)
+{
+ WARN_ON(system_state != SYSTEM_BOOTING);
+ return arch_get_random_seed_long(v);
+}
+#endif
+
+#ifndef arch_get_random_long_early
+static inline bool __init arch_get_random_long_early(unsigned long *v)
+{
+ WARN_ON(system_state != SYSTEM_BOOTING);
+ return arch_get_random_long(v);
+}
+#endif
+
#endif /* _LINUX_RANDOM_H */
From: "Jason A. Donenfeld" <[email protected]>
commit 9b29b6b20376ab64e1b043df6301d8a92378e631 upstream.
The current flow expands to:
if (crng_ready())
...
else if (...)
if (!crng_ready())
...
The second crng_ready() call is redundant, but can't so easily be
optimized out by the compiler.
This commit simplifies that to:
if (crng_ready()
...
else if (...)
...
Fixes: 560181c27b58 ("random: move initialization functions out of hot pages")
Cc: [email protected]
Cc: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -835,7 +835,7 @@ int __init random_init(const char *comma
if (crng_ready())
crng_reseed();
else if (trust_cpu)
- credit_init_bits(arch_bytes * 8);
+ _credit_init_bits(arch_bytes * 8);
return 0;
}
From: Corentin LABBE <[email protected]>
commit 079840bd13f793b915f6c8e44452eeb4a0aba8ba upstream.
This patch remove the unused PFX macro.
Signed-off-by: Corentin Labbe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 1 -
1 file changed, 1 deletion(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -25,7 +25,6 @@
#include <asm/uaccess.h>
#define RNG_MODULE_NAME "hw_random"
-#define PFX RNG_MODULE_NAME ": "
static struct hwrng *current_rng;
static struct task_struct *hwrng_fill;
From: Ingo Molnar <[email protected]>
commit b34fbaa9289328c7aec67d2b8b8b7d02bc61c67d upstream.
No need to keep preemption disabled across the whole function.
mix_pool_bytes() uses a spin_lock() to protect the pool and there are
other places like write_pool() whhich invoke mix_pool_bytes() without
disabling preemption.
credit_entropy_bits() is invoked from other places like
add_hwgenerator_randomness() without disabling preemption.
Before commit 95b709b6be49 ("random: drop trickle mode") the function
used __this_cpu_inc_return() which would require disabled preemption.
The preempt_disable() section was added in commit 43d5d3018c37 ("[PATCH]
random driver preempt robustness", history tree). It was claimed that
the code relied on "vt_ioctl() being called under BKL".
Cc: "Theodore Ts'o" <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
[bigeasy: enhance the commit message]
Signed-off-by: Sebastian Andrzej Siewior <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 4 ----
1 file changed, 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1189,8 +1189,6 @@ static void add_timer_randomness(struct
} sample;
long delta, delta2, delta3;
- preempt_disable();
-
sample.jiffies = jiffies;
sample.cycles = random_get_entropy();
sample.num = num;
@@ -1228,8 +1226,6 @@ static void add_timer_randomness(struct
* and limit entropy entimate to 12 bits.
*/
credit_entropy_bits(r, min_t(int, fls(delta>>1), 11));
-
- preempt_enable();
}
void add_input_randomness(unsigned int type, unsigned int code,
From: "Jason A. Donenfeld" <[email protected]>
commit 7c3a8a1db5e03d02cc0abb3357a84b8b326dfac3 upstream.
Before these were returning signed values, but the API is intended to be
used with unsigned values.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 195 +++++++++++++++++++++++--------------------------
include/linux/random.h | 24 +++---
2 files changed, 107 insertions(+), 112 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -211,7 +211,7 @@ static void _warn_unseeded_randomness(co
*
* There are a few exported interfaces for use by other drivers:
*
- * void get_random_bytes(void *buf, size_t nbytes)
+ * void get_random_bytes(void *buf, size_t len)
* u32 get_random_u32()
* u64 get_random_u64()
* unsigned int get_random_int()
@@ -250,7 +250,7 @@ static DEFINE_PER_CPU(struct crng, crngs
};
/* Used by crng_reseed() and crng_make_state() to extract a new seed from the input pool. */
-static void extract_entropy(void *buf, size_t nbytes);
+static void extract_entropy(void *buf, size_t len);
/* This extracts a new crng key from the input pool. */
static void crng_reseed(void)
@@ -404,24 +404,24 @@ static void crng_make_state(u32 chacha_s
local_irq_restore(flags);
}
-static void _get_random_bytes(void *buf, size_t nbytes)
+static void _get_random_bytes(void *buf, size_t len)
{
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
u8 tmp[CHACHA20_BLOCK_SIZE];
- size_t len;
+ size_t first_block_len;
- if (!nbytes)
+ if (!len)
return;
- len = min_t(size_t, 32, nbytes);
- crng_make_state(chacha_state, buf, len);
- nbytes -= len;
- buf += len;
+ first_block_len = min_t(size_t, 32, len);
+ crng_make_state(chacha_state, buf, first_block_len);
+ len -= first_block_len;
+ buf += first_block_len;
- while (nbytes) {
- if (nbytes < CHACHA20_BLOCK_SIZE) {
+ while (len) {
+ if (len < CHACHA20_BLOCK_SIZE) {
chacha20_block(chacha_state, tmp);
- memcpy(buf, tmp, nbytes);
+ memcpy(buf, tmp, len);
memzero_explicit(tmp, sizeof(tmp));
break;
}
@@ -429,7 +429,7 @@ static void _get_random_bytes(void *buf,
chacha20_block(chacha_state, buf);
if (unlikely(chacha_state[12] == 0))
++chacha_state[13];
- nbytes -= CHACHA20_BLOCK_SIZE;
+ len -= CHACHA20_BLOCK_SIZE;
buf += CHACHA20_BLOCK_SIZE;
}
@@ -446,20 +446,20 @@ static void _get_random_bytes(void *buf,
* wait_for_random_bytes() should be called and return 0 at least once
* at any point prior.
*/
-void get_random_bytes(void *buf, size_t nbytes)
+void get_random_bytes(void *buf, size_t len)
{
warn_unseeded_randomness();
- _get_random_bytes(buf, nbytes);
+ _get_random_bytes(buf, len);
}
EXPORT_SYMBOL(get_random_bytes);
-static ssize_t get_random_bytes_user(void __user *buf, size_t nbytes)
+static ssize_t get_random_bytes_user(void __user *ubuf, size_t len)
{
- size_t len, left, ret = 0;
+ size_t block_len, left, ret = 0;
u32 chacha_state[CHACHA20_BLOCK_SIZE / sizeof(u32)];
u8 output[CHACHA20_BLOCK_SIZE];
- if (!nbytes)
+ if (!len)
return 0;
/*
@@ -473,8 +473,8 @@ static ssize_t get_random_bytes_user(voi
* use chacha_state after, so we can simply return those bytes to
* the user directly.
*/
- if (nbytes <= CHACHA20_KEY_SIZE) {
- ret = nbytes - copy_to_user(buf, &chacha_state[4], nbytes);
+ if (len <= CHACHA20_KEY_SIZE) {
+ ret = len - copy_to_user(ubuf, &chacha_state[4], len);
goto out_zero_chacha;
}
@@ -483,17 +483,17 @@ static ssize_t get_random_bytes_user(voi
if (unlikely(chacha_state[12] == 0))
++chacha_state[13];
- len = min_t(size_t, nbytes, CHACHA20_BLOCK_SIZE);
- left = copy_to_user(buf, output, len);
+ block_len = min_t(size_t, len, CHACHA20_BLOCK_SIZE);
+ left = copy_to_user(ubuf, output, block_len);
if (left) {
- ret += len - left;
+ ret += block_len - left;
break;
}
- buf += len;
- ret += len;
- nbytes -= len;
- if (!nbytes)
+ ubuf += block_len;
+ ret += block_len;
+ len -= block_len;
+ if (!len)
break;
BUILD_BUG_ON(PAGE_SIZE % CHACHA20_BLOCK_SIZE != 0);
@@ -664,24 +664,24 @@ unsigned long randomize_page(unsigned lo
* use. Use get_random_bytes() instead. It returns the number of
* bytes filled in.
*/
-size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes)
+size_t __must_check get_random_bytes_arch(void *buf, size_t len)
{
- size_t left = nbytes;
+ size_t left = len;
u8 *p = buf;
while (left) {
unsigned long v;
- size_t chunk = min_t(size_t, left, sizeof(unsigned long));
+ size_t block_len = min_t(size_t, left, sizeof(unsigned long));
if (!arch_get_random_long(&v))
break;
- memcpy(p, &v, chunk);
- p += chunk;
- left -= chunk;
+ memcpy(p, &v, block_len);
+ p += block_len;
+ left -= block_len;
}
- return nbytes - left;
+ return len - left;
}
EXPORT_SYMBOL(get_random_bytes_arch);
@@ -692,15 +692,15 @@ EXPORT_SYMBOL(get_random_bytes_arch);
*
* Callers may add entropy via:
*
- * static void mix_pool_bytes(const void *in, size_t nbytes)
+ * static void mix_pool_bytes(const void *buf, size_t len)
*
* After which, if added entropy should be credited:
*
- * static void credit_init_bits(size_t nbits)
+ * static void credit_init_bits(size_t bits)
*
* Finally, extract entropy via:
*
- * static void extract_entropy(void *buf, size_t nbytes)
+ * static void extract_entropy(void *buf, size_t len)
*
**********************************************************************/
@@ -722,9 +722,9 @@ static struct {
.lock = __SPIN_LOCK_UNLOCKED(input_pool.lock),
};
-static void _mix_pool_bytes(const void *in, size_t nbytes)
+static void _mix_pool_bytes(const void *buf, size_t len)
{
- blake2s_update(&input_pool.hash, in, nbytes);
+ blake2s_update(&input_pool.hash, buf, len);
}
/*
@@ -732,12 +732,12 @@ static void _mix_pool_bytes(const void *
* update the initialization bit counter; the caller should call
* credit_init_bits if this is appropriate.
*/
-static void mix_pool_bytes(const void *in, size_t nbytes)
+static void mix_pool_bytes(const void *buf, size_t len)
{
unsigned long flags;
spin_lock_irqsave(&input_pool.lock, flags);
- _mix_pool_bytes(in, nbytes);
+ _mix_pool_bytes(buf, len);
spin_unlock_irqrestore(&input_pool.lock, flags);
}
@@ -745,7 +745,7 @@ static void mix_pool_bytes(const void *i
* This is an HKDF-like construction for using the hashed collected entropy
* as a PRF key, that's then expanded block-by-block.
*/
-static void extract_entropy(void *buf, size_t nbytes)
+static void extract_entropy(void *buf, size_t len)
{
unsigned long flags;
u8 seed[BLAKE2S_HASH_SIZE], next_key[BLAKE2S_HASH_SIZE];
@@ -774,12 +774,12 @@ static void extract_entropy(void *buf, s
spin_unlock_irqrestore(&input_pool.lock, flags);
memzero_explicit(next_key, sizeof(next_key));
- while (nbytes) {
- i = min_t(size_t, nbytes, BLAKE2S_HASH_SIZE);
+ while (len) {
+ i = min_t(size_t, len, BLAKE2S_HASH_SIZE);
/* output = HASHPRF(seed, RDSEED || ++counter) */
++block.counter;
blake2s(buf, (u8 *)&block, seed, i, sizeof(block), sizeof(seed));
- nbytes -= i;
+ len -= i;
buf += i;
}
@@ -787,16 +787,16 @@ static void extract_entropy(void *buf, s
memzero_explicit(&block, sizeof(block));
}
-static void credit_init_bits(size_t nbits)
+static void credit_init_bits(size_t bits)
{
static struct execute_work set_ready;
unsigned int new, orig, add;
unsigned long flags;
- if (crng_ready() || !nbits)
+ if (crng_ready() || !bits)
return;
- add = min_t(size_t, nbits, POOL_BITS);
+ add = min_t(size_t, bits, POOL_BITS);
do {
orig = READ_ONCE(input_pool.init_bits);
@@ -832,13 +832,11 @@ static void credit_init_bits(size_t nbit
* The following exported functions are used for pushing entropy into
* the above entropy accumulation routines:
*
- * void add_device_randomness(const void *buf, size_t size);
- * void add_hwgenerator_randomness(const void *buffer, size_t count,
- * size_t entropy);
- * void add_bootloader_randomness(const void *buf, size_t size);
+ * void add_device_randomness(const void *buf, size_t len);
+ * void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy);
+ * void add_bootloader_randomness(const void *buf, size_t len);
* void add_interrupt_randomness(int irq);
- * void add_input_randomness(unsigned int type, unsigned int code,
- * unsigned int value);
+ * void add_input_randomness(unsigned int type, unsigned int code, unsigned int value);
* void add_disk_randomness(struct gendisk *disk);
*
* add_device_randomness() adds data to the input pool that
@@ -902,7 +900,7 @@ int __init random_init(const char *comma
{
ktime_t now = ktime_get_real();
unsigned int i, arch_bytes;
- unsigned long rv;
+ unsigned long entropy;
#if defined(LATENT_ENTROPY_PLUGIN)
static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy;
@@ -910,13 +908,13 @@ int __init random_init(const char *comma
#endif
for (i = 0, arch_bytes = BLAKE2S_BLOCK_SIZE;
- i < BLAKE2S_BLOCK_SIZE; i += sizeof(rv)) {
- if (!arch_get_random_seed_long_early(&rv) &&
- !arch_get_random_long_early(&rv)) {
- rv = random_get_entropy();
- arch_bytes -= sizeof(rv);
+ i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) {
+ if (!arch_get_random_seed_long_early(&entropy) &&
+ !arch_get_random_long_early(&entropy)) {
+ entropy = random_get_entropy();
+ arch_bytes -= sizeof(entropy);
}
- _mix_pool_bytes(&rv, sizeof(rv));
+ _mix_pool_bytes(&entropy, sizeof(entropy));
}
_mix_pool_bytes(&now, sizeof(now));
_mix_pool_bytes(utsname(), sizeof(*(utsname())));
@@ -939,14 +937,14 @@ int __init random_init(const char *comma
* the entropy pool having similar initial state across largely
* identical devices.
*/
-void add_device_randomness(const void *buf, size_t size)
+void add_device_randomness(const void *buf, size_t len)
{
unsigned long entropy = random_get_entropy();
unsigned long flags;
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(&entropy, sizeof(entropy));
- _mix_pool_bytes(buf, size);
+ _mix_pool_bytes(buf, len);
spin_unlock_irqrestore(&input_pool.lock, flags);
}
EXPORT_SYMBOL(add_device_randomness);
@@ -956,10 +954,9 @@ EXPORT_SYMBOL(add_device_randomness);
* Those devices may produce endless random bits and will be throttled
* when our pool is full.
*/
-void add_hwgenerator_randomness(const void *buffer, size_t count,
- size_t entropy)
+void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy)
{
- mix_pool_bytes(buffer, count);
+ mix_pool_bytes(buf, len);
credit_init_bits(entropy);
/*
@@ -975,11 +972,11 @@ EXPORT_SYMBOL_GPL(add_hwgenerator_random
* Handle random seed passed by bootloader, and credit it if
* CONFIG_RANDOM_TRUST_BOOTLOADER is set.
*/
-void add_bootloader_randomness(const void *buf, size_t size)
+void add_bootloader_randomness(const void *buf, size_t len)
{
- mix_pool_bytes(buf, size);
+ mix_pool_bytes(buf, len);
if (trust_bootloader)
- credit_init_bits(size * 8);
+ credit_init_bits(len * 8);
}
EXPORT_SYMBOL_GPL(add_bootloader_randomness);
@@ -1179,8 +1176,7 @@ static void add_timer_randomness(struct
credit_init_bits(bits);
}
-void add_input_randomness(unsigned int type, unsigned int code,
- unsigned int value)
+void add_input_randomness(unsigned int type, unsigned int code, unsigned int value)
{
static unsigned char last_value;
static struct timer_rand_state input_timer_state = { INITIAL_JIFFIES };
@@ -1299,8 +1295,7 @@ static void try_to_generate_entropy(void
*
**********************************************************************/
-SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count, unsigned int,
- flags)
+SYSCALL_DEFINE3(getrandom, char __user *, ubuf, size_t, len, unsigned int, flags)
{
if (flags & ~(GRND_NONBLOCK | GRND_RANDOM | GRND_INSECURE))
return -EINVAL;
@@ -1312,8 +1307,8 @@ SYSCALL_DEFINE3(getrandom, char __user *
if ((flags & (GRND_INSECURE | GRND_RANDOM)) == (GRND_INSECURE | GRND_RANDOM))
return -EINVAL;
- if (count > INT_MAX)
- count = INT_MAX;
+ if (len > INT_MAX)
+ len = INT_MAX;
if (!crng_ready() && !(flags & GRND_INSECURE)) {
int ret;
@@ -1324,7 +1319,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
if (unlikely(ret))
return ret;
}
- return get_random_bytes_user(buf, count);
+ return get_random_bytes_user(ubuf, len);
}
static unsigned int random_poll(struct file *file, poll_table *wait)
@@ -1333,21 +1328,21 @@ static unsigned int random_poll(struct f
return crng_ready() ? POLLIN | POLLRDNORM : POLLOUT | POLLWRNORM;
}
-static int write_pool(const char __user *ubuf, size_t count)
+static int write_pool(const char __user *ubuf, size_t len)
{
- size_t len;
+ size_t block_len;
int ret = 0;
u8 block[BLAKE2S_BLOCK_SIZE];
- while (count) {
- len = min(count, sizeof(block));
- if (copy_from_user(block, ubuf, len)) {
+ while (len) {
+ block_len = min(len, sizeof(block));
+ if (copy_from_user(block, ubuf, block_len)) {
ret = -EFAULT;
goto out;
}
- count -= len;
- ubuf += len;
- mix_pool_bytes(block, len);
+ len -= block_len;
+ ubuf += block_len;
+ mix_pool_bytes(block, block_len);
cond_resched();
}
@@ -1356,20 +1351,20 @@ out:
return ret;
}
-static ssize_t random_write(struct file *file, const char __user *buffer,
- size_t count, loff_t *ppos)
+static ssize_t random_write(struct file *file, const char __user *ubuf,
+ size_t len, loff_t *ppos)
{
int ret;
- ret = write_pool(buffer, count);
+ ret = write_pool(ubuf, len);
if (ret)
return ret;
- return (ssize_t)count;
+ return (ssize_t)len;
}
-static ssize_t urandom_read(struct file *file, char __user *buf, size_t nbytes,
- loff_t *ppos)
+static ssize_t urandom_read(struct file *file, char __user *ubuf,
+ size_t len, loff_t *ppos)
{
static int maxwarn = 10;
@@ -1379,22 +1374,22 @@ static ssize_t urandom_read(struct file
else if (ratelimit_disable || __ratelimit(&urandom_warning)) {
--maxwarn;
pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
- current->comm, nbytes);
+ current->comm, len);
}
}
- return get_random_bytes_user(buf, nbytes);
+ return get_random_bytes_user(ubuf, len);
}
-static ssize_t random_read(struct file *file, char __user *buf, size_t nbytes,
- loff_t *ppos)
+static ssize_t random_read(struct file *file, char __user *ubuf,
+ size_t len, loff_t *ppos)
{
int ret;
ret = wait_for_random_bytes();
if (ret != 0)
return ret;
- return get_random_bytes_user(buf, nbytes);
+ return get_random_bytes_user(ubuf, len);
}
static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
@@ -1517,8 +1512,8 @@ static u8 sysctl_bootid[UUID_SIZE];
* UUID. The difference is in whether table->data is NULL; if it is,
* then a new UUID is generated and returned to the user.
*/
-static int proc_do_uuid(struct ctl_table *table, int write,
- void __user *buffer, size_t *lenp, loff_t *ppos)
+static int proc_do_uuid(struct ctl_table *table, int write, void __user *buf,
+ size_t *lenp, loff_t *ppos)
{
u8 tmp_uuid[UUID_SIZE], *uuid;
char uuid_string[UUID_STRING_LEN + 1];
@@ -1544,14 +1539,14 @@ static int proc_do_uuid(struct ctl_table
}
snprintf(uuid_string, sizeof(uuid_string), "%pU", uuid);
- return proc_dostring(&fake_table, 0, buffer, lenp, ppos);
+ return proc_dostring(&fake_table, 0, buf, lenp, ppos);
}
/* The same as proc_dointvec, but writes don't change anything. */
-static int proc_do_rointvec(struct ctl_table *table, int write, void __user *buffer,
+static int proc_do_rointvec(struct ctl_table *table, int write, void __user *buf,
size_t *lenp, loff_t *ppos)
{
- return write ? 0 : proc_dointvec(table, 0, buffer, lenp, ppos);
+ return write ? 0 : proc_dointvec(table, 0, buf, lenp, ppos);
}
extern struct ctl_table random_table[];
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -12,12 +12,12 @@
struct notifier_block;
-void add_device_randomness(const void *, size_t);
-void add_bootloader_randomness(const void *, size_t);
+void add_device_randomness(const void *buf, size_t len);
+void add_bootloader_randomness(const void *buf, size_t len);
void add_input_randomness(unsigned int type, unsigned int code,
unsigned int value) __latent_entropy;
void add_interrupt_randomness(int irq) __latent_entropy;
-void add_hwgenerator_randomness(const void *buffer, size_t count, size_t entropy);
+void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy);
#if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__)
static inline void add_latent_entropy(void)
@@ -28,8 +28,8 @@ static inline void add_latent_entropy(vo
static inline void add_latent_entropy(void) { }
#endif
-void get_random_bytes(void *buf, size_t nbytes);
-size_t __must_check get_random_bytes_arch(void *buf, size_t nbytes);
+void get_random_bytes(void *buf, size_t len);
+size_t __must_check get_random_bytes_arch(void *buf, size_t len);
u32 get_random_u32(void);
u64 get_random_u64(void);
static inline unsigned int get_random_int(void)
@@ -62,18 +62,18 @@ static inline int get_random_bytes_wait(
return ret;
}
-#define declare_get_random_var_wait(var) \
- static inline int get_random_ ## var ## _wait(var *out) { \
+#define declare_get_random_var_wait(name, ret_type) \
+ static inline int get_random_ ## name ## _wait(ret_type *out) { \
int ret = wait_for_random_bytes(); \
if (unlikely(ret)) \
return ret; \
- *out = get_random_ ## var(); \
+ *out = get_random_ ## name(); \
return 0; \
}
-declare_get_random_var_wait(u32)
-declare_get_random_var_wait(u64)
-declare_get_random_var_wait(int)
-declare_get_random_var_wait(long)
+declare_get_random_var_wait(u32, u32)
+declare_get_random_var_wait(u64, u32)
+declare_get_random_var_wait(int, unsigned int)
+declare_get_random_var_wait(long, unsigned long)
#undef declare_get_random_var
/*
From: "Jason A. Donenfeld" <[email protected]>
commit abded93ec1e9692920fe309f07f40bd1035f2940 upstream.
random_get_entropy() returns a cycles_t, not an unsigned long, which is
sometimes 64 bits on various 32-bit platforms, including x86.
Conversely, jiffies is always unsigned long. This commit fixes things to
use cycles_t for fields that use random_get_entropy(), named "cycles",
and unsigned long for fields that use jiffies, named "now". It's also
good to mix in a cycles_t and a jiffies in the same way for both
add_device_randomness and add_timer_randomness, rather than using xor in
one case. Finally, we unify the order of these volatile reads, always
reading the more precise cycles counter, and then jiffies, so that the
cycle counter is as close to the event as possible.
Cc: Theodore Ts'o <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 56 ++++++++++++++++++++++++--------------------------
1 file changed, 27 insertions(+), 29 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1016,12 +1016,6 @@ int __init rand_initialize(void)
return 0;
}
-/* There is one of these per entropy source */
-struct timer_rand_state {
- cycles_t last_time;
- long last_delta, last_delta2;
-};
-
/*
* Add device- or boot-specific data to the input pool to help
* initialize it.
@@ -1032,19 +1026,26 @@ struct timer_rand_state {
*/
void add_device_randomness(const void *buf, size_t size)
{
- unsigned long time = random_get_entropy() ^ jiffies;
- unsigned long flags;
+ cycles_t cycles = random_get_entropy();
+ unsigned long flags, now = jiffies;
if (crng_init == 0 && size)
crng_pre_init_inject(buf, size, false, false);
spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(&cycles, sizeof(cycles));
+ _mix_pool_bytes(&now, sizeof(now));
_mix_pool_bytes(buf, size);
- _mix_pool_bytes(&time, sizeof(time));
spin_unlock_irqrestore(&input_pool.lock, flags);
}
EXPORT_SYMBOL(add_device_randomness);
+/* There is one of these per entropy source */
+struct timer_rand_state {
+ unsigned long last_time;
+ long last_delta, last_delta2;
+};
+
/*
* This function adds entropy to the entropy "pool" by using timing
* delays. It uses the timer_rand_state structure to make an estimate
@@ -1053,29 +1054,26 @@ EXPORT_SYMBOL(add_device_randomness);
* The number "num" is also added to the pool - it should somehow describe
* the type of event which just happened. This is currently 0-255 for
* keyboard scan codes, and 256 upwards for interrupts.
- *
*/
static void add_timer_randomness(struct timer_rand_state *state, unsigned int num)
{
- struct {
- long jiffies;
- unsigned int cycles;
- unsigned int num;
- } sample;
+ cycles_t cycles = random_get_entropy();
+ unsigned long flags, now = jiffies;
long delta, delta2, delta3;
- sample.jiffies = jiffies;
- sample.cycles = random_get_entropy();
- sample.num = num;
- mix_pool_bytes(&sample, sizeof(sample));
+ spin_lock_irqsave(&input_pool.lock, flags);
+ _mix_pool_bytes(&cycles, sizeof(cycles));
+ _mix_pool_bytes(&now, sizeof(now));
+ _mix_pool_bytes(&num, sizeof(num));
+ spin_unlock_irqrestore(&input_pool.lock, flags);
/*
* Calculate number of bits of randomness we probably added.
* We take into account the first, second and third-order deltas
* in order to make our estimate.
*/
- delta = sample.jiffies - state->last_time;
- state->last_time = sample.jiffies;
+ delta = now - READ_ONCE(state->last_time);
+ WRITE_ONCE(state->last_time, now);
delta2 = delta - state->last_delta;
state->last_delta = delta;
@@ -1301,10 +1299,10 @@ static void mix_interrupt_randomness(str
void add_interrupt_randomness(int irq)
{
enum { MIX_INFLIGHT = 1U << 31 };
+ cycles_t cycles = random_get_entropy();
+ unsigned long now = jiffies;
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
struct pt_regs *regs = get_irq_regs();
- unsigned long now = jiffies;
- cycles_t cycles = random_get_entropy();
unsigned int new_count;
if (cycles == 0)
@@ -1379,28 +1377,28 @@ static void entropy_timer(unsigned long
static void try_to_generate_entropy(void)
{
struct {
- unsigned long now;
+ cycles_t cycles;
struct timer_list timer;
} stack;
- stack.now = random_get_entropy();
+ stack.cycles = random_get_entropy();
/* Slow counter - or none. Don't even bother */
- if (stack.now == random_get_entropy())
+ if (stack.cycles == random_get_entropy())
return;
__setup_timer_on_stack(&stack.timer, entropy_timer, 0, 0);
while (!crng_ready()) {
if (!timer_pending(&stack.timer))
mod_timer(&stack.timer, jiffies + 1);
- mix_pool_bytes(&stack.now, sizeof(stack.now));
+ mix_pool_bytes(&stack.cycles, sizeof(stack.cycles));
schedule();
- stack.now = random_get_entropy();
+ stack.cycles = random_get_entropy();
}
del_timer_sync(&stack.timer);
destroy_timer_on_stack(&stack.timer);
- mix_pool_bytes(&stack.now, sizeof(stack.now));
+ mix_pool_bytes(&stack.cycles, sizeof(stack.cycles));
}
From: Corentin LABBE <[email protected]>
commit 2a971e3b248775f808950bdc0ac75f12a2853eff upstream.
This patch fix the checkpatch warning "Comparison to NULL could be written "!ptr"
Signed-off-by: Corentin Labbe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/hw_random/core.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -439,8 +439,7 @@ int hwrng_register(struct hwrng *rng)
int err = -EINVAL;
struct hwrng *old_rng, *tmp;
- if (rng->name == NULL ||
- (rng->data_read == NULL && rng->read == NULL))
+ if (!rng->name || (!rng->data_read && !rng->read))
goto out;
mutex_lock(&rng_mutex);
From: "Jason A. Donenfeld" <[email protected]>
commit 3191dd5a1179ef0fad5a050a1702ae98b6251e8f upstream.
For the irq randomness fast pool, rather than having to use expensive
atomics, which were visibly the most expensive thing in the entire irq
handler, simply take care of the extreme edge case of resetting count to
zero in the cpuhp online handler, just after workqueues have been
reenabled. This simplifies the code a bit and lets us use vanilla
variables rather than atomics, and performance should be improved.
As well, very early on when the CPU comes up, while interrupts are still
disabled, we clear out the per-cpu crng and its batches, so that it
always starts with fresh randomness.
Cc: Thomas Gleixner <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Cc: Sultan Alsawaf <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Acked-by: Sebastian Andrzej Siewior <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 62 ++++++++++++++++++++++++++++++++++-----------
include/linux/cpuhotplug.h | 2 +
include/linux/random.h | 5 +++
kernel/cpu.c | 11 +++++++
4 files changed, 65 insertions(+), 15 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -694,6 +694,25 @@ u32 get_random_u32(void)
}
EXPORT_SYMBOL(get_random_u32);
+#ifdef CONFIG_SMP
+/*
+ * This function is called when the CPU is coming up, with entry
+ * CPUHP_RANDOM_PREPARE, which comes before CPUHP_WORKQUEUE_PREP.
+ */
+int random_prepare_cpu(unsigned int cpu)
+{
+ /*
+ * When the cpu comes back online, immediately invalidate both
+ * the per-cpu crng and all batches, so that we serve fresh
+ * randomness.
+ */
+ per_cpu_ptr(&crngs, cpu)->generation = ULONG_MAX;
+ per_cpu_ptr(&batched_entropy_u32, cpu)->position = UINT_MAX;
+ per_cpu_ptr(&batched_entropy_u64, cpu)->position = UINT_MAX;
+ return 0;
+}
+#endif
+
/**
* randomize_page - Generate a random, page aligned address
* @start: The smallest acceptable address the caller will take.
@@ -1179,7 +1198,7 @@ struct fast_pool {
};
struct work_struct mix;
unsigned long last;
- atomic_t count;
+ unsigned int count;
u16 reg_idx;
};
@@ -1215,6 +1234,29 @@ static void fast_mix(u32 pool[4])
static DEFINE_PER_CPU(struct fast_pool, irq_randomness);
+#ifdef CONFIG_SMP
+/*
+ * This function is called when the CPU has just come online, with
+ * entry CPUHP_AP_RANDOM_ONLINE, just after CPUHP_AP_WORKQUEUE_ONLINE.
+ */
+int random_online_cpu(unsigned int cpu)
+{
+ /*
+ * During CPU shutdown and before CPU onlining, add_interrupt_
+ * randomness() may schedule mix_interrupt_randomness(), and
+ * set the MIX_INFLIGHT flag. However, because the worker can
+ * be scheduled on a different CPU during this period, that
+ * flag will never be cleared. For that reason, we zero out
+ * the flag here, which runs just after workqueues are onlined
+ * for the CPU again. This also has the effect of setting the
+ * irq randomness count to zero so that new accumulated irqs
+ * are fresh.
+ */
+ per_cpu_ptr(&irq_randomness, cpu)->count = 0;
+ return 0;
+}
+#endif
+
static u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
{
u32 *ptr = (u32 *)regs;
@@ -1239,15 +1281,6 @@ static void mix_interrupt_randomness(str
local_irq_disable();
if (fast_pool != this_cpu_ptr(&irq_randomness)) {
local_irq_enable();
- /*
- * If we are unlucky enough to have been moved to another CPU,
- * during CPU hotplug while the CPU was shutdown then we set
- * our count to zero atomically so that when the CPU comes
- * back online, it can enqueue work again. The _release here
- * pairs with the atomic_inc_return_acquire in
- * add_interrupt_randomness().
- */
- atomic_set_release(&fast_pool->count, 0);
return;
}
@@ -1256,7 +1289,7 @@ static void mix_interrupt_randomness(str
* consistent view, before we reenable irqs again.
*/
memcpy(pool, fast_pool->pool32, sizeof(pool));
- atomic_set(&fast_pool->count, 0);
+ fast_pool->count = 0;
fast_pool->last = jiffies;
local_irq_enable();
@@ -1292,14 +1325,13 @@ void add_interrupt_randomness(int irq)
}
fast_mix(fast_pool->pool32);
- /* The _acquire here pairs with the atomic_set_release in mix_interrupt_randomness(). */
- new_count = (unsigned int)atomic_inc_return_acquire(&fast_pool->count);
+ new_count = ++fast_pool->count;
if (unlikely(crng_init == 0)) {
if (new_count >= 64 &&
crng_pre_init_inject(fast_pool->pool32, sizeof(fast_pool->pool32),
true, true) > 0) {
- atomic_set(&fast_pool->count, 0);
+ fast_pool->count = 0;
fast_pool->last = now;
if (spin_trylock(&input_pool.lock)) {
_mix_pool_bytes(&fast_pool->pool32, sizeof(fast_pool->pool32));
@@ -1317,7 +1349,7 @@ void add_interrupt_randomness(int irq)
if (unlikely(!fast_pool->mix.func))
INIT_WORK(&fast_pool->mix, mix_interrupt_randomness);
- atomic_or(MIX_INFLIGHT, &fast_pool->count);
+ fast_pool->count |= MIX_INFLIGHT;
queue_work_on(raw_smp_processor_id(), system_highpri_wq, &fast_pool->mix);
}
EXPORT_SYMBOL_GPL(add_interrupt_randomness);
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -29,6 +29,7 @@ enum cpuhp_state {
CPUHP_ACPI_CPUDRV_DEAD,
CPUHP_S390_PFAULT_DEAD,
CPUHP_BLK_MQ_DEAD,
+ CPUHP_RANDOM_PREPARE,
CPUHP_WORKQUEUE_PREP,
CPUHP_POWER_NUMA_PREPARE,
CPUHP_HRTIMERS_PREPARE,
@@ -119,6 +120,7 @@ enum cpuhp_state {
CPUHP_AP_PERF_ARM_CCN_ONLINE,
CPUHP_AP_PERF_ARM_L2X0_ONLINE,
CPUHP_AP_WORKQUEUE_ONLINE,
+ CPUHP_AP_RANDOM_ONLINE,
CPUHP_AP_RCUTREE_ONLINE,
CPUHP_AP_NOTIFY_ONLINE,
CPUHP_AP_ONLINE_DYN,
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -135,4 +135,9 @@ static inline bool __init arch_get_rando
}
#endif
+#ifdef CONFIG_SMP
+extern int random_prepare_cpu(unsigned int cpu);
+extern int random_online_cpu(unsigned int cpu);
+#endif
+
#endif /* _LINUX_RANDOM_H */
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -26,6 +26,7 @@
#include <linux/smpboot.h>
#include <linux/relay.h>
#include <linux/slab.h>
+#include <linux/random.h>
#include <trace/events/power.h>
#define CREATE_TRACE_POINTS
@@ -1404,6 +1405,11 @@ static struct cpuhp_step cpuhp_bp_states
.startup.single = perf_event_init_cpu,
.teardown.single = perf_event_exit_cpu,
},
+ [CPUHP_RANDOM_PREPARE] = {
+ .name = "random:prepare",
+ .startup.single = random_prepare_cpu,
+ .teardown.single = NULL,
+ },
[CPUHP_WORKQUEUE_PREP] = {
.name = "workqueue:prepare",
.startup.single = workqueue_prepare_cpu,
@@ -1529,6 +1535,11 @@ static struct cpuhp_step cpuhp_ap_states
.startup.single = workqueue_online_cpu,
.teardown.single = workqueue_offline_cpu,
},
+ [CPUHP_AP_RANDOM_ONLINE] = {
+ .name = "random:online",
+ .startup.single = random_online_cpu,
+ .teardown.single = NULL,
+ },
[CPUHP_AP_RCUTREE_ONLINE] = {
.name = "RCU/tree:online",
.startup.single = rcutree_online_cpu,
From: "Jason A. Donenfeld" <[email protected]>
commit f5bda35fba615ace70a656d4700423fa6c9bebee upstream.
Since crng_ready() is only false briefly during initialization and then
forever after becomes true, we don't need to evaluate it after, making
it a prime candidate for a static branch.
One complication, however, is that it changes state in a particular call
to credit_init_bits(), which might be made from atomic context, which
means we must kick off a workqueue to change the static key. Further
complicating things, credit_init_bits() may be called sufficiently early
on in system initialization such that system_wq is NULL.
Fortunately, there exists the nice function execute_in_process_context(),
which will immediately execute the function if !in_interrupt(), and
otherwise defer it to a workqueue. During early init, before workqueues
are available, in_interrupt() is always false, because interrupts
haven't even been enabled yet, which means the function in that case
executes immediately. Later on, after workqueues are available,
in_interrupt() might be true, but in that case, the work is queued in
system_wq and all goes well.
Cc: Theodore Ts'o <[email protected]>
Cc: Sultan Alsawaf <[email protected]>
Reviewed-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -78,8 +78,9 @@ static enum {
CRNG_EMPTY = 0, /* Little to no entropy collected */
CRNG_EARLY = 1, /* At least POOL_EARLY_BITS collected */
CRNG_READY = 2 /* Fully initialized with POOL_READY_BITS collected */
-} crng_init = CRNG_EMPTY;
-#define crng_ready() (likely(crng_init >= CRNG_READY))
+} crng_init __read_mostly = CRNG_EMPTY;
+static DEFINE_STATIC_KEY_FALSE(crng_is_ready);
+#define crng_ready() (static_branch_likely(&crng_is_ready) || crng_init >= CRNG_READY)
/* Various types of waiters for crng_init->CRNG_READY transition. */
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
static struct fasync_struct *fasync;
@@ -109,6 +110,11 @@ bool rng_is_initialized(void)
}
EXPORT_SYMBOL(rng_is_initialized);
+static void crng_set_ready(struct work_struct *work)
+{
+ static_branch_enable(&crng_is_ready);
+}
+
/* Used by wait_for_random_bytes(), and considered an entropy collector, below. */
static void try_to_generate_entropy(void);
@@ -268,7 +274,7 @@ static void crng_reseed(void)
++next_gen;
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
- if (!crng_ready())
+ if (!static_branch_likely(&crng_is_ready))
crng_init = CRNG_READY;
spin_unlock_irqrestore(&base_crng.lock, flags);
memzero_explicit(key, sizeof(key));
@@ -783,6 +789,7 @@ static void extract_entropy(void *buf, s
static void credit_init_bits(size_t nbits)
{
+ static struct execute_work set_ready;
unsigned int new, orig, add;
unsigned long flags;
@@ -798,6 +805,7 @@ static void credit_init_bits(size_t nbit
if (orig < POOL_READY_BITS && new >= POOL_READY_BITS) {
crng_reseed(); /* Sets crng_init to CRNG_READY under base_crng.lock. */
+ execute_in_process_context(crng_set_ready, &set_ready);
process_random_ready_list();
wake_up_interruptible(&crng_init_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
@@ -1307,7 +1315,7 @@ SYSCALL_DEFINE3(getrandom, char __user *
if (count > INT_MAX)
count = INT_MAX;
- if (!(flags & GRND_INSECURE) && !crng_ready()) {
+ if (!crng_ready() && !(flags & GRND_INSECURE)) {
int ret;
if (flags & GRND_NONBLOCK)
From: "Jason A. Donenfeld" <[email protected]>
commit 560181c27b582557d633ecb608110075433383af upstream.
Much of random.c is devoted to initializing the rng and accounting for
when a sufficient amount of entropy has been added. In a perfect world,
this would all happen during init, and so we could mark these functions
as __init. But in reality, this isn't the case: sometimes the rng only
finishes initializing some seconds after system init is finished.
For this reason, at the moment, a whole host of functions that are only
used relatively close to system init and then never again are intermixed
with functions that are used in hot code all the time. This creates more
cache misses than necessary.
In order to pack the hot code closer together, this commit moves the
initialization functions that can't be marked as __init into
.text.unlikely by way of the __cold attribute.
Of particular note is moving credit_init_bits() into a macro wrapper
that inlines the crng_ready() static branch check. This avoids a
function call to a nop+ret, and most notably prevents extra entropy
arithmetic from being computed in mix_interrupt_randomness().
Reviewed-by: Dominik Brodowski <[email protected]>
[ Jason: for stable, made sure the printk_deferred was a pr_notice,
because those caused problems on ≤ 4.19 according to commit logs. ]
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 40 ++++++++++++++++++----------------------
1 file changed, 18 insertions(+), 22 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -110,7 +110,7 @@ bool rng_is_initialized(void)
}
EXPORT_SYMBOL(rng_is_initialized);
-static void crng_set_ready(struct work_struct *work)
+static void __cold crng_set_ready(struct work_struct *work)
{
static_branch_enable(&crng_is_ready);
}
@@ -149,7 +149,7 @@ EXPORT_SYMBOL(wait_for_random_bytes);
* returns: 0 if callback is successfully added
* -EALREADY if pool is already initialised (callback not called)
*/
-int register_random_ready_notifier(struct notifier_block *nb)
+int __cold register_random_ready_notifier(struct notifier_block *nb)
{
unsigned long flags;
int ret = -EALREADY;
@@ -168,7 +168,7 @@ EXPORT_SYMBOL(register_random_ready_noti
/*
* Delete a previously registered readiness callback function.
*/
-int unregister_random_ready_notifier(struct notifier_block *nb)
+int __cold unregister_random_ready_notifier(struct notifier_block *nb)
{
unsigned long flags;
int ret;
@@ -180,7 +180,7 @@ int unregister_random_ready_notifier(str
}
EXPORT_SYMBOL(unregister_random_ready_notifier);
-static void process_random_ready_list(void)
+static void __cold process_random_ready_list(void)
{
unsigned long flags;
@@ -190,15 +190,9 @@ static void process_random_ready_list(vo
}
#define warn_unseeded_randomness() \
- _warn_unseeded_randomness(__func__, (void *)_RET_IP_)
-
-static void _warn_unseeded_randomness(const char *func_name, void *caller)
-{
- if (!IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM) || crng_ready())
- return;
- printk_deferred(KERN_NOTICE "random: %s called from %pS with crng_init=%d\n",
- func_name, caller, crng_init);
-}
+ if (IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM) && !crng_ready()) \
+ pr_notice("%s called from %pS with crng_init=%d\n", \
+ __func__, (void *)_RET_IP_, crng_init)
/*********************************************************************
@@ -612,7 +606,7 @@ EXPORT_SYMBOL(get_random_u32);
* This function is called when the CPU is coming up, with entry
* CPUHP_RANDOM_PREPARE, which comes before CPUHP_WORKQUEUE_PREP.
*/
-int random_prepare_cpu(unsigned int cpu)
+int __cold random_prepare_cpu(unsigned int cpu)
{
/*
* When the cpu comes back online, immediately invalidate both
@@ -787,13 +781,15 @@ static void extract_entropy(void *buf, s
memzero_explicit(&block, sizeof(block));
}
-static void credit_init_bits(size_t bits)
+#define credit_init_bits(bits) if (!crng_ready()) _credit_init_bits(bits)
+
+static void __cold _credit_init_bits(size_t bits)
{
static struct execute_work set_ready;
unsigned int new, orig, add;
unsigned long flags;
- if (crng_ready() || !bits)
+ if (!bits)
return;
add = min_t(size_t, bits, POOL_BITS);
@@ -972,7 +968,7 @@ EXPORT_SYMBOL_GPL(add_hwgenerator_random
* Handle random seed passed by bootloader, and credit it if
* CONFIG_RANDOM_TRUST_BOOTLOADER is set.
*/
-void add_bootloader_randomness(const void *buf, size_t len)
+void __cold add_bootloader_randomness(const void *buf, size_t len)
{
mix_pool_bytes(buf, len);
if (trust_bootloader)
@@ -1018,7 +1014,7 @@ static void fast_mix(unsigned long s[4],
* This function is called when the CPU has just come online, with
* entry CPUHP_AP_RANDOM_ONLINE, just after CPUHP_AP_WORKQUEUE_ONLINE.
*/
-int random_online_cpu(unsigned int cpu)
+int __cold random_online_cpu(unsigned int cpu)
{
/*
* During CPU shutdown and before CPU onlining, add_interrupt_
@@ -1173,7 +1169,7 @@ static void add_timer_randomness(struct
if (in_irq())
this_cpu_ptr(&irq_randomness)->count += max(1u, bits * 64) - 1;
else
- credit_init_bits(bits);
+ _credit_init_bits(bits);
}
void add_input_randomness(unsigned int type, unsigned int code, unsigned int value)
@@ -1201,7 +1197,7 @@ void add_disk_randomness(struct gendisk
}
EXPORT_SYMBOL_GPL(add_disk_randomness);
-void rand_initialize_disk(struct gendisk *disk)
+void __cold rand_initialize_disk(struct gendisk *disk)
{
struct timer_rand_state *state;
@@ -1230,7 +1226,7 @@ void rand_initialize_disk(struct gendisk
*
* So the re-arming always happens in the entropy loop itself.
*/
-static void entropy_timer(unsigned long data)
+static void __cold entropy_timer(unsigned long data)
{
credit_init_bits(1);
}
@@ -1239,7 +1235,7 @@ static void entropy_timer(unsigned long
* If we have an actual cycle counter, see if we can
* generate enough entropy with timing noise
*/
-static void try_to_generate_entropy(void)
+static void __cold try_to_generate_entropy(void)
{
struct {
unsigned long entropy;
From: "Jason A. Donenfeld" <[email protected]>
commit 791332b3cbb080510954a4c152ce02af8832eac9 upstream.
Now that fast_mix() has more than one caller, gcc no longer inlines it.
That's fine. But it also doesn't handle the compound literal argument we
pass it very efficiently, nor does it handle the loop as well as it
could. So just expand the code to spell out this function so that it
generates the same code as it did before. Performance-wise, this now
behaves as it did before the last commit. The difference in actual code
size on x86 is 45 bytes, which is less than a cache line.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 44 +++++++++++++++++++++++---------------------
1 file changed, 23 insertions(+), 21 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1027,25 +1027,30 @@ static DEFINE_PER_CPU(struct fast_pool,
* and therefore this has no security on its own. s represents the
* four-word SipHash state, while v represents a two-word input.
*/
-static void fast_mix(unsigned long s[4], const unsigned long v[2])
+static void fast_mix(unsigned long s[4], unsigned long v1, unsigned long v2)
{
- size_t i;
-
- for (i = 0; i < 2; ++i) {
- s[3] ^= v[i];
#ifdef CONFIG_64BIT
- s[0] += s[1]; s[1] = rol64(s[1], 13); s[1] ^= s[0]; s[0] = rol64(s[0], 32);
- s[2] += s[3]; s[3] = rol64(s[3], 16); s[3] ^= s[2];
- s[0] += s[3]; s[3] = rol64(s[3], 21); s[3] ^= s[0];
- s[2] += s[1]; s[1] = rol64(s[1], 17); s[1] ^= s[2]; s[2] = rol64(s[2], 32);
+#define PERM() do { \
+ s[0] += s[1]; s[1] = rol64(s[1], 13); s[1] ^= s[0]; s[0] = rol64(s[0], 32); \
+ s[2] += s[3]; s[3] = rol64(s[3], 16); s[3] ^= s[2]; \
+ s[0] += s[3]; s[3] = rol64(s[3], 21); s[3] ^= s[0]; \
+ s[2] += s[1]; s[1] = rol64(s[1], 17); s[1] ^= s[2]; s[2] = rol64(s[2], 32); \
+} while (0)
#else
- s[0] += s[1]; s[1] = rol32(s[1], 5); s[1] ^= s[0]; s[0] = rol32(s[0], 16);
- s[2] += s[3]; s[3] = rol32(s[3], 8); s[3] ^= s[2];
- s[0] += s[3]; s[3] = rol32(s[3], 7); s[3] ^= s[0];
- s[2] += s[1]; s[1] = rol32(s[1], 13); s[1] ^= s[2]; s[2] = rol32(s[2], 16);
+#define PERM() do { \
+ s[0] += s[1]; s[1] = rol32(s[1], 5); s[1] ^= s[0]; s[0] = rol32(s[0], 16); \
+ s[2] += s[3]; s[3] = rol32(s[3], 8); s[3] ^= s[2]; \
+ s[0] += s[3]; s[3] = rol32(s[3], 7); s[3] ^= s[0]; \
+ s[2] += s[1]; s[1] = rol32(s[1], 13); s[1] ^= s[2]; s[2] = rol32(s[2], 16); \
+} while (0)
#endif
- s[0] ^= v[i];
- }
+
+ s[3] ^= v1;
+ PERM();
+ s[0] ^= v1;
+ s[3] ^= v2;
+ PERM();
+ s[0] ^= v2;
}
#ifdef CONFIG_SMP
@@ -1115,10 +1120,8 @@ void add_interrupt_randomness(int irq)
struct pt_regs *regs = get_irq_regs();
unsigned int new_count;
- fast_mix(fast_pool->pool, (unsigned long[2]){
- entropy,
- (regs ? instruction_pointer(regs) : _RET_IP_) ^ swab(irq)
- });
+ fast_mix(fast_pool->pool, entropy,
+ (regs ? instruction_pointer(regs) : _RET_IP_) ^ swab(irq));
new_count = ++fast_pool->count;
if (new_count & MIX_INFLIGHT)
@@ -1158,8 +1161,7 @@ static void add_timer_randomness(struct
* sometime after, so mix into the fast pool.
*/
if (in_irq()) {
- fast_mix(this_cpu_ptr(&irq_randomness)->pool,
- (unsigned long[2]){ entropy, num });
+ fast_mix(this_cpu_ptr(&irq_randomness)->pool, entropy, num);
} else {
spin_lock_irqsave(&input_pool.lock, flags);
_mix_pool_bytes(&entropy, sizeof(entropy));
From: "Jason A. Donenfeld" <[email protected]>
This reverts upstream commit f5bda35fba615ace70a656d4700423fa6c9bebee
from stable. It's not essential and will take some time during 5.19 to
work out properly.
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -80,8 +80,7 @@ static enum {
CRNG_EARLY = 1, /* At least POOL_EARLY_BITS collected */
CRNG_READY = 2 /* Fully initialized with POOL_READY_BITS collected */
} crng_init __read_mostly = CRNG_EMPTY;
-static DEFINE_STATIC_KEY_FALSE(crng_is_ready);
-#define crng_ready() (static_branch_likely(&crng_is_ready) || crng_init >= CRNG_READY)
+#define crng_ready() (likely(crng_init >= CRNG_READY))
/* Various types of waiters for crng_init->CRNG_READY transition. */
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
static struct fasync_struct *fasync;
@@ -111,11 +110,6 @@ bool rng_is_initialized(void)
}
EXPORT_SYMBOL(rng_is_initialized);
-static void __cold crng_set_ready(struct work_struct *work)
-{
- static_branch_enable(&crng_is_ready);
-}
-
/* Used by wait_for_random_bytes(), and considered an entropy collector, below. */
static void try_to_generate_entropy(void);
@@ -269,7 +263,7 @@ static void crng_reseed(void)
++next_gen;
WRITE_ONCE(base_crng.generation, next_gen);
WRITE_ONCE(base_crng.birth, jiffies);
- if (!static_branch_likely(&crng_is_ready))
+ if (!crng_ready())
crng_init = CRNG_READY;
spin_unlock_irqrestore(&base_crng.lock, flags);
memzero_explicit(key, sizeof(key));
@@ -710,7 +704,6 @@ static void extract_entropy(void *buf, s
static void __cold _credit_init_bits(size_t bits)
{
- static struct execute_work set_ready;
unsigned int new, orig, add;
unsigned long flags;
@@ -726,7 +719,6 @@ static void __cold _credit_init_bits(siz
if (orig < POOL_READY_BITS && new >= POOL_READY_BITS) {
crng_reseed(); /* Sets crng_init to CRNG_READY under base_crng.lock. */
- execute_in_process_context(crng_set_ready, &set_ready);
process_random_ready_list();
wake_up_interruptible(&crng_init_wait);
kill_fasync(&fasync, SIGIO, POLL_IN);
From: Dominik Brodowski <[email protected]>
commit c321e907aa4803d562d6e70ebed9444ad082f953 upstream.
The rngd kernel thread may sleep indefinitely if the entropy count is
kept above random_write_wakeup_bits by other entropy sources. To make
best use of multiple sources of randomness, mix entropy from hardware
RNGs into the pool at least once within CRNG_RESEED_INTERVAL.
Cc: Herbert Xu <[email protected]>
Cc: Jason A. Donenfeld <[email protected]>
Signed-off-by: Dominik Brodowski <[email protected]>
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/char/random.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2244,13 +2244,15 @@ void add_hwgenerator_randomness(const ch
return;
}
- /* Suspend writing if we're above the trickle threshold.
+ /* Throttle writing if we're above the trickle threshold.
* We'll be woken up again once below random_write_wakeup_thresh,
- * or when the calling thread is about to terminate.
+ * when the calling thread is about to terminate, or once
+ * CRNG_RESEED_INTERVAL has lapsed.
*/
- wait_event_interruptible(random_write_wait,
+ wait_event_interruptible_timeout(random_write_wait,
!system_wq || kthread_should_stop() ||
- POOL_ENTROPY_BITS() <= random_write_wakeup_bits);
+ POOL_ENTROPY_BITS() <= random_write_wakeup_bits,
+ CRNG_RESEED_INTERVAL);
mix_pool_bytes(buffer, count);
credit_entropy_bits(entropy);
}
On 6/23/22 09:39, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.9.320 release.
> There are 264 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Sat, 25 Jun 2022 16:43:11 +0000.
> Anything received after that time might be too late.
>
> The whole patch series can be found in one patch at:
> https://www.kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.9.320-rc1.gz
> or in the git tree and branch at:
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-4.9.y
> and the diffstat can be found below.
>
> thanks,
>
> greg k-h
On ARCH_BRCMSTB using 32-bit and 64-bit ARM kernels:
Tested-by: Florian Fainelli <[email protected]>
--
Florian
Hi!
> This is the start of the stable review cycle for the 4.9.320 release.
> There are 264 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
CIP testing did not find any problems here:
https://gitlab.com/cip-project/cip-testing/linux-stable-rc-ci/-/tree/linux-4.9.y
Tested-by: Pavel Machek (CIP) <[email protected]>
Best regards,
Pavel
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
On 6/23/22 10:39 AM, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.9.320 release.
> There are 264 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Sat, 25 Jun 2022 16:43:11 +0000.
> Anything received after that time might be too late.
>
> The whole patch series can be found in one patch at:
> https://www.kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.9.320-rc1.gz
> or in the git tree and branch at:
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-4.9.y
> and the diffstat can be found below.
>
> thanks,
>
> greg k-h
>
Compiled and booted on my test system. No dmesg regressions.
Tested-by: Shuah Khan <[email protected]>
thanks,
-- Shuah
On Thu, Jun 23, 2022 at 06:39:53PM +0200, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.9.320 release.
> There are 264 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Sat, 25 Jun 2022 16:43:11 +0000.
> Anything received after that time might be too late.
>
Build results:
total: 164 pass: 164 fail: 0
Qemu test results:
total: 397 pass: 397 fail: 0
Tested-by: Guenter Roeck <[email protected]>
Guenter
On Thu, 23 Jun 2022 at 22:17, Greg Kroah-Hartman
<[email protected]> wrote:
>
> This is the start of the stable review cycle for the 4.9.320 release.
> There are 264 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Sat, 25 Jun 2022 16:43:11 +0000.
> Anything received after that time might be too late.
>
> The whole patch series can be found in one patch at:
> https://www.kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.9.320-rc1.gz
> or in the git tree and branch at:
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-4.9.y
> and the diffstat can be found below.
>
> thanks,
>
> greg k-h
Results from Linaro’s test farm.
No regressions on arm64, arm, x86_64, and i386.
Tested-by: Linux Kernel Functional Testing <[email protected]>
## Build
* kernel: 4.9.320-rc1
* git: https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable-rc
* git branch: linux-4.9.y
* git commit: 00d9858d20e4c4b5988b85df09c79010e037e456
* git describe: v4.9.319-265-g00d9858d20e4
* test details:
https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-4.9.y/build/v4.9.319-265-g00d9858d20e4
## Test Regressions (compared to v4.9.319-251-g5de156af25f6)
No test regressions found.
## Metric Regressions (compared to v4.9.319-251-g5de156af25f6)
No metric regressions found.
## Test Fixes (compared to v4.9.319-251-g5de156af25f6)
No test fixes found.
## Metric Fixes (compared to v4.9.319-251-g5de156af25f6)
No metric fixes found.
## Test result summary
total: 104551, pass: 91428, fail: 204, skip: 11519, xfail: 1400
## Build Summary
* arc: 10 total, 10 passed, 0 failed
* arm: 261 total, 255 passed, 6 failed
* arm64: 50 total, 39 passed, 11 failed
* i386: 27 total, 23 passed, 4 failed
* mips: 22 total, 22 passed, 0 failed
* parisc: 12 total, 0 passed, 12 failed
* powerpc: 36 total, 16 passed, 20 failed
* s390: 12 total, 9 passed, 3 failed
* sh: 24 total, 24 passed, 0 failed
* sparc: 12 total, 12 passed, 0 failed
* x86_64: 46 total, 44 passed, 2 failed
## Test suites summary
* fwts
* igt-gpu-tools
* kunit
* kvm-unit-tests
* libhugetlbfs
* log-parser-boot
* log-parser-test
* ltp-cap_bounds
* ltp-cap_bounds-tests
* ltp-commands
* ltp-commands-tests
* ltp-containers
* ltp-containers-tests
* ltp-controllers-tests
* ltp-cpuhotplug-tests
* ltp-crypto
* ltp-crypto-tests
* ltp-cve-tests
* ltp-dio-tests
* ltp-fcntl-locktests
* ltp-fcntl-locktests-tests
* ltp-filecaps
* ltp-filecaps-tests
* ltp-fs
* ltp-fs-tests
* ltp-fs_bind
* ltp-fs_bind-tests
* ltp-fs_perms_simple
* ltp-fs_perms_simple-tests
* ltp-fsx
* ltp-fsx-tests
* ltp-hugetlb
* ltp-hugetlb-tests
* ltp-io
* ltp-io-tests
* ltp-ipc
* ltp-ipc-tests
* ltp-math-tests
* ltp-mm-tests
* ltp-nptl
* ltp-nptl-tests
* ltp-open-posix-tests
* ltp-pty
* ltp-pty-tests
* ltp-sched
* ltp-sched-tests
* ltp-securebits
* ltp-securebits-tests
* ltp-smoke
* ltp-syscalls-tests
* ltp-tracing-tests
* network-basic-tests
* packetdrill
* rcutorture
* ssuite
* v4l2-compliance
* vdso
--
Linaro LKFT
https://lkft.linaro.org