2005-03-15 13:41:15

by David McCullough

[permalink] [raw]
Subject: ocf-linux-20050315 - Asynchronous Crypto support for linux


Hi all,

The latest release of ocf-linux (20050315) is available for download
from the project page:

http://ocf-linux.sourceforge.net/

This release includes the following changes:

* Hifn PLL bug fixes
* Makefile fixes for 2.6 builds
* 2.6.11 and 2.4.29 patches
* cleanups for x86 builds

OCF-Linux is a Linux port of the OpenBSD/FreeBSD Cryptographic Framework
(OCF). This port aims to bring full asynchronous HW/SW crypto
acceleration to the Linux kernel and applications running under Linux.
Results have shown improvements of up to 7 times that of software crypto
for bulk crypto throughput using OpenSSL.

At this point in time OCF-Linux provides acceleration for OpenSSL,
OpenSSH (scp, ssh, ...) and also supports the BSD crypto testing
applications. It can accelerate DES, 3DES, AES, MD5, SHA, and Public Key
operations with plans to include Random Number generators and more. This
project is being actively developed as a high performance crypto
solution for embedded devices but also applies equally well to any linux
based server or desktop.

Cheers,
Davidm

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org


2005-03-24 04:27:33

by David McCullough

[permalink] [raw]
Subject: [PATCH] API for true Random Number Generators to add entropy (2.6.11)


Hi all,

Here is a small patch for 2.6.11 that adds a routine:

add_true_randomness(__u32 *buf, int nwords);

so that true random number generator device drivers can add a entropy
to the system. Drivers that use this can be found in the latest release
of ocf-linux, an asynchronous crypto implementation for linux based on
the *BSD Cryptographic Framework.

http://ocf-linux.sourceforge.net/

Adding this can dramatically improve the performance of /dev/random on
small embedded systems which do not generate much entropy.

Cheers,
Davidm

Signed-off-by: David McCullough <[email protected]>

Index: linux-2.6.11/include/linux/random.h
===================================================================
RCS file: linux-2.6.11/include/linux/random.h,v
retrieving revision 1.1.1.6
diff -u -r1.1.1.6 random.h
--- linux-2.6.11/include/linux/random.h 3 Mar 2005 00:45:49 -0000 1.1.1.6
+++ linux-2.6.11/include/linux/random.h 18 Mar 2005 01:46:16 -0000
@@ -48,6 +48,8 @@
unsigned int value);
extern void add_interrupt_randomness(int irq);

+extern void add_true_randomness(__u32 *buf, int nwords);
+
extern void get_random_bytes(void *buf, int nbytes);
void generate_random_uuid(unsigned char uuid_out[16]);

Index: linux-2.6.11/drivers/char/random.c
===================================================================
RCS file: linux-2.6.x/drivers/char/random.c,v
retrieving revision 1.1.1.28
diff -u -r1.1.1.28 random.c
--- linux-2.6.11/drivers/char/random.c 3 Mar 2005 00:45:31 -0000 1.1.1.28
+++ linux-2.6.11/drivers/char/random.c 18 Mar 2005 01:46:16 -0000
@@ -902,6 +902,39 @@

EXPORT_SYMBOL(add_disk_randomness);

+/*
+ * provide a mechanism for HW to add entropy that is of
+ * very good quality from a true random number generator
+ */
+void add_true_randomness(__u32 *buf, int nwords)
+{
+ struct entropy_store *r;
+ int wakeup_check = 0;
+
+
+ if (!random_state || !sec_random_state)
+ return;
+
+ /*
+ * if we have too much entropy, put some in the secondary pool
+ */
+ r = random_state;
+ if (r->entropy_count >= r->poolinfo.POOLBITS)
+ r = sec_random_state;
+ else
+ wakeup_check = (r->entropy_count < random_read_wakeup_thresh);
+
+ add_entropy_words(r, buf, nwords);
+ credit_entropy_store(r, nwords * 32);
+
+ /*
+ * wakeup if we added enough entropy to cross the threshold
+ */
+ if (wakeup_check && r->entropy_count >= random_read_wakeup_thresh)
+ wake_up_interruptible(&random_read_wait);
+}
+EXPORT_SYMBOL(add_true_randomness);
+
/******************************************************************
*
* Hash function definition

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 04:30:45

by David McCullough

[permalink] [raw]
Subject: [PATCH] API for true Random Number Generators to add entropy (2.4.29)


Hi all,

Here is a small patch for 2.4.29 that adds a routine:

add_true_randomness(__u32 *buf, int nwords);

so that true random number generator device drivers can add a entropy
to the system. Drivers that use this can be found in the latest release
of ocf-linux, an asynchronous crypto implementation for linux based on
the *BSD Cryptographic Framework.

http://ocf-linux.sourceforge.net/

Adding this can dramatically improve the performance of /dev/random on
small embedded systems which do not generate much entropy.

Cheers,
Davidm

Signed-off-by: David McCullough <[email protected]>

Index: linux-2.4.29/include/linux/random.h
===================================================================
RCS file: /cvs/sw/linux-2.4.29/include/linux/random.h,v
retrieving revision 1.1.1.1
retrieving revision 1.3
diff -u -r1.1.1.1 -r1.3
--- linux-2.4.29/include/linux/random.h 9 Jul 2001 00:39:16 -0000 1.1.1.1
+++ linux-2.4.29/include/linux/random.h 18 Mar 2005 04:33:35 -0000 1.3
@@ -52,6 +52,7 @@
extern void add_mouse_randomness(__u32 mouse_data);
extern void add_interrupt_randomness(int irq);
extern void add_blkdev_randomness(int major);
+extern void add_true_randomness(__u32 *buf, int nwords);

extern void get_random_bytes(void *buf, int nbytes);
void generate_random_uuid(unsigned char uuid_out[16]);
Index: linux-2.4.29/drivers/char/random.c
===================================================================
RCS file: /cvs/sw/linux-2.4.29/drivers/char/random.c,v
retrieving revision 1.1.1.9
retrieving revision 1.11
diff -u -r1.1.1.9 -r1.11
--- linux-2.4.29/drivers/char/random.c 4 Feb 2005 00:28:36 -0000 1.1.1.9
+++ linux-2.4.29/drivers/char/random.c 18 Mar 2005 04:33:35 -0000 1.11
@@ -842,6 +837,39 @@
add_timer_randomness(blkdev_timer_state[major], 0x200+major);
}

+/*
+ * provide a mechanism for HW RNG's to add entropy that is of
+ * very good quality.
+ */
+void add_true_randomness(__u32 *buf, int nwords)
+{
+ struct entropy_store *r;
+ int wakeup_check = 0;
+
+
+ if (!random_state || !sec_random_state)
+ return;
+
+ /*
+ * if we have too much entropy, put some in the secondary pool
+ */
+ r = random_state;
+ if (r->entropy_count >= r->poolinfo.POOLBITS)
+ r = sec_random_state;
+ else
+ wakeup_check = (r->entropy_count < random_read_wakeup_thresh);
+
+ add_entropy_words(r, buf, nwords);
+ credit_entropy_store(r, nwords * 32);
+
+ /*
+ * wakeup if we added enough entropy to cross the threshold
+ */
+ if (wakeup_check && r->entropy_count >= random_read_wakeup_thresh)
+ wake_up_interruptible(&random_read_wait);
+}
+EXPORT_SYMBOL(add_true_randomness);
+
/******************************************************************
*
* Hash function definition

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 04:33:46

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, Mar 24, 2005 at 02:27:08PM +1000, David McCullough wrote:
>
> Hi all,
>
> Here is a small patch for 2.6.11 that adds a routine:
>
> add_true_randomness(__u32 *buf, int nwords);
>
> so that true random number generator device drivers can add a entropy
> to the system. Drivers that use this can be found in the latest release
> of ocf-linux, an asynchronous crypto implementation for linux based on
> the *BSD Cryptographic Framework.
>
> http://ocf-linux.sourceforge.net/
>
> Adding this can dramatically improve the performance of /dev/random on
> small embedded systems which do not generate much entropy.

We've already had hardware RNG support for a while now.

No kernel patching needed.

Jeff



2005-03-24 04:40:06

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

David McCullough <[email protected]> wrote:
>
> Here is a small patch for 2.6.11 that adds a routine:
>
> add_true_randomness(__u32 *buf, int nwords);

It neither applies correctly nor compiles in current kernels. 2.6.11 is
very old in kernel time.

Are we likely to see any in-kernel users of this?

2005-03-24 04:46:37

by David McCullough

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)


Jivin Jeff Garzik lays it down ...
> On Thu, Mar 24, 2005 at 02:27:08PM +1000, David McCullough wrote:
> >
> > Hi all,
> >
> > Here is a small patch for 2.6.11 that adds a routine:
> >
> > add_true_randomness(__u32 *buf, int nwords);
> >
> > so that true random number generator device drivers can add a entropy
> > to the system. Drivers that use this can be found in the latest release
> > of ocf-linux, an asynchronous crypto implementation for linux based on
> > the *BSD Cryptographic Framework.
> >
> > http://ocf-linux.sourceforge.net/
> >
> > Adding this can dramatically improve the performance of /dev/random on
> > small embedded systems which do not generate much entropy.
>
> We've already had hardware RNG support for a while now.
>
> No kernel patching needed.

Are you talking about /dev/hw_random ? If not then sorry I didn't see it :-(

On a lot of the small systems I work on, /dev/random is completely
unresponsive, and all the apps use /dev/random, not /dev/hw_random.

Would you suggest making /dev/random point to /dev/hw_random then ?

Thanks,
Davidm

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 04:49:18

by Michal Ludvig

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

David McCullough wrote:

> Are you talking about /dev/hw_random ? If not then sorry I didn't see it :-(
>
> On a lot of the small systems I work on, /dev/random is completely
> unresponsive, and all the apps use /dev/random, not /dev/hw_random.
>
> Would you suggest making /dev/random point to /dev/hw_random then ?

Or feed the entropy pool from hw_random?

Michal Ludvig

2005-03-24 05:14:20

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

David McCullough wrote:
> Jivin Jeff Garzik lays it down ...
>
>>On Thu, Mar 24, 2005 at 02:27:08PM +1000, David McCullough wrote:
>>
>>>Hi all,
>>>
>>>Here is a small patch for 2.6.11 that adds a routine:
>>>
>>> add_true_randomness(__u32 *buf, int nwords);
>>>
>>>so that true random number generator device drivers can add a entropy
>>>to the system. Drivers that use this can be found in the latest release
>>>of ocf-linux, an asynchronous crypto implementation for linux based on
>>>the *BSD Cryptographic Framework.
>>>
>>> http://ocf-linux.sourceforge.net/
>>>
>>>Adding this can dramatically improve the performance of /dev/random on
>>>small embedded systems which do not generate much entropy.
>>
>>We've already had hardware RNG support for a while now.
>>
>>No kernel patching needed.
>
>
> Are you talking about /dev/hw_random ? If not then sorry I didn't see it :-(
>
> On a lot of the small systems I work on, /dev/random is completely
> unresponsive, and all the apps use /dev/random, not /dev/hw_random.
>
> Would you suggest making /dev/random point to /dev/hw_random then ?

All the apps are supposed to use /dev/random, so that's correct.

For Hardware RNGs, userspace rngd daemon obtains entropy, checks it
(mainly checking for hardware failures), and then stuffs entropy into
the kernel random device. http://sf.net/projects/gkernel/

On the "to do" list is making rngd directly generate entropy use
'xstore' on VIA CPUs, rather than going kernel -> userland -> kernel.

Also, there are other entropy daemons floating about. I think there is
one that obtains noise from an audio device.

Jeff



2005-03-24 05:17:25

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Andrew Morton wrote:
> David McCullough <[email protected]> wrote:
>
>>Here is a small patch for 2.6.11 that adds a routine:
>>
>> add_true_randomness(__u32 *buf, int nwords);
>
>
> It neither applies correctly nor compiles in current kernels. 2.6.11 is
> very old in kernel time.

Hrm. This is getting pretty lame, if you can't take patches from the
-latest- stable release. It's pretty easy in BK:

bk clone -ql -rv2.6.11 linux-2.6 rng-2.6.11
cd rng-2.6.11
{ apply patch }
bk pull ../linux-2.6

Can you set up something like that?


> Are we likely to see any in-kernel users of this?

We already have a hardware RNG system. This is completely unneeded.

Jeff


2005-03-24 05:33:52

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Jeff Garzik <[email protected]> wrote:
>
> > It neither applies correctly nor compiles in current kernels. 2.6.11 is
> > very old in kernel time.
>
> Hrm. This is getting pretty lame, if you can't take patches from the
> -latest- stable release. It's pretty easy in BK:
>
> bk clone -ql -rv2.6.11 linux-2.6 rng-2.6.11
> cd rng-2.6.11
> { apply patch }
> bk pull ../linux-2.6
>
> Can you set up something like that?

About thirty patches have gone into random.c since 2.6.11. But the patch
was easy enough to apply anyway.

And then, it didn't compile. I don't think bk will fix that.

2005-03-24 05:44:34

by Randy.Dunlap

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Jeff Garzik wrote:
> Andrew Morton wrote:
>
>> David McCullough <[email protected]> wrote:
>>
>>> Here is a small patch for 2.6.11 that adds a routine:
>>>
>>> add_true_randomness(__u32 *buf, int nwords);
>>
>>
>>
>> It neither applies correctly nor compiles in current kernels. 2.6.11 is
>> very old in kernel time.
>
>
> Hrm. This is getting pretty lame, if you can't take patches from the
> -latest- stable release. It's pretty easy in BK:
>
> bk clone -ql -rv2.6.11 linux-2.6 rng-2.6.11
> cd rng-2.6.11
> { apply patch }
> bk pull ../linux-2.6
>
> Can you set up something like that?

I thought that the latest stable release was 2.6.11.5.

However, what I really want to do is ask what patches should be
made against. I suggested on linux-scsi a day or 2 ago that
they should be made against the latest linus-bk (or snapshot)
unless the patch only applies to -mm, then they should obviously
be made against -mm. 2.6.11 plain is relatively aged IMO also....

--
~Randy

2005-03-24 07:18:56

by Jan Engelhardt

[permalink] [raw]
Subject: Re: API for true Random Number Generators to add entropy (2.6.11)


>Would you suggest making /dev/random point to /dev/hw_random then ?

No. I for example do not have a hardware RNG, so `modprobe hw_random` fails
with No Such Device. Making it a symlink would make it a dangling one.


Jan Engelhardt
--

2005-03-24 07:37:14

by Dave Jones

[permalink] [raw]
Subject: Re: API for true Random Number Generators to add entropy (2.6.11)

On Thu, Mar 24, 2005 at 08:18:35AM +0100, Jan Engelhardt wrote:
>
> >Would you suggest making /dev/random point to /dev/hw_random then ?
>
> No. I for example do not have a hardware RNG, so `modprobe hw_random` fails
> with No Such Device. Making it a symlink would make it a dangling one.

It shouldn't be a symlink. Something like rngd should read from it
and feed it into /dev/random's entropy pool.

Dave

2005-03-24 11:54:55

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
> Hi all,
>
> Here is a small patch for 2.6.11 that adds a routine:
>
> add_true_randomness(__u32 *buf, int nwords);
>
> so that true random number generator device drivers can add a entropy
> to the system. Drivers that use this can be found in the latest release
> of ocf-linux, an asynchronous crypto implementation for linux based on
> the *BSD Cryptographic Framework.
>
> http://ocf-linux.sourceforge.net/
>
> Adding this can dramatically improve the performance of /dev/random on
> small embedded systems which do not generate much entropy.

People will not apply any kind of such changes.
Both OCF and acrypto already handle all RNG cases - no need for any kind
of userspace daemon or entropy (re)injection mechanism.
Anyone who want to use HW randomness may use OCF/acrypto mechanism.
For example here is patch to enable acrypto support for hw_random.c
It is very simple and support only upto 4 bytes request, of course it
is
not interested for anyone, but it is only 2-minutes example:

--- ./drivers/char/hw_random.c.orig 2005-03-24 13:36:05.000000000 +0300
+++ ./drivers/char/hw_random.c 2005-03-24 14:48:30.470407432 +0300
@@ -34,6 +34,10 @@
#include <linux/smp_lock.h>
#include <linux/mm.h>
#include <linux/delay.h>
+#include <linux/acrypto.h>
+#include <linux/crypto_def.h>
+#include <linux/crypto_stat.h>
+#include <linux/highmem.h>

#ifdef __i386__
#include <asm/msr.h>
@@ -73,6 +77,8 @@
#endif

#define RNG_MISCDEV_MINOR 183 /* official */
+
+static DEFINE_SPINLOCK(rng_lock);

static int rng_dev_open (struct inode *inode, struct file *filp);
static ssize_t rng_dev_read (struct file *filp, char __user *buf, size_t size,
@@ -482,7 +488,6 @@
static ssize_t rng_dev_read (struct file *filp, char __user *buf, size_t size,
loff_t * offp)
{
- static DEFINE_SPINLOCK(rng_lock);
unsigned int have_data;
u32 data = 0;
ssize_t ret = 0;
@@ -526,7 +531,109 @@
return ret;
}

+#define CONFIG_ACRYPTO
+#ifdef CONFIG_ACRYPTO
+static struct crypto_device *hwr_cdev;
+static struct crypto_capability hwr_caps[] = {
+ {CRYPTO_OP_RNG, 0, 0, 100},
+};
+

+static void hwr_data_ready(struct crypto_device *dev)
+{
+ struct crypto_session *s, *n;
+ unsigned long flags;
+ u32 data = 0;
+ unsigned int have_data;
+ u8 *ptr, *optr;
+
+ local_irq_save(flags);
+ if (spin_trylock(&dev->session_lock)) {
+ local_irq_restore(flags);
+ return;
+ }
+
+ list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry) {
+ if (!session_completed(s) && !session_is_processed(s)) {
+ start_process_session(s);
+ spin_lock(&rng_lock);
+ have_data = 0;
+ if (rng_ops->data_present()) {
+ data = rng_ops->data_read();
+ have_data = rng_ops->n_bytes;
+ }
+ spin_unlock (&rng_lock);
+
+ optr = ptr = kmap_atomic(s->data.sg_dst[0].page, KM_USER0) + s->data.sg_dst[0].offset;
+ while (s->data.sg_dst[0].length && have_data) {
+ *ptr++ = data & 0xff;
+ s->data.sg_dst[0].length--;
+ have_data--;
+ data >>= 8;
+ }
+ kunmap_atomic(optr, KM_USER0);
+
+ if (s->data.sg_dst[0].length)
+ broke_session(s); /* Need proper work deferring, see async_provider.c as example. */
+
+ crypto_stat_complete_inc(s);
+ crypto_session_dequeue_route(s);
+ complete_session(s);
+ stop_process_session(s);
+ }
+ }
+
+ spin_unlock(&dev->session_lock);
+ local_irq_restore(flags);
+}
+
+static int hwr_acrypto_init(struct rng_operations *ops)
+{
+ int err;
+
+ hwr_cdev = kmalloc(sizeof(*hwr_cdev), GFP_KERNEL);
+ if (!hwr_cdev) {
+ printk(KERN_ERR "Failed to allocate new crypto_device structure.\n");
+ return -ENOMEM;
+ }
+
+ memset(hwr_cdev, 0, sizeof(*hwr_cdev));
+
+ hwr_cdev->cap = hwr_caps;
+ hwr_cdev->cap_number = sizeof(hwr_caps)/sizeof(hwr_caps[0]);
+ hwr_cdev->priv = ops;
+ hwr_cdev->data_ready = &hwr_data_ready;
+ snprintf(hwr_cdev->name, sizeof(hwr_cdev->name), "%s", "hwr");
+
+ err = crypto_device_add(hwr_cdev);
+ if (err) {
+ kfree(hwr_cdev);
+ hwr_cdev = NULL;
+ return err;
+ }
+
+ printk(KERN_INFO "%s acrypto support is turned on.\n", hwr_cdev->name);
+
+ return err;
+}
+
+static void hwr_acrypto_fini(void)
+{
+ crypto_device_remove(hwr_cdev);
+ printk(KERN_INFO "%s acrypto support is turned off.\n", hwr_cdev->name);
+ kfree(hwr_cdev);
+ hwr_cdev = NULL;
+}
+#else
+static int hwr_acrypto_init(struct rng_operations *ops)
+{
+ return 0;
+}
+
+static void hwr_acrypto_fini(void)
+{
+}
+#endif

/*
* rng_init_one - look for and attempt to init a single RNG
@@ -549,9 +656,15 @@
goto err_out_cleanup_hw;
}

+ rc = hwr_acrypto_init(rng_ops);
+ if (rc)
+ goto err_out_misc_unregister;
+
DPRINTK ("EXIT, returning 0\n");
return 0;

+err_out_misc_unregister:
+ misc_deregister(&rng_miscdev);
err_out_cleanup_hw:
rng_ops->cleanup();
err_out:
@@ -617,6 +730,8 @@
{
DPRINTK ("ENTER\n");

+ hwr_acrypto_fini();
+
misc_deregister (&rng_miscdev);

if (rng_ops->cleanup)


So, one may use crypto_session_alloc(gimme_real_randomness) and that is
all
(either from userspace or kernelspace).
As far as I can see, OCF support is also very simple.

What I want to say, is that since OCF and acrypto already support
all RNG cases, it is better to port existing drivers to them, but
not the reverse.
That will allow to use both old hw_random.c driver and
any new one(HIFN, VIA, safenet, any other) without breaking any
interfaces from both userspace and kernelspace.

If one of the asynchronous crypto layers will be included,
/dev/random implementation can be changed to support it.

> Cheers,
> Davidm

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-24 12:15:53

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Wed, 2005-03-23 at 20:38 -0800, Andrew Morton wrote:
> David McCullough <[email protected]> wrote:
> >
> > Here is a small patch for 2.6.11 that adds a routine:
> >
> > add_true_randomness(__u32 *buf, int nwords);
>
> It neither applies correctly nor compiles in current kernels. 2.6.11 is
> very old in kernel time.
>
> Are we likely to see any in-kernel users of this?

Any external crtypto framework can add entropy using that routing.
Currently it can be
- OCF
- acrypto
- hw_random.c

hw_random.c already does it using userspace daemons,
which is bad idea for very fast HW - like VIA xstore/xcrypt
instructions.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-24 12:30:15

by David McCullough

[permalink] [raw]
Subject: [PATCH 2.6.12-rc1] API for true Random Number Generators to add entropy


Hi all,

Here is a revised patch for 2.6.12-rc1 that adds a routine:

add_true_randomness(__u32 *buf, int nwords);

so that true random number generator device drivers can add a entropy
to the system.

Cheers,
Davidm

Signed-off-by: David McCullough <[email protected]>

--- linux-2.6.12-rc1.orig/drivers/char/random.c 2005-03-18 11:33:49.000000000 +1000
+++ linux-2.6.12-rc1/drivers/char/random.c 2005-03-24 16:16:40.000000000 +1000
@@ -1148,6 +1148,36 @@
EXPORT_SYMBOL(generate_random_uuid);

/********************************************************************
+ * provide a mechanism for HW to add entropy that is of
+ * very good quality from a true random number generator
+ ***************************************************************/
+
+void add_true_randomness(__u32 *buf, int nwords)
+{
+ struct entropy_store *r;
+ int wakeup_check = 0;
+
+ /*
+ * if we have too much entropy, put some in the secondary pool
+ */
+ r = &blocking_pool;
+ if (r->entropy_count >= r->poolinfo->POOLBITS)
+ r = &nonblocking_pool;
+ else
+ wakeup_check = (r->entropy_count < random_read_wakeup_thresh);
+
+ add_entropy_words(r, buf, nwords);
+ credit_entropy_store(r, nwords * 32);
+
+ /*
+ * wakeup if we added enough entropy to cross the threshold
+ */
+ if (wakeup_check && r->entropy_count >= random_read_wakeup_thresh)
+ wake_up_interruptible(&random_read_wait);
+}
+EXPORT_SYMBOL(add_true_randomness);
+
+/********************************************************************
*
* Sysctl interface
*
--- linux-2.6.12-rc1.orig/include/linux/random.h 2005-03-18 11:34:37.000000000 +1000
+++ linux-2.6.12-rc1/include/linux/random.h 2005-03-24 15:59:42.000000000 +1000
@@ -48,6 +48,8 @@
unsigned int value);
extern void add_interrupt_randomness(int irq);

+extern void add_true_randomness(__u32 *buf, int nwords);
+
extern void get_random_bytes(void *buf, int nbytes);
void generate_random_uuid(unsigned char uuid_out[16]);


--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 12:37:29

by folkert

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

> Also, there are other entropy daemons floating about. I think there is
> one that obtains noise from an audio device.

That's correct: http://www.vanheusden.com/aed/ audio-entropyd

There's also one for doing the same with video4linux devices:
http://www.vanheusden.com/ved/


Folkert van Heusden

Auto te koop! Zie: http://www.vanheusden.com/daihatsu.php
Op zoek naar een IT of Finance baan? Mail me voor de mogelijkheden!
+------------------------------------------------------------------+
|UNIX admin? Then give MultiTail (http://vanheusden.com/multitail/)|
|a try, it brings monitoring logfiles to a different level! See |
|http://vanheusden.com/multitail/features.html for a feature list. |
+------------------------------------------= http://www.unixsoftware.nl =-+
Phone: +31-6-41278122, PGP-key: 1F28D8AE
Get your PGP/GPG key signed at http://www.biglumber.com!

2005-03-24 12:39:29

by David McCullough

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)


Jivin Andrew Morton lays it down ...
> David McCullough <[email protected]> wrote:
> >
> > Here is a small patch for 2.6.11 that adds a routine:
> >
> > add_true_randomness(__u32 *buf, int nwords);
>
> It neither applies correctly nor compiles in current kernels. 2.6.11 is
> very old in kernel time.

Sorry about that, I had actually checked a fairly recent bk version
and noticed quite a few changes. I used that to figure out what I could
do that would apply reasonably to both 2.4 and 2.6 kernels, and then
forgot about all those new changes and used the older release kernel.

See the new patch.

> Are we likely to see any in-kernel users of this?

Both the OCF port that I am working on and Evgeniy Polyakov's acrypto
support devices that could use such an API. The OCF port has already
has two drivers (hifn and safenet) that are using this and, depending
on how this pans out, there will be another for Xscale soon.

Whether or not these users of it end up in the kernel is out of my hands
somewhat :-)

Cheers,
Davidm

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 12:50:40

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
>
>>Hi all,
>>
>>Here is a small patch for 2.6.11 that adds a routine:
>>
>> add_true_randomness(__u32 *buf, int nwords);
>>
>>so that true random number generator device drivers can add a entropy
>>to the system. Drivers that use this can be found in the latest release
>>of ocf-linux, an asynchronous crypto implementation for linux based on
>>the *BSD Cryptographic Framework.
>>
>> http://ocf-linux.sourceforge.net/
>>
>>Adding this can dramatically improve the performance of /dev/random on
>>small embedded systems which do not generate much entropy.
>
>
> People will not apply any kind of such changes.
> Both OCF and acrypto already handle all RNG cases - no need for any kind
> of userspace daemon or entropy (re)injection mechanism.
> Anyone who want to use HW randomness may use OCF/acrypto mechanism.
> For example here is patch to enable acrypto support for hw_random.c
> It is very simple and support only upto 4 bytes request, of course it
> is
> not interested for anyone, but it is only 2-minutes example:

If you want to add entropy to the kernel entropy pool from hardware RNG,
you should use the userland daemon, which detects non-random (broken)
hardware and provides throttling, so that RNG data collection does not
consume 100% CPU.

If you want to use the hardware RNG directly, it's simple: just open
/dev/hw_random.

Hardware RNG should not go kernel->kernel without adding FIPS tests and
such.

Jeff


2005-03-24 12:53:46

by David McCullough

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)


Jivin Jeff Garzik lays it down ...
> David McCullough wrote:
> >Jivin Jeff Garzik lays it down ...
> >
> >>On Thu, Mar 24, 2005 at 02:27:08PM +1000, David McCullough wrote:
> >>
> >>>Hi all,
> >>>
> >>>Here is a small patch for 2.6.11 that adds a routine:
> >>>
> >>> add_true_randomness(__u32 *buf, int nwords);
> >>>
> >>>so that true random number generator device drivers can add a entropy
> >>>to the system. Drivers that use this can be found in the latest release
> >>>of ocf-linux, an asynchronous crypto implementation for linux based on
> >>>the *BSD Cryptographic Framework.
> >>>
> >>> http://ocf-linux.sourceforge.net/
> >>>
> >>>Adding this can dramatically improve the performance of /dev/random on
> >>>small embedded systems which do not generate much entropy.
> >>
> >>We've already had hardware RNG support for a while now.
> >>
> >>No kernel patching needed.
> >
> >
> >Are you talking about /dev/hw_random ? If not then sorry I didn't see it
> >:-(
> >
> >On a lot of the small systems I work on, /dev/random is completely
> >unresponsive, and all the apps use /dev/random, not /dev/hw_random.
> >
> >Would you suggest making /dev/random point to /dev/hw_random then ?
>
> All the apps are supposed to use /dev/random, so that's correct.

Ok

> For Hardware RNGs, userspace rngd daemon obtains entropy, checks it
> (mainly checking for hardware failures), and then stuffs entropy into
> the kernel random device. http://sf.net/projects/gkernel/
>
> On the "to do" list is making rngd directly generate entropy use
> 'xstore' on VIA CPUs, rather than going kernel -> userland -> kernel.
>
> Also, there are other entropy daemons floating about. I think there is
> one that obtains noise from an audio device.

I had looked at hw_random, but it seemed a little platform specific (x86),
and it doesn't currently have a way for RNG providers to register themselves.
Admittedly I did not know how it's output was being used in practice.

The drivers I am working with do crypto/public key and RNG. Not all of
them can easily have the RNG support taken from the driver and plugged
into hw_random.c, since it is (in most cases) a single PCI chip with
it's own registers, initialisation and configuration, that, IMO
belongs in the driver for the particular chip.

Not that it isn't possible, but hw_random would start supporting a
much larger number of HW variants and I think it would get ugly.

It would be possible to add a "register" interface to hw_random so that
you can register other RNG's easily. This would seem reasonable.

I work on fairly resource constrained embedded devices a lot of the
time, so when I can avoid adding applications and reduce kernel size,
I do. Thus my motivation to add a simple API for adding entropy to
/dev/random.

Cheers,
Davidm

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 13:02:47

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 07:48 -0500, Jeff Garzik wrote:
> Evgeniy Polyakov wrote:
> > On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
> >
> >>Hi all,
> >>
> >>Here is a small patch for 2.6.11 that adds a routine:
> >>
> >> add_true_randomness(__u32 *buf, int nwords);
> >>
> >>so that true random number generator device drivers can add a entropy
> >>to the system. Drivers that use this can be found in the latest release
> >>of ocf-linux, an asynchronous crypto implementation for linux based on
> >>the *BSD Cryptographic Framework.
> >>
> >> http://ocf-linux.sourceforge.net/
> >>
> >>Adding this can dramatically improve the performance of /dev/random on
> >>small embedded systems which do not generate much entropy.
> >
> >
> > People will not apply any kind of such changes.
> > Both OCF and acrypto already handle all RNG cases - no need for any kind
> > of userspace daemon or entropy (re)injection mechanism.
> > Anyone who want to use HW randomness may use OCF/acrypto mechanism.
> > For example here is patch to enable acrypto support for hw_random.c
> > It is very simple and support only upto 4 bytes request, of course it
> > is
> > not interested for anyone, but it is only 2-minutes example:
>
> If you want to add entropy to the kernel entropy pool from hardware RNG,
> you should use the userland daemon, which detects non-random (broken)
> hardware and provides throttling, so that RNG data collection does not
> consume 100% CPU.
>
> If you want to use the hardware RNG directly, it's simple: just open
> /dev/hw_random.
>
> Hardware RNG should not go kernel->kernel without adding FIPS tests and
> such.

hw_random can not and will not support HIFN, freescale, ixp and
great majority of the existing and future HW crypto devices.
I mean that userspace daemon(or any other one) which want to contribute
entropy
should use crypto framwork to obtain all it's data, but not different
access methods for each separate driver.

> Jeff

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-24 13:24:10

by David McCullough

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)


Jivin Jeff Garzik lays it down ...
> Evgeniy Polyakov wrote:
> >On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
> >
> >>Hi all,
> >>
> >>Here is a small patch for 2.6.11 that adds a routine:
> >>
> >> add_true_randomness(__u32 *buf, int nwords);
> >>
> >>so that true random number generator device drivers can add a entropy
> >>to the system. Drivers that use this can be found in the latest release
> >>of ocf-linux, an asynchronous crypto implementation for linux based on
> >>the *BSD Cryptographic Framework.
> >>
> >> http://ocf-linux.sourceforge.net/
> >>
> >>Adding this can dramatically improve the performance of /dev/random on
> >>small embedded systems which do not generate much entropy.
> >
> >
> >People will not apply any kind of such changes.
> >Both OCF and acrypto already handle all RNG cases - no need for any kind
> >of userspace daemon or entropy (re)injection mechanism.
> >Anyone who want to use HW randomness may use OCF/acrypto mechanism.
> >For example here is patch to enable acrypto support for hw_random.c
> >It is very simple and support only upto 4 bytes request, of course it
> >is
> >not interested for anyone, but it is only 2-minutes example:
>
> If you want to add entropy to the kernel entropy pool from hardware RNG,
> you should use the userland daemon, which detects non-random (broken)
> hardware and provides throttling, so that RNG data collection does not
> consume 100% CPU.
>
> If you want to use the hardware RNG directly, it's simple: just open
> /dev/hw_random.
>
> Hardware RNG should not go kernel->kernel without adding FIPS tests and
> such.

For reference, the RNG on the Safenet I am using this with is
FIPS140 certified. I believe the HIFN part is also but I place the doc that
says so.

Cheers,
Davidm

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-24 13:40:54

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 23:23 +1000, David McCullough wrote:
> Jivin Jeff Garzik lays it down ...
> > Evgeniy Polyakov wrote:
> > >On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
> > >
> > >>Hi all,
> > >>
> > >>Here is a small patch for 2.6.11 that adds a routine:
> > >>
> > >> add_true_randomness(__u32 *buf, int nwords);
> > >>
> > >>so that true random number generator device drivers can add a entropy
> > >>to the system. Drivers that use this can be found in the latest release
> > >>of ocf-linux, an asynchronous crypto implementation for linux based on
> > >>the *BSD Cryptographic Framework.
> > >>
> > >> http://ocf-linux.sourceforge.net/
> > >>
> > >>Adding this can dramatically improve the performance of /dev/random on
> > >>small embedded systems which do not generate much entropy.
> > >
> > >
> > >People will not apply any kind of such changes.
> > >Both OCF and acrypto already handle all RNG cases - no need for any kind
> > >of userspace daemon or entropy (re)injection mechanism.
> > >Anyone who want to use HW randomness may use OCF/acrypto mechanism.
> > >For example here is patch to enable acrypto support for hw_random.c
> > >It is very simple and support only upto 4 bytes request, of course it
> > >is
> > >not interested for anyone, but it is only 2-minutes example:
> >
> > If you want to add entropy to the kernel entropy pool from hardware RNG,
> > you should use the userland daemon, which detects non-random (broken)
> > hardware and provides throttling, so that RNG data collection does not
> > consume 100% CPU.
> >
> > If you want to use the hardware RNG directly, it's simple: just open
> > /dev/hw_random.
> >
> > Hardware RNG should not go kernel->kernel without adding FIPS tests and
> > such.
>
> For reference, the RNG on the Safenet I am using this with is
> FIPS140 certified. I believe the HIFN part is also but I place the doc that
> says so.

At least HIFN 795x is certified.

Idea to validate entropy data is good in general,
but it should be implemented in a way allowing external both in-kernel
and userspace
processes to contribute data.
So for in-kernel use we need such a mechanism, and userspace gkernel
daemon
should use it(as the latest "step") too.

Your changes are correct, ioctl(RNDADDENTROPY) could even use it instead
of direct
add_entropy_words()/credit_entropy_store(), but without external entropy
contributors
it will not be applied by maintainers.

> Cheers,
> Davidm
>
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-24 14:26:52

by Jean-Luc Cooke

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, Mar 24, 2005 at 07:48:18AM -0500, Jeff Garzik wrote:
> If you want to add entropy to the kernel entropy pool from hardware RNG,
> you should use the userland daemon, which detects non-random (broken)
> hardware and provides throttling, so that RNG data collection does not
> consume 100% CPU.
>
> If you want to use the hardware RNG directly, it's simple: just open
> /dev/hw_random.
>
> Hardware RNG should not go kernel->kernel without adding FIPS tests and
> such.

If your RNG were properly written, it shouldn't matter if the data you're
pumping into /dev/random passed muster or not. If you're tracking entropy
count, then that's a different story of course.

I've been commissioned to write Fortuna RNG for Linux and weddings, houses and
cars not withstanding, I should I it ready soon to be given to LKML for
digestion.

JLC

2005-03-24 18:52:59

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Andrew Morton <[email protected]> writes:

> David McCullough <[email protected]> wrote:
>>
>> Here is a small patch for 2.6.11 that adds a routine:
>>
>> add_true_randomness(__u32 *buf, int nwords);
>
> It neither applies correctly nor compiles in current kernels. 2.6.11 is
> very old in kernel time.
>
> Are we likely to see any in-kernel users of this?

I added similar support to the pre hw_random AMD8111 driver
a long time ago. Basically a timer that regularly read some
dat from the hw random generator and feed it into the random
code.

I think it is a good idea, because it doesnt make much sense
imho to run a daemon for something that can be done in 20 lines
of code in the kernel.

-Andi

2005-03-24 20:37:34

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Andi Kleen wrote:
> Andrew Morton <[email protected]> writes:
>
>
>>David McCullough <[email protected]> wrote:
>>
>>>Here is a small patch for 2.6.11 that adds a routine:
>>>
>>> add_true_randomness(__u32 *buf, int nwords);
>>
>>It neither applies correctly nor compiles in current kernels. 2.6.11 is
>>very old in kernel time.
>>
>>Are we likely to see any in-kernel users of this?
>
>
> I added similar support to the pre hw_random AMD8111 driver
> a long time ago. Basically a timer that regularly read some
> dat from the hw random generator and feed it into the random
> code.
>
> I think it is a good idea, because it doesnt make much sense
> imho to run a daemon for something that can be done in 20 lines
> of code in the kernel.

Check your kernel history.

We -used- to need data from RNG directly into the kernel randomness
pool. The consensus was that the FIPS testing should be moved to userspace.

Jeff



2005-03-24 20:44:40

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> hw_random.c already does it using userspace daemons,
> which is bad idea for very fast HW - like VIA xstore/xcrypt
> instructions.

This is incorrect, because it implies that a user would want to use the
'xstore' feature at full speed -- which would dominate the CPU,
drastically slowing down the applications that are actually doing work.

As I mentioned in another message, VIA xstore support should be removed
from hw_random.c and moved completely to userspace rngd.

Jeff


2005-03-24 20:54:02

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> hw_random can not and will not support HIFN, freescale, ixp and
> great majority of the existing and future HW crypto devices.
> I mean that userspace daemon(or any other one) which want to contribute
> entropy
> should use crypto framwork to obtain all it's data, but not different
> access methods for each separate driver.


I don't mean to imply that hw_random should be modified to support all
hardware RNGs. Separate drivers are perfectly OK with me. I would even
take patches that split up hw_random into an RNG registration interface,
and AMD/Intel/VIA support modules.

The part I disagree with is direct kernel->kernel RNG usage, with no
intervening checks.

This has all been discussed years ago, when the FIPS testing was in the
kernel (and then removed, per consensus).

Jeff


2005-03-24 20:52:51

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

David McCullough wrote:
> Jivin Jeff Garzik lays it down ...
>
>>David McCullough wrote:
>>
>>>Jivin Jeff Garzik lays it down ...
>>>
>>>
>>>>On Thu, Mar 24, 2005 at 02:27:08PM +1000, David McCullough wrote:
>>>>
>>>>
>>>>>Hi all,
>>>>>
>>>>>Here is a small patch for 2.6.11 that adds a routine:
>>>>>
>>>>> add_true_randomness(__u32 *buf, int nwords);
>>>>>
>>>>>so that true random number generator device drivers can add a entropy
>>>>>to the system. Drivers that use this can be found in the latest release
>>>>>of ocf-linux, an asynchronous crypto implementation for linux based on
>>>>>the *BSD Cryptographic Framework.
>>>>>
>>>>> http://ocf-linux.sourceforge.net/
>>>>>
>>>>>Adding this can dramatically improve the performance of /dev/random on
>>>>>small embedded systems which do not generate much entropy.
>>>>
>>>>We've already had hardware RNG support for a while now.
>>>>
>>>>No kernel patching needed.
>>>
>>>
>>>Are you talking about /dev/hw_random ? If not then sorry I didn't see it
>>>:-(
>>>
>>>On a lot of the small systems I work on, /dev/random is completely
>>>unresponsive, and all the apps use /dev/random, not /dev/hw_random.
>>>
>>>Would you suggest making /dev/random point to /dev/hw_random then ?
>>
>>All the apps are supposed to use /dev/random, so that's correct.
>
>
> Ok
>
>
>>For Hardware RNGs, userspace rngd daemon obtains entropy, checks it
>>(mainly checking for hardware failures), and then stuffs entropy into
>>the kernel random device. http://sf.net/projects/gkernel/
>>
>>On the "to do" list is making rngd directly generate entropy use
>>'xstore' on VIA CPUs, rather than going kernel -> userland -> kernel.
>>
>>Also, there are other entropy daemons floating about. I think there is
>>one that obtains noise from an audio device.
>
>
> I had looked at hw_random, but it seemed a little platform specific (x86),
> and it doesn't currently have a way for RNG providers to register themselves.
> Admittedly I did not know how it's output was being used in practice.
>
> The drivers I am working with do crypto/public key and RNG. Not all of
> them can easily have the RNG support taken from the driver and plugged
> into hw_random.c, since it is (in most cases) a single PCI chip with
> it's own registers, initialisation and configuration, that, IMO
> belongs in the driver for the particular chip.

Agreed.


> Not that it isn't possible, but hw_random would start supporting a
> much larger number of HW variants and I think it would get ugly.

Agreed.


> It would be possible to add a "register" interface to hw_random so that
> you can register other RNG's easily. This would seem reasonable.

Agreed.


> I work on fairly resource constrained embedded devices a lot of the
> time, so when I can avoid adding applications and reduce kernel size,
> I do. Thus my motivation to add a simple API for adding entropy to
> /dev/random.

We already have the facilities to add entropy, as current use of
hw_random+rngd shows.

Jeff


2005-03-24 20:57:41

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> On Thu, 2005-03-24 at 23:23 +1000, David McCullough wrote:
>
>>Jivin Jeff Garzik lays it down ...
>>
>>>Evgeniy Polyakov wrote:
>>>
>>>>On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
>>>>
>>>>
>>>>>Hi all,
>>>>>
>>>>>Here is a small patch for 2.6.11 that adds a routine:
>>>>>
>>>>> add_true_randomness(__u32 *buf, int nwords);
>>>>>
>>>>>so that true random number generator device drivers can add a entropy
>>>>>to the system. Drivers that use this can be found in the latest release
>>>>>of ocf-linux, an asynchronous crypto implementation for linux based on
>>>>>the *BSD Cryptographic Framework.
>>>>>
>>>>> http://ocf-linux.sourceforge.net/
>>>>>
>>>>>Adding this can dramatically improve the performance of /dev/random on
>>>>>small embedded systems which do not generate much entropy.
>>>>
>>>>
>>>>People will not apply any kind of such changes.
>>>>Both OCF and acrypto already handle all RNG cases - no need for any kind
>>>>of userspace daemon or entropy (re)injection mechanism.
>>>>Anyone who want to use HW randomness may use OCF/acrypto mechanism.
>>>>For example here is patch to enable acrypto support for hw_random.c
>>>>It is very simple and support only upto 4 bytes request, of course it
>>>>is
>>>>not interested for anyone, but it is only 2-minutes example:
>>>
>>>If you want to add entropy to the kernel entropy pool from hardware RNG,
>>>you should use the userland daemon, which detects non-random (broken)
>>>hardware and provides throttling, so that RNG data collection does not
>>>consume 100% CPU.
>>>
>>>If you want to use the hardware RNG directly, it's simple: just open
>>>/dev/hw_random.
>>>
>>>Hardware RNG should not go kernel->kernel without adding FIPS tests and
>>>such.
>>
>>For reference, the RNG on the Safenet I am using this with is
>>FIPS140 certified. I believe the HIFN part is also but I place the doc that
>>says so.
>
>
> At least HIFN 795x is certified.
>
> Idea to validate entropy data is good in general,
> but it should be implemented in a way allowing external both in-kernel
> and userspace
> processes to contribute data.
> So for in-kernel use we need such a mechanism, and userspace gkernel
> daemon
> should use it(as the latest "step") too.

See the earlier discussion, when data validation was -removed- from the
original Intel RNG driver, and moved to userspace.


> Your changes are correct, ioctl(RNDADDENTROPY) could even use it instead
> of direct
> add_entropy_words()/credit_entropy_store(), but without external entropy
> contributors
> it will not be applied by maintainers.

No changes are needed, as the system is currently functional without
these changes.

Jeff


2005-03-24 20:58:29

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

David McCullough wrote:
> Jivin Jeff Garzik lays it down ...
>
>>Evgeniy Polyakov wrote:
>>
>>>On Thu, 2005-03-24 at 14:27 +1000, David McCullough wrote:
>>>
>>>
>>>>Hi all,
>>>>
>>>>Here is a small patch for 2.6.11 that adds a routine:
>>>>
>>>> add_true_randomness(__u32 *buf, int nwords);
>>>>
>>>>so that true random number generator device drivers can add a entropy
>>>>to the system. Drivers that use this can be found in the latest release
>>>>of ocf-linux, an asynchronous crypto implementation for linux based on
>>>>the *BSD Cryptographic Framework.
>>>>
>>>> http://ocf-linux.sourceforge.net/
>>>>
>>>>Adding this can dramatically improve the performance of /dev/random on
>>>>small embedded systems which do not generate much entropy.
>>>
>>>
>>>People will not apply any kind of such changes.
>>>Both OCF and acrypto already handle all RNG cases - no need for any kind
>>>of userspace daemon or entropy (re)injection mechanism.
>>>Anyone who want to use HW randomness may use OCF/acrypto mechanism.
>>>For example here is patch to enable acrypto support for hw_random.c
>>>It is very simple and support only upto 4 bytes request, of course it
>>>is
>>>not interested for anyone, but it is only 2-minutes example:
>>
>>If you want to add entropy to the kernel entropy pool from hardware RNG,
>>you should use the userland daemon, which detects non-random (broken)
>>hardware and provides throttling, so that RNG data collection does not
>>consume 100% CPU.
>>
>>If you want to use the hardware RNG directly, it's simple: just open
>>/dev/hw_random.
>>
>>Hardware RNG should not go kernel->kernel without adding FIPS tests and
>>such.
>
>
> For reference, the RNG on the Safenet I am using this with is
> FIPS140 certified. I believe the HIFN part is also but I place the doc that
> says so.


FIPS certification is not relevant to the discussion.

I am talking about active testing of -all data- that is produced by an
RNG, before making use of it. Please read the source code of rngd.

Jeff


2005-03-24 21:00:57

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Jean-Luc Cooke wrote:
> On Thu, Mar 24, 2005 at 07:48:18AM -0500, Jeff Garzik wrote:
>
>>If you want to add entropy to the kernel entropy pool from hardware RNG,
>>you should use the userland daemon, which detects non-random (broken)
>>hardware and provides throttling, so that RNG data collection does not
>>consume 100% CPU.
>>
>>If you want to use the hardware RNG directly, it's simple: just open
>>/dev/hw_random.
>>
>>Hardware RNG should not go kernel->kernel without adding FIPS tests and
>>such.
>
>
> If your RNG were properly written, it shouldn't matter if the data you're
> pumping into /dev/random passed muster or not. If you're tracking entropy
> count, then that's a different story of course.

It's rather lame to add data, without also crediting entropy.

Further, it wastes many CPU cycles in many places, if you are doing
nothing but pumping bad data (all 1's, for example) into /dev/random.


> I've been commissioned to write Fortuna RNG for Linux and weddings, houses and
> cars not withstanding, I should I it ready soon to be given to LKML for
> digestion.

Sounds great.

Jeff


2005-03-24 21:24:32

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, Mar 24, 2005 at 09:25:40AM -0500, Jean-Luc Cooke wrote:
>
> If your RNG were properly written, it shouldn't matter if the data you're
> pumping into /dev/random passed muster or not. If you're tracking entropy
> count, then that's a different story of course.

We're talking about hardware RNGs here so we need to take hardware faults
into account.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-25 04:19:52

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 15:39 -0500, Jeff Garzik wrote:
> Evgeniy Polyakov wrote:
> > hw_random.c already does it using userspace daemons,
> > which is bad idea for very fast HW - like VIA xstore/xcrypt
> > instructions.
>
> This is incorrect, because it implies that a user would want to use the
> 'xstore' feature at full speed -- which would dominate the CPU,
> drastically slowing down the applications that are actually doing work.

If user want to get RNG data at full speed we do not want to allow it?
Something changed in the world...

User actually do not want to use xstore, but only read from /dev/random.

If kernelspace can assist and driver _knows_ in advance that data
produced is cryptographically strong, why not allow it directly
access pools?

> As I mentioned in another message, VIA xstore support should be removed
> from hw_random.c and moved completely to userspace rngd.
>
> Jeff
>
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 04:28:13

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 15:56 -0500, Jeff Garzik wrote:

> > Idea to validate entropy data is good in general,
> > but it should be implemented in a way allowing external both in-kernel
> > and userspace
> > processes to contribute data.
> > So for in-kernel use we need such a mechanism, and userspace gkernel
> > daemon
> > should use it(as the latest "step") too.
>
> See the earlier discussion, when data validation was -removed- from the
> original Intel RNG driver, and moved to userspace.
>

I'm not arguing against userspace validation, but if data produced
_is_ cryptographically strong, why revalidate it again?

I definitely prefer such mechanism to be implemented, and if you want,
it
can be turned by default off, and can be turned on using new ioctl()
over /dev/random.

> > Your changes are correct, ioctl(RNDADDENTROPY) could even use it instead
> > of direct
> > add_entropy_words()/credit_entropy_store(), but without external entropy
> > contributors
> > it will not be applied by maintainers.
>
> No changes are needed, as the system is currently functional without
> these changes.

And how HIFN driver can contribute entropy?
It needs to create /dev/hifn_random, and rngd should be patched too?
And even using any existing crypto framework why add 2 kernel/user
copying, if we _want_ fast RNG and _know_ that it is valid?

You may say, that hardware can be broken and thus produces
wrong data, but if user want, it can turn it on or off.

What about new ioctl() that will enable/disable entropy contribution
from kernelspace without validating it? I can create a patch on
top of David's.

> Jeff
>
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 04:46:15

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> On Thu, 2005-03-24 at 15:39 -0500, Jeff Garzik wrote:
>
>>Evgeniy Polyakov wrote:
>>
>>>hw_random.c already does it using userspace daemons,
>>>which is bad idea for very fast HW - like VIA xstore/xcrypt
>>>instructions.
>>
>>This is incorrect, because it implies that a user would want to use the
>>'xstore' feature at full speed -- which would dominate the CPU,
>>drastically slowing down the applications that are actually doing work.
>
>
> If user want to get RNG data at full speed we do not want to allow it?
> Something changed in the world...

I agree with this sentiment; this is mainly a policy decision that
kernel programmers should not make.

Certainly _by default_ the RNG should not be run "full blast" all the
time. This is a needless CPU soaker.

This is another example of why the userspace rngd is useful: it is
trivial to implement "CPU soaker" policy if you wish, or use the default
"don't eat all my CPU" policy.


> User actually do not want to use xstore, but only read from /dev/random.

That's a policy decision to be made by the user, not you.

Some users may wish to use RNG directly.


> If kernelspace can assist and driver _knows_ in advance that data
> produced is cryptographically strong, why not allow it directly
> access pools?

A kernel driver cannot know in advance that the data from a hardware RNG
is truly random, unless the data itself is 100% validated beforehand.

Jeff


2005-03-25 04:49:32

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> On Thu, 2005-03-24 at 15:56 -0500, Jeff Garzik wrote:
>>See the earlier discussion, when data validation was -removed- from the
>>original Intel RNG driver, and moved to userspace.
>
> I'm not arguing against userspace validation, but if data produced
> _is_ cryptographically strong, why revalidate it again?

You cannot prove this without validating the data in software.

Otherwise, you are not handling the hardware-fault case.

It is foolish to presume that hardware always works correctly. It is
-very- foolish to presume this, in cryptography.


> And how HIFN driver can contribute entropy?

Use the current chrdev->rngd method.


> You may say, that hardware can be broken and thus produces
> wrong data, but if user want, it can turn it on or off.

The user cannot know the data is bad unless it is constantly being
validated.

Jeff


2005-03-25 05:27:12

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 23:48 -0500, Jeff Garzik wrote:

> > And how HIFN driver can contribute entropy?
>
> Use the current chrdev->rngd method.

Why HIFN must be chardev?

> > You may say, that hardware can be broken and thus produces
> > wrong data, but if user want, it can turn it on or off.
>
> The user cannot know the data is bad unless it is constantly being
> validated.

The user can not use HW crypto processors, since he does not
know if HW is broken or not, and thus must validate each crypto
operation, i.e. reencrypt data in SW.

Not the point.

Validation can be performed in other HW
(like Xilinx which routes HW requests to the real devices),
or in driver (if it is not FIPS validation).

So I still insist on creating ability to contribute entropy directly,
without userspace validation.
It will be turned off by default.

> Jeff

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 05:47:59

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, 2005-03-24 at 14:59 +0300, Evgeniy Polyakov wrote:

> For example here is patch to enable acrypto support for hw_random.c
> It is very simple and support only upto 4 bytes request, of course it
> is not interested for anyone, but it is only 2-minutes example:

Full port.

--- ./drivers/char/hw_random.c.orig 2005-03-24 13:36:05.000000000 +0300
+++ ./drivers/char/hw_random.c 2005-03-25 08:46:03.841601032 +0300
@@ -34,6 +34,10 @@
#include <linux/smp_lock.h>
#include <linux/mm.h>
#include <linux/delay.h>
+#include <linux/acrypto.h>
+#include <linux/crypto_def.h>
+#include <linux/crypto_stat.h>
+#include <linux/highmem.h>

#ifdef __i386__
#include <asm/msr.h>
@@ -73,6 +77,8 @@
#endif

#define RNG_MISCDEV_MINOR 183 /* official */
+
+static DEFINE_SPINLOCK(rng_lock);

static int rng_dev_open (struct inode *inode, struct file *filp);
static ssize_t rng_dev_read (struct file *filp, char __user *buf, size_t size,
@@ -482,7 +488,6 @@
static ssize_t rng_dev_read (struct file *filp, char __user *buf, size_t size,
loff_t * offp)
{
- static DEFINE_SPINLOCK(rng_lock);
unsigned int have_data;
u32 data = 0;
ssize_t ret = 0;
@@ -526,7 +531,163 @@
return ret;
}

+#ifdef CONFIG_ACRYPTO
+static struct crypto_device *hwr_cdev;
+static struct crypto_capability hwr_caps[] = {
+ {CRYPTO_OP_RNG, 0, 0, 100},
+};
+static int hwr_pid, hwr_need_exit;
+static struct completion hwr_thread_exited;
+static DECLARE_WAIT_QUEUE_HEAD(hwr_wait);
+
+
+static void hwr_data_ready(struct crypto_device *dev)
+{
+ wake_up(&hwr_wait);
+}
+
+static int hwr_process(void *data)
+{
+ struct crypto_device *dev = data;
+ struct crypto_session *s, *n;
+ u32 rng_data = 0;
+ unsigned int have_data, size;
+ int i;
+ u8 *ptr;
+
+ daemonize("%s", dev->name);
+ allow_signal(SIGTERM);
+
+ while (!hwr_need_exit) {
+ interruptible_sleep_on_timeout(&hwr_wait, 10);
+
+ list_for_each_entry_safe(s, n, &dev->session_list, dev_queue_entry) {
+ if (!session_completed(s) && !session_is_processed(s)) {
+ start_process_session(s);
+
+ if (s->data.sg_src_num != s->data.sg_dst_num) {
+ dprintk("%s: session %llu [%llu]: different src/dst sg numbers: %d %d.\n",
+ dev->name, s->ci.id, s->ci.dev_id,
+ s->data.sg_src_num, s->data.sg_dst_num);
+ broke_session(s);
+ goto out_complete_session;
+ }
+
+ for (i=0; i<s->data.sg_src_num; ++i) {
+ if (s->data.sg_dst[i].length != s->data.sg_src[i].length) {
+ dprintk("%s: session %llu [%llu]: sg %d different src/dst lengths: %u %u.\n",
+ dev->name, s->ci.id, s->ci.dev_id, i,
+ s->data.sg_src[i].length, s->data.sg_dst[i].length);
+ if (s->data.sg_dst[i].length)
+ s->data.sg_src[i].length = s->data.sg_dst[i].length;
+ else
+ s->data.sg_dst[i].length = s->data.sg_src[i].length;
+
+ }
+
+ size = s->data.sg_dst[i].length;
+
+ while (size) {
+ spin_lock(&rng_lock);
+ have_data = 0;
+ if (rng_ops->data_present()) {
+ rng_data = rng_ops->data_read();
+ have_data = rng_ops->n_bytes;
+ }
+ spin_unlock (&rng_lock);
+
+ ptr = kmap_atomic(s->data.sg_dst[i].page, KM_USER0) + s->data.sg_dst[i].offset +
+ s->data.sg_dst[i].length - size;
+
+ while (size && have_data) {
+ *ptr = rng_data & 0xff;
+ size--;
+ have_data--;
+ rng_data >>= 8;
+ }
+ kunmap_atomic(ptr, KM_USER0);
+
+ if (size)
+ msleep_interruptible(1);
+ }
+ }
+
+ crypto_stat_complete_inc(s);
+
+out_complete_session:
+ crypto_session_dequeue_route(s);
+ complete_session(s);
+ stop_process_session(s);
+ }
+ }
+ }
+
+ complete_and_exit(&hwr_thread_exited, 0);
+}
+
+static int hwr_acrypto_init(struct rng_operations *ops)
+{
+ int err;
+
+ hwr_cdev = kmalloc(sizeof(*hwr_cdev), GFP_KERNEL);
+ if (!hwr_cdev) {
+ printk(KERN_ERR "Failed to allocate new crypto_device structure.\n");
+ return -ENOMEM;
+ }
+
+ memset(hwr_cdev, 0, sizeof(*hwr_cdev));
+
+ hwr_cdev->cap = hwr_caps;
+ hwr_cdev->cap_number = sizeof(hwr_caps)/sizeof(hwr_caps[0]);
+ hwr_cdev->priv = ops;
+ hwr_cdev->data_ready = &hwr_data_ready;
+ snprintf(hwr_cdev->name, sizeof(hwr_cdev->name), "%s", "hwr");
+
+ init_completion(&hwr_thread_exited);
+ hwr_pid = kernel_thread(hwr_process, hwr_cdev, CLONE_FS | CLONE_FILES);
+ if (hwr_pid < 0) {
+ err = hwr_pid;
+ goto err_out_free_cdev;
+ }

+ err = crypto_device_add(hwr_cdev);
+ if (err)
+ goto err_out_remove_thread;
+
+ printk(KERN_INFO "%s acrypto support is turned on.\n", hwr_cdev->name);
+
+err_out_remove_thread:
+ hwr_need_exit = 1;
+ kill_proc(hwr_pid, SIGTERM, 0);
+ wait_for_completion(&hwr_thread_exited);
+err_out_free_cdev:
+ kfree(hwr_cdev);
+ hwr_cdev = NULL;
+ return err;
+}
+
+static void hwr_acrypto_fini(void)
+{
+ crypto_device_remove(hwr_cdev);
+ printk(KERN_INFO "%s acrypto support is turned off.\n", hwr_cdev->name);
+
+ hwr_need_exit = 1;
+ kill_proc(hwr_pid, SIGTERM, 0);
+ wait_for_completion(&hwr_thread_exited);
+
+ kfree(hwr_cdev);
+ hwr_cdev = NULL;
+}
+#else
+static int hwr_acrypto_init(struct rng_operations *ops)
+{
+ return 0;
+}
+
+static void hwr_acrypto_fini(void)
+{
+}
+#endif

/*
* rng_init_one - look for and attempt to init a single RNG
@@ -549,9 +710,15 @@
goto err_out_cleanup_hw;
}

+ rc = hwr_acrypto_init(rng_ops);
+ if (rc)
+ goto err_out_misc_unregister;
+
DPRINTK ("EXIT, returning 0\n");
return 0;

+err_out_misc_unregister:
+ misc_deregister(&rng_miscdev);
err_out_cleanup_hw:
rng_ops->cleanup();
err_out:
@@ -617,6 +784,8 @@
{
DPRINTK ("ENTER\n");

+ hwr_acrypto_fini();
+
misc_deregister (&rng_miscdev);

if (rng_ops->cleanup)


--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 05:49:30

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Thu, Mar 24, 2005 at 11:45:53PM -0500, Jeff Garzik wrote:
>
> I agree with this sentiment; this is mainly a policy decision that
> kernel programmers should not make.

Exactly. Policy decisions like this as well as entropy checking
should be done in user-space.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-25 06:01:24

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> So I still insist on creating ability to contribute entropy directly,
> without userspace validation.
> It will be turned off by default.

If its disabled by default, then you and 2-3 other people will use this
feature. Not enough justification for a kernel API at that point.

Jeff


2005-03-25 06:18:31

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 09:16:01AM +0300, Evgeniy Polyakov wrote:
> On Fri, 2005-03-25 at 00:58 -0500, Jeff Garzik wrote:
>
> > If its disabled by default, then you and 2-3 other people will use this
> > feature. Not enough justification for a kernel API at that point.
>
> It is only because there are only couple of HW crypto devices
> in the tree, with one crypto framework inclusion there will be
> at least redouble.

You missed the point. This has nothing to do with the crypto API.
Jeff is saying that if this is disabled by default, then only a few
users will enable it and therefore use this API.

Since we can't afford to enable it by default as hardware RNG may
fail which can lead to catastrophic consequences, there is no point
for this API at all.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-25 06:21:54

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, 2005-03-25 at 00:58 -0500, Jeff Garzik wrote:
> Evgeniy Polyakov wrote:
> > So I still insist on creating ability to contribute entropy directly,
> > without userspace validation.
> > It will be turned off by default.
>
> If its disabled by default, then you and 2-3 other people will use this
> feature. Not enough justification for a kernel API at that point.

It is only because there are only couple of HW crypto devices
in the tree, with one crypto framework inclusion there will be
at least redouble.
Let's return to this discussion after it.

Thank you.

> Jeff
>
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 06:29:08

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, 2005-03-25 at 17:13 +1100, Herbert Xu wrote:
> On Fri, Mar 25, 2005 at 09:16:01AM +0300, Evgeniy Polyakov wrote:
> > On Fri, 2005-03-25 at 00:58 -0500, Jeff Garzik wrote:
> >
> > > If its disabled by default, then you and 2-3 other people will use this
> > > feature. Not enough justification for a kernel API at that point.
> >
> > It is only because there are only couple of HW crypto devices
> > in the tree, with one crypto framework inclusion there will be
> > at least redouble.
>
> You missed the point. This has nothing to do with the crypto API.
> Jeff is saying that if this is disabled by default, then only a few
> users will enable it and therefore use this API.
>
> Since we can't afford to enable it by default as hardware RNG may
> fail which can lead to catastrophic consequences, there is no point
> for this API at all.

Currently implemented in-tree drivers(hw_random.c, do not have spec
about
VIA) do not perform any kind of validation, drivers created for
OCF/acrypto
have HW validated RNG.
Such hardware is used mostly in embedded world where SW crypto
processing
is too expensive, so users of such HW likely want to trust to
theirs hardware and likely will turn in on.
That would be even be a good idea to have two way of turning it on -
kernel config option and ioctl() one - to allow embedded systems
with too limited userspace not change it's applications.
Of course with big fat warning about possible dramatical consequences.

> Cheers,
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 06:35:20

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 09:34:19AM +0300, Evgeniy Polyakov wrote:
>
> Such hardware is used mostly in embedded world where SW crypto
> processing
> is too expensive, so users of such HW likely want to trust to
> theirs hardware and likely will turn in on.

That's fine. All you need for these embedded users is a user-space
daemon that feeds data from the hardware directly into /dev/random.
No matter how small your system is, I'm sure you can spare a few
hundred bytes for such a thing.

In fact most of these systems will have some sort of a general-purpose
daemon that sits around which can perform such a task.

System calls on Linux are fast enough that there is really no
advantage in doing this in the kernel.

But if you're really desparate, write a kernel module that does this
in a kernel thread.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-25 06:53:38

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, 2005-03-25 at 17:33 +1100, Herbert Xu wrote:
> On Fri, Mar 25, 2005 at 09:34:19AM +0300, Evgeniy Polyakov wrote:
> >
> > Such hardware is used mostly in embedded world where SW crypto
> > processing
> > is too expensive, so users of such HW likely want to trust to
> > theirs hardware and likely will turn in on.
>
> That's fine. All you need for these embedded users is a user-space
> daemon that feeds data from the hardware directly into /dev/random.
> No matter how small your system is, I'm sure you can spare a few
> hundred bytes for such a thing.
>
> In fact most of these systems will have some sort of a general-purpose
> daemon that sits around which can perform such a task.
>
> System calls on Linux are fast enough that there is really no
> advantage in doing this in the kernel.
>
> But if you're really desparate, write a kernel module that does this
> in a kernel thread.

It is not only about userspace/kernelspace system calls and data
copying,
but about whole revalidation process, which can and is quite expensive,
due to system calls, copying and validating itself,
I even think that using userspace rng daemon is completely useless for
crypto HW devices - it is faster to obtain entropy from interrupts,
than revalidating it in that way.
And what about initial bootup? When system needs to create randoom
IP/dhcp/any ids? What about small router?
There are too many cases where userspace validation is just making
things worse.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 06:58:41

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 09:59:18AM +0300, Evgeniy Polyakov wrote:
>
> It is not only about userspace/kernelspace system calls and data
> copying,
> but about whole revalidation process, which can and is quite expensive,
> due to system calls, copying and validating itself,

What I meant is if you don't need the revalidation then don't do it.
That's the advantage of having it in user-space, *you* get to decide,
not us.

> And what about initial bootup? When system needs to create randoom
> IP/dhcp/any ids? What about small router?

Let's not reinvent the wheel, this is exactly what initramfs is for.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-25 07:14:11

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, 2005-03-25 at 17:56 +1100, Herbert Xu wrote:
> On Fri, Mar 25, 2005 at 09:59:18AM +0300, Evgeniy Polyakov wrote:
> >
> > It is not only about userspace/kernelspace system calls and data
> > copying,
> > but about whole revalidation process, which can and is quite expensive,
> > due to system calls, copying and validating itself,
>
> What I meant is if you don't need the revalidation then don't do it.
> That's the advantage of having it in user-space, *you* get to decide,
> not us.

One can not add entropy data directly to the pool from kernelspace now.
But now noone may do it, since all presented data is not validated.
So when there will be validated HW RNG they still need to pass
it's data through userspace validation daemon
(which btw makes tens to hundreds operations per bit according to FIPS).

> > And what about initial bootup? When system needs to create randoom
> > IP/dhcp/any ids? What about small router?
>
> Let's not reinvent the wheel, this is exactly what initramfs is for.

It is not panacea and even not always working solution.

If user turn that feature on - he is on his own.
Noone will complain on Linux if NIC is broken and produces wrong
checksum
and HW checksum offloading is enabled using ethtools.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 07:19:47

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Evgeniy Polyakov wrote:
> Noone will complain on Linux if NIC is broken and produces wrong
> checksum
> and HW checksum offloading is enabled using ethtools.


Actually, that is a problem and people have definitely complained about
it in the past.

Jeff


2005-03-25 07:26:49

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 10:19:55AM +0300, Evgeniy Polyakov wrote:
>
> Noone will complain on Linux if NIC is broken and produces wrong
> checksum
> and HW checksum offloading is enabled using ethtools.

This is completely different. The worst that can happen with checksum
offloading is that the packet is dropped. That's something people deal
with on a daily basis since the Internet as a whole does not guarantee
the delivery of packets.

On the other hand, /dev/random is something that has always promised
to deliver random numbers that are totally unpredictable. People out
there *depend* on this.

If that assumption is violated the result could be catastrophic.

That's why it's OK to have hardware RNG spit out unverified numbers
in /dev/hw_random, but it's absolutely unaccpetable for the same
numbers to add entropy to /dev/random without verification.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-25 07:31:53

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, 2005-03-25 at 02:19 -0500, Jeff Garzik wrote:
> Evgeniy Polyakov wrote:
> > Noone will complain on Linux if NIC is broken and produces wrong
> > checksum
> > and HW checksum offloading is enabled using ethtools.
>
>
> Actually, that is a problem and people have definitely complained about
> it in the past.

And what they were recommended to do? :)
I believe not changing drivers and stack, but only disable it using
ethtool.

And people of course should be able to turn kernlspace <-> kernelspace
RNG dataflow off if they fill it is insecure.

> Jeff
>
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 07:52:27

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, 2005-03-25 at 18:25 +1100, Herbert Xu wrote:
> On Fri, Mar 25, 2005 at 10:19:55AM +0300, Evgeniy Polyakov wrote:
> >
> > Noone will complain on Linux if NIC is broken and produces wrong
> > checksum
> > and HW checksum offloading is enabled using ethtools.
>
> This is completely different. The worst that can happen with checksum
> offloading is that the packet is dropped. That's something people deal
> with on a daily basis since the Internet as a whole does not guarantee
> the delivery of packets.

It will just completely stop network dataflow.
It is of course not as catastrophic as removing strong random numbers
from system.
But nevertheless - write cahce in disks may corrupt data on power-down,
but people do not turn it off, crypto HW can be broken and does not
encrypt dataflow, but people want it, broken NIC can corrupt data with
various sg/offload combinations, but it can be enabled.

It is a feature, that _may_ broke thing badly.
But if all is ok - it is extremly usefull.

And as I said there may be other [HW/driver] validating techniques,
not only userspace daemon.

> On the other hand, /dev/random is something that has always promised
> to deliver random numbers that are totally unpredictable. People out
> there *depend* on this.
>
> If that assumption is violated the result could be catastrophic.
>
> That's why it's OK to have hardware RNG spit out unverified numbers
> in /dev/hw_random, but it's absolutely unaccpetable for the same
> numbers to add entropy to /dev/random without verification.

Userspace daemon can read data from /dev/random and validate it
in background, if it fells it is broken - turn feature off.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-25 23:44:31

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 04:50:16PM -0600, Kim Phillips wrote:
> I did some tests and found that the udelay(200) call in hw_random.c is
> not good for extreme RNG consumption. Whether or not such extremes
> occur in real life, on the mpc8541, /dev/hwrandom is an order of
> magnitude slower than /dev/random, and two orders of magnitude slower
> than /dev/urandom. The hardware RNG is capable of exceeding software
> /dev/random speeds plus it would alleviate the core to do other, more
> useful things (that's being realistic).

Consider what an RNG does: spews garbage.

In practical applications, you -do not- want to dedicate the machine to
spewing garbage. The vast majority of users would prefer to use their
machines for real stuff. Thus, "extreme RNG consumption" is largely
irrelevant to sane usage.

That said, I cannot remember if the udelay() is hardcoding a policy
decision (bad), or required for the hardware.

In any case, it is the wrong solution to simply "turn on the tap" and
let the RNG spew data. There needs to be a limiting factor... typically
rngd should figure out when /dev/random needs more entropy, or simply
delay a little bit between entropy collection/stuffing runs.

Jeff



2005-03-25 23:51:02

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 06:43:49PM -0500, Jeff Garzik wrote:
>
> In any case, it is the wrong solution to simply "turn on the tap" and
> let the RNG spew data. There needs to be a limiting factor... typically
> rngd should figure out when /dev/random needs more entropy, or simply
> delay a little bit between entropy collection/stuffing runs.

Completely agreed. Having it in rngd also allows the scheduler to
do its job.

When applications need entropy from /dev/random and they can't get it,
they'll simply block which allows rngd to run to refill the tank.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-26 00:21:02

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Sat, 26 Mar 2005 10:47:45 +1100
Herbert Xu <[email protected]> wrote:

> On Fri, Mar 25, 2005 at 06:43:49PM -0500, Jeff Garzik wrote:
> >
> > In any case, it is the wrong solution to simply "turn on the tap" and
> > let the RNG spew data. There needs to be a limiting factor... typically
> > rngd should figure out when /dev/random needs more entropy, or simply
> > delay a little bit between entropy collection/stuffing runs.
>
> Completely agreed. Having it in rngd also allows the scheduler to
> do its job.

It looks like we all misunderstand each other -
why do you think that if there will be kernel <-> kernel
RNG dataflow, then system will continuously spent all
it's time to produce that data?
_Ability_ existence does not mean that only it must be used.
Userspace daemon should be able to turn it on or off,
but it is too expensive to allow it to be not only dataflow
controller, but the only random numbers dataflow initiator.

I can create following patch on top of rngd -
it will read from /dev/random, if read succeds too fast
(or even better just to check pool counts), then rngd
will turn HW RNG assist off and examine received data
to check if it is valid.
Later it can be turned on again.

> When applications need entropy from /dev/random and they can't get it,
> they'll simply block which allows rngd to run to refill the tank.

Such a blocking will be definitely a sign to turn
HW RNG assist on.

> --
> Visit Openswan at http://www.openswan.org/
> Email: Herbert Xu ~{PmV>HI~} <[email protected]>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Evgeniy Polyakov

Only failure makes us experts. -- Theo de Raadt

2005-03-26 00:38:38

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Sat, Mar 26, 2005 at 03:47:33AM +0300, Evgeniy Polyakov wrote:
>
> It looks like we all misunderstand each other -
> why do you think that if there will be kernel <-> kernel
> RNG dataflow, then system will continuously spent all
> it's time to produce that data?

It doesn't matter whether it's like that or not.

The point is if you do it in the kernel then either you'll have very
coarse controls over the rate of data coming out of the hardware RNG,
e.g., only on/off, or you'll have to put more code in to set the rate
appropriately.

Either way it's a loss compared to doing it in user-space.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-26 08:26:32

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Sat, 26 Mar 2005 11:36:02 +1100
Herbert Xu <[email protected]> wrote:

> On Sat, Mar 26, 2005 at 03:47:33AM +0300, Evgeniy Polyakov wrote:
> >
> > It looks like we all misunderstand each other -
> > why do you think that if there will be kernel <-> kernel
> > RNG dataflow, then system will continuously spent all
> > it's time to produce that data?
>
> It doesn't matter whether it's like that or not.
>
> The point is if you do it in the kernel then either you'll have very
> coarse controls over the rate of data coming out of the hardware RNG,
> e.g., only on/off, or you'll have to put more code in to set the rate
> appropriately.

It is not the problem absolutely - just sleep if pool is full.
It will limit usage but this is better than nothing.

> Either way it's a loss compared to doing it in user-space.
>
>
> --
> Visit Openswan at http://www.openswan.org/
> Email: Herbert Xu ~{PmV>HI~} <[email protected]>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


Evgeniy Polyakov

Only failure makes us experts. -- Theo de Raadt

2005-03-27 17:19:40

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

> We -used- to need data from RNG directly into the kernel randomness

Are you sure? I dont think there was ever code to do this in
mainline. There might have been something in -ac*, but not mainline.

> pool. The consensus was that the FIPS testing should be moved to userspace.

Consensus from whom? And who says the FIPS testing is useful anyways?

I think you just need to trust the random generator, it is like
you need to trust any other piece of hardware in your machine. Or do you
check regularly if you mov instruction still works? @)

I think it is a trade off between easy to use and saving of
resources and overly paranoia. With an user space solution
which near nobody uses currently (I am not aware of
any distribution that runs that daemon)
it means most people wont have hardware supported randomness
in their ssh, and I think that is a big drawback.

Also I dont like the memory consumption of the daemon. It needs
at least 20+k for kernel stack, page tables etc. I know
a lot of people dont care about memory usage anymore, but I still
do. It is not a lot of memory, but bloat does usually not come in big
pieces but in small amounts of a time. And the code to do it
from kernel space is really simple.

And it would suddenly make a lot of peoples ssh/https etc. more secure,
which is a good thing. Probably would help Linux security a lot
more than all these crazy - "ABI, what ABI?" - buffer overflow
workarounds.

If you are really paranoid you can always turn off the sysctl
and do it from userspace.

-Andi

2005-03-27 18:55:30

by folkert

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

> > pool. The consensus was that the FIPS testing should be moved to userspace.
> Consensus from whom? And who says the FIPS testing is useful anyways?
> I think you just need to trust the random generator, it is like
> you need to trust any other piece of hardware in your machine. Or do you
> check regularly if you mov instruction still works? @)

For joe-user imho it's better to do a check from a cronjob once a day. But for
high demand security, maybe make it pluggable? Like that a user can plug-in some
module which does the testing? Then you can have several kinds of tests
depending on your needs.


Folkert van Heusden

Auto te koop! Zie: http://www.vanheusden.com/daihatsu.php
Op zoek naar een IT of Finance baan? Mail me voor de mogelijkheden!
+------------------------------------------------------------------+
|UNIX admin? Then give MultiTail (http://vanheusden.com/multitail/)|
|a try, it brings monitoring logfiles to a different level! See |
|http://vanheusden.com/multitail/features.html for a feature list. |
+------------------------------------------= http://www.unixsoftware.nl =-+
Phone: +31-6-41278122, PGP-key: 1F28D8AE
Get your PGP/GPG key signed at http://www.biglumber.com!


Attachments:
(No filename) (1.17 kB)
signature.asc (189.00 B)
Digital signature
Download all attachments

2005-03-28 13:57:49

by Jean-Luc Cooke

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Fri, Mar 25, 2005 at 06:43:49PM -0500, Jeff Garzik wrote:
>
> Consider what an RNG does: spews garbage.
>
> In practical applications, you -do not- want to dedicate the machine to
> spewing garbage. The vast majority of users would prefer to use their
> machines for real stuff. Thus, "extreme RNG consumption" is largely
> irrelevant to sane usage.

I have clients who run online Casinos. So spewing lots of reliable garbage
is a good thing. That's why they chose Fortuna. Bad random data input is
not a factor as long as there is enough trustwothy random data coming in.

JLC

2005-03-28 15:20:51

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Sun, Mar 27, 2005 at 08:55:03PM +0200, [email protected] wrote:
> > > pool. The consensus was that the FIPS testing should be moved to userspace.
> > Consensus from whom? And who says the FIPS testing is useful anyways?
> > I think you just need to trust the random generator, it is like
> > you need to trust any other piece of hardware in your machine. Or do you
> > check regularly if you mov instruction still works? @)
>
> For joe-user imho it's better to do a check from a cronjob once a day. But for
> high demand security, maybe make it pluggable? Like that a user can plug-in some
> module which does the testing? Then you can have several kinds of tests
> depending on your needs.

In my old 2.4 patch there was a sysctl to turn off the kernel reseeding.
If you turn it off you can do it in user space. That might be
an option for the clinical paranoid.

BTW what do you do when the FIPS test fails? I dont see a good fallback
path for this case.

-Andi

2005-03-28 15:26:19

by folkert

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

> > For joe-user imho it's better to do a check from a cronjob once a day. But for
> > high demand security, maybe make it pluggable? Like that a user can plug-in some
> > module which does the testing? Then you can have several kinds of tests
> > depending on your needs.
> In my old 2.4 patch there was a sysctl to turn off the kernel reseeding.
> If you turn it off you can do it in user space. That might be
> an option for the clinical paranoid.
> BTW what do you do when the FIPS test fails? I dont see a good fallback
> path for this case.

Send a message to klogd and let read() block untill the test no longer fails.


Folkert van Heusden

Auto te koop! Zie: http://www.vanheusden.com/daihatsu.php
Op zoek naar een IT of Finance baan? Mail me voor de mogelijkheden!
+------------------------------------------------------------------+
|UNIX admin? Then give MultiTail (http://vanheusden.com/multitail/)|
|a try, it brings monitoring logfiles to a different level! See |
|http://vanheusden.com/multitail/features.html for a feature list. |
+------------------------------------------= http://www.unixsoftware.nl =-+
Phone: +31-6-41278122, PGP-key: 1F28D8AE
Get your PGP/GPG key signed at http://www.biglumber.com!

2005-03-28 21:35:34

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Mon, Mar 28, 2005 at 08:45:10AM -0500, Jean-Luc Cooke wrote:
>
> I have clients who run online Casinos. So spewing lots of reliable garbage
> is a good thing. That's why they chose Fortuna. Bad random data input is
> not a factor as long as there is enough trustwothy random data coming in.

I don't think the Casino operators will be very happy when their supposed
hardware RNGs degenerate into software RNGs because it's feeding all zeros
into /dev/random and nobody noticed...
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 01:35:34

by Matt Mackall

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Wed, Mar 23, 2005 at 09:32:26PM -0800, Andrew Morton wrote:
> Jeff Garzik <[email protected]> wrote:
> >
> > > It neither applies correctly nor compiles in current kernels. 2.6.11 is
> > > very old in kernel time.
> >
> > Hrm. This is getting pretty lame, if you can't take patches from the
> > -latest- stable release. It's pretty easy in BK:
> >
> > bk clone -ql -rv2.6.11 linux-2.6 rng-2.6.11
> > cd rng-2.6.11
> > { apply patch }
> > bk pull ../linux-2.6
> >
> > Can you set up something like that?
>
> About thirty patches have gone into random.c since 2.6.11. But the patch
> was easy enough to apply anyway.
>
> And then, it didn't compile. I don't think bk will fix that.

No, the names of all the pools changed. I agree with Jeff, this patch
is unnecessary. If we actually wanted such an interface, I'd rather it
refactor things so as not to reproduce the wake up logic.

--
Mathematics is the supreme nostalgia of our time.

2005-03-29 07:28:37

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

[email protected] wrote:
>>>pool. The consensus was that the FIPS testing should be moved to userspace.
>>
>>Consensus from whom? And who says the FIPS testing is useful anyways?
>>I think you just need to trust the random generator, it is like
>>you need to trust any other piece of hardware in your machine. Or do you
>>check regularly if you mov instruction still works? @)
>
>
> For joe-user imho it's better to do a check from a cronjob once a day. But for

That would not catch the hardware failures seen in the field, in the past.

Jeff



2005-03-29 07:37:49

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Andi Kleen wrote:
> BTW what do you do when the FIPS test fails? I dont see a good fallback
> path for this case.

If the FIPS test fails, do the obvious: don't feed that data to the
kernel (and credit entropy), and possibly stop using the hardware RNG
under a human has intervened.

This is not rocket science. The fallback path is "use software", which
is what most users do right now anyway.

Jeff

2005-03-29 07:47:46

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Andi Kleen wrote:
>>We -used- to need data from RNG directly into the kernel randomness
>
>
> Are you sure? I dont think there was ever code to do this in
> mainline. There might have been something in -ac*, but not mainline.

Yes, I am positive. I wrote the code. Look at the old Intel RNG driver
code, before it grew AMD and VIA support, and became hw_random.


>>pool. The consensus was that the FIPS testing should be moved to userspace.
>
>
> Consensus from whom? And who says the FIPS testing is useful anyways?

lkml. Read the archives.

> I think you just need to trust the random generator, it is like
> you need to trust any other piece of hardware in your machine. Or do you
> check regularly if you mov instruction still works? @)

Hardware RNGs -have- failed in the past. And if you are going to credit
entropy to the data -- a very big deal -- it damn well better be random
data. Otherwise failures cascade through the system.


> I think it is a trade off between easy to use and saving of
> resources and overly paranoia. With an user space solution
> which near nobody uses currently (I am not aware of
> any distribution that runs that daemon)

Debian does.

It's under-use is mainly because nobody has an RNG.


> it means most people wont have hardware supported randomness
> in their ssh, and I think that is a big drawback.

"big drawback" == 99% of users right now.

Jeff


2005-03-29 10:19:20

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Hi!


> >>See the earlier discussion, when data validation was -removed- from the
> >>original Intel RNG driver, and moved to userspace.
> >
> >I'm not arguing against userspace validation, but if data produced
> >_is_ cryptographically strong, why revalidate it again?
>
> You cannot prove this without validating the data in software.
>
> Otherwise, you are not handling the hardware-fault case.
>
> It is foolish to presume that hardware always works correctly. It is
> -very- foolish to presume this, in cryptography.

We trust hardware, anyway. Like your disk *could* accidentaly turn on
setuid bit on /bin/bash, and we do not insist on userspace
disk-validator.

I do not think paranoia about random generators is neccessary. If
vendor provides you with random generator, it should be ok to just use
it. [Did anyone see failing hw random generator, *at all*?] I can
provide you with plenty of failing hdds....
Pavel
--
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

2005-03-29 10:25:04

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Hi!

On P? 25-03-05 18:25:31, Herbert Xu wrote:
> On Fri, Mar 25, 2005 at 10:19:55AM +0300, Evgeniy Polyakov wrote:
> >
> > Noone will complain on Linux if NIC is broken and produces wrong
> > checksum
> > and HW checksum offloading is enabled using ethtools.
>
> This is completely different. The worst that can happen with checksum
> offloading is that the packet is dropped. That's something people deal
> with on a daily basis since the Internet as a whole does not guarantee
> the delivery of packets.
>
> On the other hand, /dev/random is something that has always promised
> to deliver random numbers that are totally unpredictable. People out
> there *depend* on this.
>
> If that assumption is violated the result could be catastrophic.

Yes, and we had huge security hole where it sometimes gave same random
data to two different processes on smp, and noone noticed for 10
years.... /dev/random is not as critical as you paint it.

Pavel
--
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

2005-03-29 10:30:35

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 12:18:16PM +0200, Pavel Machek wrote:
>
> I do not think paranoia about random generators is neccessary. If
> vendor provides you with random generator, it should be ok to just use
> it. [Did anyone see failing hw random generator, *at all*?] I can
> provide you with plenty of failing hdds....

We've been through this before. It's not a question of trusting the
hardware to be /dev/hw_random, it's a question of trusting it to be
/dev/random.

/dev/random is special in that we've gone to extraordinary lengths to
make sure that it contains the amount of entropy that we say it does.

So while it'd be perfectly OK to feed unverified data through
/dev/hw_random, the same cannot be done for /dev/random.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 10:34:29

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 12:21:04PM +0200, Pavel Machek wrote:
>
> What catastrophic consequences? Noone is likely to even *notice*, and
> it does not help practical attack at all. Unless hardware RNGs are
> *very* flakey (like, more flakey than harddrives), this is not a problem.

The reason some people use hardware RNGs in the first place is because
they don't trust the software RNGs. When the hardware RNG fails but
continues to send data to /dev/random, /dev/random essentially degenerates
into a software RNG. Now granted /dev/random is a pretty good software
RNG, however, for some purposes it just isn't good enough.

Otherwise we can just do away with it and always use /dev/urandom.

Someone else raised the example of Casinos using hardware RNGs. Some
of them are doing this to comply with government regulation. In that
case, using data from the software RNG when the hardware has failed
would be violating the law.

> I can assure you that failing hdd will have more catastrophic
> consequences.

That's we have things like RAID and backups.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 10:42:22

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Hi!

> > What catastrophic consequences? Noone is likely to even *notice*, and
> > it does not help practical attack at all. Unless hardware RNGs are
> > *very* flakey (like, more flakey than harddrives), this is not a problem.
>
> The reason some people use hardware RNGs in the first place is because
> they don't trust the software RNGs. When the hardware RNG fails but
> continues to send data to /dev/random, /dev/random essentially degenerates
> into a software RNG. Now granted /dev/random is a pretty good software
> RNG, however, for some purposes it just isn't good enough.

It seems to me that people wanting this level of assurance should do
their own FIPS (or whatever) tests.

Interrupts are not totally unpredictable, either, yet noone runs FIPS
tests on them. I'd say that hardware generator is still better than
interrupt timing. If someone really wants casino-level of randomness,
they should do it all in userspace, probably off interrupt entropy
sources, and do their own FIPS testing.

> Otherwise we can just do away with it and always use /dev/urandom.
>
> Someone else raised the example of Casinos using hardware RNGs. Some
> of them are doing this to comply with government regulation. In that
> case, using data from the software RNG when the hardware has failed
> would be violating the law.

Well, you are still using hardware RNG, but failed one. I do not see
how you can break law with that... given that hardware RNG had proper
certification in the first place.
Pavel
--
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

2005-03-29 10:43:55

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On P? 25-03-05 17:13:11, Herbert Xu wrote:
> On Fri, Mar 25, 2005 at 09:16:01AM +0300, Evgeniy Polyakov wrote:
> > On Fri, 2005-03-25 at 00:58 -0500, Jeff Garzik wrote:
> >
> > > If its disabled by default, then you and 2-3 other people will use this
> > > feature. Not enough justification for a kernel API at that point.
> >
> > It is only because there are only couple of HW crypto devices
> > in the tree, with one crypto framework inclusion there will be
> > at least redouble.
>
> You missed the point. This has nothing to do with the crypto API.
> Jeff is saying that if this is disabled by default, then only a few
> users will enable it and therefore use this API.
>
> Since we can't afford to enable it by default as hardware RNG may
> fail which can lead to catastrophic consequences, there is no point
> for this API at all.

What catastrophic consequences? Noone is likely to even *notice*, and
it does not help practical attack at all. Unless hardware RNGs are
*very* flakey (like, more flakey than harddrives), this is not a problem.

I can assure you that failing hdd will have more catastrophic
consequences.
Pavel
--
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

2005-03-29 10:48:57

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, 2005-03-29 at 20:30 +1000, Herbert Xu wrote:
> On Tue, Mar 29, 2005 at 12:21:04PM +0200, Pavel Machek wrote:
> >
> > What catastrophic consequences? Noone is likely to even *notice*, and
> > it does not help practical attack at all. Unless hardware RNGs are
> > *very* flakey (like, more flakey than harddrives), this is not a problem.
>
> The reason some people use hardware RNGs in the first place is because
> they don't trust the software RNGs. When the hardware RNG fails but
> continues to send data to /dev/random, /dev/random essentially degenerates
> into a software RNG. Now granted /dev/random is a pretty good software
> RNG, however, for some purposes it just isn't good enough.

I think the most people use hardware accelerated devices to
speed up theirs calculations - embedded world is the best example -
applications that are written to use /dev/random
will work just too slow, so hardware vendors
place HW assistant chips to unload that very cpu-intencive work
from main CPU.
Without ability speed this up in kernel, we completely [ok, almost]
loose all RNG advantages.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-29 10:48:15

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 12:38:01PM +0200, Pavel Machek wrote:
>
> It seems to me that people wanting this level of assurance should do
> their own FIPS (or whatever) tests.

That's exactly what the current scheme of driver + rngd allows you
to do. For those that require high assurance, they can let rngd
do the checks. Otherwise they can disable the checks and just let
rngd feed data from /dev/hw_random into /dev/random.

It's not just about the quality checks. It's also about deciding the
rate at which data should be fed into /dev/random.

Feeding too much means that you might be wasting system resources
(CPU/PCI bus/etc.). Feeding too little means that /dev/random users
may block unnecessarily.

Doing the feeding in a process means that you can feed exactly the
right amount. You feed only when the /dev/random's pool is depleted
and you feed exactly the amount of data that is needed to replenish
the pool.

> Interrupts are not totally unpredictable, either, yet noone runs FIPS

I haven't looked at this but if that's the case we might want to
look at lowering the entropy count for that.

> > Someone else raised the example of Casinos using hardware RNGs. Some
> > of them are doing this to comply with government regulation. In that
> > case, using data from the software RNG when the hardware has failed
> > would be violating the law.
>
> Well, you are still using hardware RNG, but failed one. I do not see
> how you can break law with that... given that hardware RNG had proper
> certification in the first place.

Actually in the scheme proposed in this thread the hardware RNG could
be feeding a stream of zeros into /dev/random which means that the
/dev/random user will be using a software RNG.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 10:52:08

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 02:50:28PM +0400, Evgeniy Polyakov wrote:
>
> Without ability speed this up in kernel, we completely [ok, almost]
> loose all RNG advantages.

Well if you can demonstrate that you're getting a higher rate of
throughput from your RNG by doing this in kernel space vs. doing
it in user space please let me know.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 10:56:59

by Martin Mares

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Hello!

> We trust hardware, anyway. Like your disk *could* accidentaly turn on
> setuid bit on /bin/bash, and we do not insist on userspace
> disk-validator.

But there is a very important difference: the most likely (both in theory
and practice) failure of a disk is clearly visible, while failures of HW RNG's
are likely to be silent (the data are just less random than they should be).

Have a nice fortnight
--
Martin `MJ' Mares <[email protected]> http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
Top ten reasons to procrastinate: 1.

2005-03-29 11:37:05

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, 2005-03-29 at 20:46 +1000, Herbert Xu wrote:
> On Tue, Mar 29, 2005 at 02:50:28PM +0400, Evgeniy Polyakov wrote:
> >
> > Without ability speed this up in kernel, we completely [ok, almost]
> > loose all RNG advantages.
>
> Well if you can demonstrate that you're getting a higher rate of
> throughput from your RNG by doing this in kernel space vs. doing
> it in user space please let me know.

Quote from VIA RNG crypto analysis:

"Bitrate. The RNG generates output at significantly
higher rates than most PC-based randomness resources.
Raw bits are produced at rates of 30 to 50 Mbits/sec,
and whitened bits were observed at rates of 4 to 9
Mbits/sec. Variations in output rates depend on the
RNG configuration and the oscillator rates. PRNGs
seeded with the Nehemiah RNG should be able to
easily sustain output in excess of 2 Mbits of entropy per
second, which should eliminate blocked PRNG reads in
virtually all applications."


While raw bits reading from hw_random on the fastest
VIA boards can exceed 55mbits per second
[above quite was taken from VIA C3 Nehemiah analysis],
it is not evaluated in rngd and is not written
back to the /dev/random.

David provided his patch exactly because of
"Adding this can dramatically improve the performance of
/dev/random on small embedded systems which do not
generate much entropy."
and it works.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-29 11:43:16

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 03:42:05PM +0400, Evgeniy Polyakov wrote:
> On Tue, 2005-03-29 at 20:46 +1000, Herbert Xu wrote:

> > Well if you can demonstrate that you're getting a higher rate of
> > throughput from your RNG by doing this in kernel space vs. doing
> > it in user space please let me know.
>
> While raw bits reading from hw_random on the fastest
> VIA boards can exceed 55mbits per second
> [above quite was taken from VIA C3 Nehemiah analysis],
> it is not evaluated in rngd and is not written
> back to the /dev/random.

Well when you get 55mb/s from /dev/random please get back to me.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 12:11:12

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, 2005-03-29 at 21:39 +1000, Herbert Xu wrote:
> On Tue, Mar 29, 2005 at 03:42:05PM +0400, Evgeniy Polyakov wrote:
> > On Tue, 2005-03-29 at 20:46 +1000, Herbert Xu wrote:
>
> > > Well if you can demonstrate that you're getting a higher rate of
> > > throughput from your RNG by doing this in kernel space vs. doing
> > > it in user space please let me know.
> >
> > While raw bits reading from hw_random on the fastest
> > VIA boards can exceed 55mbits per second
> > [above quite was taken from VIA C3 Nehemiah analysis],
> > it is not evaluated in rngd and is not written
> > back to the /dev/random.
>
> Well when you get 55mb/s from /dev/random please get back to me.

I cant, noone writes 55mbit into it, but HW RNG drivers could. :)

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-29 12:18:14

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Hi!


> > > > Well if you can demonstrate that you're getting a higher rate of
> > > > throughput from your RNG by doing this in kernel space vs. doing
> > > > it in user space please let me know.
> > >
> > > While raw bits reading from hw_random on the fastest
> > > VIA boards can exceed 55mbits per second
> > > [above quite was taken from VIA C3 Nehemiah analysis],
> > > it is not evaluated in rngd and is not written
> > > back to the /dev/random.
> >
> > Well when you get 55mb/s from /dev/random please get back to me.
>
> I cant, noone writes 55mbit into it, but HW RNG drivers could. :)

Exactly, on via machines, it might be good idea to hook /dev/random
directly to hardware random generator... that should give you
55mbit/sec.
Pavel
--
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

2005-03-29 12:46:05

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 04:15:17PM +0400, Evgeniy Polyakov wrote:
> > > On Tue, 2005-03-29 at 20:46 +1000, Herbert Xu wrote:
> >
> > > > Well if you can demonstrate that you're getting a higher rate of
> > > > throughput from your RNG by doing this in kernel space vs. doing
> > > > it in user space please let me know.
>
> > Well when you get 55mb/s from /dev/random please get back to me.
>
> I cant, noone writes 55mbit into it, but HW RNG drivers could. :)

Are you intending to feed that into /dev/random at 55mb/s?

If not then how is this an argument against doing it in userspace
through rngd?
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-29 13:06:15

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, 2005-03-29 at 22:43 +1000, Herbert Xu wrote:
> On Tue, Mar 29, 2005 at 04:15:17PM +0400, Evgeniy Polyakov wrote:
> > > > On Tue, 2005-03-29 at 20:46 +1000, Herbert Xu wrote:
> > >
> > > > > Well if you can demonstrate that you're getting a higher rate of
> > > > > throughput from your RNG by doing this in kernel space vs. doing
> > > > > it in user space please let me know.
> >
> > > Well when you get 55mb/s from /dev/random please get back to me.
> >
> > I cant, noone writes 55mbit into it, but HW RNG drivers could. :)
>
> Are you intending to feed that into /dev/random at 55mb/s?
>
> If not then how is this an argument against doing it in userspace
> through rngd?

Yes.
Untill pool is filled and then sleep there.
When someone wants to draw from the pool - awake and fill it again.

I clearly see your point here and I agree that it must be
default method for entropy producing, but if there are
possibility to speed that up, such techniques should be
allowed to be used.

It is really faster to fill pool from the kernelspace
without copying/validating it in userspace.

--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-29 13:49:40

by Jean-Luc Cooke

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 09:39:21PM +1000, Herbert Xu wrote:
> Well when you get 55mb/s from /dev/random please get back to me.

I will add you to my list for notification of Fortuna Patch to /dev/random
(/dev/urandom)

JLC

2005-03-29 14:34:17

by Evgeniy Polyakov

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, 2005-03-29 at 17:11 +0400, Evgeniy Polyakov wrote:
> On Tue, 2005-03-29 at 22:43 +1000, Herbert Xu wrote:
> > On Tue, Mar 29, 2005 at 04:15:17PM +0400, Evgeniy Polyakov wrote:
> > > > > On Tue, 2005-03-29 at 20:46 +1000, Herbert Xu wrote:
> > > >
> > > > > > Well if you can demonstrate that you're getting a higher rate of
> > > > > > throughput from your RNG by doing this in kernel space vs. doing
> > > > > > it in user space please let me know.
> > >
> > > > Well when you get 55mb/s from /dev/random please get back to me.
> > >
> > > I cant, noone writes 55mbit into it, but HW RNG drivers could. :)
> >
> > Are you intending to feed that into /dev/random at 55mb/s?
> >
> > If not then how is this an argument against doing it in userspace
> > through rngd?
>
> Yes.
> Untill pool is filled and then sleep there.
> When someone wants to draw from the pool - awake and fill it again.
>
> I clearly see your point here and I agree that it must be

I mean userspace validating and load balancing must be
default method for entropy producing.

> default method for entropy producing, but if there are
> possibility to speed that up, such techniques should be
> allowed to be used.
>
> It is really faster to fill pool from the kernelspace
> without copying/validating it in userspace.
>
--
Evgeniy Polyakov

Crash is better than data corruption -- Arthur Grabowski


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2005-03-29 15:03:32

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 02:17:24AM -0500, Jeff Garzik wrote:
> Andi Kleen wrote:
> >BTW what do you do when the FIPS test fails? I dont see a good fallback
> >path for this case.
>
> If the FIPS test fails, do the obvious: don't feed that data to the
> kernel (and credit entropy), and possibly stop using the hardware RNG

This will just cause hangs; basically it is a DOS.

> under a human has intervened.
>
> This is not rocket science. The fallback path is "use software", which
> is what most users do right now anyway.

Just that use software does not work on a headless machine sitting
in a rack with not much disk IO.

-Andi

2005-03-29 15:07:50

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Tue, Mar 29, 2005 at 02:16:12AM -0500, Jeff Garzik wrote:
> [email protected] wrote:
> >>>pool. The consensus was that the FIPS testing should be moved to
> >>>userspace.
> >>
> >>Consensus from whom? And who says the FIPS testing is useful anyways?
> >>I think you just need to trust the random generator, it is like
> >>you need to trust any other piece of hardware in your machine. Or do you
> >>check regularly if you mov instruction still works? @)
> >
> >
> >For joe-user imho it's better to do a check from a cronjob once a day. But
> >for
>
> That would not catch the hardware failures seen in the field, in the past.

I am sure that all hardware in the field has failed in the past
at some point - but that still does not mean we dont trust
hardware in Linux. There simply is no choice.

I think we have a case here of the perfect being the enemy of the good.

I just want a simple solution that works for near all users out of
the box without needing resource eating and hard to configure daemons
of dubious value.

You are extrapolating from extremly unlikely events and think
it makes sense to handle them - which I think is wrong.

The cronjob once a day or once an hour setup should catch
the hardware failures anyways, and for shorter time amounts
the random buffer in the kernel can handle bad input.

-Andi (who should probably just resubmit the patch)

2005-03-29 21:57:32

by Bill Davidsen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Herbert Xu wrote:

> You missed the point. This has nothing to do with the crypto API.
> Jeff is saying that if this is disabled by default, then only a few
> users will enable it and therefore use this API.
>
> Since we can't afford to enable it by default as hardware RNG may
> fail which can lead to catastrophic consequences, there is no point
> for this API at all.

Wait a minute, if it fails the system drops back to software, which is
not as good in a pedantic analysis, but perhaps falls a good bit short
of "catastrophic consequences" as most people would characterize that
phrase. And more to the point, now that many CPUs and chipsets are the
RNG of choice, what is the actual probability of a failure of the RNG
leaving a functional system (that's a real question seeking response
from someone who has some actual data).

It would be desirable for the kernel to detect a failure and do
something appropriate, but I have to feel that if an RNG is in the CPU
or chipset, it would serve users better to use it. By default. People
who need quality entropy would be better served by a hardware source,
and people who don't (or fail to realize they do) would not be hurt by
use of better numbers.

I'm not sure you would get people to agree what should be done if a
hardware RNG fails, other than make the failure information available to
user space.

--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me

2005-03-29 22:24:51

by Kyle Moffett

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On Mar 29, 2005, at 17:02, Bill Davidsen wrote:
> Wait a minute, if it fails the system drops back to software,

Does it? It would seem that if it fails and begins returning
all zeroes, then the seed function would (depending on the
implementation) be called like this:

add_random_bytes("\0\0\0\0".... , 4096);
add_random_bytes("\0\0\0\0".... , 4096);
[...]

Or:

add_random_bytes("\0\0\0\0".... , 4096);
add_random_bytes(soft_random() , 64);
add_random_bytes("\0\0\0\0".... , 4096);
add_random_bytes(soft_random() , 64);
[...]

In either case, it's very bad, and will likely return cause
some _very_ predictable data to be emitted.

> I'm not sure you would get people to agree what should be
> done if a hardware RNG fails, other than make the failure
> information available to user space.

How do you know if it fails? You know when your disk fails
and begins giving bad data because the filesystem detects
that the data is invalid, but how do you tell when your
random number generator starts giving bad data? AFAIK, the
only way to do that is to continuously monitor the random
data produced and _immediately_ stop the data flow when
you start getting bad data. That should only be done from
a userspace rngd-type daemon.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r
!y?(-)
------END GEEK CODE BLOCK------


2005-03-29 22:48:53

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Bill Davidsen wrote:
> Herbert Xu wrote:
>
>> You missed the point. This has nothing to do with the crypto API.
>> Jeff is saying that if this is disabled by default, then only a few
>> users will enable it and therefore use this API.
>>
>> Since we can't afford to enable it by default as hardware RNG may
>> fail which can lead to catastrophic consequences, there is no point
>> for this API at all.
>
>
> Wait a minute, if it fails the system drops back to software, which is
> not as good in a pedantic analysis, but perhaps falls a good bit short
> of "catastrophic consequences" as most people would characterize that
> phrase. And more to the point, now that many CPUs and chipsets are the
> RNG of choice, what is the actual probability of a failure of the RNG
> leaving a functional system (that's a real question seeking response
> from someone who has some actual data).

As I've said, in the past the Intel RNGs in particular -have- failed,
but the rest of the system keeps on working just fine.

It probably depends on the hardware implementation; I think the Intel
RNG was based on a thermal diode, or somesuch.

In the cases where an RNG has failed in the past, the system has worked
as expected: rngd stopped feeding data into the entropy pool.

If the VIA RNG (on-CPU) fails, that's probably indicative of a larger
problem, though.

Jeff


2005-03-29 23:43:57

by Andrew James Wade

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

On March 29, 2005 05:50 am, Evgeniy Polyakov wrote:
> I think the most people use hardware accelerated devices to
> speed up theirs calculations - embedded world is the best example -
> applications that are written to use /dev/random
> will work just too slow, so hardware vendors
> place HW assistant chips to unload that very cpu-intencive work
> from main CPU.
> Without ability speed this up in kernel, we completely [ok, almost]
> loose all RNG advantages.

The reason for hardware random number generators is that computers
are pretty deterministic machines and random number sources tend to be
few, far between, very low bitrate, and of uncertain randomness. So much
so that without a user (a decent entropy source), a computer might take
minutes to collect a few hundred bits of entropy.[1] The advantage of a
hardware RNG is that it is random in the first place, high bitrates are
just icing on the cake.

[1] Vague recollection from a hardware RNG article.

The thing is few applications need truly random data, and even fewer
need much. (Maybe casinos). Even cryptographic applications don't need
much; they can be served by a carefully crafted pseudo-random number
generator, so long as that generator is seeded with enough entropy. (512
bits of entropy is plenty). And while a crypographically strong
pseudo-random number generator is pretty cpu-intensive, I would be
quite surprised to learn that a hardware RNG is faster.

2005-03-30 21:20:48

by Bill Davidsen

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Jeff Garzik wrote:
> Bill Davidsen wrote:
>
>> Herbert Xu wrote:
>>
>>> You missed the point. This has nothing to do with the crypto API.
>>> Jeff is saying that if this is disabled by default, then only a few
>>> users will enable it and therefore use this API.
>>>
>>> Since we can't afford to enable it by default as hardware RNG may
>>> fail which can lead to catastrophic consequences, there is no point
>>> for this API at all.
>>
>>
>>
>> Wait a minute, if it fails the system drops back to software, which is
>> not as good in a pedantic analysis, but perhaps falls a good bit short
>> of "catastrophic consequences" as most people would characterize that
>> phrase. And more to the point, now that many CPUs and chipsets are the
>> RNG of choice, what is the actual probability of a failure of the RNG
>> leaving a functional system (that's a real question seeking response
>> from someone who has some actual data).
>
>
> As I've said, in the past the Intel RNGs in particular -have- failed,
> but the rest of the system keeps on working just fine.

People have been hit by meteors, too. I wasn't questioning that it was
possible in practice, or even that it did happen, but trying to get some
quantitative values for failure rates. I guess that wasn't clear, I know
entropy is scarse on embedded systems, I don't know what the magnitude
of the RNG failure is, so I'm coming from the "router in a cigar box"
point of view.
>
> It probably depends on the hardware implementation; I think the Intel
> RNG was based on a thermal diode, or somesuch.
>
> In the cases where an RNG has failed in the past, the system has worked
> as expected: rngd stopped feeding data into the entropy pool.

If the hardware RNG always fails to all zeros it should be possible to
detect that it failed with the need for userspace daemons. While true
random bits might legitimately have 10k zeros in a row, I will bet that
if it happens the device isn't working.

I'm assuming that the hardware RNG is only read when a read on
/dev/random occurs and there isn't enough entropy available to satisfy
the read. I realize that may not be true, I just haven't had time to go
study the code.
>
> If the VIA RNG (on-CPU) fails, that's probably indicative of a larger
> problem, though.


--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me

2005-03-30 21:52:13

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Bill Davidsen wrote:
> If the hardware RNG always fails to all zeros it should be possible to
> detect that it failed with the need for userspace daemons. While true
> random bits might legitimately have 10k zeros in a row, I will bet that
> if it happens the device isn't working.


Or all 1's (more likely), or all 0x55's, or...

Jeff


2005-03-30 22:31:44

by Paul Jackson

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)

Jeff wrote:
> Or all 1's (more likely), or all 0x55's, or...

Stray thought - to follow up stray thought:

Hmmm ... run some numbers through a good compression program,
and complain if they compress much.

--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373, 1.925.600.0401

2005-03-31 03:52:56

by David McCullough

[permalink] [raw]
Subject: Re: [PATCH] API for true Random Number Generators to add entropy (2.6.11)


Jivin Jeff Garzik lays it down ...
...
> >If kernelspace can assist and driver _knows_ in advance that data
> >produced is cryptographically strong, why not allow it directly
> >access pools?
>
> A kernel driver cannot know in advance that the data from a hardware RNG
> is truly random, unless the data itself is 100% validated beforehand.

You can also say that it cannot know that data written to /dev/random
is truly random unless it is also validated ?

For argument you could just run "cat < /dev/hwrandom > /dev/random"
instead of using rngd.

If /dev/random demands a level of randomness, shouldn't it enforce it ?

If the HW is using 2 random sources, a non-linear mixer and a FIPS140
post processor before handing you a random number it would be nice to
take advantage of that IMO.

Cheers,
Davidm

--
David McCullough, [email protected] Ph:+61 7 34352815 http://www.SnapGear.com
Custom Embedded Solutions + Security Fx:+61 7 38913630 http://www.uCdot.org

2005-03-31 14:02:35

by Jean-Luc Cooke

[permalink] [raw]
Subject: Re: [PATCH] API for TRNG (2.6.11) [Fortuna]

On Thu, Mar 31, 2005 at 01:52:14PM +1000, David McCullough wrote:
>
> Jivin Jeff Garzik lays it down ...
> ...
> > >If kernelspace can assist and driver _knows_ in advance that data
> > >produced is cryptographically strong, why not allow it directly
> > >access pools?
> >
> > A kernel driver cannot know in advance that the data from a hardware RNG
> > is truly random, unless the data itself is 100% validated beforehand.
>
> You can also say that it cannot know that data written to /dev/random
> is truly random unless it is also validated ?
>
> For argument you could just run "cat < /dev/hwrandom > /dev/random"
> instead of using rngd.
>
> If /dev/random demands a level of randomness, shouldn't it enforce it ?
>
> If the HW is using 2 random sources, a non-linear mixer and a FIPS140
> post processor before handing you a random number it would be nice to
> take advantage of that IMO.


For those who are interested, my Fortuna patch to the Linux RNG (/dev/random,
/dev/urandom) is available here (2.6.12-rc1, works on kernels as low as
2.6.11.4):
http://jlcooke.ca/random/

Fortuna is a Cryptographically Secure Random Number Generator (CSRNG)
developed by Neils Ferguson and Bruce Schnier and published in their book
Applied Cryptography.

By most regards, it is the state of the art as far as software based CSRNGs
go. The website gives an over view of the design, here is a summary:
Fortuna uses a block cipher (AES128) in CTR mode to generate output.
Fortuna uses a 32 hash states (SHA-256) which collect event data from
sources of randomness (as usual in Linux).
Once every 0.1sec or so, some of the hash states are finalised and the
digests are collected.
These digests are hashed together with with the current block cipher key to
produce the new block cipher key.

Ferguson goes into detail in Practical Cryptography as to why this design is
superior to Yarrow based RNG (like the existing Linux RNG) and also delves
into why entropy estimation is impossible and is infact a liability in RNG
design.

My patch keep the entropy estimation from the current Linux RNG since this is
a very controversial issue with most people. Disabling entropy estimation
and /dev/random blocking can be done by changing the RANDOM_NO_ENTROPY_COUNT
macro to 1.

I have not tested the syncookie code yet. But networking works smoothly
after I echo "1" to /proc/sys/net/ipv4/tcp_syncookies. Any help on this
would be great.

JLC

2005-04-13 15:47:23

by Jean-Luc Cooke

[permalink] [raw]
Subject: Re: [PATCH] API for TRNG (2.6.11) [Fortuna]

Patch attached.

JLC

On Thu, Mar 31, 2005 at 08:58:22AM -0500, Jean-Luc Cooke wrote:
> On Thu, Mar 31, 2005 at 01:52:14PM +1000, David McCullough wrote:
> >
> > Jivin Jeff Garzik lays it down ...
> > ...
> > > >If kernelspace can assist and driver _knows_ in advance that data
> > > >produced is cryptographically strong, why not allow it directly
> > > >access pools?
> > >
> > > A kernel driver cannot know in advance that the data from a hardware RNG
> > > is truly random, unless the data itself is 100% validated beforehand.
> >
> > You can also say that it cannot know that data written to /dev/random
> > is truly random unless it is also validated ?
> >
> > For argument you could just run "cat < /dev/hwrandom > /dev/random"
> > instead of using rngd.
> >
> > If /dev/random demands a level of randomness, shouldn't it enforce it ?
> >
> > If the HW is using 2 random sources, a non-linear mixer and a FIPS140
> > post processor before handing you a random number it would be nice to
> > take advantage of that IMO.
>
>
> For those who are interested, my Fortuna patch to the Linux RNG (/dev/random,
> /dev/urandom) is available here (2.6.12-rc1, works on kernels as low as
> 2.6.11.4):
> http://jlcooke.ca/random/
>
> Fortuna is a Cryptographically Secure Random Number Generator (CSRNG)
> developed by Neils Ferguson and Bruce Schnier and published in their book
> Applied Cryptography.
>
> By most regards, it is the state of the art as far as software based CSRNGs
> go. The website gives an over view of the design, here is a summary:
> Fortuna uses a block cipher (AES128) in CTR mode to generate output.
> Fortuna uses a 32 hash states (SHA-256) which collect event data from
> sources of randomness (as usual in Linux).
> Once every 0.1sec or so, some of the hash states are finalised and the
> digests are collected.
> These digests are hashed together with with the current block cipher key to
> produce the new block cipher key.
>
> Ferguson goes into detail in Practical Cryptography as to why this design is
> superior to Yarrow based RNG (like the existing Linux RNG) and also delves
> into why entropy estimation is impossible and is infact a liability in RNG
> design.
>
> My patch keep the entropy estimation from the current Linux RNG since this is
> a very controversial issue with most people. Disabling entropy estimation
> and /dev/random blocking can be done by changing the RANDOM_NO_ENTROPY_COUNT
> macro to 1.
>
> I have not tested the syncookie code yet. But networking works smoothly
> after I echo "1" to /proc/sys/net/ipv4/tcp_syncookies. Any help on this
> would be great.
>
> JLC
> _______________________________________________
>
> Subscription: http://lists.logix.cz/mailman/listinfo/cryptoapi
> List archive: http://lists.logix.cz/pipermail/cryptoapi


Attachments:
(No filename) (2.77 kB)
fortuna-random-2.6.11.4.patch (63.77 kB)
Download all attachments