2006-01-21 19:40:44

by Shaun Savage

[permalink] [raw]
Subject: CBD Compressed Block Device, New embedded block device

HI

Here is a patch for 2.6.14.5 of CBD
CBD is a compressed block device that is designed to shrink the file
system size to 1/3 the original size. CBD is a block device on a file
system so, it also allows for in-field upgrade of file system. If
necessary is also allows for secure booting, with a GRUB patch.

Reply to email please.

Shaun Savage


Attachments:
cbd-012106.patch (73.82 kB)

2006-01-21 19:53:19

by Arjan van de Ven

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

On Sun, 2006-01-22 at 00:46 -0800, Shaun Savage wrote:
> HI
>
> Here is a patch for 2.6.14.5 of CBD
> CBD is a compressed block device that is designed to shrink the file
> system size to 1/3 the original size. CBD is a block device on a file
> system so, it also allows for in-field upgrade of file system. If
> necessary is also allows for secure booting, with a GRUB patch.
>
> Reply to email please.


could you convert it to use mutexes instead of semaphores? ;)

2006-01-21 21:09:07

by Randy Dunlap

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

On Sun, 22 Jan 2006 00:46:52 -0800 Shaun Savage wrote:

> HI
>
> Here is a patch for 2.6.14.5 of CBD
> CBD is a compressed block device that is designed to shrink the file
> system size to 1/3 the original size. CBD is a block device on a file
> system so, it also allows for in-field upgrade of file system. If
> necessary is also allows for secure booting, with a GRUB patch.
>
> Reply to email please.

No need to send an entire .config file with it.

In Kconfig help, use "compressed".

I need more help on the BLK_CBD_DEVICE option.
What does it do?

Please fix the indentation after
+#ifndef CBD_PARTITION
in cbd_int.c

Try to limit lines to < 80 characters each.

Use spaces around operators (=, <, >, <=, >=, etc).

Use space after "if", "for", and "while".

Has some funky indenting (use tabs instead of spaces).

Don't use typedefs.
Don't use typedef struct S { ... } S_t;

struct partition_info &
struct cbd_part_hdr seem to expect that ushort, uint,
etc., have fixed sizes, but they can actually vary by
architecture, so you should use known fixed sizes
anywhere that field & struct sizes matter.

Use ALIGN() from kernel.h instead of your own align() macro.

Lots of trailing whitespace. Don't do that:
Warning: trailing whitespace in lines 41,58,102,104,106,137,198,234 of include/linux/cbd.h
Warning: trailing whitespace in lines 348,355 of drivers/block/Kconfig
Warning: trailing whitespace in lines 32,72,138,158 of drivers/block/cbd_int.c
Warning: trailing whitespace in lines 7,50,95,162,240,318,428,516,613,657,658,669,704,747,797,808,813,817,827,872 of drivers/block/cbd_main.c

Has about 25 gcc warnings when I build it.

---
~Randy

2006-01-21 21:47:50

by Alexey Dobriyan

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

On Sun, Jan 22, 2006 at 12:46:52AM -0800, Shaun Savage wrote:
> Here is a patch for 2.6.14.5 of CBD
> CBD is a compressed block device that is designed to shrink the file
> system size to 1/3 the original size. CBD is a block device on a file
> system so, it also allows for in-field upgrade of file system. If
> necessary is also allows for secure booting, with a GRUB patch.
> +/*
> + * IOCTL stuff, maybe new file

/dev/null ?

> + switch (cmd) {
> + case HDIO_GETGEO:
> + {
> + struct hd_geometry loc;
> + loc.heads = 4;
> + loc.sectors = 32;
> + loc.cylinders = cbd->nr_sections;
> + loc.start = 0;
> + return _COPYOUT(loc);
> + }

Already in generic code.

2006-01-21 22:21:13

by Matt Mackall

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

On Sun, Jan 22, 2006 at 12:46:52AM -0800, Shaun Savage wrote:
> HI
>
> Here is a patch for 2.6.14.5 of CBD
> CBD is a compressed block device that is designed to shrink the file
> system size to 1/3 the original size. CBD is a block device on a file
> system so, it also allows for in-field upgrade of file system. If
> necessary is also allows for secure booting, with a GRUB patch.

How does it work? Does it remap one block device and present it as a
new one? Or does it work more like loopback? In either case, we might
prefer a device mapper plug-in.

Please add a brief write-up under Documentation/, including a general
overview and example usage so we can get some idea of what this
actually does. Or URLs to existing docs, of course.

--
Mathematics is the supreme nostalgia of our time.

2006-01-22 08:26:32

by Pavel Machek

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

On Ne 22-01-06 00:46:52, Shaun Savage wrote:
> HI
>
> Here is a patch for 2.6.14.5 of CBD
> CBD is a compressed block device that is designed to shrink the file
> system size to 1/3 the original size. CBD is a block device on a file
> system so, it also allows for in-field upgrade of file system. If
> necessary is also allows for secure booting, with a GRUB patch.

What does compression have to do with secure booting?


--
Thanks, Sharp!

2006-01-22 13:20:44

by Shaun Savage

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

Hi

Compression is just one feature. Each partition is wrapped in a package
that has a header. even the kernel and boot loader are wrapped in the
package. Each package can be hashed and signed. There is a patch for
GRUB that boots CBD. GRUB patch does 3 things, boot the kernel, find
the latest version of each package, verify hash and signature. This
allow for in robust field update of packages.

shaun

Pavel Machek wrote:

>On Ne 22-01-06 00:46:52, Shaun Savage wrote:
>
>
>>HI
>>
>>Here is a patch for 2.6.14.5 of CBD
>>CBD is a compressed block device that is designed to shrink the file
>>system size to 1/3 the original size. CBD is a block device on a file
>>system so, it also allows for in-field upgrade of file system. If
>>necessary is also allows for secure booting, with a GRUB patch.
>>
>>
>
>What does compression have to do with secure booting?
>
>
>
>

2006-01-22 15:50:50

by Phillip Susi

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

How is this different from cloop or dm-crypt?

Shaun Savage wrote:
> HI
>
> Here is a patch for 2.6.14.5 of CBD
> CBD is a compressed block device that is designed to shrink the file
> system size to 1/3 the original size. CBD is a block device on a file
> system so, it also allows for in-field upgrade of file system. If
> necessary is also allows for secure booting, with a GRUB patch.
>
> Reply to email please.
>
> Shaun Savage

2006-01-22 18:26:15

by Jan Engelhardt

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device


> How is this different from cloop or dm-crypt?

(dm-crypt is for crypting, CBD is for compression, I'd say.)
Compared to cloop, I'd say "the same" without giving too much thought away,
but as always - he who comes first, draws first.


Jan Engelhardt
--

2006-01-22 22:27:49

by Shaun Savage

[permalink] [raw]
Subject: Re: CBD Compressed Block Device, New embedded block device

CBD is designed for embedded systems. The compression starts off the
similar to cloop, The filesystem partitions are created in files, These
partition files are broken into 32K blocks( the window size of gzip).
Now there is a compressed file and a array of start locations. This is
placed into a CBD partition package (CBDpp), by adding a header. This
header includes version, hash, signature,....

The flash device is divided 64K blocks, you can think of these as
sectors. There is a user program that writes new CBDpp to these
blocks. The CBDpp are run length encoded. There is a table of which
blocks are used by the CBDpp in the header of the CBDpp. On startup the
driver searches the blocks for the MAGIC header. When it finds one it
read the header and maps which blocks holds the CBDpp. The driver then
does not search thoses blocks. Empty block devices are slow to boot
while full ones are fast.

There is a patch for GRUB that know about the CBD headers. It search
for the latest and greatest version of the CBDpp. The allows for in the
field system update.

partition 0 is the bootloader and kernel, and othe global system stuff.
partition 1 is the root file system

The one unique feature is any filesystem can be on top AND it support
writes!! Now the write never make it to the physical device, but the
write data is locked in buffer cache. Yes I know this can be a memory
leak, but in an embedded device the writes are configuration and patches.

I received lots of email on how to improve the code, Which I am doing.
I will answer your emails in how fast I can make those changes.

shaun


Phillip Susi wrote:

> How is this different from cloop or dm-crypt?
>
> Shaun Savage wrote:
>
>> HI
>>
>> Here is a patch for 2.6.14.5 of CBD
>> CBD is a compressed block device that is designed to shrink the file
>> system size to 1/3 the original size. CBD is a block device on a
>> file system so, it also allows for in-field upgrade of file system.
>> If necessary is also allows for secure booting, with a GRUB patch.
>>
>> Reply to email please.
>>
>> Shaun Savage
>
>