This is a second attempt at mainlining Pramfs. The first attempt was
back in early 2004 by MontaVista. Since then the kernel code has almost
been completely rewritten. So my first item on the list was porting the
code on a recent kernel version. After that I added the XIP support.
Now some FAQs:
What is the goal of this filesystem?
Many embedded systems have a block of non-volatile RAM separate from
normal system memory, i.e. of which the kernel maintains no memory page
descriptors. For such systems it would be beneficial to mount a
fast read/write filesystem over this "I/O memory", for storing
frequently accessed data that must survive system reboots and power
cycles. An example usage might be system logs under /var/log, or a user
address book in a cell phone or PDA.
Why this kind of filesystem should be "mainlined"?
Linux traditionally had no support for a persistent, non-volatile
RAM-based filesystem, persistent meaning the filesystem survives a
system reboot or power cycle intact. The RAM-based filesystems such as
tmpfs and ramfs have no actual backing store but exist entirely in the
page and buffer caches, hence the filesystem disappears after a system
reboot or power cycle.
Are there any pending patents on this code?
NO, there aren't patents pending on this code. MontaVista had a pending
patent application but now it has abandoned this way. Daniel Walker can
confirm that.
Marco
On Sat, 2009-06-13 at 15:20 +0200, Marco wrote:
> Are there any pending patents on this code?
>
> NO, there aren't patents pending on this code. MontaVista had a pending
> patent application but now it has abandoned this way. Daniel Walker can
> confirm that.
Confirmed , there are no patents on this code.
Daniel
Marco wrote:
> Linux traditionally had no support for a persistent, non-volatile
> RAM-based filesystem, persistent meaning the filesystem survives a
> system reboot or power cycle intact. The RAM-based filesystems such as
> tmpfs and ramfs have no actual backing store but exist entirely in the
> page and buffer caches, hence the filesystem disappears after a system
> reboot or power cycle.
Why is a ramdisk not sufficient for this?
Why is an entire filesystem needed, instead of simply a block driver
if the ramdisk driver cannot be used?
It just struck me as a lot of code which might be completely
unnecessary for the desired functionality.
-- Jamie
Jamie Lokier wrote:
> Marco wrote:
>> Linux traditionally had no support for a persistent, non-volatile
>> RAM-based filesystem, persistent meaning the filesystem survives a
>> system reboot or power cycle intact. The RAM-based filesystems such as
>> tmpfs and ramfs have no actual backing store but exist entirely in the
>> page and buffer caches, hence the filesystem disappears after a system
>> reboot or power cycle.
>
> Why is a ramdisk not sufficient for this?
>
Simply because the ramdisk was not designed to work in a persistent
environment. In addition this kind of filesystem has been designed to
work not only with classic ram. You can think at the situation where you
have got an external SRAM with a battery for example. With it you can
"remap" in an easy way the SRAM. Moreover there's the issue of memory
protection that this filesystem takes care.
> Why is an entire filesystem needed, instead of simply a block driver
> if the ramdisk driver cannot be used?
>
>From documentation:
"A relatively straight-forward solution is to write a simple block
driver for the non-volatile RAM, and mount over it any disk-based
filesystem such as ext2/ext3, reiserfs, etc.
But the disk-based fs over non-volatile RAM block driver approach has
some drawbacks:
1. Disk-based filesystems such as ext2/ext3 were designed for optimum
performance on spinning disk media, so they implement features such
as block groups, which attempts to group inode data into a contiguous
set of data blocks to minimize disk seeking when accessing files. For
RAM there is no such concern; a file's data blocks can be scattered
throughout the media with no access speed penalty at all. So block
groups in a filesystem mounted over RAM just adds unnecessary
complexity. A better approach is to use a filesystem specifically
tailored to RAM media which does away with these disk-based features.
This increases the efficient use of space on the media, i.e. more
space is dedicated to actual file data storage and less to meta-data
needed to maintain that file data.
2. If the backing-store RAM is comparable in access speed to system memory,
there's really no point in caching the file I/O data in the page
cache. Better to move file data directly between the user buffers
and the backing store RAM, i.e. use direct I/O. This prevents the
unnecessary populating of the page cache with dirty pages. However
direct I/O has to be enabled at every file open. To enable direct
I/O at all times for all regular files requires either that
applications be modified to include the O_DIRECT flag on all file
opens, or that a new filesystem be used that always performs direct
I/O by default."
On this point I'd like to hear other embedded guys.
> It just struck me as a lot of code which might be completely
> unnecessary for the desired functionality.
>
> -- Jamie
>
Marco wrote:
> To enable direct
> I/O at all times for all regular files requires either that
> applications be modified to include the O_DIRECT flag on all file
> opens, or that a new filesystem be used that always performs direct
> I/O by default."
This could be done as well by just introducing a "direct_io_only"
mount option to a file-system which would need this feature.
--
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
Marco wrote:
> Simply because the ramdisk was not designed to work in a persistent
> environment.
One thing with persistent RAM disks is you _really_ want it to be
robust if the system crashes for any reason while it is being
modified. The last thing you want is to reboot, and find various
directories containing configuration files or application files have
been corrupted or disappeared as a side effect of writing something else.
That's one of the advantages of using a log-structured filesystem such
as Nilfs, JFFS2, Logfs, UBIFS, Btrfs, ext3, reiserfs, XFS or JFS on a
ramdisk :-)
Does PRAMFS have this kind of robustness?
> In addition this kind of filesystem has been designed to work not
> only with classic ram. You can think at the situation where you have
> got an external SRAM with a battery for example. With it you can
> "remap" in an easy way the SRAM. Moreover there's the issue of
> memory protection that this filesystem takes care. > Why is an
> entire filesystem needed, instead of simply a block driver > if the
> ramdisk driver cannot be used? > >From documentation: "A relatively
> straight-forward solution is to write a simple block driver for the
> non-volatile RAM, and mount over it any disk-based filesystem such
> as ext2/ext3, reiserfs, etc. But the disk-based fs over
> non-volatile RAM block driver approach has some drawbacks:
>
> 1. Disk-based filesystems such as ext2/ext3 were designed for
> optimum performance on spinning disk media, so they implement
> features such as block groups, which attempts to group inode data
> into a contiguous set of data blocks to minimize disk seeking when
> accessing files. For RAM there is no such concern; a file's data
> blocks can be scattered throughout the media with no access speed
> penalty at all. So block groups in a filesystem mounted over RAM
> just adds unnecessary complexity. A better approach is to use a
> filesystem specifically tailored to RAM media which does away with
> these disk-based features. This increases the efficient use of
> space on the media, i.e. more space is dedicated to actual file data
> storage and less to meta-data needed to maintain that file data.
All true, I agree. RAM-based databases use different structures to
disk-based databases for the same reasons.
Isn't there any good RAM-based filesystem already? Some of the flash
filesystems and Nilfs seem promising, using fake MTD with a small
erase size. All are robust on crashes.
> 2. If the backing-store RAM is comparable in access speed to system
> memory, there's really no point in caching the file I/O data in the
> page cache.
>
> Better to move file data directly between the user buffers
> and the backing store RAM, i.e. use direct I/O. This prevents the
> unnecessary populating of the page cache with dirty pages.
Good idea.
> However direct I/O has to be enabled at every file open. To
> enable direct I/O at all times for all regular files requires
> either that applications be modified to include the O_DIRECT flag
> on all file opens, or that a new filesystem be used that always
> performs direct I/O by default."
There are other ways to include the O_DIRECT flag automatically. A
generic mount option would be enough. I've seen other OSes with such
an option. That code for that would be tiny.
But standard O_DIRECT direct I/O doesn't work for all applications: it
has to be aligned: device offset, application memory address and size
all have to be aligned.
(It would be a nice touch to produce a generic mount option
o_direct_when_possible, which turns on direct I/O but permits
unaligned I/O. That could be used with all applications.)
As you say PRAMFS can work with special SRAMs needing memory
protection (and maybe cache coherence?), if you mmap() a file does it
need to use the page cache then? If so, do you have issues with
coherency between mmap() and direct read/write?
> On this point I'd like to hear other embedded guys.
As one, I'd like to say if it can checksum the RAM at boot as well,
then I might like to use a small one in ordinary SRAM (at a fixed
reserved address) for those occasions when a reboot happens
(intentional or not) and I'd like to pass a little data to the next
running kernel about why the reboot happened, without touching flash
every time.
-- Jamie
Jamie Lokier wrote:
> Marco wrote:
>> Simply because the ramdisk was not designed to work in a persistent
>> environment.
>
> One thing with persistent RAM disks is you _really_ want it to be
> robust if the system crashes for any reason while it is being
> modified. The last thing you want is to reboot, and find various
> directories containing configuration files or application files have
> been corrupted or disappeared as a side effect of writing something else.
>
> That's one of the advantages of using a log-structured filesystem such
> as Nilfs, JFFS2, Logfs, UBIFS, Btrfs, ext3, reiserfs, XFS or JFS on a
> ramdisk :-)
>
> Does PRAMFS have this kind of robustness?
There's the checksum, but the most important feature of this fs is the
write protection. The page table entries that map the
backing-store RAM are normally marked read-only. Write operations into
the filesystem temporarily mark the affected pages as writeable, the
write operation is carried out with locks held, and then the pte is
marked read-only again. This feature provides protection against
filesystem corruption caused by errant writes into the RAM due to
kernel bugs for instance. I provided a test module for this. When the
module is loaded tries to do a dirty write in the superblock, at this
point you should see an error on the write.
>
>> In addition this kind of filesystem has been designed to work not
>> only with classic ram. You can think at the situation where you have
>> got an external SRAM with a battery for example. With it you can
>> "remap" in an easy way the SRAM. Moreover there's the issue of
>> memory protection that this filesystem takes care. > Why is an
>> entire filesystem needed, instead of simply a block driver > if the
>> ramdisk driver cannot be used? > >From documentation: "A relatively
>> straight-forward solution is to write a simple block driver for the
>> non-volatile RAM, and mount over it any disk-based filesystem such
>> as ext2/ext3, reiserfs, etc. But the disk-based fs over
>> non-volatile RAM block driver approach has some drawbacks:
>>
>> 1. Disk-based filesystems such as ext2/ext3 were designed for
>> optimum performance on spinning disk media, so they implement
>> features such as block groups, which attempts to group inode data
>> into a contiguous set of data blocks to minimize disk seeking when
>> accessing files. For RAM there is no such concern; a file's data
>> blocks can be scattered throughout the media with no access speed
>> penalty at all. So block groups in a filesystem mounted over RAM
>> just adds unnecessary complexity. A better approach is to use a
>> filesystem specifically tailored to RAM media which does away with
>> these disk-based features. This increases the efficient use of
>> space on the media, i.e. more space is dedicated to actual file data
>> storage and less to meta-data needed to maintain that file data.
>
> All true, I agree. RAM-based databases use different structures to
> disk-based databases for the same reasons.
>
> Isn't there any good RAM-based filesystem already? Some of the flash
> filesystems and Nilfs seem promising, using fake MTD with a small
> erase size. All are robust on crashes.
>
Good question. The only similar thing that I know it's a patch called
pmem provided by WindRiver. It's the main reason that I led to develop
this kind of fs. In addition in my projects to have this feature has
always been very useful.
>> However direct I/O has to be enabled at every file open. To
>> enable direct I/O at all times for all regular files requires
>> either that applications be modified to include the O_DIRECT flag
>> on all file opens, or that a new filesystem be used that always
>> performs direct I/O by default."
>
> There are other ways to include the O_DIRECT flag automatically. A
> generic mount option would be enough. I've seen other OSes with such
> an option. That code for that would be tiny.
>
> But standard O_DIRECT direct I/O doesn't work for all applications: it
> has to be aligned: device offset, application memory address and size
> all have to be aligned.
>
> (It would be a nice touch to produce a generic mount option
> o_direct_when_possible, which turns on direct I/O but permits
> unaligned I/O. That could be used with all applications.)
>
> As you say PRAMFS can work with special SRAMs needing memory
> protection (and maybe cache coherence?), if you mmap() a file does it
> need to use the page cache then? If so, do you have issues with
> coherency between mmap() and direct read/write?
See my response above about my concept of protection. However the mmap
it's a similar approach. I can "mmap" the SRAM and I can write into it
my data, but I think the possibility to have a fs it's great. We can use
the device as normal disk, i.e. we can use cp, mv and so on.
>
>> On this point I'd like to hear other embedded guys.
>
> As one, I'd like to say if it can checksum the RAM at boot as well,
> then I might like to use a small one in ordinary SRAM (at a fixed
> reserved address) for those occasions when a reboot happens
> (intentional or not) and I'd like to pass a little data to the next
> running kernel about why the reboot happened, without touching flash
> every time.
>
> -- Jamie
>
Yeah Jamie, the goal of this fs is exactly that!
Marco
> Marco wrote:
> > To enable direct
> > I/O at all times for all regular files requires either that
> > applications be modified to include the O_DIRECT flag on all file
> > opens, or that a new filesystem be used that always performs direct
> > I/O by default."
>
> This could be done as well by just introducing a "direct_io_only"
> mount option to a file-system which would need this feature.
A mount option would not be the right way. Mount options are for things
that are characteristic of the way you're going to access the files.
_This_ is a characteristic of the block device. So if one were to make
this memory accessible with a block device, it would make more sense to
have a block device ioctl. And it wouldn't ask the question, "should I
use direct I/O only," but "does this device have the performance
characteristics of a classic disk drive?"
But it's possible that there's just no advantage to having a block device
in the stack here. When unix block devices were invented, their main
purpose was that they could reorder reads and writes and do buffering and
caching -- all things essential for disk drives. We don't want to stretch
the concept too far.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Storage Systems
Marco wrote:
> This is a second attempt at mainlining Pramfs. The first attempt was
> back in early 2004 by MontaVista. Since then the kernel code has almost
> been completely rewritten. So my first item on the list was porting the
> code on a recent kernel version. After that I added the XIP support.
It's very nice to see this technology revived.
Is the information at:
http://pramfs.sourceforge.net/
and
http://pramfs.sourceforge.net/pramfs-spec.html
still valid - particularly the latter?
It would be very nice to see this get mainlined. I believe that
one of the main uses for this is to store crash information
over a reboot so the next kernel (not in crashing state) can have
a better chance of dealing with it. As such, I think
it's important to keep the code paths for Pramfs short, synchronous,
and unentangled with other kernel systems (block IO, page cache, etc.).
Thanks,
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
Bryan Henderson wrote:
>> Marco wrote:
>>> To enable direct
>>> I/O at all times for all regular files requires either that
>>> applications be modified to include the O_DIRECT flag on all file
>>> opens, or that a new filesystem be used that always performs direct
>>> I/O by default."
>> This could be done as well by just introducing a "direct_io_only"
>> mount option to a file-system which would need this feature.
>
> But it's possible that there's just no advantage to having a block device
> in the stack here. When unix block devices were invented, their main
> purpose was that they could reorder reads and writes and do buffering and
> caching -- all things essential for disk drives. We don't want to stretch
> the concept too far.
>
Yes I agree, we can't in this case talk about read and write reordering,
buffering and caching because we're talking about something completely
different from a classic disk. The issues of this kind of fs are more
similar to the tmpfs issues.
Marco
Tim Bird wrote:
> Marco wrote:
>> This is a second attempt at mainlining Pramfs. The first attempt was
>> back in early 2004 by MontaVista. Since then the kernel code has almost
>> been completely rewritten. So my first item on the list was porting the
>> code on a recent kernel version. After that I added the XIP support.
>
> It's very nice to see this technology revived.
>
> Is the information at:
> http://pramfs.sourceforge.net/
> and
> http://pramfs.sourceforge.net/pramfs-spec.html
> still valid - particularly the latter?
Yep. at 99%. I've done some modifications due to the porting and there
will be some ones due to this review. I tried to talk with Steve
Longerbeam to update the site but without success. I'd like to update it.
>
> It would be very nice to see this get mainlined. I believe that
> one of the main uses for this is to store crash information
> over a reboot so the next kernel (not in crashing state) can have
> a better chance of dealing with it. As such, I think
> it's important to keep the code paths for Pramfs short, synchronous,
> and unentangled with other kernel systems (block IO, page cache, etc.).
>
Yes, I quite agree. I think that this kind of feature would be very
useful especially for the embedded world.
Marco
Marco wrote:
> Tim Bird wrote:
>> Marco wrote:
>>> This is a second attempt at mainlining Pramfs. The first attempt was
>>> back in early 2004 by MontaVista. Since then the kernel code has almost
>>> been completely rewritten. So my first item on the list was porting the
>>> code on a recent kernel version. After that I added the XIP support.
>> It's very nice to see this technology revived.
>>
>> Is the information at:
>> http://pramfs.sourceforge.net/
>> and
>> http://pramfs.sourceforge.net/pramfs-spec.html
>> still valid - particularly the latter?
>
> Yep. at 99%. I've done some modifications due to the porting and there
> will be some ones due to this review. I tried to talk with Steve
> Longerbeam to update the site but without success. I'd like to update it.
>
>> It would be very nice to see this get mainlined. I believe that
>> one of the main uses for this is to store crash information
>> over a reboot so the next kernel (not in crashing state) can have
>> a better chance of dealing with it. As such, I think
>> it's important to keep the code paths for Pramfs short, synchronous,
>> and unentangled with other kernel systems (block IO, page cache, etc.).
>>
>
> Yes, I quite agree. I think that this kind of feature would be very
> useful especially for the embedded world.
Just FYI - we have an "exception monitor" at Sony
which is used in several projects, that records application and
kernel crash information into the file system, for subsequent
(often in-field) analysis. However, the data currently gets written
to a flash filesystem and the logs sometimes get truncated or otherwise
corrupted. This seems like a perfect match for what we're trying
to do.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
Marco wrote:
> There's the checksum, but the most important feature of this fs is the
> write protection. The page table entries that map the
> backing-store RAM are normally marked read-only. Write operations into
> the filesystem temporarily mark the affected pages as writeable, the
> write operation is carried out with locks held, and then the pte is
> marked read-only again. This feature provides protection against
> filesystem corruption caused by errant writes into the RAM due to
> kernel bugs for instance. I provided a test module for this. When the
> module is loaded tries to do a dirty write in the superblock, at this
> point you should see an error on the write.
Ok. Random question: does it work with NOMMU? :-) (I'm biased, my
devices are NOMMU).
Second question: what happens if the system crashing _during_ a write
to a file. Does it mean that file will fail it's checksum when it's
read at the next boot?
Maybe files aren't so important. What about when you write a file,
and then rename it over an existing file to replace it. (E.g. a
config file), and the system crashes _during_ the rename? At the next
boot, is it guaranteed to see either the old or the new file, or can
the directory be corrupt / fail it's checksum?
> > As you say PRAMFS can work with special SRAMs needing memory
> > protection (and maybe cache coherence?), if you mmap() a file does it
> > need to use the page cache then? If so, do you have issues with
> > coherency between mmap() and direct read/write?
>
> See my response above about my concept of protection. However the mmap
> it's a similar approach. I can "mmap" the SRAM and I can write into it
> my data, but I think the possibility to have a fs it's great. We can use
> the device as normal disk, i.e. we can use cp, mv and so on.
I meant when you mmap() a file on the filesystem, like you do when
running an executable, for example. Does mmap() on a file work or is
it forbidden? Just curious, I'd guess it's forbidden, and you
wouldn't want _direct_ mappings to the backing SRAM anyway so you can
keep those checksums up to date.
> >> On this point I'd like to hear other embedded guys.
> >
> > As one, I'd like to say if it can checksum the RAM at boot as well,
> > then I might like to use a small one in ordinary SRAM (at a fixed
> > reserved address) for those occasions when a reboot happens
> > (intentional or not) and I'd like to pass a little data to the next
> > running kernel about why the reboot happened, without touching flash
> > every time.
> >
> > -- Jamie
>
> Yeah Jamie, the goal of this fs is exactly that!
Great :-)
-- Jamie
Jamie Lokier wrote:
> Marco wrote:
>> There's the checksum, but the most important feature of this fs is the
>> write protection. The page table entries that map the
>> backing-store RAM are normally marked read-only. Write operations into
>> the filesystem temporarily mark the affected pages as writeable, the
>> write operation is carried out with locks held, and then the pte is
>> marked read-only again. This feature provides protection against
>> filesystem corruption caused by errant writes into the RAM due to
>> kernel bugs for instance. I provided a test module for this. When the
>> module is loaded tries to do a dirty write in the superblock, at this
>> point you should see an error on the write.
>
> Ok. Random question: does it work with NOMMU? :-) (I'm biased, my
> devices are NOMMU).
>
I have to say no. Or at least you can use it but you should select the
option to disable the write protection. Of course, from this point of
view there are some archs that they couldn't have this feature. I had to
disable when I worked with a board with avr32.
> Second question: what happens if the system crashing _during_ a write
> to a file. Does it mean that file will fail it's checksum when it's
> read at the next boot?
>
> Maybe files aren't so important. What about when you write a file,
> and then rename it over an existing file to replace it. (E.g. a
> config file), and the system crashes _during_ the rename? At the next
> boot, is it guaranteed to see either the old or the new file, or can
> the directory be corrupt / fail it's checksum?
First of all I have to explain better the current policy: the checksum
works at inode and superblock level and currently there isn't a recovery
function as the journaling. About the superblock it's easy to use a
redundant policy to be more robust. About the inode, at the moment when
the checksum doesn't match the inode it's marked as bad calling the
function make_bad_inode().
>>> As you say PRAMFS can work with special SRAMs needing memory
>>> protection (and maybe cache coherence?), if you mmap() a file does it
>>> need to use the page cache then? If so, do you have issues with
>>> coherency between mmap() and direct read/write?
>> See my response above about my concept of protection. However the mmap
>> it's a similar approach. I can "mmap" the SRAM and I can write into it
>> my data, but I think the possibility to have a fs it's great. We can use
>> the device as normal disk, i.e. we can use cp, mv and so on.
>
> I meant when you mmap() a file on the filesystem, like you do when
> running an executable, for example. Does mmap() on a file work or is
> it forbidden? Just curious, I'd guess it's forbidden, and you
> wouldn't want _direct_ mappings to the backing SRAM anyway so you can
> keep those checksums up to date.
>
Because pramfs attempts to avoid filesystem corruption caused by kernel
bugs, dirty pages in the page cache are not allowed to be written back
to the backing-store RAM. This means that only private file mappings are
supported. This way, an errant write into the page cache will not get
written back to the filesystem.
>>>> On this point I'd like to hear other embedded guys.
>>> As one, I'd like to say if it can checksum the RAM at boot as well,
>>> then I might like to use a small one in ordinary SRAM (at a fixed
>>> reserved address) for those occasions when a reboot happens
>>> (intentional or not) and I'd like to pass a little data to the next
>>> running kernel about why the reboot happened, without touching flash
>>> every time.
>>>
>>> -- Jamie
>> Yeah Jamie, the goal of this fs is exactly that!
>
> Great :-)
>
> -- Jamie
>
Marco
Marco wrote:
> This is a second attempt at mainlining Pramfs. The first attempt was
> back in early 2004 by MontaVista. Since then the kernel code has almost
> been completely rewritten. So my first item on the list was porting the
> code on a recent kernel version. After that I added the XIP support.
>
> Now some FAQs:
>
> What is the goal of this filesystem?
>
> Many embedded systems have a block of non-volatile RAM separate from
> normal system memory, i.e. of which the kernel maintains no memory page
> descriptors. For such systems it would be beneficial to mount a
> fast read/write filesystem over this "I/O memory", for storing
> frequently accessed data that must survive system reboots and power
> cycles. An example usage might be system logs under /var/log, or a user
> address book in a cell phone or PDA.
Nice to see something like this submitted to mainline. We use something
similar to provide persistent storage for crash recovery debug data for
boards which don't have local storage.
In many cases kdump can provide good information, but it's not
sufficient for "flight recorder" type data if the kernel gets rebooted
by a hardware mechanism (watchdog, for instance) that doesn't give a
pre-interrupt.
I'm a bit concerned about your PTE modifications on every write
though...we do things like log every exception and scheduler operation
to persistent memory, and I think the overhead of changing the
protection on every write would be a killer. Instead, we make extensive
use of checksums at various different levels so that the recovery app
can determine which data is valid.
Also, I'd like to ensure that direct memory access to the memory area
would be available. There are some things (like the sched/exception
logging mentioned above) where we want to make accesses as fast as possible.
Chris
2009/6/17 Chris Friesen <[email protected]>:
> Marco wrote:
>> This is a second attempt at mainlining Pramfs. The first attempt was
>> back in early 2004 by MontaVista. Since then the kernel code has almost
>> been completely rewritten. So my first item on the list was porting the
>> code on a recent kernel version. After that I added the XIP support.
>>
>> Now some FAQs:
>>
>> What is the goal of this filesystem?
>>
>> Many embedded systems have a block of non-volatile RAM separate from
>> normal system memory, i.e. of which the kernel maintains no memory page
>> descriptors. For such systems it would be beneficial to mount a
>> fast read/write filesystem over this "I/O memory", for storing
>> frequently accessed data that must survive system reboots and power
>> cycles. An example usage might be system logs under /var/log, or a user
>> address book in a cell phone or PDA.
>
> Nice to see something like this submitted to mainline. ?We use something
> similar to provide persistent storage for crash recovery debug data for
> boards which don't have local storage.
>
> In many cases kdump can provide good information, but it's not
> sufficient for "flight recorder" type data if the kernel gets rebooted
> by a hardware mechanism (watchdog, for instance) that doesn't give a
> pre-interrupt.
I'm very happy that this fs has the approval of the kernel community. :)
>
> I'm a bit concerned about your PTE modifications on every write
> though...we do things like log every exception and scheduler operation
> to persistent memory, and I think the overhead of changing the
> protection on every write would be a killer. ?Instead, we make extensive
> use of checksums at various different levels so that the recovery app
> can determine which data is valid.
>
It's a trade-off between security and performance. Checksum it's a
good way to understand if a data is valid or not (indeed it's used in
this fs), but with this schema you can prevent the system to do
something wrong. This option, however, can be enabled/disabled via
kconfig.
> Also, I'd like to ensure that direct memory access to the memory area
> would be available.
What do you exactly mean with this? Can you explain to me a bit deeper?
> There are some things (like the sched/exception logging mentioned above) where we want to make accesses as fast as possible.
>
> Chris
>
>
Marco
> > Why is an entire filesystem needed, instead of simply a block driver
> > if the ramdisk driver cannot be used?
> >
>
> >From documentation:
>
> "A relatively straight-forward solution is to write a simple block
> driver for the non-volatile RAM, and mount over it any disk-based
> filesystem such as ext2/ext3, reiserfs, etc.
>
> But the disk-based fs over non-volatile RAM block driver approach has
> some drawbacks:
>
> 1. Disk-based filesystems such as ext2/ext3 were designed for optimum
> performance on spinning disk media, so they implement features such
> as block groups, which attempts to group inode data into a contiguous
> set of data blocks to minimize disk seeking when accessing files. For
> RAM there is no such concern; a file's data blocks can be scattered
> throughout the media with no access speed penalty at all. So block
> groups in a filesystem mounted over RAM just adds unnecessary
> complexity. A better approach is to use a filesystem specifically
> tailored to RAM media which does away with these disk-based features.
> This increases the efficient use of space on the media, i.e. more
> space is dedicated to actual file data storage and less to meta-data
> needed to maintain that file data.
So... what is the performance difference between ext2 and your new
filesystem?
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
>>> Why is an entire filesystem needed, instead of simply a block driver
>>> if the ramdisk driver cannot be used?
>>>
>> >From documentation:
>>
>> "A relatively straight-forward solution is to write a simple block
>> driver for the non-volatile RAM, and mount over it any disk-based
>> filesystem such as ext2/ext3, reiserfs, etc.
>>
>> But the disk-based fs over non-volatile RAM block driver approach has
>> some drawbacks:
>>
>> 1. Disk-based filesystems such as ext2/ext3 were designed for optimum
>> performance on spinning disk media, so they implement features such
>> as block groups, which attempts to group inode data into a contiguous
>> set of data blocks to minimize disk seeking when accessing files. For
>> RAM there is no such concern; a file's data blocks can be scattered
>> throughout the media with no access speed penalty at all. So block
>> groups in a filesystem mounted over RAM just adds unnecessary
>> complexity. A better approach is to use a filesystem specifically
>> tailored to RAM media which does away with these disk-based features.
>> This increases the efficient use of space on the media, i.e. more
>> space is dedicated to actual file data storage and less to meta-data
>> needed to maintain that file data.
>
> So... what is the performance difference between ext2 and your new
> filesystem?
>
About the "space" you can read a detailed documentation on the site:
http://pramfs.sourceforge.net/pramfs-spec.html
In addition I can do an example of "compact" information: ext2 uses
directory entry objects ("dentries") to associate file names to inodes,
and these dentries are located in data blocks owned by the parent
directory. In pramfs, directory inode's do not need to own any data
blocks, because all dentry information is contained within the inode's
themselves.
>From performance point of view:
Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark
results to compare the performance with and without XIP in a real
embedded environment with bonnie++. You could use it as reference point.
I hope I've answered to your question.
Regards,
Marco
> >> 1. Disk-based filesystems such as ext2/ext3 were designed for optimum
> >> performance on spinning disk media, so they implement features such
> >> as block groups, which attempts to group inode data into a contiguous
> >> set of data blocks to minimize disk seeking when accessing files. For
> >> RAM there is no such concern; a file's data blocks can be scattered
> >> throughout the media with no access speed penalty at all. So block
> >> groups in a filesystem mounted over RAM just adds unnecessary
> >> complexity. A better approach is to use a filesystem specifically
> >> tailored to RAM media which does away with these disk-based features.
> >> This increases the efficient use of space on the media, i.e. more
> >> space is dedicated to actual file data storage and less to meta-data
> >> needed to maintain that file data.
> >
> > So... what is the performance difference between ext2 and your new
> > filesystem?
> >
>
> About the "space" you can read a detailed documentation on the site:
>
> http://pramfs.sourceforge.net/pramfs-spec.html
I do not see any numbers there. Do you think you can save significant
memory when storing for example kernel source trees?
> In addition I can do an example of "compact" information: ext2 uses
> directory entry objects ("dentries") to associate file names to
> inodes,
I'm not sure that on-disk directory entry == dentry.
> and these dentries are located in data blocks owned by the parent
> directory. In pramfs, directory inode's do not need to own any data
> blocks, because all dentry information is contained within the inode's
> themselves.
How do you handle hard-links, then?
> >From performance point of view:
>
> Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark
> results to compare the performance with and without XIP in a real
> embedded environment with bonnie++. You could use it as reference point.
Well, so XIP helps. ext2 can do XIP too, IIRC. Is your performance
better than ext2?
Wait... those numbers you pointed me... claim to be as slow as
13MB/sec. That's very very bad. My harddrive is faster than that.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
2009/6/21 Pavel Machek <[email protected]>:
>
>> >> 1. Disk-based filesystems such as ext2/ext3 were designed for optimum
>> >> ? ?performance on spinning disk media, so they implement features such
>> >> ? ?as block groups, which attempts to group inode data into a contiguous
>> >> ? ?set of data blocks to minimize disk seeking when accessing files. For
>> >> ? ?RAM there is no such concern; a file's data blocks can be scattered
>> >> ? ?throughout the media with no access speed penalty at all. So block
>> >> ? ?groups in a filesystem mounted over RAM just adds unnecessary
>> >> ? ?complexity. A better approach is to use a filesystem specifically
>> >> ? ?tailored to RAM media which does away with these disk-based features.
>> >> ? ?This increases the efficient use of space on the media, i.e. more
>> >> ? ?space is dedicated to actual file data storage and less to meta-data
>> >> ? ?needed to maintain that file data.
>> >
>> > So... what is the performance difference between ext2 and your new
>> > filesystem?
>> >
>>
>> About the "space" you can read a detailed documentation on the site:
>>
>> http://pramfs.sourceforge.net/pramfs-spec.html
>
> I do not see any numbers there. Do you think you can save significant
> memory when storing for example kernel source trees?
There aren't benchmark, but I pointed it out because if you know ext2
you can do a comparison.
>
>> In addition I can do an example of "compact" information: ext2 uses
>> directory entry objects ("dentries") to associate file names to
>> inodes,
>
> I'm not sure that on-disk directory entry == dentry.
>
>> and these dentries are located in data blocks owned by the parent
>> directory. In pramfs, directory inode's do not need to own any data
>> blocks, because all dentry information is contained within the inode's
>> themselves.
>
> How do you handle hard-links, then?
Indeed hard-links are not supported :) Due to the design of this fs
there are some limitations explained in the documentation as not
hard-link, only private memory mapping and so on. However this
limitations don't limit the fs itself because you must consider the
special goal of this fs.
>
>> >From performance point of view:
>>
>> Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark
>> results to compare the performance with and without XIP in a real
>> embedded environment with bonnie++. You could use it as reference point.
>
> Well, so XIP helps. ext2 can do XIP too, IIRC. Is your performance
> better than ext2?
>
> Wait... those numbers you pointed me... claim to be as slow as
> 13MB/sec. That's very very bad. My harddrive is faster than that.
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Pavel
>
As I said I did the test in a real embedded environment so to have
comparable result you should use the same environmente with the same
tools, with the same workload and so on.
Marco
> > How do you handle hard-links, then?
>
> Indeed hard-links are not supported :) Due to the design of this fs
> there are some limitations explained in the documentation as not
> hard-link, only private memory mapping and so on. However this
> limitations don't limit the fs itself because you must consider the
> special goal of this fs.
I did not see that in the changelog. If it is not general purpose
filesystem, it is lot less interesting.
> >> >From performance point of view:
> >>
> >> Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark
> >> results to compare the performance with and without XIP in a real
> >> embedded environment with bonnie++. You could use it as reference point.
> >
> > Well, so XIP helps. ext2 can do XIP too, IIRC. Is your performance
> > better than ext2?
> >
> > Wait... those numbers you pointed me... claim to be as slow as
> > 13MB/sec. That's very very bad. My harddrive is faster than that.
>
> As I said I did the test in a real embedded environment so to have
> comparable result you should use the same environmente with the same
> tools, with the same workload and so on.
Even on real embedded hardware you should get better than 13MB/sec
writing to _RAM_. I guess something is seriously wrong with pramfs.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
>>> How do you handle hard-links, then?
>> Indeed hard-links are not supported :) Due to the design of this fs
>> there are some limitations explained in the documentation as not
>> hard-link, only private memory mapping and so on. However this
>> limitations don't limit the fs itself because you must consider the
>> special goal of this fs.
>
> I did not see that in the changelog. If it is not general purpose
> filesystem, it is lot less interesting.
PRAMFS is not a general purpose filesystem. Please read
the introductory post to this thread, or look at
http://pramfs.sourceforge.net/ for more information.
Since the purpose of PRAMFS is to provide a filesystem
that is persistent across kernel instantions, it is not
designed for high speed. Robustness in the face of
kernel crashes or bugs is the highest priority, so
PRAMFS has significant overhead to make the window
of writability to the filesystem RAM as small as possible.
This is not a file system one would do kernel compiles on.
This is where someone would keep a small amount of sensitive
data, or crash logs that one needed to preserve over kernel
invocations.
>
>>>> >From performance point of view:
>>>>
>>>> Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark
>>>> results to compare the performance with and without XIP in a real
>>>> embedded environment with bonnie++. You could use it as reference point.
>>> Well, so XIP helps. ext2 can do XIP too, IIRC. Is your performance
>>> better than ext2?
>>>
>>> Wait... those numbers you pointed me... claim to be as slow as
>>> 13MB/sec. That's very very bad. My harddrive is faster than that.
>> As I said I did the test in a real embedded environment so to have
>> comparable result you should use the same environmente with the same
>> tools, with the same workload and so on.
>
> Even on real embedded hardware you should get better than 13MB/sec
> writing to _RAM_. I guess something is seriously wrong with pramfs.
See above.
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
On Mon 2009-06-22 10:31:28, Tim Bird wrote:
> Pavel Machek wrote:
> >>> How do you handle hard-links, then?
> >> Indeed hard-links are not supported :) Due to the design of this fs
> >> there are some limitations explained in the documentation as not
> >> hard-link, only private memory mapping and so on. However this
> >> limitations don't limit the fs itself because you must consider the
> >> special goal of this fs.
> >
> > I did not see that in the changelog. If it is not general purpose
> > filesystem, it is lot less interesting.
>
> PRAMFS is not a general purpose filesystem. Please read
> the introductory post to this thread, or look at
> http://pramfs.sourceforge.net/ for more information.
Yeah, I seen that. It directly contradicts what you say.
> Since the purpose of PRAMFS is to provide a filesystem
> that is persistent across kernel instantions, it is not
> designed for high speed. Robustness in the face of
> kernel crashes or bugs is the highest priority, so
> PRAMFS has significant overhead to make the window
> of writability to the filesystem RAM as small as possible.
Really? So why don't you use well known, reliable fs like ext3?
> This is not a file system one would do kernel compiles on.
> This is where someone would keep a small amount of sensitive
> data, or crash logs that one needed to preserve over kernel
> invocations.
Really? Web page says:
#2. If the backing-store RAM is comparable in access speed to system
#memory, there's really no point in caching the file I/O data in the
#page cache. Better to move file data directly between the user buffers
#and the backing store RAM, i.e. use direct I/O. This prevents the
#unnecessary
So you don't cache it "because its fast", and then it is 13MB/sec?
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
> On Mon 2009-06-22 10:31:28, Tim Bird wrote:
>> Pavel Machek wrote:
>>>>> How do you handle hard-links, then?
>>>> Indeed hard-links are not supported :) Due to the design of this fs
>>>> there are some limitations explained in the documentation as not
>>>> hard-link, only private memory mapping and so on. However this
>>>> limitations don't limit the fs itself because you must consider the
>>>> special goal of this fs.
>>> I did not see that in the changelog. If it is not general purpose
>>> filesystem, it is lot less interesting.
>> PRAMFS is not a general purpose filesystem. Please read
>> the introductory post to this thread, or look at
>> http://pramfs.sourceforge.net/ for more information.
>
> Yeah, I seen that. It directly contradicts what you say.
>
I don't think, I think it's very clear:
"In summary, PRAMFS is a light-weight, full-featured, and
space-efficient special filesystem that is ideal for systems with a
block of fast non-volatile RAM that need to access data on it using a
standard filesytem interface."
>> Since the purpose of PRAMFS is to provide a filesystem
>> that is persistent across kernel instantions, it is not
>> designed for high speed. Robustness in the face of
>> kernel crashes or bugs is the highest priority, so
>> PRAMFS has significant overhead to make the window
>> of writability to the filesystem RAM as small as possible.
>
> Really? So why don't you use well known, reliable fs like ext3?
>
>> This is not a file system one would do kernel compiles on.
>> This is where someone would keep a small amount of sensitive
>> data, or crash logs that one needed to preserve over kernel
>> invocations.
>
> Really? Web page says:
>
> #2. If the backing-store RAM is comparable in access speed to system
> #memory, there's really no point in caching the file I/O data in the
> #page cache. Better to move file data directly between the user buffers
> #and the backing store RAM, i.e. use direct I/O. This prevents the
> #unnecessary
>
> So you don't cache it "because its fast", and then it is 13MB/sec?
>
> Pavel
Tim Bird wrote:
> Pavel Machek wrote:
>>>> How do you handle hard-links, then?
>>> Indeed hard-links are not supported :) Due to the design of this fs
>>> there are some limitations explained in the documentation as not
>>> hard-link, only private memory mapping and so on. However this
>>> limitations don't limit the fs itself because you must consider the
>>> special goal of this fs.
>> I did not see that in the changelog. If it is not general purpose
>> filesystem, it is lot less interesting.
>
> PRAMFS is not a general purpose filesystem. Please read
> the introductory post to this thread, or look at
> http://pramfs.sourceforge.net/ for more information.
>
> Since the purpose of PRAMFS is to provide a filesystem
> that is persistent across kernel instantions, it is not
> designed for high speed. Robustness in the face of
> kernel crashes or bugs is the highest priority, so
> PRAMFS has significant overhead to make the window
> of writability to the filesystem RAM as small as possible.
>
> This is not a file system one would do kernel compiles on.
> This is where someone would keep a small amount of sensitive
> data, or crash logs that one needed to preserve over kernel
> invocations.
>
Yep, I quite agree.
Marco
Pavel Machek wrote:
> On Mon 2009-06-22 10:31:28, Tim Bird wrote:
>> Pavel Machek wrote:
>>> I did not see that in the changelog. If it is not general purpose
>>> filesystem, it is lot less interesting.
>> PRAMFS is not a general purpose filesystem. Please read
>> the introductory post to this thread, or look at
>> http://pramfs.sourceforge.net/ for more information.
>
> Yeah, I seen that. It directly contradicts what you say.
Could you be more specific? In what way does the
description on the website contradict what I said?
>> Since the purpose of PRAMFS is to provide a filesystem
>> that is persistent across kernel instantions, it is not
>> designed for high speed. Robustness in the face of
>> kernel crashes or bugs is the highest priority, so
>> PRAMFS has significant overhead to make the window
>> of writability to the filesystem RAM as small as possible.
>
> Really? So why don't you use well known, reliable fs like ext3?
Are you sure you read the web site? It directly addresses this
question. From the web site: "1. Disk-based filesystems such as
ext2/ext3 were designed for optimum performance on spinning disk
media, so they implement features such as block groups, which
attempts to group inode data into a contiguous set of data blocks
to minimize disk seeking when accessing files. For RAM there is
no such concern; a file's data blocks can be scattered throughout
the media with no access speed penalty at all. So block groups in a
filesystem mounted over RAM just adds unnecessary complexity. A
better approach is to use a filesystem specifically tailored to
RAM media which does away with these disk-based features. This
increases the efficient use of space on the media, i.e. more
space is dedicated to actual file data storage and less to
meta-data needed to maintain that file data."
>> This is not a file system one would do kernel compiles on.
>> This is where someone would keep a small amount of sensitive
>> data, or crash logs that one needed to preserve over kernel
>> invocations.
>
> Really? Web page says:
>
> #2. If the backing-store RAM is comparable in access speed to system
> #memory, there's really no point in caching the file I/O data in the
> #page cache. Better to move file data directly between the user buffers
> #and the backing store RAM, i.e. use direct I/O. This prevents the
> #unnecessary
>
> So you don't cache it "because its fast", and then it is 13MB/sec?
I'm not sure what you're quoting. The paragraph you quoted doesn't
have the words "because its fast" [sic]. Speed is not a primary
consideration for this filesystem. Due to the overhead of changing
the page flags, it's possible that using the page cache would
speed up overall access to this filesystem. (This would depend on
the cost of changing the access flags, and the pattern of operations
on the filesystem.) However, using the page cache would add
a level of indirection which would detract from the intended
robustness.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
On Mon 2009-06-22 20:07:06, Marco wrote:
> Pavel Machek wrote:
> > On Mon 2009-06-22 10:31:28, Tim Bird wrote:
> >> Pavel Machek wrote:
> >>>>> How do you handle hard-links, then?
> >>>> Indeed hard-links are not supported :) Due to the design of this fs
> >>>> there are some limitations explained in the documentation as not
> >>>> hard-link, only private memory mapping and so on. However this
> >>>> limitations don't limit the fs itself because you must consider the
> >>>> special goal of this fs.
> >>> I did not see that in the changelog. If it is not general purpose
> >>> filesystem, it is lot less interesting.
> >> PRAMFS is not a general purpose filesystem. Please read
> >> the introductory post to this thread, or look at
> >> http://pramfs.sourceforge.net/ for more information.
> >
> > Yeah, I seen that. It directly contradicts what you say.
> >
>
> I don't think, I think it's very clear:
>
> "In summary, PRAMFS is a light-weight, full-featured, and
> space-efficient special filesystem that is ideal for systems with a
Except that it is not full-featured. No hardlinks. (What about ACLs, etc?)
> block of fast non-volatile RAM that need to access data on it using a
> standard filesytem interface."
Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
better with ext2.
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Mon, 22 Jun 2009, Marco wrote:
> I don't think, I think it's very clear:
>
> "In summary, PRAMFS is a light-weight, full-featured, and
> space-efficient special filesystem that is ideal for systems with a
> block of fast non-volatile RAM that need to access data on it using a
> standard filesytem interface."
It is not full-featured if it doesn't have support for hardlinks, security
labels, extended attributes, etc. Please call it a specialized filesystem
instead, that seems to be much more in line with the comments about pramfs
use cases in this thread...
Oh, and that should be in the Kconfig help text, as well as the stuff that
goes in Documentation/*, obviously. Maybe it already is, I didn't look.
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
On Mon 2009-06-22 11:55:04, Tim Bird wrote:
> Pavel Machek wrote:
> > On Mon 2009-06-22 10:31:28, Tim Bird wrote:
> >> Pavel Machek wrote:
> >>> I did not see that in the changelog. If it is not general purpose
> >>> filesystem, it is lot less interesting.
> >> PRAMFS is not a general purpose filesystem. Please read
> >> the introductory post to this thread, or look at
> >> http://pramfs.sourceforge.net/ for more information.
> >
> > Yeah, I seen that. It directly contradicts what you say.
>
> Could you be more specific? In what way does the
> description on the website contradict what I said?
You are saying top goal is robustness, while the web page says (home
page, stop using frames!):
"embedded systems have a block of non-volatile RAM seperate from
normal system memory, i.e. of which the kernel maintains no memory
page descriptors. For such systems it would be beneficial to mount a
fast read/write filesystem over this "I/O memory", for storing
frequently accessed data that must survive system reboots and power
cycles"
Note the "frequently accessed" and "fast".
IOW the web page is confusing. It does not talk about robustness at
all.
> >> Since the purpose of PRAMFS is to provide a filesystem
> >> that is persistent across kernel instantions, it is not
> >> designed for high speed. Robustness in the face of
> >> kernel crashes or bugs is the highest priority, so
> >> PRAMFS has significant overhead to make the window
> >> of writability to the filesystem RAM as small as possible.
> >
> > Really? So why don't you use well known, reliable fs like ext3?
>
> Are you sure you read the web site? It directly addresses this
> question. From the web site: "1. Disk-based filesystems such as
No, it does not. It explains that ext2 would be too slow on this, and
explains that it will eat too much disk space. Please back that claims
with numbers.
If reliability is top concern, explain how you get away w/o
journalling.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
>> block of fast non-volatile RAM that need to access data on it using a
>> standard filesytem interface."
>
> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
> better with ext2.
Not if you want the RAM-based filesystem to persist over a kernel
invocation.
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
On Mon 2009-06-22 14:50:01, Tim Bird wrote:
> Pavel Machek wrote:
> >> block of fast non-volatile RAM that need to access data on it using a
> >> standard filesytem interface."
> >
> > Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
> > better with ext2.
>
> Not if you want the RAM-based filesystem to persist over a kernel
> invocation.
Yes, you'll need to code Persistent, RAM-based _block_device_.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
> You are saying top goal is robustness, while the web page says (home
> page, stop using frames!):
>
> "embedded systems have a block of non-volatile RAM seperate from
> normal system memory, i.e. of which the kernel maintains no memory
> page descriptors. For such systems it would be beneficial to mount a
> fast read/write filesystem over this "I/O memory", for storing
> frequently accessed data that must survive system reboots and power
> cycles"
>
> Note the "frequently accessed" and "fast".
>
> IOW the web page is confusing. It does not talk about robustness at
> all.
OK, thanks.
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
On Mon 2009-06-22 23:57:53, Pavel Machek wrote:
> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
> > Pavel Machek wrote:
> > >> block of fast non-volatile RAM that need to access data on it using a
> > >> standard filesytem interface."
> > >
> > > Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
> > > better with ext2.
> >
> > Not if you want the RAM-based filesystem to persist over a kernel
> > invocation.
>
> Yes, you'll need to code Persistent, RAM-based _block_device_.
More politely said: "I believe you would be better off modifying
ramdisk to include the functionality for persistence." New filesystem
should not really be neccessary. ext2 for performance, ext3 if you
need robustness from journalling, maybe something else makes sense,
too.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
> More politely said: "I believe you would be better off modifying
> ramdisk to include the functionality for persistence." New filesystem
> should not really be neccessary. ext2 for performance, ext3 if you
> need robustness from journalling, maybe something else makes sense,
> too.
I agree that a persistent block device makes more sense.
However, as someone with some experience in using this type of
mechanism, a special filesystem (separate from the special block device)
could also be useful. We often use things like circular buffers,
per-cpu areas, both log-structured and variable-length records, etc. A
filesystem that would take care of this type of thing under the hood
might simplify a few things.
Also note that it's very useful for the kernel itself to be able to
access the contents of this persistent area...we use it for parts of the
log stream, various forms of "flight recorder" information, panic
tracebacks, etc. as well as making it available to select userspace apps.
Chris
On Mon, Jun 22, 2009 at 05:26:45PM -0600, Chris Friesen wrote:
> Pavel Machek wrote:
>
> > More politely said: "I believe you would be better off modifying
> > ramdisk to include the functionality for persistence." New filesystem
> > should not really be neccessary. ext2 for performance, ext3 if you
> > need robustness from journalling, maybe something else makes sense,
> > too.
>
> I agree that a persistent block device makes more sense.
>
> However, as someone with some experience in using this type of
> mechanism, a special filesystem (separate from the special block device)
> could also be useful. We often use things like circular buffers,
> per-cpu areas, both log-structured and variable-length records, etc. A
> filesystem that would take care of this type of thing under the hood
> might simplify a few things.
>
> Also note that it's very useful for the kernel itself to be able to
> access the contents of this persistent area...we use it for parts of the
> log stream, various forms of "flight recorder" information, panic
> tracebacks, etc. as well as making it available to select userspace apps.
I think we might step back from this particular patch and look at what people
actually want. From my standpoint, we use, or intend to use, persistent memory
in three ways:
1. Recording crash logs from user space
2. Recording crash logs from kernel space
3. Flight recorder/continuous logging.
The first two uses share the same chunk of memory. In the first case, the
interface is through filesystem operations from user space, but simple memcpys
in kernel space. Neither of these uses is performance critical
The third use is a different memory chunk and, depending on just what you log,
can have a significant performance impact. It is helpful if this can be a
single chunk of pram that is contiguous in kernel virtual space. Depending
on how you use it, you might even want it to be uncached. I'm not clear on
whether Pramfs supports this.
All three use cases can be satisfied with a persistent memory block device,
as others have mentioned, so long as multiple pram partitions are supported.
We might steal the mtdparts syntax for this (see drivers/mtd/cmdlinepart.c).
Another consideration--we actually stopped using a checksum on our crash logs.
The crash logs are in text, so if they are a little bit stomped on, they
are still usable. If you require a matching checksum, you lose the ability
to read those slightly dented logs. So, all the work that's being done to
ensure data integrity in the pram filesystem is actually harmful in this case.
I don't know whether a pram block device helps at all with a pramfs, but I
wouldn't want to lose ability to choose to use the block device directly
or switch to a pramfs.
> Chris
Pavel Machek wrote:
> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
>> Pavel Machek wrote:
>>>> block of fast non-volatile RAM that need to access data on it using a
>>>> standard filesytem interface."
>>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
>>> better with ext2.
>> Not if you want the RAM-based filesystem to persist over a kernel
>> invocation.
>
> Yes, you'll need to code Persistent, RAM-based _block_device_.
> Pavel
First of all I have to say that I'd like to update the site and make it
clearer but at the moment it's not possible because I'm not the admin
and I've already asked to the sourceforge support to have this possibility.
About the comments: sincerely I don't understand the comments. We have
*already* a fs that takes care to remap a piace of ram (ram, sram,
nvram, etc.), takes care of caching problems, takes care of write
protection, takes care to be "persistent", can use xip and it's my
intention to add in the next future other features like acl, xattr and
so on.
You are talked about journaling. This schema works well for a disk, but
what about a piece of ram? What about a crazy kernel that write in that
area for a bug? Do you remember for example the e1000e bug? It's not
casual that this fs use an hw protection schema. I mean, this fs is not
only a "yet another fs", but a fs born with a specific target. So I
think modifying a ramdisk to have these behaviors isn't a little
modification.
Marco
On Tue 2009-06-23 20:07:23, Marco wrote:
> Pavel Machek wrote:
> > On Mon 2009-06-22 14:50:01, Tim Bird wrote:
> >> Pavel Machek wrote:
> >>>> block of fast non-volatile RAM that need to access data on it using a
> >>>> standard filesytem interface."
> >>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
> >>> better with ext2.
> >> Not if you want the RAM-based filesystem to persist over a kernel
> >> invocation.
> >
> > Yes, you'll need to code Persistent, RAM-based _block_device_.
>
> First of all I have to say that I'd like to update the site and make it
> clearer but at the moment it's not possible because I'm not the admin
> and I've already asked to the sourceforge support to have this possibility.
>
> About the comments: sincerely I don't understand the comments. We have
> *already* a fs that takes care to remap a piace of ram (ram, sram,
> nvram, etc.), takes care of caching problems, takes care of write
Well, it looks pramfs design is confused. 13MB/sec shows that caching
_is_ useful for pramfs. So...?
> You are talked about journaling. This schema works well for a disk, but
> what about a piece of ram? What about a crazy kernel that write in that
> area for a bug? Do you remember for example the e1000e bug? It's not
I believe you need both journaling *and* write protection. How do you
handle power fault while writing data?
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
>> Pavel Machek wrote:
>>> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
>>>> Pavel Machek wrote:
>>>>>> block of fast non-volatile RAM that need to access data on it using a
>>>>>> standard filesytem interface."
>>>>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
>>>>> better with ext2.
>>>> Not if you want the RAM-based filesystem to persist over a kernel
>>>> invocation.
>>> Yes, you'll need to code Persistent, RAM-based _block_device_.
>> First of all I have to say that I'd like to update the site and make it
>> clearer but at the moment it's not possible because I'm not the admin
>> and I've already asked to the sourceforge support to have this possibility.
>>
>> About the comments: sincerely I don't understand the comments. We have
>> *already* a fs that takes care to remap a piace of ram (ram, sram,
>> nvram, etc.), takes care of caching problems, takes care of write
>
> Well, it looks pramfs design is confused. 13MB/sec shows that caching
> _is_ useful for pramfs. So...?
caching problems means to avoid filesystem corruption, so dirty pages in
the page cache are not allowed to be written back to the backing-store
RAM. It's clear that there is a performance penalty. This penalty should
be reduced by the access speed of the RAM, however the performance are
not important for this special fs as Tim Bird said, so this question is
not relevant for me. If this issue is not clear enough on the web site,
I hope I can update the information in the future.
>
>> You are talked about journaling. This schema works well for a disk, but
>> what about a piece of ram? What about a crazy kernel that write in that
>> area for a bug? Do you remember for example the e1000e bug? It's not
>
> I believe you need both journaling *and* write protection. How do you
> handle power fault while writing data?
> Pavel
Ah now the write protection is a "needed feature", in your previous
comment you talked about why not use ext2/3.......
Marco
Marco wrote:
> > Second question: what happens if the system crashing _during_ a write
> > to a file. Does it mean that file will fail it's checksum when it's
> > read at the next boot?
> >
> > Maybe files aren't so important. What about when you write a file,
> > and then rename it over an existing file to replace it. (E.g. a
> > config file), and the system crashes _during_ the rename? At the next
> > boot, is it guaranteed to see either the old or the new file, or can
> > the directory be corrupt / fail it's checksum?
>
> First of all I have to explain better the current policy: the checksum
> works at inode and superblock level and currently there isn't a recovery
> function as the journaling. About the superblock it's easy to use a
> redundant policy to be more robust.
To be honest, superblock robustness is less of a concern. The real
concern is losing file or directory contents, so it can't be used to
store persistent configuration data, only debugging logs.
> About the inode, at the moment when the checksum doesn't match the
> inode it's marked as bad calling the function make_bad_inode().
Let's see if I understand right.
If it lose power when writing to a file, after boot the file is likely
to be marked bad and so return -EIO instead of any file contents?
If it loses power when doing atomic rename (to replace config files,
for example), it's likely that the whole /pramfs/configs/ directory
will be corrupt, because the rename is writing to the directory inode,
so you lose access to all names in that directory?
That sounds like it can't be used for persistent configuration data.
If a directory is marked as bad, or a file-inode in it is marked bad,
can you even rmdir it to clean up and start again?
Thanks,
-- Jamie
>>> Pavel Machek wrote:
>>>> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
>>>>> Pavel Machek wrote:
>>>>>>> block of fast non-volatile RAM that need to access data on it using a
>>>>>>> standard filesytem interface."
>>>>>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
>>>>>> better with ext2.
>>>>> Not if you want the RAM-based filesystem to persist over a kernel
>>>>> invocation.
>>>> Yes, you'll need to code Persistent, RAM-based _block_device_.
>>> First of all I have to say that I'd like to update the site and make it
>>> clearer but at the moment it's not possible because I'm not the admin
>>> and I've already asked to the sourceforge support to have this possibility.
>>>
>>> About the comments: sincerely I don't understand the comments. We have
>>> *already* a fs that takes care to remap a piace of ram (ram, sram,
>>> nvram, etc.), takes care of caching problems, takes care of write
>> Well, it looks pramfs design is confused. 13MB/sec shows that caching
>> _is_ useful for pramfs. So...?
>
> caching problems means to avoid filesystem corruption, so dirty pages in
> the page cache are not allowed to be written back to the backing-store
> RAM. It's clear that there is a performance penalty. This penalty should
> be reduced by the access speed of the RAM, however the performance are
> not important for this special fs as Tim Bird said, so this question is
> not relevant for me. If this issue is not clear enough on the web site,
> I hope I can update the information in the future.
>
>>> You are talked about journaling. This schema works well for a disk, but
>>> what about a piece of ram? What about a crazy kernel that write in that
>>> area for a bug? Do you remember for example the e1000e bug? It's not
>> I believe you need both journaling *and* write protection. How do you
>> handle power fault while writing data?
>> Pavel
>
> Ah now the write protection is a "needed feature", in your previous
> comment you talked about why not use ext2/3.......
>
> Marco
>
Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
hostname 15M:1k 14156 99 128779 100 92240 100 11669 100 166242 99 80058 82
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
4 2842 99 133506 104 45088 101 2787 99 79581 101 58212 102
These data are the proof of the importance of the environment, workload and so on when we talk
about benchmark. Your consideration are really superficial.
Marco
Pavel Machek wrote:
> On Tue 2009-06-23 20:07:23, Marco wrote:
> > You are talked about journaling. This schema works well for a disk, but
> > what about a piece of ram? What about a crazy kernel that write in that
> > area for a bug? Do you remember for example the e1000e bug? It's not
>
> I believe you need both journaling *and* write protection. How do you
> handle power fault while writing data?
I think this is basically right.
write protection for the crazy kernels, and journalling for
powerfail/crash during updates.
Journalling can be extremely simple. It can be just one memory block
at a fixed location, double-buffering all writes.
Pramfs already has checksums, which makes that easier. You just write
to the buffer area first, with checksum, then write to the final area.
Mount looks at the buffer area, and if the checksum is fine, copies
the contents to the destination block.
That's all it takes to be resistant against power failures and crashes
during writes. Probably <100 lines of code.
-- Jamie
On Wed 2009-06-24 18:49:11, Marco wrote:
> >> Pavel Machek wrote:
> >>> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
> >>>> Pavel Machek wrote:
> >>>>>> block of fast non-volatile RAM that need to access data on it using a
> >>>>>> standard filesytem interface."
> >>>>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
> >>>>> better with ext2.
> >>>> Not if you want the RAM-based filesystem to persist over a kernel
> >>>> invocation.
> >>> Yes, you'll need to code Persistent, RAM-based _block_device_.
> >> First of all I have to say that I'd like to update the site and make it
> >> clearer but at the moment it's not possible because I'm not the admin
> >> and I've already asked to the sourceforge support to have this possibility.
> >>
> >> About the comments: sincerely I don't understand the comments. We have
> >> *already* a fs that takes care to remap a piace of ram (ram, sram,
> >> nvram, etc.), takes care of caching problems, takes care of write
> >
> > Well, it looks pramfs design is confused. 13MB/sec shows that caching
> > _is_ useful for pramfs. So...?
>
> caching problems means to avoid filesystem corruption, so dirty pages in
> the page cache are not allowed to be written back to the backing-store
> RAM. It's clear that there is a performance penalty. This penalty should
> be reduced by the access speed of the RAM, however the performance are
> not important for this special fs as Tim Bird said, so this question is
> not relevant for me. If this issue is not clear enough on the web site,
> I hope I can update the information in the future.
Yes, please update the pages...
> >> You are talked about journaling. This schema works well for a disk, but
> >> what about a piece of ram? What about a crazy kernel that write in that
> >> area for a bug? Do you remember for example the e1000e bug? It's not
> >
> > I believe you need both journaling *and* write protection. How do you
> > handle power fault while writing data?
>
> Ah now the write protection is a "needed feature", in your previous
> comment you talked about why not use ext2/3.......
write protection should be handled at block device layer, not
filesystem layer. So yes, use ext2.
You still did not explain how you avoid the need for journalling...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Wed 2009-06-24 19:38:37, Marco wrote:
> >>> Pavel Machek wrote:
> >>>> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
> >>>>> Pavel Machek wrote:
> >>>>>>> block of fast non-volatile RAM that need to access data on it using a
> >>>>>>> standard filesytem interface."
> >>>>>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
> >>>>>> better with ext2.
> >>>>> Not if you want the RAM-based filesystem to persist over a kernel
> >>>>> invocation.
> >>>> Yes, you'll need to code Persistent, RAM-based _block_device_.
> >>> First of all I have to say that I'd like to update the site and make it
> >>> clearer but at the moment it's not possible because I'm not the admin
> >>> and I've already asked to the sourceforge support to have this possibility.
> >>>
> >>> About the comments: sincerely I don't understand the comments. We have
> >>> *already* a fs that takes care to remap a piace of ram (ram, sram,
> >>> nvram, etc.), takes care of caching problems, takes care of write
> >> Well, it looks pramfs design is confused. 13MB/sec shows that caching
> >> _is_ useful for pramfs. So...?
> >
> > caching problems means to avoid filesystem corruption, so dirty pages in
> > the page cache are not allowed to be written back to the backing-store
> > RAM. It's clear that there is a performance penalty. This penalty should
> > be reduced by the access speed of the RAM, however the performance are
> > not important for this special fs as Tim Bird said, so this question is
> > not relevant for me. If this issue is not clear enough on the web site,
> > I hope I can update the information in the future.
> >
> >>> You are talked about journaling. This schema works well for a disk, but
> >>> what about a piece of ram? What about a crazy kernel that write in that
> >>> area for a bug? Do you remember for example the e1000e bug? It's not
> >> I believe you need both journaling *and* write protection. How do you
> >> handle power fault while writing data?
> >> Pavel
> >
> > Ah now the write protection is a "needed feature", in your previous
> > comment you talked about why not use ext2/3.......
> >
> > Marco
> >
>
> Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
>
> Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
> hostname 15M:1k 14156 99 128779 100 92240 100 11669 100 166242 99 80058 82
> ------Sequential Create------ --------Random Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> 4 2842 99 133506 104 45088 101 2787 99 79581 101 58212 102
>
> These data are the proof of the importance of the environment, workload and so on when we talk
> about benchmark. Your consideration are really superficial.
Unfortunately, your numbers are meaningless.
Pavel
(PCs should have cca 3GB/sec RAM transfer rates; and you demosstrated
cca 166MB/sec read rate; disk is 80MB/sec, so that's too slow. If you
want to prove your filesystem the filesystem is reasonably fast,
compare it with ext2 on ramdisk.)
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
2009/6/24 Pavel Machek <[email protected]>:
> On Wed 2009-06-24 19:38:37, Marco wrote:
>> >>> Pavel Machek wrote:
>> >>>> On Mon 2009-06-22 14:50:01, Tim Bird wrote:
>> >>>>> Pavel Machek wrote:
>> >>>>>>> block of fast non-volatile RAM that need to access data on it using a
>> >>>>>>> standard filesytem interface."
>> >>>>>> Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are
>> >>>>>> better with ext2.
>> >>>>> Not if you want the RAM-based filesystem to persist over a kernel
>> >>>>> invocation.
>> >>>> Yes, you'll need to code Persistent, RAM-based _block_device_.
>> >>> First of all I have to say that I'd like to update the site and make it
>> >>> clearer but at the moment it's not possible because I'm not the admin
>> >>> and I've already asked to the sourceforge support to have this possibility.
>> >>>
>> >>> About the comments: sincerely I don't understand the comments. We have
>> >>> *already* a fs that takes care to remap a piace of ram (ram, sram,
>> >>> nvram, etc.), takes care of caching problems, takes care of write
>> >> Well, it looks pramfs design is confused. 13MB/sec shows that caching
>> >> _is_ useful for pramfs. So...?
>> >
>> > caching problems means to avoid filesystem corruption, so dirty pages in
>> > the page cache are not allowed to be written back to the backing-store
>> > RAM. It's clear that there is a performance penalty. This penalty should
>> > be reduced by the access speed of the RAM, however the performance are
>> > not important for this special fs as Tim Bird said, so this question is
>> > not relevant for me. If this issue is not clear enough on the web site,
>> > I hope I can update the information in the future.
>> >
>> >>> You are talked about journaling. This schema works well for a disk, but
>> >>> what about a piece of ram? What about a crazy kernel that write in that
>> >>> area for a bug? Do you remember for example the e1000e bug? It's not
>> >> I believe you need both journaling *and* write protection. How do you
>> >> handle power fault while writing data?
>> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Pavel
>> >
>> > Ah now the write protection is a "needed feature", in your previous
>> > comment you talked about why not use ext2/3.......
>> >
>> > Marco
>> >
>>
>> Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
>>
>> Version 1.03e ? ? ? ------Sequential Output------ --Sequential Input- --Random-
>> ? ? ? ? ? ? ? ? ? ? -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>> Machine ? Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP ?/sec %CP
>> hostname ? ? 15M:1k 14156 ?99 128779 100 92240 100 11669 100 166242 ?99 80058 ?82
>> ? ? ? ? ? ? ? ? ? ? ------Sequential Create------ --------Random Create--------
>> ? ? ? ? ? ? ? ? ? ? -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
>> ? ? ? ? ? ? ? files ?/sec %CP ?/sec %CP ?/sec %CP ?/sec %CP ?/sec %CP ?/sec %CP
>> ? ? ? ? ? ? ? ? ? 4 ?2842 ?99 133506 104 45088 101 ?2787 ?99 79581 101 58212 102
>>
>> These data are the proof of the importance of the environment, workload and so on when we talk
>> about benchmark. Your consideration are really superficial.
>
> Unfortunately, your numbers are meaningless.
I don't think so.
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Pavel
>
> (PCs should have cca 3GB/sec RAM transfer rates; and you demosstrated
> cca 166MB/sec read rate; disk is 80MB/sec, so that's too slow. If you
> want to prove your filesystem the filesystem is reasonably fast,
> compare it with ext2 on ramdisk.)
>
This is the point. I don't want compare it with ext2 from performance
point of view. This comparison makes no sense for me. I've done this
test to prove that if you change environment you can change in a
purposeful way the results.
Marco
2009/6/24 Jamie Lokier <[email protected]>:
> Pavel Machek wrote:
>> On Tue 2009-06-23 20:07:23, Marco wrote:
>> > You are talked about journaling. This schema works well for a disk, but
>> > what about a piece of ram? What about a crazy kernel that write in that
>> > area for a bug? Do you remember for example the e1000e bug? It's not
>>
>> I believe you need both journaling *and* write protection. How do you
>> handle power fault while writing data?
>
> I think this is basically right.
>
> write protection for the crazy kernels, and journalling for
> powerfail/crash during updates.
>
> Journalling can be extremely simple. ?It can be just one memory block
> at a fixed location, double-buffering all writes.
>
> Pramfs already has checksums, which makes that easier. ?You just write
> to the buffer area first, with checksum, then write to the final area.
> Mount looks at the buffer area, and if the checksum is fine, copies
> the contents to the destination block.
>
> That's all it takes to be resistant against power failures and crashes
> during writes. ?Probably <100 lines of code.
>
> -- Jamie
>
It seems a reasonable request.
Marco
2009/6/24 Jamie Lokier <[email protected]>:
> Marco wrote:
>> > Second question: what happens if the system crashing _during_ a write
>> > to a file. ?Does it mean that file will fail it's checksum when it's
>> > read at the next boot?
>> >
>> > Maybe files aren't so important. ?What about when you write a file,
>> > and then rename it over an existing file to replace it. ?(E.g. a
>> > config file), and the system crashes _during_ the rename? ?At the next
>> > boot, is it guaranteed to see either the old or the new file, or can
>> > the directory be corrupt / fail it's checksum?
>>
>> First of all I have to explain better the current policy: the checksum
>> works at inode and superblock level and currently there isn't a recovery
>> function as the journaling. About the superblock it's easy to use a
>> redundant policy to be more robust.
>
> To be honest, superblock robustness is less of a concern. ?The real
> concern is losing file or directory contents, so it can't be used to
> store persistent configuration data, only debugging logs.
>
>> About the inode, at the moment when the checksum doesn't match the
>> inode it's marked as bad calling the function make_bad_inode().
>
> Let's see if I understand right.
>
> If it lose power when writing to a file, after boot the file is likely
> to be marked bad and so return -EIO instead of any file contents?
Depends on the checksum. If you lose power before the checksum update
of the inode
you'll have a bad inode and then an -EIO at the next access.
>
> If it loses power when doing atomic rename (to replace config files,
> for example), it's likely that the whole /pramfs/configs/ directory
> will be corrupt, because the rename is writing to the directory inode,
> so you lose access to all names in that directory?
>
> That sounds like it can't be used for persistent configuration data.
It's true from this point of view currently there is a lack for this
and it needs a bit of effort to resolve this problem.
>From this point of view I'd like to point out that I know that there
was some aspects to study in a deeper way, so I'll need
of more then one review :) but since this fs has been abandoned since
2004 and it hadn't ever reviewed, it was important
to do a serious review with the kernel community to understand all the
problems.
>
> If a directory is marked as bad, or a file-inode in it is marked bad,
> can you even rmdir it to clean up and start again?
>
You can start again always. You can remount the fs with the init
option and then you'll have a new fs.
Marco
Marco Stornelli wrote:
> 2009/6/24 Jamie Lokier <[email protected]>:
> > Marco wrote:
> >> > Second question: what happens if the system crashing _during_ a write
> >> > to a file. ?Does it mean that file will fail it's checksum when it's
> >> > read at the next boot?
> >> >
> >> > Maybe files aren't so important. ?What about when you write a file,
> >> > and then rename it over an existing file to replace it. ?(E.g. a
> >> > config file), and the system crashes _during_ the rename? ?At the next
> >> > boot, is it guaranteed to see either the old or the new file, or can
> >> > the directory be corrupt / fail it's checksum?
> >>
> >> First of all I have to explain better the current policy: the checksum
> >> works at inode and superblock level and currently there isn't a recovery
> >> function as the journaling. About the superblock it's easy to use a
> >> redundant policy to be more robust.
> >
> > To be honest, superblock robustness is less of a concern. ?The real
> > concern is losing file or directory contents, so it can't be used to
> > store persistent configuration data, only debugging logs.
> >
> >> About the inode, at the moment when the checksum doesn't match the
> >> inode it's marked as bad calling the function make_bad_inode().
> >
> > Let's see if I understand right.
> >
> > If it lose power when writing to a file, after boot the file is likely
> > to be marked bad and so return -EIO instead of any file contents?
>
> Depends on the checksum. If you lose power before the checksum update
> of the inode
> you'll have a bad inode and then an -EIO at the next access.
>
> >
> > If it loses power when doing atomic rename (to replace config files,
> > for example), it's likely that the whole /pramfs/configs/ directory
> > will be corrupt, because the rename is writing to the directory inode,
> > so you lose access to all names in that directory?
> >
> > That sounds like it can't be used for persistent configuration data.
>
> It's true from this point of view currently there is a lack for this
> and it needs a bit of effort to resolve this problem. >From this
> point of view I'd like to point out that I know that there was some
> aspects to study in a deeper way, so I'll need of more then one
> review :) but since this fs has been abandoned since 2004 and it
> hadn't ever reviewed, it was important to do a serious review with
> the kernel community to understand all the problems.
That's reasonable.
What do you think of my suggestion to double-buffer writes using a
single fixed position block, as explained elsewhere in this thread?
It should give the power fail safety with very little code. I don't
know how much it would slwo down writing. That probably depends on
whether it's the checksum which is slow (which only needs to be done
once when double-buffering), or the writing.
-- Jamie
Jamie Lokier wrote:
> Marco Stornelli wrote:
>> 2009/6/24 Jamie Lokier <[email protected]>:
>>> Marco wrote:
>>> If it loses power when doing atomic rename (to replace config files,
>>> for example), it's likely that the whole /pramfs/configs/ directory
>>> will be corrupt, because the rename is writing to the directory inode,
>>> so you lose access to all names in that directory?
>>>
>>> That sounds like it can't be used for persistent configuration data.
>> It's true from this point of view currently there is a lack for this
>> and it needs a bit of effort to resolve this problem. >From this
>> point of view I'd like to point out that I know that there was some
>> aspects to study in a deeper way, so I'll need of more then one
>> review :) but since this fs has been abandoned since 2004 and it
>> hadn't ever reviewed, it was important to do a serious review with
>> the kernel community to understand all the problems.
>
> That's reasonable.
>
> What do you think of my suggestion to double-buffer writes using a
> single fixed position block, as explained elsewhere in this thread?
>
> It should give the power fail safety with very little code. I don't
> know how much it would slwo down writing. That probably depends on
> whether it's the checksum which is slow (which only needs to be done
> once when double-buffering), or the writing.
>
> -- Jamie
>
Yeah it can be a choice. For this fs it's important to use "simple" but
useful mechanism. What do you exactly mean with "fixed position block"?
A fixed position in the fs? For example superblock+inodetable+in-use
bitmap+blocks+"double-buffering block"? Using a temp block of the same
size of blocks used, isn't it? I agree, but I think it needs more then
100 lines of code. Even with this simple schema it needs a mechanism
with a timeout to do the "commit" of the temp block, it needs a
mechanism to read the temp block instead of the "old" block or a
mechanism to write-back the temp block. So it can be implemented but it
needs a bit of effort. I think I'll implement it in the next release.
Marco
> >>> If it loses power when doing atomic rename (to replace config files,
> >>> for example), it's likely that the whole /pramfs/configs/ directory
> >>> will be corrupt, because the rename is writing to the directory inode,
> >>> so you lose access to all names in that directory?
> >>>
> >>> That sounds like it can't be used for persistent configuration data.
> >> It's true from this point of view currently there is a lack for this
> >> and it needs a bit of effort to resolve this problem. >From this
> >> point of view I'd like to point out that I know that there was some
> >> aspects to study in a deeper way, so I'll need of more then one
> >> review :) but since this fs has been abandoned since 2004 and it
> >> hadn't ever reviewed, it was important to do a serious review with
> >> the kernel community to understand all the problems.
> >
> > That's reasonable.
> >
> > What do you think of my suggestion to double-buffer writes using a
> > single fixed position block, as explained elsewhere in this thread?
> >
> > It should give the power fail safety with very little code. I don't
> > know how much it would slwo down writing. That probably depends on
> > whether it's the checksum which is slow (which only needs to be done
> > once when double-buffering), or the writing.
>
> Yeah it can be a choice. For this fs it's important to use "simple" but
> useful mechanism. What do you exactly mean with "fixed position block"?
> A fixed position in the fs? For example superblock+inodetable+in-use
> bitmap+blocks+"double-buffering block"? Using a temp block of the same
> size of blocks used, isn't it? I agree, but I think it needs more then
> 100 lines of code. Even with this simple schema it needs a mechanism
> with a timeout to do the "commit" of the temp block, it needs a
> mechanism to read the temp block instead of the "old" block or a
> mechanism to write-back the temp block. So it can be implemented but it
> needs a bit of effort. I think I'll implement it in the next release.
I think you should really use ext3 over special block device.
Initially this filesystem was presented as fast (it is not) then you
claim it is reliable (it is not). If you want it to get reliable
you'll need journalling+fsck. Good luck.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
> >> > Ah now the write protection is a "needed feature", in your previous
> >> > comment you talked about why not use ext2/3.......
> >> >
> >> > Marco
> >> >
> >>
> >> Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
> >>
> >> Version 1.03e ? ? ? ------Sequential Output------ --Sequential Input- --Random-
> >> ? ? ? ? ? ? ? ? ? ? -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> >> Machine ? Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP ?/sec %CP
> >> hostname ? ? 15M:1k 14156 ?99 128779 100 92240 100 11669 100 166242 ?99 80058 ?82
> >> ? ? ? ? ? ? ? ? ? ? ------Sequential Create------ --------Random Create--------
> >> ? ? ? ? ? ? ? ? ? ? -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> >> ? ? ? ? ? ? ? files ?/sec %CP ?/sec %CP ?/sec %CP ?/sec %CP ?/sec %CP ?/sec %CP
> >> ? ? ? ? ? ? ? ? ? 4 ?2842 ?99 133506 104 45088 101 ?2787 ?99 79581 101 58212 102
> >>
> >> These data are the proof of the importance of the environment, workload and so on when we talk
> >> about benchmark. Your consideration are really superficial.
> >
> > Unfortunately, your numbers are meaningless.
>
> I don't think so.
>
> > (PCs should have cca 3GB/sec RAM transfer rates; and you demosstrated
> > cca 166MB/sec read rate; disk is 80MB/sec, so that's too slow. If you
> > want to prove your filesystem the filesystem is reasonably fast,
> > compare it with ext2 on ramdisk.)
> >
> This is the point. I don't want compare it with ext2 from performance
> point of view. This comparison makes no sense for me. I've done this
> test to prove that if you change environment you can change in a
> purposeful way the results.
Yes, IOW you demonstrated that the numbers are machine-dependend and
really meaningless.
ext2 comparison would tell you how much pramfs sucks (or not).
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
>>>>> Ah now the write protection is a "needed feature", in your previous
>>>>> comment you talked about why not use ext2/3.......
>>>>>
>>>>> Marco
>>>>>
>>>> Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
>>>>
>>>> Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
>>>> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>>>> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
>>>> hostname 15M:1k 14156 99 128779 100 92240 100 11669 100 166242 99 80058 82
>>>> ------Sequential Create------ --------Random Create--------
>>>> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
>>>> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
>>>> 4 2842 99 133506 104 45088 101 2787 99 79581 101 58212 102
>>>>
>>>> These data are the proof of the importance of the environment, workload and so on when we talk
>>>> about benchmark. Your consideration are really superficial.
>>> Unfortunately, your numbers are meaningless.
>> I don't think so.
>>
>>> (PCs should have cca 3GB/sec RAM transfer rates; and you demosstrated
>>> cca 166MB/sec read rate; disk is 80MB/sec, so that's too slow. If you
>>> want to prove your filesystem the filesystem is reasonably fast,
>>> compare it with ext2 on ramdisk.)
>>>
>> This is the point. I don't want compare it with ext2 from performance
>> point of view. This comparison makes no sense for me. I've done this
>> test to prove that if you change environment you can change in a
>> purposeful way the results.
>
> Yes, IOW you demonstrated that the numbers are machine-dependend and
> really meaningless.
>
> ext2 comparison would tell you how much pramfs sucks (or not).
> Pavel
If you knew that the results were machine-dependent I don't understand
why you were so upset by my previous benchmark.
Marco
Pavel Machek wrote:
>>>>> Ah now the write protection is a "needed feature", in your previous
>>>>> comment you talked about why not use ext2/3.......
>>>>>
>>>>> Marco
>>>>>
>>>> Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
>>>>
>>>> Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
>>>> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>>>> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
>>>> hostname 15M:1k 14156 99 128779 100 92240 100 11669 100 166242 99 80058 82
>>>> ------Sequential Create------ --------Random Create--------
>>>> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
>>>> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
>>>> 4 2842 99 133506 104 45088 101 2787 99 79581 101 58212 102
>>>>
>>>> These data are the proof of the importance of the environment, workload and so on when we talk
>>>> about benchmark. Your consideration are really superficial.
>>> Unfortunately, your numbers are meaningless.
>> I don't think so.
>>
>>> (PCs should have cca 3GB/sec RAM transfer rates; and you demosstrated
>>> cca 166MB/sec read rate; disk is 80MB/sec, so that's too slow. If you
>>> want to prove your filesystem the filesystem is reasonably fast,
>>> compare it with ext2 on ramdisk.)
>>>
>> This is the point. I don't want compare it with ext2 from performance
>> point of view. This comparison makes no sense for me. I've done this
>> test to prove that if you change environment you can change in a
>> purposeful way the results.
>
> Yes, IOW you demonstrated that the numbers are machine-dependend and
> really meaningless.
>
> ext2 comparison would tell you how much pramfs sucks (or not).
> Pavel
Here the test with ext2 (same environment):
Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
hostname 15M:1k 10262 83 40847 82 38574 82 9866 92 62252 98 25204 81
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
1 19859 98 44804 61 68830 100 13566 99 157129 100 30431 98
Marco
On Sun 2009-06-28 19:33:02, Marco Stornelli wrote:
> Pavel Machek wrote:
> >>>>> Ah now the write protection is a "needed feature", in your previous
> >>>>> comment you talked about why not use ext2/3.......
> >>>>>
> >>>>> Marco
> >>>>>
> >>>> Just for your information I tried the same test with pc in a virtual machine with 32MB of RAM:
> >>>>
> >>>> Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
> >>>> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> >>>> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
> >>>> hostname 15M:1k 14156 99 128779 100 92240 100 11669 100 166242 99 80058 82
> >>>> ------Sequential Create------ --------Random Create--------
> >>>> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> >>>> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> >>>> 4 2842 99 133506 104 45088 101 2787 99 79581 101 58212 102
> >>>>
> >>>> These data are the proof of the importance of the environment, workload and so on when we talk
> >>>> about benchmark. Your consideration are really superficial.
> >>> Unfortunately, your numbers are meaningless.
> >> I don't think so.
> >>
> >>> (PCs should have cca 3GB/sec RAM transfer rates; and you demosstrated
> >>> cca 166MB/sec read rate; disk is 80MB/sec, so that's too slow. If you
> >>> want to prove your filesystem the filesystem is reasonably fast,
> >>> compare it with ext2 on ramdisk.)
> >>>
> >> This is the point. I don't want compare it with ext2 from performance
> >> point of view. This comparison makes no sense for me. I've done this
> >> test to prove that if you change environment you can change in a
> >> purposeful way the results.
> >
> > Yes, IOW you demonstrated that the numbers are machine-dependend and
> > really meaningless.
> >
> > ext2 comparison would tell you how much pramfs sucks (or not).
>
> Here the test with ext2 (same environment):
>
> Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
> hostname 15M:1k 10262 83 40847 82 38574 82 9866 92 62252 98 25204 81
> ------Sequential Create------ --------Random Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> 1 19859 98 44804 61 68830 100 13566 99 157129 100 30431 98
>
Ok, so pramfs is significantly faster than ext2. Interesting, and good
for pramfs.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html