Hello list,
I intend to build a diskless linux system (root over NFS). Because it
has 1GB of embedded flash storage, I'm thinking of using this as swap
(I've been bitten many times by the problems linux has with *no*
swap...). And to avoid wearing out the flash storage too fast, I 'm
thinking to format the 1GB partition as JFFS2, and create the swapfile
on top of it.
I'm not so experienced with JFFS and I don't know if it's too heavy for
the CPU, for swapping. Or if there are other issues I 'll face. What do
you think about it? Any other ways you 'd propose?
Sorry for sending this at LKML but jffs-dev mailing list seems to be
off. And JFFS is the only in-kernel filesystem that does wear-leveling,
right?
Thanks in advance,
Dimitris
P.S. Please CC replies to me, as I'm not subscribed to the list.
On Fri, 29 February 2008 04:50:17 +0200, Dimitrios Apostolou wrote:
>
> I intend to build a diskless linux system (root over NFS). Because it
> has 1GB of embedded flash storage, I'm thinking of using this as swap
> (I've been bitten many times by the problems linux has with *no*
> swap...). And to avoid wearing out the flash storage too fast, I 'm
> thinking to format the 1GB partition as JFFS2, and create the swapfile
> on top of it.
>
> I'm not so experienced with JFFS and I don't know if it's too heavy for
> the CPU, for swapping. Or if there are other issues I 'll face. What do
> you think about it? Any other ways you 'd propose?
>
> Sorry for sending this at LKML but jffs-dev mailing list seems to be
> off. And JFFS is the only in-kernel filesystem that does wear-leveling,
> right?
Replying in reverse order...
The relevant mailing list is linux-mtd, added to Cc:. JFFS and JFFS2
are two different things, JFFS is older and was removed from the kernel
not too long ago.
The real fun comes not from CPU usage, but from interactions with the
memory management subsystem. In a nutshell, JFFS2 may require memory in
order to write data. When the system is under memory pressure, it needs
JFFS2 to write out pages, which will try to allocate memory. It is
theoretically possible to deadlock the system in this way.
On the plus side, the write path of JFFS2 is relatively simple and
extremely low-latency. It shouldn't be too hard to review the code and
handle all problem cases wrt. memory allocations.
One issue that is hard to solve is space reservation. JFFS2 compresses
data and allows users to write as long as there is space remaining. It
is possible to swap out data that compresses well, have some other
process fill up the filesystem, then try to swap out data that
compresses badly and get -ENOSPC in return. As a system administrator
you can prevent others from ever writing to JFFS2 - and you better do!
Jörn
--
Joern's library part 5:
http://www.faqs.org/faqs/compression-faq/part2/section-9.html
On Fre, 2008-02-29 at 04:50 +0200, Dimitrios Apostolou wrote:
> Hello list,
>
> I intend to build a diskless linux system (root over NFS). Because it
> has 1GB of embedded flash storage, I'm thinking of using this as swap
> (I've been bitten many times by the problems linux has with *no*
Put more RAM on the hardware and/or reduce the software on it.
If that means heavily customizing the distribution: Do it.
If that means you can't run some specific common distribution[0]: Throw
it out and roll your own minimalistic one with just the tools and
especially libraries you need.
We run Linux on embedded hardware all the time without swapspace without
any problems (related to the "missing" swapspace).
> swap...). And to avoid wearing out the flash storage too fast, I 'm
Swapping (and constantly writing logfiles) will wear it out much faster
than without. Perhaps not at development time but for sure in the field.
[....]
Bernd
[0]: Which I highly doubt.
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services
Jörn Engel wrote:
> On Fri, 29 February 2008 04:50:17 +0200, Dimitrios Apostolou wrote:
>> I intend to build a diskless linux system (root over NFS). Because it
>> has 1GB of embedded flash storage, I'm thinking of using this as swap
>> (I've been bitten many times by the problems linux has with *no*
>> swap...). And to avoid wearing out the flash storage too fast, I 'm
>> thinking to format the 1GB partition as JFFS2, and create the swapfile
>> on top of it.
>>
>> I'm not so experienced with JFFS and I don't know if it's too heavy for
>> the CPU, for swapping. Or if there are other issues I 'll face. What do
>> you think about it? Any other ways you 'd propose?
>>
>> Sorry for sending this at LKML but jffs-dev mailing list seems to be
>> off. And JFFS is the only in-kernel filesystem that does wear-leveling,
>> right?
>
> Replying in reverse order...
>
> The relevant mailing list is linux-mtd, added to Cc:. JFFS and JFFS2
> are two different things, JFFS is older and was removed from the kernel
> not too long ago.
Thanks and sorry for intruding LKML. It seems that even wikipedia has
wrong address for the mailing list, see the last link of the article:
http://en.wikipedia.org/wiki/JFFS2
>
> The real fun comes not from CPU usage, but from interactions with the
> memory management subsystem. In a nutshell, JFFS2 may require memory in
> order to write data. When the system is under memory pressure, it needs
> JFFS2 to write out pages, which will try to allocate memory. It is
> theoretically possible to deadlock the system in this way.
Interesting. I guess nobody has experimented with it yet so I'll try.
Unfortunately it seems I'll face another problem, that JFFS2 doesn't
support having a swap-file at all. Why would this happen? More info:
http://dev.laptop.org/ticket/6469
>
> On the plus side, the write path of JFFS2 is relatively simple and
> extremely low-latency. It shouldn't be too hard to review the code and
> handle all problem cases wrt. memory allocations.
>
> One issue that is hard to solve is space reservation. JFFS2 compresses
> data and allows users to write as long as there is space remaining. It
> is possible to swap out data that compresses well, have some other
> process fill up the filesystem, then try to swap out data that
> compresses badly and get -ENOSPC in return. As a system administrator
> you can prevent others from ever writing to JFFS2 - and you better do!
Of course! I intend to use all the 1GB of flash only for swap, the
system will be practically diskless. And I don't think enabling
compression for such a task would be wise.
>
> Jörn
>
Thanks for the help,
Dimitris
Bernd Petrovitsch wrote:
> On Fre, 2008-02-29 at 04:50 +0200, Dimitrios Apostolou wrote:
>> Hello list,
>>
>> I intend to build a diskless linux system (root over NFS). Because it
>> has 1GB of embedded flash storage, I'm thinking of using this as swap
>> (I've been bitten many times by the problems linux has with *no*
>
> Put more RAM on the hardware and/or reduce the software on it.
> If that means heavily customizing the distribution: Do it.
> If that means you can't run some specific common distribution[0]: Throw
> it out and roll your own minimalistic one with just the tools and
> especially libraries you need.
> We run Linux on embedded hardware all the time without swapspace without
> any problems (related to the "missing" swapspace).
I have been running swapless linux systems a long time ago, but I think
I have had more problems than not. I have tried setting swappiness and
overcommit settings but these don't help much. Let me explain:
When it is not certain that your system will run a specific task and
only that, it is almost certain that sometime you will run out of
memory. For example in the thin client case a user might open too many
windows and fill the RAM with X server pixmaps...
So when the out of memory condition happens the system will start
swapping. And if you have no swap enabled, the system will swap even
harder: it will swap out all mmap'ed read-only memory segments, which
mostly are the executables themselves. And because these will be needed
immediately they will be swapped in immediately afterwards...
This renders the system *unusable* for all cases. Sometimes it even
stops responding to ping... On the other hand if you have a swap
partition then the kernel will choose to swap out other memory segments,
not the executable ones that are so much needed.
>
>> swap...). And to avoid wearing out the flash storage too fast, I 'm
>
> Swapping (and constantly writing logfiles) will wear it out much faster
> than without. Perhaps not at development time but for sure in the field.
That's why I want to do it over a wear-levelling layer. Given that the
partition is large enough it will hopefully take a while to wear it out.
Thanks,
Dimitris
>
> [....]
>
> Bernd
>
> [0]: Which I highly doubt.
On Fri, 29 February 2008 19:29:53 +0200, Dimitrios Apostolou wrote:
>
> Thanks and sorry for intruding LKML. It seems that even wikipedia has
> wrong address for the mailing list, see the last link of the article:
>
> http://en.wikipedia.org/wiki/JFFS2
Fixed. Not that I actually read the page and checked for other
errors...
> Interesting. I guess nobody has experimented with it yet so I'll try.
> Unfortunately it seems I'll face another problem, that JFFS2 doesn't
> support having a swap-file at all. Why would this happen? More info:
>
> http://dev.laptop.org/ticket/6469
Yes, this still requires a bit of work. Peter Zijlstra has a patch for
the memory-management side of it, someone has to fix up jffs2 as well.
I'll add support to logfs fairly soon.
> Of course! I intend to use all the 1GB of flash only for swap, the
> system will be practically diskless. And I don't think enabling
> compression for such a task would be wise.
It might, actually. Assuming your average swap page compresses to 50%,
it means that you use half the bandwidth to flash and get twice the
lifetime before blocks wear out. Whether the performance gets better or
worse depends on the relative speeds of your CPU and flash.
Jörn
--
Joern's library part 10:
http://blogs.msdn.com/David_Gristwood/archive/2004/06/24/164849.aspx
On Fre, 2008-02-29 at 20:13 +0200, Dimitrios Apostolou wrote:
> Bernd Petrovitsch wrote:
> > On Fre, 2008-02-29 at 04:50 +0200, Dimitrios Apostolou wrote:
[....]
> >> swap...). And to avoid wearing out the flash storage too fast, I 'm
> >
> > Swapping (and constantly writing logfiles) will wear it out much faster
> > than without. Perhaps not at development time but for sure in the field.
>
> That's why I want to do it over a wear-levelling layer. Given that the
> partition is large enough it will hopefully take a while to wear it out.
The wear-levelling only makes all erase blocks/chunks/... wear out at
the same rate (at least more or less). It doesn't avoid or reduce
wearing out, it just distributes it.
Even if you do it: Make sure you have the better flash chips, not the
cheap ones with only 5000 (or so) write cycles. Or even MLCs ....
Bernd
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services
Bernd Petrovitsch wrote:
> The wear-levelling only makes all erase blocks/chunks/... wear out at
> the same rate (at least more or less). It doesn't avoid or reduce
> wearing out, it just distributes it.
>
Surely a unit with one defective block and a squillion good blocks is
faulty. Sure it is. It has a fault, which is what it means to be
faulty. Spread the wear of that one block over all squillion and one
blocks and you get a much longer lifetime. It seems intuitive that
distributing wear significantly increases time to failure.
On Mon, 2008-03-03 at 00:27 +1030, David Newall wrote:
> Bernd Petrovitsch wrote:
> > The wear-levelling only makes all erase blocks/chunks/... wear out at
> > the same rate (at least more or less). It doesn't avoid or reduce
> > wearing out, it just distributes it.
>
> Surely a unit with one defective block and a squillion good blocks is
> faulty. Sure it is. It has a fault, which is what it means to be
Several are very probably faulty as NAND flashes initially already have
bad blocks (but there a re some more on that chip to compensate for
that).
Even if just 1 of a squillion (erase-)blocks is gone. But that's now and
only the first. More to follow sooner or later .....
I don't think that "writing to swap space" is in any way limited or
minimized but assumes normal harddisks. So it is not like you are
writing to the flash chip only on "save in some UI or on a firmware
update.
> faulty. Spread the wear of that one block over all squillion and one
> blocks and you get a much longer lifetime. It seems intuitive that
> distributing wear significantly increases time to failure.
Yes, that's the idea. Nevertheless it places "load" on the flash chips.
Actually I'm curious how fats it is and how well it works (and with
which flash chips exactly) if one simply tries it out. If it actually
works.
Bernd
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services