I seem to remember seeing some patches go by at some point that allowed
one of the rom type embeded system filesystems to directly execute
binaries out of the original rom memory rather than copying them to ram
first, then executing from there. I was wondering if rootfs or tmpfs
support such execute in place today, or if binaries executed from there
have their code segments duplicated in ram?
On Tue, 1 May 2007, Phillip Susi wrote:
> I seem to remember seeing some patches go by at some point that allowed one of
> the rom type embeded system filesystems to directly execute binaries out of
> the original rom memory rather than copying them to ram first, then executing
> from there. I was wondering if rootfs or tmpfs support such execute in place
> today, or if binaries executed from there have their code segments duplicated
> in ram?
Only ext2 supports it today: see Documentation/filesystems/xip.txt
Hugh
On 2007.05.02 15:04:44 +0100, Hugh Dickins wrote:
> On Tue, 1 May 2007, Phillip Susi wrote:
> > I seem to remember seeing some patches go by at some point that
> > allowed one of the rom type embeded system filesystems to directly
> > execute binaries out of the original rom memory rather than copying
> > them to ram first, then executing from there. I was wondering if
> > rootfs or tmpfs support such execute in place today, or if binaries
> > executed from there have their code segments duplicated in ram?
>
> Only ext2 supports it today: see Documentation/filesystems/xip.txt
As I understand it, xip avoids the page cache copy. But tmpfs already
lives in the page cache (or swap), so avoiding that "copy" is
impossible. But I always expected tmpfs to implicitly due its own kind
of xip, i.e. that it doesn't have to store its stuff in the page cache
twice. Are you saying that this isn't true?
According to Documentation/filesystems/xip.txt the ramdisk driver does
something similar, pushing the data into the page cache and discarding
the original data.
thanks,
Bj?rn
On Wed, 2 May 2007, Björn Steinbrink wrote:
> On 2007.05.02 15:04:44 +0100, Hugh Dickins wrote:
> > On Tue, 1 May 2007, Phillip Susi wrote:
> > > I seem to remember seeing some patches go by at some point that
> > > allowed one of the rom type embeded system filesystems to directly
> > > execute binaries out of the original rom memory rather than copying
> > > them to ram first, then executing from there. I was wondering if
> > > rootfs or tmpfs support such execute in place today, or if binaries
> > > executed from there have their code segments duplicated in ram?
> >
> > Only ext2 supports it today: see Documentation/filesystems/xip.txt
>
> As I understand it, xip avoids the page cache copy. But tmpfs already
> lives in the page cache (or swap), so avoiding that "copy" is
> impossible. But I always expected tmpfs to implicitly due its own kind
> of xip, i.e. that it doesn't have to store its stuff in the page cache
> twice. Are you saying that this isn't true?
tmpfs doesn't store its stuff in the page cache twice: that's true,
and I didn't mean to imply otherwise. But tmpfs doesn't contain any
support for rom memory: you'd have to copy from rom to tmpfs to use it.
Hugh
Hugh Dickins wrote:
> tmpfs doesn't store its stuff in the page cache twice: that's true,
> and I didn't mean to imply otherwise. But tmpfs doesn't contain any
> support for rom memory: you'd have to copy from rom to tmpfs to use it.
>
> Hugh
The question is, when you execute a binary on tmpfs, does its code
segment get mapped directly where it's at in the buffer cache, or does
it get copied to another page for the executing process? At least,
assuming this is possible due to the vma and file offsets of the segment
being aligned.
On Wed, 2 May 2007, Phillip Susi wrote:
> Hugh Dickins wrote:
> > tmpfs doesn't store its stuff in the page cache twice: that's true,
> > and I didn't mean to imply otherwise. But tmpfs doesn't contain any
> > support for rom memory: you'd have to copy from rom to tmpfs to use it.
>
> The question is, when you execute a binary on tmpfs, does its code segment get
> mapped directly where it's at in the buffer cache, or does it get copied to
> another page for the executing process? At least, assuming this is possible
> due to the vma and file offsets of the segment being aligned.
Its pages are mapped directly into the executing process, without copying.
Hugh
Hugh Dickins wrote:
> On Wed, 2 May 2007, Phillip Susi wrote:
> > Hugh Dickins wrote:
> > > tmpfs doesn't store its stuff in the page cache twice: that's true,
> > > and I didn't mean to imply otherwise. But tmpfs doesn't contain any
> > > support for rom memory: you'd have to copy from rom to tmpfs to use
> > > it.
> >
> > The question is, when you execute a binary on tmpfs, does its code
> > segment get mapped directly where it's at in the buffer cache, or does
> > it get copied to another page for the executing process? At least,
> > assuming this is possible due to the vma and file offsets of the segment
> > being aligned.
>
> Its pages are mapped directly into the executing process, without copying.
Thank GOD! Boy, was I worried there for a second.
Now, if there were only an easy way to make tmpfs persistent?
Thanks!
--
Al
Al Boldi wrote:
> Hugh Dickins wrote:
>> On Wed, 2 May 2007, Phillip Susi wrote:
>>> Hugh Dickins wrote:
>>>> tmpfs doesn't store its stuff in the page cache twice: that's true,
>>>> and I didn't mean to imply otherwise. But tmpfs doesn't contain any
>>>> support for rom memory: you'd have to copy from rom to tmpfs to use
>>>> it.
>>> The question is, when you execute a binary on tmpfs, does its code
>>> segment get mapped directly where it's at in the buffer cache, or does
>>> it get copied to another page for the executing process? At least,
>>> assuming this is possible due to the vma and file offsets of the segment
>>> being aligned.
>> Its pages are mapped directly into the executing process, without copying.
>
> Thank GOD! Boy, was I worried there for a second.
>
> Now, if there were only an easy way to make tmpfs persistent?
>
>
It would be not a tmpfs (*temporary* fs)then, but something like this
http://pramfs.sourceforge.net/
Regards,
Dmitry
On Wed, May 02, 2007 at 03:04:44PM +0100, Hugh Dickins wrote:
> On Tue, 1 May 2007, Phillip Susi wrote:
> > I seem to remember seeing some patches go by at some point that allowed one of
> > the rom type embeded system filesystems to directly execute binaries out of
> > the original rom memory rather than copying them to ram first, then executing
> > from there. I was wondering if rootfs or tmpfs support such execute in place
> > today, or if binaries executed from there have their code segments duplicated
> > in ram?
>
> Only ext2 supports it today: see Documentation/filesystems/xip.txt
IIRC JFFS2 also supports XIP.
Erik
--
They're all fools. Don't worry. Darwin may be slow, but he'll
eventually get them. -- Matthew Lammers in alt.sysadmin.recovery
Dmitry Krivoschekov wrote:
> Al Boldi wrote:
> > Now, if there were only an easy way to make tmpfs persistent?
>
> It would be not a tmpfs (*temporary* fs)then,
Isn't everything really just temporary?
> but something like this
>
> http://pramfs.sourceforge.net/
Thanks a lot, but this seems to rely on a non-volatile RAM.
Would something like an mmap'd tmpfs be possible?
Thanks!
--
Al
On Wed 2 May 2007 10:04, Hugh Dickins pondered:
> On Tue, 1 May 2007, Phillip Susi wrote:
> > I seem to remember seeing some patches go by at some point that
> > allowed one of the rom type embeded system filesystems to directly
> > execute binaries out of the original rom memory rather than copying
> > them to ram first, then executing from there. I was wondering if
> > rootfs or tmpfs support such execute in place today, or if
> > binaries executed from there have their code segments duplicated
> > in ram?
>
> Only ext2 supports it today: see Documentation/filesystems/xip.txt
>
Depends on if it is a noMMU or MMU platform.
Since noMMU platforms can't re-arrange non-contiguous blocks (which appears in
a read/write ext2 file system) we need to use a read only romfs which
applications are guaranteed to be contiguous by design.
I don't think the noMMU case is documented in xip.txt
-Robin
On Thu, 3 May 2007 13:38:22 +0200, Erik Mouw wrote:
> On Wed, May 02, 2007 at 03:04:44PM +0100, Hugh Dickins wrote:
> >
> > Only ext2 supports it today: see Documentation/filesystems/xip.txt
>
> IIRC JFFS2 also supports XIP.
Definitely not. AXFS does, if you want to consider out-of-tree patches.
Jörn
--
Fools ignore complexity. Pragmatists suffer it.
Some can avoid it. Geniuses remove it.
-- Perlis's Programming Proverb #58, SIGPLAN Notices, Sept. 1982
Al Boldi wrote:
> Dmitry Krivoschekov wrote:
>> Al Boldi wrote:
>>> Now, if there were only an easy way to make tmpfs persistent?
>> It would be not a tmpfs (*temporary* fs)then,
>
> Isn't everything really just temporary?
Would you like to talk about this?
Not with me, I'm not a psychoanalyst :)
>
>> but something like this
>>
>> http://pramfs.sourceforge.net/
>
> Thanks a lot, but this seems to rely on a non-volatile RAM.
No, it relies on a battery-backed normal RAM, which of course
may be considered as non-volatile.
Thanks,
Dmitry
Al Boldi wrote:
>
> Isn't everything really just temporary?
>
> Would something like an mmap'd tmpfs be possible?
>
No. tmpfs relies on being able to leave data structures in the running
kernel. In particular, it has no metadata store at all.
The needs for a persistent filesystem are very different, regardless of
what the underlying medium is.
-hpa
H. Peter Anvin wrote:
> Al Boldi wrote:
> > Isn't everything really just temporary?
> >
> > Would something like an mmap'd tmpfs be possible?
>
> No. tmpfs relies on being able to leave data structures in the running
> kernel. In particular, it has no metadata store at all.
>
> The needs for a persistent filesystem are very different, regardless of
> what the underlying medium is.
Think Suspend-To-Disk; in that case tmpfs looks pretty persistent to me.
So what does STD do? It pushes memory out to swap.
Now, tmpfs could probably do the same thing on its own either to a private
swap or an mmap. I am suggesting mmap, as swap is currently really slow
with tmpfs, and switching it to use mmap may buy us 2 for the price of 1.
Thanks!
--
Al
Al Boldi wrote:
> H. Peter Anvin wrote:
>> Al Boldi wrote:
>>> Isn't everything really just temporary?
>>>
>>> Would something like an mmap'd tmpfs be possible?
>> No. tmpfs relies on being able to leave data structures in the running
>> kernel. In particular, it has no metadata store at all.
>>
>> The needs for a persistent filesystem are very different, regardless of
>> what the underlying medium is.
>
> Think Suspend-To-Disk; in that case tmpfs looks pretty persistent to me.
>
> So what does STD do? It pushes memory out to swap.
It pushes ALL of (used) memory out to swap.
> Now, tmpfs could probably do the same thing on its own either to a private
> swap or an mmap. I am suggesting mmap, as swap is currently really slow
> with tmpfs, and switching it to use mmap may buy us 2 for the price of 1.
No.
First of all, it would have to map ALL of kernel memory, or you would
have to change the way things like dentries, inodes, and namespaces are
allocated in the kernel itself.
Second, you still would have no stability across kernel versions.
Third, you would have to accept total data loss on unclean shutdown,
because tmpfs doesn't care about coherency, *NOR SHOULD IT*. This is
the fundamental reason why allocating a large swap partition and making
/tmp a tmpfs can give a *huge* performance boost, even though it still
hits disk.
What you're talking about is, *and should be*, a different filesystem.
You will relatively quickly find that you have to deal with the same
kind of stuff that you have to in any filesystem.
-hpa
H. Peter Anvin wrote:
> Al Boldi wrote:
> > H. Peter Anvin wrote:
> >> Al Boldi wrote:
> >>> Isn't everything really just temporary?
> >>>
> >>> Would something like an mmap'd tmpfs be possible?
> >>
> >> No. tmpfs relies on being able to leave data structures in the running
> >> kernel. In particular, it has no metadata store at all.
> >>
> >> The needs for a persistent filesystem are very different, regardless of
> >> what the underlying medium is.
> >
> > Think Suspend-To-Disk; in that case tmpfs looks pretty persistent to me.
> >
> > So what does STD do? It pushes memory out to swap.
>
> It pushes ALL of (used) memory out to swap.
Ok.
> > Now, tmpfs could probably do the same thing on its own either to a
> > private swap or an mmap. I am suggesting mmap, as swap is currently
> > really slow with tmpfs, and switching it to use mmap may buy us 2 for
> > the price of 1.
>
> No.
>
> First of all, it would have to map ALL of kernel memory, or you would
> have to change the way things like dentries, inodes, and namespaces are
> allocated in the kernel itself.
That's probably one way of doing it.
> Second, you still would have no stability across kernel versions.
No problem. You could probably convince the user to do a tmpfs backup before
an upgrade.
> Third, you would have to accept total data loss on unclean shutdown,
> because tmpfs doesn't care about coherency, *NOR SHOULD IT*.
I think that's understood. But this risk could be reduced by instantiating
the latest sync'd mmap/swap.
> This is
> the fundamental reason why allocating a large swap partition and making
> /tmp a tmpfs can give a *huge* performance boost, even though it still
> hits disk.
Don't know what you mean here. Do you mean that tmpfs performance is
dependent on free swap-space being larger than some threshold. If so, what
threshold causes a tmpfs slowdown?
> What you're talking about is, *and should be*, a different filesystem.
> You will relatively quickly find that you have to deal with the same
> kind of stuff that you have to in any filesystem.
That's exactly what I want to avoid, as this would introduce a performance
penalty.
All we need is a periodically synced tmpfs to mmap, with a minimal stream
into page-cache algo on mount.
Thanks!
--
Al
Al Boldi wrote:
>
>> What you're talking about is, *and should be*, a different filesystem.
>> You will relatively quickly find that you have to deal with the same
>> kind of stuff that you have to in any filesystem.
>
> That's exactly what I want to avoid, as this would introduce a performance
> penalty.
>
> All we need is a periodically synced tmpfs to mmap, with a minimal stream
> into page-cache algo on mount.
>
... which would be completely useless, because you wouldn't get any sort
of coherency; you would have pointers pointing into space, etc. etc.
Sorry, it's useless.
-hpa
H. Peter Anvin wrote:
> Al Boldi wrote:
> >> What you're talking about is, *and should be*, a different filesystem.
> >> You will relatively quickly find that you have to deal with the same
> >> kind of stuff that you have to in any filesystem.
> >
> > That's exactly what I want to avoid, as this would introduce a
> > performance penalty.
> >
> > All we need is a periodically synced tmpfs to mmap, with a minimal
> > stream into page-cache algo on mount.
>
> ... which would be completely useless, because you wouldn't get any sort
> of coherency; you would have pointers pointing into space, etc. etc.
You don't really think that anybody is suggesting to store the tmpfs data
without any coherency, do you?
I am suggesting that you can easily isolate tmpfs coherency from the rest of
the page-cache, by simply streaming tmpfs data out to an mmap and plugging
it with the tmpfs fat on umount. And streaming things back in on mount.
> Sorry, it's useless.
Persisting tmpfs is useful for the mere case of saving you the need to
repopulate on mount.
Thanks!
--
Al
Al Boldi wrote:
>
> You don't really think that anybody is suggesting to store the tmpfs data
> without any coherency, do you?
>
> I am suggesting that you can easily isolate tmpfs coherency from the rest of
> the page-cache, by simply streaming tmpfs data out to an mmap and plugging
> it with the tmpfs fat on umount. And streaming things back in on mount.
>
You still don't get it, do you?
tmpfs metadata doesn't live in the pagecache.
-hpa
H. Peter Anvin wrote:
> Al Boldi wrote:
> > You don't really think that anybody is suggesting to store the tmpfs
> > data without any coherency, do you?
> >
> > I am suggesting that you can easily isolate tmpfs coherency from the
> > rest of the page-cache, by simply streaming tmpfs data out to an mmap
> > and plugging it with the tmpfs fat on umount. And streaming things back
> > in on mount.
>
> You still don't get it, do you?
>
> tmpfs metadata doesn't live in the pagecache.
It does not matter; you can deduce it on umount.
Thanks!
--
Al