> Hello Kernel Users,
>
> is there a size limit for tmpfs for the /dev/shm filesystem?
> Normally its default size is set to 2 GB. Is it possible to create a 2
> TB (Terrabyte) filesystem with tmpfs?
> Or is there a maximum size defined in the linux kernel?
Depends on your arch.
If it's 32 bit, it's limited to 16TB:
# mount -o size=16383G -t tmpfs tmpfs /mnt/2
# df -h
(...)
tmpfs 16T 0 16T 0% /mnt/2
# umount /mnt/2
# mount -o size=16385G -t tmpfs tmpfs /mnt/2
# df -h
(...)
tmpfs 1.0G 0 1.0G 0% /mnt/2
So 16384G would mean the same as 0.
If you're 64 bit, you need to have really loads of storage and/or RAM to
accumulate 16EB:
# mount -t tmpfs -o size=171798691839G tmpfs /mnt/2
# df -h
(...)
tmpfs 16E 0 16E 0% /mnt/2
--
Tomasz Chmielewski
http://lists.wpkg.org
Tomasz Chmielewski schrieb:
>> Hello Kernel Users,
>>
>> is there a size limit for tmpfs for the /dev/shm filesystem?
>> Normally its default size is set to 2 GB. Is it possible to create a
>> 2 TB (Terrabyte) filesystem with tmpfs?
>> Or is there a maximum size defined in the linux kernel?
>
> Depends on your arch.
I have a 64 Bits architecture (amd64/x86_64).
> If you're 64 bit, you need to have really loads of storage and/or RAM
> to accumulate 16EB:
>
> # mount -t tmpfs -o size=171798691839G tmpfs /mnt/2
> # df -h
> (...)
> tmpfs 16E 0 16E 0% /mnt/2
Nice, I will try this out.
I have not the money for 16E of memory of RAM! :-) lol
Already to provide 1 TB it would cost at least for the memory about 20
000 EUR + the server costs which support this.
--
Best regards
On Wed, 6 Feb 2008, Tomasz Chmielewski wrote:
> > Hello Kernel Users,
> >
> > is there a size limit for tmpfs for the /dev/shm filesystem?
There shouldn't be any artificial size limit on the /dev/shm filesystem,
it's an "internal" mount, and those are unlimited by default. Which is
not to say that you magically receive unlimited memory along with it!
The practical limit would be somewhere round about the total size of
your swap plus, ooh, finger in the air, 75% of your RAM - though that
all depends on what else you'd be wanting to use RAM+swap for.
> > Normally its default size is set to 2 GB.
Hmm, where did that number come from, maybe I'm forgetting something.
The user-visible mounts are limited by default to half of ram, so if
you've 4GB of ram, then 2GB would be the default for those tmpfs mounts.
> > Is it possible to create a 2 TB (Terrabyte) filesystem with tmpfs?
Yes.
> > Or is there a maximum size defined in the linux kernel?
>
> Depends on your arch.
>
> If it's 32 bit, it's limited to 16TB:
>
> # mount -o size=16383G -t tmpfs tmpfs /mnt/2
> # df -h
> (...)
> tmpfs 16T 0 16T 0% /mnt/2
>
> # umount /mnt/2
>
> # mount -o size=16385G -t tmpfs tmpfs /mnt/2
> # df -h
> (...)
> tmpfs 1.0G 0 1.0G 0% /mnt/2
>
> So 16384G would mean the same as 0.
Hah! Nice investigation. That's more of a bug than anything,
though not one I feel urgently compelled to fix.
> If you're 64 bit, you need to have really loads of storage and/or RAM to
> accumulate 16EB:
>
> # mount -t tmpfs -o size=171798691839G tmpfs /mnt/2
> # df -h
> (...)
> tmpfs 16E 0 16E 0% /mnt/2
Mika, you answered:
> Nice, I will try this out.
> I have not the money for 16E of memory of RAM! :-) lol
> Already to provide 1 TB it would cost at least for the memory about 20 000
> EUR + the server costs which support this.
Don't forget that tmpfs overflows into swap, so you could save money
by adding adding more swap and cutting down on the RAM: though of
course that will perform very poorly once it's actually using the
swap, probably not the direction you want to go in.
Hugh
Hugh Dickins wrote:
>
> Don't forget that tmpfs overflows into swap, so you could save money
> by adding adding more swap and cutting down on the RAM: though of
> course that will perform very poorly once it's actually using the
> swap, probably not the direction you want to go in.
>
It can still outperform conventional disk filesystems, however, mostly
because it doesn't have to worry one iota about consistency.
-hpa
On Wed, 6 Feb 2008, H. Peter Anvin wrote:
> Hugh Dickins wrote:
> >
> > Don't forget that tmpfs overflows into swap, so you could save money
> > by adding adding more swap and cutting down on the RAM: though of
> > course that will perform very poorly once it's actually using the
> > swap, probably not the direction you want to go in.
>
> It can still outperform conventional disk filesystems, however, mostly because
> it doesn't have to worry one iota about consistency.
In theory, yes, and should be true in practice before it hits swap.
But I think you'll find our swap handling is too primitive for tmpfs
to perform well once we hit swap. Most filesystems pay considerable
attention to good performance within their constraints of correctness.
Whereas with tmpfs we've just never worried about the performance once
swapping. It's used so you don't lose your data, but if you're really
expecting to be going to disk very much, better start with a filesystem
really designed for that.
Hugh
Hugh Dickins wrote:
>
> In theory, yes, and should be true in practice before it hits swap.
> But I think you'll find our swap handling is too primitive for tmpfs
> to perform well once we hit swap. Most filesystems pay considerable
> attention to good performance within their constraints of correctness.
> Whereas with tmpfs we've just never worried about the performance once
> swapping. It's used so you don't lose your data, but if you're really
> expecting to be going to disk very much, better start with a filesystem
> really designed for that.
>
That sounds like a problem in our overall swap handling, not
specifically in tmpfs. Now, I can't say anything concrete about heavy
swap conditions, but in light swap conditions I have measured a 20x
performance improvement(!) over ext3 on real workloads.
-hpa
On Wed, 6 Feb 2008, H. Peter Anvin wrote:
>
> That sounds like a problem in our overall swap handling, not specifically in
> tmpfs. Now, I can't say anything concrete about heavy swap conditions, but in
> light swap conditions I have measured a 20x performance improvement(!) over
> ext3 on real workloads.
Wow, I'm surprised. I suppose I do jump to thinking of heavy swapping
when light swapping won't be so bad; but even so, 20x ext3 astonishes
me - ext3 wouldn't be anyone's choice for fastest, but even so...
I certainly guess too much and measure too little:
is there a useful test you could point me to? TIA
Hugh
Hugh Dickins wrote:
> On Wed, 6 Feb 2008, H. Peter Anvin wrote:
>> That sounds like a problem in our overall swap handling, not specifically in
>> tmpfs. Now, I can't say anything concrete about heavy swap conditions, but in
>> light swap conditions I have measured a 20x performance improvement(!) over
>> ext3 on real workloads.
>
> Wow, I'm surprised. I suppose I do jump to thinking of heavy swapping
> when light swapping won't be so bad; but even so, 20x ext3 astonishes
> me - ext3 wouldn't be anyone's choice for fastest, but even so...
>
> I certainly guess too much and measure too little:
> is there a useful test you could point me to? TIA
>
The specific application was this:
- extract a kernel tarball
- "make distclean"
- cp -al the resulting tree
- apply a patch to each tree
- do a diff between the trees
- delete all files
... repeat something like 20,000 times.
-hpa
On Wed, 6 Feb 2008, H. Peter Anvin wrote:
>
> The specific application was this:
>
> - extract a kernel tarball
> - "make distclean"
> - cp -al the resulting tree
> - apply a patch to each tree
> - do a diff between the trees
> - delete all files
>
> ... repeat something like 20,000 times.
Right, thanks.
Certainly extracting a tarball can be a surprising challenge
for some filesystems.
And you remind me that of course, to the extent that a load is
about metadata (lots of creates for example), tmpfs can hardly
lose since its metadata never goes out to swap. But there is
plenty of data in your test above.
Hugh