I would like to better understand ext2/3's performance characteristics.
I'm specifically interested in how ext2/3 will handle a /var/spool/mail
directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
75,000 messages daily. Virtually all access is via imap, w/ approx
~1000 imapd processes running during peak load. Local delivery is via
procmail, which by default uses both kernel-supported locking calls and
.lock files.
I understand that various tuning parameters will have an impact,
e.g. putting the journal on a separate device, setting the noatime mount
option, etc. I also understand that there are other mailbox formats and
other strategies for locating mail spools (e.g. in user's home
directories).
I'm interested in people's thoughts on these issues, but I'm mostly
interested in whether or not the scenario I described falls within
ext2/3's designed capabilities.
Best.
--
Ron Peterson
Network & Systems Manager
Mount Holyoke College
http://www.mtholyoke.edu/~rpeterso
On Wed, 23 Feb 2005, Ron Peterson wrote:
> I would like to better understand ext2/3's performance characteristics.
>
> I'm specifically interested in how ext2/3 will handle a /var/spool/mail
> directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
> 75,000 messages daily. Virtually all access is via imap, w/ approx
> ~1000 imapd processes running during peak load. Local delivery is via
> procmail, which by default uses both kernel-supported locking calls and
> .lock files.
At some point it makes sense to subdivide you mail load because
serialization of i/o on that one filesystem becomes a bigger issue than
the performance of your filesystem... We deliver into mbox formatted
mailboxes inside users homedirs, some folks do a similar thing with
maildir. In the end you can on make one filesystem so fast. beyond that
you need more filesystems to acheive any kind of reasonable scaling...
> I understand that various tuning parameters will have an impact,
> e.g. putting the journal on a separate device, setting the noatime mount
> option, etc. I also understand that there are other mailbox formats and
> other strategies for locating mail spools (e.g. in user's home
> directories).
>
> I'm interested in people's thoughts on these issues, but I'm mostly
> interested in whether or not the scenario I described falls within
> ext2/3's designed capabilities.
>
> Best.
>
>
--
--------------------------------------------------------------------------
Joel Jaeggli Unix Consulting [email protected]
GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
On Wed, 2005-02-23 at 22:11 -0500, Ron Peterson wrote:
> I would like to better understand ext2/3's performance characteristics.
>
> I'm specifically interested in how ext2/3 will handle a /var/spool/mail
> directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
> 75,000 messages daily. Virtually all access is via imap, w/ approx
> ~1000 imapd processes running during peak load. Local delivery is via
> procmail, which by default uses both kernel-supported locking calls and
> .lock files.
>
> I understand that various tuning parameters will have an impact,
> e.g. putting the journal on a separate device, setting the noatime mount
> option, etc. I also understand that there are other mailbox formats and
> other strategies for locating mail spools (e.g. in user's home
> directories).
>
> I'm interested in people's thoughts on these issues, but I'm mostly
> interested in whether or not the scenario I described falls within
> ext2/3's designed capabilities.
Yes, ext2 and ext3 can handle that load easily. You should not have to
do any special tuning.
The real question is why in the world you would want to use mbox format
for this. It simply does not scale. Use maildir.
Lee
Ron Peterson <[email protected]> wrote:
>
> I would like to better understand ext2/3's performance characteristics.
>
> I'm specifically interested in how ext2/3 will handle a /var/spool/mail
> directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
> 75,000 messages daily. Virtually all access is via imap, w/ approx
> ~1000 imapd processes running during peak load. Local delivery is via
> procmail, which by default uses both kernel-supported locking calls and
> .lock files.
>
> I understand that various tuning parameters will have an impact,
> e.g. putting the journal on a separate device, setting the noatime mount
> option, etc. I also understand that there are other mailbox formats and
> other strategies for locating mail spools (e.g. in user's home
> directories).
>
> I'm interested in people's thoughts on these issues, but I'm mostly
> interested in whether or not the scenario I described falls within
> ext2/3's designed capabilities.
>
noatime will help.
increasing the journal size _may_ help.
With 6k files per directory you'll benefit from indexed directories
(htree). Use `tune2fs -O dir_index'. dir_index isn't available for ext2.