From: "Joseph D. Wagner" <wagnerjd@prodigy.net>
To: "'David S. Miller'" <davem@redhat.com>
Cc: <robm@fastmail.fm>, <hahn@physics.mcmaster.ca>,
       <linux-kernel@vger.kernel.org>, <jhoward@fastmail.fm>
Subject: RE: Strange load spikes on 2.4.19 kernel
Date: Sun, 13 Oct 2002 02:49:55 -0500
Message-ID: <000c01c2728d$263c0ca0$7443f4d1@joe>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
In-Reply-To: <20021013.000127.43007739.davem@redhat.com>
Importance: Normal
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2382
Lines: 51

>> I'll let you in on a dirty little secret.  The Linux file
>> system does not utilize SMP.  That's right.  All file
>> processes go through one and only one processor.  It has
>> to do with the fact that the Linux kernel is a non-preemptive
>> kernel.

> Not true, page cache accesses (translation: read and write)
> go through the page cache which is fully multi-threaded.

> Allocating blocks and inodes, yes that is currently single
> threaded on SMP.

Now wait a minute!  Allocating blocks and inodes is an integral part of
a write.  Oh sure the actual writing of file data is SMP, but that
process is bottlenecked by single threaded allocation of blocks and
inodes.  Perhaps I could phrase what I said to be more technically
accurate by saying, "Writing makes such poor use of multi-threading on
SMP that in terms of performance it's as if it was single threaded."

> But there is no fundamental reason for that, we just haven't
> gotten around to threading that bit yet.

Oh yes there is.  What if an allocation of blocks and/or inodes is
preempted?  Another thread could attempt to allocate the same set of
blocks and/or inodes.

This isn't a problem on a uniprocessor system because only one processor
can access any data structure at any given time.

However, on SMP two kernel control paths running on different CPUs could
concurrently access the same data structure.  There's two ways to deal
with this error: 1) the lazy way and the way Linus decided to go was to
run block and inode allocation through one single thread, or 2) the
better way is to preempt the other process which would require a) a
preemptive kernel and b) synchronization (and as a programmer I can tell
you that synchronization gets messy if not thoroughly designed and
implemented).

Rather than go through all the work of rewriting the kernel to be
preemptive and significantly improving synchronization routines (a lot
of work), Linus chose to solve the problem by avoiding it, rather than
dealing with it as he should have.

If you don't believe me, prove me wrong.  Write the code.  If you ever
got your @$$ around to it, you'd see that I'm right.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/