MIME-Version: 1.0
Date: Mon, 30 Apr 2012 23:03:23 -0700
Message-ID: <CANkdjvZinRvHYPKLCFsd6z3FXgM4ScS1Ar9OjO3s1SLDFnridw@mail.gmail.com>
Subject: read() syscall slowing down due to other threads?
From: George Porter <gmporter@cs.ucsd.edu>
To: linux-kernel@vger.kernel.org
Content-Type: text/plain; charset=ISO-8859-1
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1386
Lines: 30

Hi all,

I've run into a weird phenomenon I was hoping someone could help me pin down.

I've got a multithreaded program which reads data off of 8 input
disks, and does some processing on it.  I have 8 Reader threads, each
of which reads data off of one of the eight input disks.  That data
gets passed to other threads, which do some processing (this is an 8
core machine).  If I do no or minimal processing in those other
threads, the read() calls go at the speed of the disks (~100 MBps).  I
measure that speed by taking a timestamp before the read syscall, then
afterwards, and dividing that into the read size (5MB).  All seems
good.

However, if I start doing more computation on those other threads, the
read() syscalls take longer to read the same amount of data,
eventually slowing down to 50 MBps (50% slower).  I've used
setaffinity() to isolate the Reader threads to one set of cores, and
the compute threads to a different set of cores, and so I don't think
it is CPU/scheduling interference.

Thoughts?  Has anyone run into this before?

Also, if you could CC me directly on any responses I would appreciate it.
Thanks, George
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/