Message-ID: <4CEB91FA.3040209@inria.fr>
Date: Tue, 23 Nov 2010 11:05:46 +0100
From: Brice Goglin <Brice.Goglin@inria.fr>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101030 Icedove/3.0.10
MIME-Version: 1.0
To: Christopher Yeoh <cyeoh@au1.ibm.com>
CC: Andrew Morton <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org,
        Linus Torvalds <torvalds@linux-foundation.org>, linux-mm@kvack.org,
        Ingo Molnar <mingo@elte.hu>
Subject: Re: [RFC][PATCH] Cross Memory Attach v2 (resend)
References: <20101122122847.3585b447@lilo> <20101122130527.c13c99d3.akpm@linux-foundation.org> <20101123195523.46e6addb@lilo>
In-Reply-To: <20101123195523.46e6addb@lilo>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1960
Lines: 42

Le 23/11/2010 10:25, Christopher Yeoh a ?crit :
> On Mon, 22 Nov 2010 13:05:27 -0800
> Andrew Morton <akpm@linux-foundation.org> wrote:
>   
>> We have a bit of a track record of adding cool-looking syscalls and
>> then regretting it a few years later.  Few people use them, and maybe
>> they weren't so cool after all, and we have to maintain them for
>> ever. Bugs (sometimes security-relevant ones) remain undiscovered for
>> long periods because few people use (or care about) the code.
>>
>> So I think the bar is a high one - higher than it used to be.
>> Convince us that this feature is so important that it's worth all
>> that overhead and risk?
>>     
> Well there are the benchmark results to show that there is
> real improvement for MPI implementations (well at least for those
> benchmarks ;-) There's also been a few papers written on something
> quite similar (KNEM) which goes into more detail on the potential gains.
>
> http://runtime.bordeaux.inria.fr/knem/
>
> I've also heard privately that something very similar has been used in
> at least one device driver to support intranode operations for quite a
> while
>   

Many HPC hardware vendors implemented something like this in their
custom drivers to avoid going through their network loopback for local
communication. Even if their loopback is very fast, going to the NIC and
back to same host isn't really optimal. And I think all of them kept the
traditional approach (double-copy across a shared-memory buffer) for
small messages and only switched to this single-copy model for large
messages (tens or hundreds of kB). CMA and KNEM are "standardizing" all
this work and making it portable across multiple HPC platform/networks.

Brice

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/