2008-08-20 14:35:16

by Martin Uecker

[permalink] [raw]
Subject: distributed shared memory / mmap



Hi!

Is there any easy way to get shared memory between to
processes on different machines? It seems possible using mmap
on a file system like GFS on top of a distributed block device.


Sincerly,
Martin Uecker



2008-08-20 14:45:40

by Steven Whitehouse

[permalink] [raw]
Subject: Re: distributed shared memory / mmap

Hi,

On Wed, 2008-08-20 at 15:18 +0200, Martin Uecker wrote:
>
> Hi!
>
> Is there any easy way to get shared memory between to
> processes on different machines? It seems possible using mmap
> on a file system like GFS on top of a distributed block device.
>
>
> Sincerly,
> Martin Uecker
>
Yes, you can do it using GFS/GFS2 and it should work. On the other hand,
don't do it. Its probably not what you want and unlikely to give you
good performance since the locking is "per inode" and thus you'll
generate a lot of cache flushing if there is any appreciable amount of
writing to the shared region. If you have a read mostly (or only)
workload, then it will work much better.

There are often better solutions though, depending on the application in
question,

Steve.


2008-08-20 15:04:22

by Louis Rilling

[permalink] [raw]
Subject: Re: distributed shared memory / mmap

On Wed, Aug 20, 2008 at 03:43:24PM +0100, Steven Whitehouse wrote:
> Hi,
>
> On Wed, 2008-08-20 at 15:18 +0200, Martin Uecker wrote:
> >
> > Hi!
> >
> > Is there any easy way to get shared memory between to
> > processes on different machines? It seems possible using mmap
> > on a file system like GFS on top of a distributed block device.
> >
> >
> > Sincerly,
> > Martin Uecker
> >
> Yes, you can do it using GFS/GFS2 and it should work. On the other hand,
> don't do it. Its probably not what you want and unlikely to give you
> good performance since the locking is "per inode" and thus you'll
> generate a lot of cache flushing if there is any appreciable amount of
> writing to the shared region. If you have a read mostly (or only)
> workload, then it will work much better.

Alternatively, you can use kernel-level DSM like provided by Kerrighed.
Kerrighed provides cluster-wide SYSV shm segments, which sharing is handled
at the page level -> far better than the inode level.

You may find this solution not-so-easy though...

Regards,

Louis

--
Dr Louis Rilling Kerlabs
Skype: louis.rilling Batiment Germanium
Phone: (+33|0) 6 80 89 08 23 80 avenue des Buttes de Coesmes
http://www.kerlabs.com/ 35700 Rennes


Attachments:
(No filename) (1.21 kB)
signature.asc (189.00 B)
Digital signature
Download all attachments

2008-08-20 16:04:59

by Mark Hounschell

[permalink] [raw]
Subject: Re: distributed shared memory / mmap

Martin Uecker wrote:
>
> Hi!
>
> Is there any easy way to get shared memory between to
> processes on different machines? It seems possible using mmap
> on a file system like GFS on top of a distributed block device.
>
>

There is hardware available, designed just for this. Reflective Memory
pci cards.

Mark

2008-08-20 17:04:11

by linux-os (Dick Johnson)

[permalink] [raw]
Subject: Re: distributed shared memory / mmap


On Wed, 20 Aug 2008, Mark Hounschell wrote:

> Martin Uecker wrote:
>>
>> Hi!
>>
>> Is there any easy way to get shared memory between to
>> processes on different machines? It seems possible using mmap
>> on a file system like GFS on top of a distributed block device.
>>
>>
>
> There is hardware available, designed just for this. Reflective Memory
> pci cards.
>
> Mark
> --

Check out "Fabric." There is even an InfiniBand driver in the
kernel. Fabric attached memory is used for HPC (High Performance
Computing). It's been around for a couple of years and is fairly
mature.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (4785.87 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.