Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Alun Evans <alun@badgerous.net>
To:     ebiederm@xmission.com (Eric W. Biederman)
Cc:     linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 05/27] containers: Open a socket inside a container
References: <m2o8z7t2w5.fsf@badgerous.net>
        <871rw1yey8.fsf@x220.int.ebiederm.org>
Date:   Sat, 28 Sep 2019 15:29:44 -0700
In-Reply-To: <871rw1yey8.fsf@x220.int.ebiederm.org> (Eric W. Biederman's
        message of "Fri, 27 Sep 2019 09:46:39 -0500")
Message-ID: <m2d0fkt5pj.fsf@badgerous.net>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (darwin)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On Fri 27 Sep '19 at 07:46 ebiederm@xmission.com (Eric W. Biederman) wrote:
>=20
> Alun Evans <alun@badgerous.net> writes:
>
>> Hi Eric,
>>
>>
>> On Tue, 19 Feb 2019, Eric W. Biederman <ebiederm@xmission.com> wrote:
>>>
>>> David Howells <dhowells@redhat.com> writes:
>>>
>>> > Provide a system call to open a socket inside of a container, using t=
hat
>>> > container's network namespace.  This allows netlink to be used to man=
age
>>> > the container.
>>> >
>>> > 	fd =3D container_socket(int container_fd,
>>> > 			      int domain, int type, int protocol);
>>> >
>>>
>>> Nacked-by: "Eric W. Biederman" <ebiederm@xmission.com>
>>>
>>> Use a namespace file descriptor if you need this.  So far we have not
>>> added this system call as it is just a performance optimization.  And it
>>> has been too niche to matter.
>>>
>>> If this that has changed we can add this separately from everything else
>>> you are doing here.
>>
>> I think I've found the niche.
>>
>>
>> I'm trying to use network namespaces from Go.
>
> Yes. Go sucks for this.

Haha... Neither confirm nor deny.

>> Since setns is thread
>> specific, I'm forced to use this pattern:
>>
>>     runtime.LockOSThread()
>>     defer runtime.UnlockOSThread()
>>     =E2=80=A6
>>     err =3D netns.Set(newns)
>>
>>
>> This is only safe recently:
>> https://github.com/vishvananda/netns/issues/17#issuecomment-367325770
>>
>> - but is still less than ideal performance wise, as it locks out other
>>   socket operations.
>>
>> The socketat() / socketns() would be ideal:
>>
>>   https://lwn.net/Articles/406684/
>>   https://lwn.net/Articles/407495/
>>   https://lkml.org/lkml/2011/10/3/220
>>
>>
>> One thing that is interesting, the LockOSThread works pretty well for
>> receiving, since I can wrap it around the socket()/bind()/listen() at
>> startup. Then accept() can run outside of the lock.
>>
>> It's creating new outbound tcp connections via socket()/connect() pairs
>> that is the issue.
>
> As I understand it you should be able to write socketat in go something l=
ike:
>
>         runtime.LockOSThread()
>         err =3D netns.Set(newns);
>         fd =3D socket(...);
>         err =3D netns.Set(defaultns);
>         runtime.UnlockOSThread()

Yeah, this is currently what I'm having to do. It's painful because due
to the Go runtime model of a single OS netpoller thread, locking the OS
thread to the current goroutine blocks out the other goroutines doing
network I/O.

> I have no real objections to a kernel system call doing that.  It has
> just never risen to the level where it was necessary to optimize
> userspace yet.

Would you be able to accept the patch from this thread with the
container API?

    fd =3D container_socket(int container_fd,
                          int domain, int type, int protocol);

I think that seems more coherent with the rest of the container world
than a follow up of https://lkml.org/lkml/2011/10/3/220 :

    int socketns(int namespace, int domain, int type, int protocol)


I could also put some up if required.


A.


--=20
Alun Evans.