Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <40bc060f-f359-081d-9ba7-fae531cf2cd6@suse.de>
Date:   Tue, 26 Apr 2022 17:58:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.8.0
Subject: Re: [PATCH RFC 4/5] net/tls: Add support for PF_TLSH (a TLS handshake
 listener)
Content-Language: en-US
To:     Jakub Kicinski <kuba@kernel.org>, Sagi Grimberg <sagi@grimberg.me>
Cc:     Chuck Lever <chuck.lever@oracle.com>, netdev@vger.kernel.org,
        linux-nfs@vger.kernel.org, linux-nvme@lists.infradead.org,
        linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
        ak@tempesta-tech.com, borisp@nvidia.com, simo@redhat.com
References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev>
 <165030059051.5073.16723746870370826608.stgit@oracle-102.nfsv4.dev>
 <20220425101459.15484d17@kernel.org>
 <66077b73-c1a4-d2ae-c8e4-3e19e9053171@suse.de>
 <1fca2eda-83e4-fe39-13c8-0e5e7553689b@grimberg.me>
 <20220426080247.19bbb64e@kernel.org>
From:   Hannes Reinecke <hare@suse.de>
In-Reply-To: <20220426080247.19bbb64e@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 4/26/22 17:02, Jakub Kicinski wrote:
> On Tue, 26 Apr 2022 17:29:03 +0300 Sagi Grimberg wrote:
>>>> Create the socket in user space, do all the handshakes you need there
>>>> and then pass it to the kernel.  This is how NBD + TLS works.  Scales
>>>> better and requires much less kernel code.
>>>>   
>>> But we can't, as the existing mechanisms (at least for NVMe) creates the
>>> socket in-kernel.
>>> Having to create the socket in userspace would require a completely new
>>> interface for nvme and will not be backwards compatible.
>>
>> And we will still need the upcall anyways when we reconnect
>> (re-establish the socket)
> 
> That totally flew over my head, I have zero familiarity with in-kernel
> storage network users :S
> 
Count yourself lucky.

> In all honesty the tls code in the kernel is a bit of a dumping ground.
> People come, dump a bunch of code and disappear. Nobody seems to care
> that the result is still (years in) not ready for production use :/
> Until a month ago it'd break connections even under moderate memory
> pressure. This set does not even have selftests.
> 
Well, I'd been surprised that it worked, too.
And even more so that Boris Piskenny @ Nvidia is actively working on it.
(Thanks, Sagi!)

> Plus there are more protocols being actively worked on (QUIC, PSP etc.)
> Having per ULP special sauce to invoke a user space helper is not the
> paradigm we chose, and the time as inopportune as ever to change that.

Which is precisely what we hope to discuss at LSF.
(Yes, I know, probably not the best venue to discuss network stuff ...)

Each approach has its drawbacks:

- Establishing sockets from userspace will cause issues during 
reconnection, as then someone (aka the kernel) will have to inform 
userspace that a new connection will need to be established.
(And that has to happen while the root filesystem is potentially 
inaccessible, so you can't just call arbitrary commands here)
(Especially call_usermodehelper() is out of the game)
- Having ULP helpers (as with this design) mitigates that problem 
somewhat in the sense that you can mlock() that daemon and having it 
polling on an intermediate socket; that solves the notification problem.
But you have to have ULP special sauce here to make it work.
- Moving everything in kernel is ... possible. But then you have yet 
another security-relevant piece of code in the kernel which needs to be 
audited, CVEd etc. Not to mention the usual policy discussion whether it 
really belongs into the kernel.

So I don't really see any obvious way to go; best we can do is to pick 
the least ugly :-(

Cheers,

Hannes