Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 27 Apr 2022 16:53:54 -0700
From:   Jakub Kicinski <kuba@kernel.org>
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     netdev <netdev@vger.kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        "linux-cifs@vger.kernel.org" <linux-cifs@vger.kernel.org>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "ak@tempesta-tech.com" <ak@tempesta-tech.com>,
        "borisp@nvidia.com" <borisp@nvidia.com>,
        "simo@redhat.com" <simo@redhat.com>
Subject: Re: [PATCH RFC 4/5] net/tls: Add support for PF_TLSH (a TLS
 handshake listener)
Message-ID: <20220427165354.2eed6c5b@kernel.org>
In-Reply-To: <7B871201-AC3C-46E2-98B0-52B44530E7BD@oracle.com>
References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev>
        <165030059051.5073.16723746870370826608.stgit@oracle-102.nfsv4.dev>
        <20220425101459.15484d17@kernel.org>
        <E8809EC2-D49A-4171-8C88-D5E24FFA4079@oracle.com>
        <20220426075504.18be4ee2@kernel.org>
        <BA6BB8F6-3A2A-427B-A5D7-30B5F778B7E0@oracle.com>
        <20220426164712.068e365c@kernel.org>
        <7B871201-AC3C-46E2-98B0-52B44530E7BD@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, 27 Apr 2022 14:42:53 +0000 Chuck Lever III wrote:
> > On Apr 26, 2022, at 7:47 PM, Jakub Kicinski <kuba@kernel.org> wrote:
> >> RPC-with-TLS requires one RPC as a "starttls" token. That could be
> >> done in user space as part of the handshake, but it is currently
> >> done in the kernel to enable the user agent to be shared with other
> >> kernel consumers of TLS. Keep in mind that we already have two
> >> real consumers: NVMe and RPC-with-TLS; and possibly QUIC.
> >> 
> >> You asserted earlier that creating sockets in user space "scales
> >> better" but did not provide any data. Can we see some? How well
> >> does it need to scale for storage protocols that use long-lived
> >> connections?  
> > 
> > I meant scale with the number of possible crypto protocols, 
> > I mentioned three there.  
> 
> I'm looking at previous emails. The "three crypto protocols"
> don't stand out to me. Which ones?

TLS, QUIC and PSP maybe that was in a different email that what you
quoted, sorry:
https://lore.kernel.org/all/20220426080247.19bbb64e@kernel.org/

PSP:
https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf

> The prototype has a "handshake type" option that enables the kernel
> to request handshakes for different transport layer security
> protocols. Is that the kind of scalability you mean?
> 
> For TLS, we expect to have at least:
> 
>  - ClientHello
>   - X509
>   - PSK
>  - ServerHello
>  - Re-key
> 
> It should be straightforward to add the ability to service
> other handshake types.
> 
> >> I can include unit tests with the prototype. Someone needs to
> >> educate me on what is the preferred unit test paradigm for this
> >> type of subsystem. Examples in the current kernel code base would
> >> help too.  
> > 
> > Whatever level of testing makes you as an engineer comfortable
> > with saying "this test suite is sufficient"? ;)
> > 
> > For TLS we have tools/testing/selftests/net/tls.c - it's hardly
> > an example of excellence but, you know, it catches bugs here and 
> > there.  
> 
> My question wasn't clear, sorry. I meant, what framework is
> appropriate to use for unit tests in this area?

Nothing area specific, tools/testing/selftests/kselftest_harness.h
is what the tls test uses.

> >> When we started discussing TLS handshake requirements with some
> >> community members several years ago, creating the socket in
> >> kernel and passing it up to a user agent was the suggested design.
> >> Has that recommendation changed since then?  
> > 
> > Hm, do you remember who you discussed it with? Would be good 
> > to loop those folks in.  
> 
> Yes, I remember. Trond Myklebust discussed this with Dave Miller
> during a hallway conversation at a conference (probably Plumbers)
> in 2018 or 2019.
> 
> Trond is Cc'd on this thread via linux-nfs@ and Dave is Cc'd via
> netdev@.
> 
> I also traded email with Boris Pismenny about this a year ago,
> and if memory serves he also recommended passing an existing
> socket up to user space. He is Cc'd on this directly.

I see.

> > I wasn't involved at the beginning of the 
> > TLS work, I know second hand that HW offload and nbd were involved 
> > and that the design went thru some serious re-architecting along 
> > the way. In the beginning there was a separate socket for control
> > records, and that was nacked.
> > 
> > But also (and perhaps most importantly) I'm not really objecting 
> > to creating the socket in the kernel. I'm primarily objecting to 
> > a second type of a special TLS socket which has TLS semantics.  
> 
> I don't understand your objection. Can you clarify?
> 
> AF_TLSH is a listen-only socket. It's just a rendezvous point
> for passing a kernel socket up to user space. It doesn't have
> any particular "TLS semantics". It's the user space agent
> listening on that endpoint that implements particular handshake
> behaviors.
> 
> In fact, if the name AF_TLSH gives you hives, that can be made
> more generic.

Yes, a more generic "user space please bake my socket" interface 
is what I'm leaning towards.

> However, that makes it harder for the kernel to
> figure out which listening endpoint handles handshake requests.

Right, the listening endpoint...

Is it possible to instead create a fd-passing-like structured message
which could carry the fd and all the relevant context (what goes 
via the getsockopt() now)? 

The user space agent can open such upcall socket, then bind to
whatever entity it wants to talk to on the kernel side and read
the notifications via recv()?