Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     "lsf-pc@lists.linux-foundation.org" 
        <lsf-pc@lists.linux-foundation.org>
CC:     "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        CIFS <linux-cifs@vger.kernel.org>
Subject: Re: [LSF/MM/BPF TOPIC][LSF/MM/BPF ATTEND] TLS handshake for in-kernel
 consumers
Thread-Topic: [LSF/MM/BPF TOPIC][LSF/MM/BPF ATTEND] TLS handshake for
 in-kernel consumers
Thread-Index: AQHYGD7d7d+2S4V6f0CEtplIA5luzayAYIwA
Date:   Wed, 2 Feb 2022 15:21:16 +0000
Message-ID: <C4E94EAA-7452-4D69-9C06-E5AD5B7A1F14@oracle.com>
References: <3a066f81-a53d-4d39-5efb-bd957443e7e2@suse.de>
In-Reply-To: <3a066f81-a53d-4d39-5efb-bd957443e7e2@suse.de>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?3qPVvEvouheW/kxwBAlSXJtURRjMYLHNRSSeqHQJh2BbrUKcbyM6sJhT5PSh?=
 =?us-ascii?Q?sC7YyU3qLnhVrz6AkGFar2oCQE7WiXDXlf6ceenNJveM5Y8ohyrYAVUJz+a1?=
 =?us-ascii?Q?WShfRhko7x9Fbeql7F0qzJlvxrIlF4Kd8o3ujRbi98H+kDA4XOwB5+vFuUBr?=
 =?us-ascii?Q?ycdL54FUrT7N+hSDPaVs8Z7c85WqRwojew6OIJjM5EadqL5zgvNDTyGlSb+V?=
 =?us-ascii?Q?zL074uvYKgaDbvLDx7pXMIwJ3l0sN2txOjyaTbjj/XSj4nYoilaJmGVgZ1dk?=
 =?us-ascii?Q?PnMF4c0yOyR0MDJ8OFDB9PTXIVT9OKAJ8sTmJBypL8yBjH/RHObyOWIotYWs?=
 =?us-ascii?Q?3nU4fucBkk9cfyw+G/d5RKhgBSbZNg7pP0KkZco2DDjIEfz8oqSveKCO3tTs?=
 =?us-ascii?Q?xuK/qRprU8oo6YKxD8yvxUVmbXFxAHPDnNFbRMQGgHzCZaZHuyJA3Gz30L+L?=
 =?us-ascii?Q?FKIYGLm94wWvh5YdesIT+079Umy4QyNPlYtA49gl5YQOFM6Oslpz1yJqsbL2?=
 =?us-ascii?Q?fONynE1qLzJEuCewiTmygevhxxtektlxVNviZfkN8bLn2tjkuuqLJd827sUg?=
 =?us-ascii?Q?G87z3htj6jTH+4yG2xUj8Sto39pLPRBSvsZPXGArdtjvuTT4NH5axmAUxABu?=
 =?us-ascii?Q?i/be1VpithDqG1Dm6kETIzuqmeaKLiskcJL37ld3N+DS/7R8jc4gZdsJ3OAl?=
 =?us-ascii?Q?rp9m36LfgZzuVDoVnrzk/uWq/uFkk+k6ifXMF4q90sK/CD4bAY3MUX3XEa/j?=
 =?us-ascii?Q?t/edU1meFV7AFOzjbwF0DGk+jXnQJM/pNOltumWXIbo/1iCmsunTHfIasXs+?=
 =?us-ascii?Q?qjrChs3JzqNBh2H/PH43lD38DyTIi2nQHE0w1TmhN2NDvUkHGrqNUeqWnXJL?=
 =?us-ascii?Q?X9DnZrbociPRkCjcrEtkRVhHIl9G2kVXEUdZx3Ni5Atfp8HEO23eH0pWkley?=
 =?us-ascii?Q?iXX8RZ7UwBkKGxkIisy8igwgRzcu2yFqWHo327KgeHLIkshiVxTPu3vqbj21?=
 =?us-ascii?Q?IPyfRChYMUwlTXwSQI+17mQhiM/Mwz2g4iwCrTOPXE8jEplC5zh8nMPYu2gp?=
 =?us-ascii?Q?8AFPT6Fc72J3sEh9Kcp/0AfLiXRhOuYUxZKsjZ0BkJ7buXNFRIsv/kk7ACSs?=
 =?us-ascii?Q?kJoTXyzXLqoqGuOqyKEx0Kv9XHy4qGOmo6iYi9Md+roo7xy260NKvDSK45AY?=
 =?us-ascii?Q?WFqsfwf/j+tGlJWCBBKkjfr42RTmcON9BcuPkdmjsi9sdhMobxxvBtDPRfT/?=
 =?us-ascii?Q?L/fNFG/y1SufSNA/Pq+LEad34HbreEhuKCPmF8oHkPLh4BLWKQtDyq7EFGkg?=
 =?us-ascii?Q?f/pOy42bR4lgMwz4OBwkP84wqfoDd5nCbkVn15uykiEGjE1hFvUuliaDJ9xj?=
 =?us-ascii?Q?PkFHUElqU6F9FbmXFgp8VFQfxIsdJO8W60AAfEa5J/mhg5nMNM3ePrzzPtIy?=
 =?us-ascii?Q?ymvWN1HMVTxkttMuE4Q14GNOd1zcaXdE1AjLWqAtUVXYSqBpXK3//BpWMtSr?=
 =?us-ascii?Q?/a2O55QNDzbx+D2pJHiwktxU76DpWLfgBiexIvJxpQb3yz3P9mdFmF7hGMyh?=
 =?us-ascii?Q?dLHxfv7B2+6zPyNLX0kNq0KYIGUi01ps/l+k83tXDugrv89r4HDrqDEkTCEj?=
 =?us-ascii?Q?38e9bUZ4ip2BTbrrTtUcncQ=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <CAE409FF6A5DCE42963E6F82F324EF2C@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CH0PR10MB4858.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fdc75198-533e-4f3b-acca-08d9e65fa84c
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Feb 2022 15:21:16.8577
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /gs57ax6gbUF+npgYNEXa1MzqjSaWOpgfRihl358F0fxSUqm7LPHh3TQb1FkczjWar/czxrWSe5Pydj3wNaJsA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1001MB2123
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10246 signatures=673430
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999
 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2201110000 definitions=main-2202020085
X-Proofpoint-GUID: YgsLR-5CqKGtVNd2cHw82K8Qrr6N7Y7l
X-Proofpoint-ORIG-GUID: YgsLR-5CqKGtVNd2cHw82K8Qrr6N7Y7l
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

[ ... adding NFS and CIFS ... ]

> On Feb 2, 2022, at 9:12 AM, Hannes Reinecke <hare@suse.de> wrote:
>=20
> Hi all,
>=20
> nvme-over-tcp has the option to utilize TLS for encrypted traffic, but du=
e to the internal design of the nvme-over-fabrics stack we cannot initiate =
the TLS connection from userspace (as the current in-kernel TLS implementat=
ion is designed).
>=20
> This leaves us with two options:
> 1) Put TLS handshake into the kernel (which will be quite some
>  discussion as it's arguably a userspace configuration)
> 2) Pass an in-kernel socket to userspace and have a userspace
>  application to run the TLS handshake.
>=20
> None of these options are quiet clear cut, as we will be have to put
> quite some complexity into the kernel to do full TLS handshake (if we
> were to go with option 1) or will have to design a mechanism to pass
> an in-kernel socket to userspace as we don't do that currently (if we wer=
e going with option 2).
>=20
> We have been discussing some ideas on how to implement option 2 (together=
 with Chuck Lever and the NFS crowd), but so far haven't been able to come =
up with a decent design.
>=20
> So I would like to discuss with interested parties on how TLS handshake c=
ould be facilitated, and what would be the best design options here.

IMO we are a bit farther along than Hannes suggests, and I had
the impression that we have already agreed on a "decent design"
(see Ben Coddington's earlier post).

We currently have several prototypes we can discuss, and there
are some important issues on the table.

First, from the start we have recognized that we have a range
of potential in-kernel TLS consumers. To name a few: NVMe/TLS,
RPC-with-TLS (for in-transit NFS encryption), CIFS/SMB, and,
when it arrives, the QUICv1 transport. We don't intend to
build something that works for only one of these, thus it
will not be based on existing security infrastructure like
rpc.gssd.

Second, we believe in-kernel consumers will hitch onto the
existing kTLS infrastructure to handle payload encryption and
decryption. This transparently enables both software-based
and offload, and in the latter case, we hope for quite
reasonable performance.

As Hannes said, the missing piece is support for the TLS
handshake protocol to boot strap each TLS session.

The security community has demanded that we stick with user
space handshake implementations because they view the TLS
handshake as complex and a broad attack surface. I question
those assumptions, but even so...

We will need to have in-kernel handshake to support NFSROOT
and NVMe/TLS with a root filesystem, which are requirements
for the storage community.

We have an in-kernel prototype based on Tempesta's TLSv1.2
offload in the works. See the "topic-rpc-with-tls" branch:

 https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git

We also have three promising user space upcall implementations
that are helping us with architectural choices. The main issue
here is how to set the correct peer authentication parameters
for each handshake. As Ben said, key rings can play a critical
part (as might netlink, but perhaps that can be avoided). We
are sensitive to containerization requirements as well.

One (not-yet-working) user space prototype is published in
the "topic-rpc-with-tls-upcall" branch in the repo above.


> The proposed configd would be an option, but then we don't have that, eit=
her :-)
>=20
> Required attendees:
>=20
> Chuck Lever
> James Bottomley
> Sagi Grimberg
> Keith Busch
> Christoph Hellwig
> David Howells

Anyone from the CIFS team? Enzo? How about Dave Miller?

Actually I think we need to have security and network folks
at the table. LSF/MM might not be the right venue for a
full-scale discussion of alternatives. We have been waiting
for an opportunity to bring this to a broad community event
such as Plumbers but the pandemic has interfered.

However, I am happy to discuss alternative upcall mechanisms,
new requirements, and anything related to securing an
in-kernel handshake against remote attack.

--
Chuck Lever