Received: by 2002:a25:1506:0:0:0:0:0 with SMTP id 6csp295151ybv; Tue, 4 Feb 2020 21:52:26 -0800 (PST) X-Google-Smtp-Source: APXvYqxKdZW5jWXJJXURT36Zvf3awr+Dnc+hqo8IEEk5viGQ+VHDBXu5LN5kZcHMZNFMBGwZdsob X-Received: by 2002:a05:6830:140f:: with SMTP id v15mr25064320otp.218.1580881946581; Tue, 04 Feb 2020 21:52:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1580881946; cv=none; d=google.com; s=arc-20160816; b=rLD12UoFyZBs4FyXRRl+Vz1bh9NThlQsQZ/Me9AvFL4+aEAwhoBAQ7aIpl6dEGxB/d J+mwQnMWE/73RhpmF3iVNktD7J1ne7RR5HG2SK/ebY3nedAFg/QZnstTqfsLErYiCK+J Z+vRCeDt+kenJjGZGKJsF3mMErYpwAPVYMPPeLVPfVK+72BMTMg8PDolnIlmq5d7aIfm u4iG3Y850X7Y1VfT33d1OhIGJGx/OxAklxL8RPnfkUMs33Hw4djbnV/4yCoHiiV/VEuO iH7ebGNqy/9sthD5hLjRXpuU6paBsqeP0eaPcEWoFg+YI+bYRnZ+wgv65zzlapYEEjQF Nv3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dkim-signature; bh=mCCBz5Rdlbkg4ojmDomdGX6wRFkIiLU8SA3hhtwlDwg=; b=mtDuTOxOPWYPhilglOzaeDZxwVjikWgGfGNKHAOOAp4vbvumOpM4HxmifOChxz3seQ nvQFDHnsCD4hQlSerPKj/EZStBlV+rbYgXW8UngVV8BoAfaChqdSmFyaNB/Bjqk397+F Bd6k4ER4bafRAspEJ/z56+gaey03j4kSJ+vdjGPvhItCB1fklBTkKhlm7Lb5FoUvftPO qvahzWBspsleJyQM/ZmbxzJMoMSc9zAEfbf0iHNsgFziPfj6rOWqsmtesFaOp6hcgmEd 6/dZ2BETe+JzArTcVOvo9Fbx4qee0e8EbrNtqfPpoC5slTIXxgEq8fknoZThSZ5dY60S GOag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=CDTocgNr; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p25si13243619oto.191.2020.02.04.21.52.13; Tue, 04 Feb 2020 21:52:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=CDTocgNr; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726509AbgBEFvT (ORCPT + 99 others); Wed, 5 Feb 2020 00:51:19 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:36822 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbgBEFvS (ORCPT ); Wed, 5 Feb 2020 00:51:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580881877; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mCCBz5Rdlbkg4ojmDomdGX6wRFkIiLU8SA3hhtwlDwg=; b=CDTocgNrNp/v/MPRdY1Z1xxqJcI6aeDY7cangVdqK8AbeIPTKYirbe48ezIr9SD4E0ycGs OE/J23fr9EYDJso0eoWIcOxpy1LXKxxWzQUYsd6BEGrPX1YQKmTOe58RydpsKGEnnlLijg afxOx7Y3aLl6N68CZckMirWa3CWjS1I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-6-BTzEyaLKMeuvyVcErxsMpw-1; Wed, 05 Feb 2020 00:51:15 -0500 X-MC-Unique: BTzEyaLKMeuvyVcErxsMpw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2C27D1034B21; Wed, 5 Feb 2020 05:51:13 +0000 (UTC) Received: from [10.72.13.188] (ovpn-13-188.pek2.redhat.com [10.72.13.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C1C548; Wed, 5 Feb 2020 05:50:36 +0000 (UTC) Subject: Re: [PATCH] vhost: introduce vDPA based backend To: "Michael S. Tsirkin" Cc: Tiwei Bie , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, shahafs@mellanox.com, jgg@mellanox.com, rob.miller@broadcom.com, haotian.wang@sifive.com, eperezma@redhat.com, lulu@redhat.com, parav@mellanox.com, rdunlap@infradead.org, hch@infradead.org, jiri@mellanox.com, hanand@xilinx.com, mhabets@solarflare.com, maxime.coquelin@redhat.com, lingshan.zhu@intel.com, dan.daly@intel.com, cunming.liang@intel.com, zhihong.wang@intel.com References: <20200131033651.103534-1-tiwei.bie@intel.com> <7aab2892-bb19-a06a-a6d3-9c28bc4c3400@redhat.com> <20200204005306-mutt-send-email-mst@kernel.org> <20200205020555.GA369236@___> <798e5644-ca28-ee46-c953-688af9bccd3b@redhat.com> <20200205003048-mutt-send-email-mst@kernel.org> From: Jason Wang Message-ID: Date: Wed, 5 Feb 2020 13:50:28 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20200205003048-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020/2/5 =E4=B8=8B=E5=8D=881:31, Michael S. Tsirkin wrote: > On Wed, Feb 05, 2020 at 11:12:21AM +0800, Jason Wang wrote: >> On 2020/2/5 =E4=B8=8A=E5=8D=8810:05, Tiwei Bie wrote: >>> On Tue, Feb 04, 2020 at 02:46:16PM +0800, Jason Wang wrote: >>>> On 2020/2/4 =E4=B8=8B=E5=8D=882:01, Michael S. Tsirkin wrote: >>>>> On Tue, Feb 04, 2020 at 11:30:11AM +0800, Jason Wang wrote: >>>>>> 5) generate diffs of memory table and using IOMMU API to setup the= dma >>>>>> mapping in this method >>>>> Frankly I think that's a bunch of work. Why not a MAP/UNMAP interfa= ce? >>>>> >>>> Sure, so that basically VHOST_IOTLB_UPDATE/INVALIDATE I think? >>> Do you mean we let userspace to only use VHOST_IOTLB_UPDATE/INVALIDAT= E >>> to do the DMA mapping in vhost-vdpa case? When vIOMMU isn't available= , >>> userspace will set msg->iova to GPA, otherwise userspace will set >>> msg->iova to GIOVA, and vhost-vdpa module will get HPA from msg->uadd= r? >>> >>> Thanks, >>> Tiwei >> I think so. Michael, do you think this makes sense? >> >> Thanks > to make sure, could you post the suggested argument format for > these ioctls? > It's the existed uapi: /* no alignment requirement */ struct vhost_iotlb_msg { =C2=A0=C2=A0=C2=A0 __u64 iova; =C2=A0=C2=A0=C2=A0 __u64 size; =C2=A0=C2=A0=C2=A0 __u64 uaddr; #define VHOST_ACCESS_RO=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x1 #define VHOST_ACCESS_WO=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x2 #define VHOST_ACCESS_RW=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x3 =C2=A0=C2=A0=C2=A0 __u8 perm; #define VHOST_IOTLB_MISS=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 1 #define VHOST_IOTLB_UPDATE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= 2 #define VHOST_IOTLB_INVALIDATE=C2=A0=C2=A0=C2=A0=C2=A0 3 #define VHOST_IOTLB_ACCESS_FAIL=C2=A0=C2=A0=C2=A0 4 =C2=A0=C2=A0=C2=A0 __u8 type; }; #define VHOST_IOTLB_MSG 0x1 #define VHOST_IOTLB_MSG_V2 0x2 struct vhost_msg { =C2=A0=C2=A0=C2=A0 int type; =C2=A0=C2=A0=C2=A0 union { =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 struct vhost_iotlb_msg iotlb; =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 __u8 padding[64]; =C2=A0=C2=A0=C2=A0 }; }; struct vhost_msg_v2 { =C2=A0=C2=A0=C2=A0 __u32 type; =C2=A0=C2=A0=C2=A0 __u32 reserved; =C2=A0=C2=A0=C2=A0 union { =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 struct vhost_iotlb_msg iotlb; =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 __u8 padding[64]; =C2=A0=C2=A0=C2=A0 }; };