Received: by 2002:a05:6a10:a0d1:0:0:0:0 with SMTP id j17csp1039403pxa; Wed, 12 Aug 2020 21:35:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzmBDrRGDB7kFLfVEr0nOQbGnrS8AswJ6eAfr1XRfU9bozt5X/oMP/agnr6PulycfDCsBhr X-Received: by 2002:a17:906:d9db:: with SMTP id qk27mr3011964ejb.37.1597293319293; Wed, 12 Aug 2020 21:35:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597293319; cv=none; d=google.com; s=arc-20160816; b=Ktow8Ux4f04W5uAEu4927IKwrwpDqHWmeJ0Rh8ucdTqi5SEwGvFPKXPtJsiBo0WIE8 6tM3JrvPwWPNQ2g3QQXfwKvWJxfS05HbAFB1F6mh+HInixV2j5bZcC8FO2Z5RVOzazy+ otEwuhDMGzQkdIq8vt3A1yqyFKwm4SceIzc9dNLc5IRtcpytYUYQ7WhyoMkoWJHCNkV9 EZ4hdGMriP0zgepuYtLsodZqLnnvluT5YsOUegFXkpZuorb51S1b+MzXpw72C8kWPJzw qV8tTmd31PMMKC+9+PRYAizUsrNEAwylATeijFXwf9gD9URF4cNtCZTZQetfEcmYYzRT lxbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-language :content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dkim-signature; bh=qvrNz4SFjTzW3E/xjnyIB3Qq4CFTBccMTkHV2o/Wvns=; b=DmeqY0aPE13+R+/Sqn7z5U5x9VyEZQrZDU7lYnuo2Qufsx0wsbuSMauZwxZOXQGQ/c 212dxCOjVaZHbIrjesKperRtNygGZF9eesKzsfRhC3EZY4HrhcyP+uA3xkQ0aO5g5Qlk ifAOcOjfwk2yqES+4CmgyvkUJehhDHsOQvyE4+V7t+sCzyuFvuaXlAHAwgKf/IMybYJ4 fkhhzXswSw9GwZsUqsPPBpa/Kx6WXxQXubYDgW0k+DY0wSZTTK+zNkPj0nXgijrUv02a VLZ+Gyhj2h+lzm2T1xy2UI2eQ2RE8xlqLRxGCoj1EZoAmID5v3CwrVN8IlBy61F5BKkT F9TQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=DGuz8A8z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h24si2617461ejt.451.2020.08.12.21.34.56; Wed, 12 Aug 2020 21:35:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=DGuz8A8z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726167AbgHMEeZ (ORCPT + 99 others); Thu, 13 Aug 2020 00:34:25 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:29287 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725446AbgHMEeX (ORCPT ); Thu, 13 Aug 2020 00:34:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597293261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qvrNz4SFjTzW3E/xjnyIB3Qq4CFTBccMTkHV2o/Wvns=; b=DGuz8A8zQHPu4cXN/GV0rvNMMGd9XKevQFRRs1IfAAJ1G1VpMGuTA9MVGBcapxk4VK557s xtZwwcx2nZmBn+dcT/b94PCQ+4q9NTizfCZVJubx6zV56U/WKFgsjpzzqYkZsT+mJ/HQ1Z VczV2qmcbIngNjfXmW7jjjxqUS6WC00= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-392-vQ5TZQD_Mw6U0R6mxvcHsw-1; Thu, 13 Aug 2020 00:34:16 -0400 X-MC-Unique: vQ5TZQD_Mw6U0R6mxvcHsw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A7D651005504; Thu, 13 Aug 2020 04:34:12 +0000 (UTC) Received: from [10.72.13.44] (ovpn-13-44.pek2.redhat.com [10.72.13.44]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BE435DA30; Thu, 13 Aug 2020 04:33:54 +0000 (UTC) Subject: Re: [PATCH RFC v2 00/18] Add VFIO mediated device support and DEV-MSI support for the idxd driver To: "Tian, Kevin" , Jason Gunthorpe , Alex Williamson Cc: "Jiang, Dave" , "vkoul@kernel.org" , "Dey, Megha" , "maz@kernel.org" , "bhelgaas@google.com" , "rafael@kernel.org" , "gregkh@linuxfoundation.org" , "tglx@linutronix.de" , "hpa@zytor.com" , "Pan, Jacob jun" , "Raj, Ashok" , "Liu, Yi L" , "Lu, Baolu" , "Kumar, Sanjay K" , "Luck, Tony" , "Lin, Jing" , "Williams, Dan J" , "kwankhede@nvidia.com" , "eric.auger@redhat.com" , "parav@mellanox.com" , "Hansen, Dave" , "netanelg@mellanox.com" , "shahafs@mellanox.com" , "yan.y.zhao@linux.intel.com" , "pbonzini@redhat.com" , "Ortiz, Samuel" , "Hossain, Mona" , "dmaengine@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "x86@kernel.org" , "linux-pci@vger.kernel.org" , "kvm@vger.kernel.org" References: <159534667974.28840.2045034360240786644.stgit@djiang5-desk3.ch.intel.com> <20200721164527.GD2021248@mellanox.com> <20200724001930.GS2021248@mellanox.com> <20200805192258.5ee7a05b@x1.home> <20200807121955.GS16789@nvidia.com> From: Jason Wang Message-ID: Date: Thu, 13 Aug 2020 12:33:52 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020/8/12 下午12:05, Tian, Kevin wrote: >> The problem is that if we tie all controls via VFIO uAPI, the other >> subsystem like vDPA is likely to duplicate them. I wonder if there is a >> way to decouple the vSVA out of VFIO uAPI? > vSVA is a per-device (either pdev or mdev) feature thus naturally should > be managed by its device driver (VFIO or vDPA). From this angle some > duplication is inevitable given VFIO and vDPA are orthogonal passthrough > frameworks. Within the kernel the majority of vSVA handling is done by > IOMMU and IOASID modules thus most logic are shared. So why not introduce vSVA uAPI at IOMMU or IOASID layer? > >>> If an userspace DMA interface can be easily >>> adapted to be a passthrough one, it might be the choice. >> It's not that easy even for VFIO which requires a lot of new uAPIs and >> infrastructures(e.g mdev) to be invented. >> >> >>> But for idxd, >>> we see mdev a much better fit here, given the big difference between >>> what userspace DMA requires and what guest driver requires in this hw. >> A weak point for mdev is that it can't serve kernel subsystem other than >> VFIO. In this case, you need some other infrastructures (like [1]) to do >> this. > mdev is not exclusive from kernel usages. It's perfectly fine for a driver > to reserve some work queues for host usages, while wrapping others > into mdevs. I meant you may want slices to be an independent device from the kernel point of view: E.g for ethernet devices, you may want 10K mdevs to be passed to guest. Similarly, you may want 10K net devices which is connected to the kernel networking subsystems. In this case it's not simply reserving queues but you need some other type of device abstraction. There could be some kind of duplication between this and mdev. Thanks > > Thanks > Kevin >