Received: by 2002:ac0:aed5:0:0:0:0:0 with SMTP id t21csp4810463imb;
        Thu, 7 Mar 2019 00:45:44 -0800 (PST)
X-Google-Smtp-Source: APXvYqxcux/+bYW2QDNJm0+mkT8syUSqhS5c9jOZ/WX1cz9/bnq4IpnhSaG1+LiKi4xv+Th+lxRH
X-Received: by 2002:a17:902:7e0f:: with SMTP id b15mr11139758plm.124.1551948344634;
        Thu, 07 Mar 2019 00:45:44 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1551948344; cv=none;
        d=google.com; s=arc-20160816;
        b=Yht/ZNrnT3T5tChB45virAU3+kGBYVEg1oVFX6Y78P8rrZiOhe3E03KlyORbQ1kHTg
         zjYcZ1w9fYZ/bvrC5EDkI5+xe6YVP9qoiVaEhH94R8H5CztLn9aDbLUrqfHC4Wk0W9UQ
         GavbfbqaXX56DkRTpc98ZO7EiArT6on7ZUzTK0aTwOCcdWjVceqn5fP7tjVYD49Zh0U9
         acYcOk+ZoRMujzQ1ePWot5qyWMOIZMxaz77KRYZI8aYYQ1xk48QKLRsWiiRx0Pgam5pI
         zUPuozWSv/lYqr1rk1EK/UM8Z4SoYncvcf+WhxBvdtltGzaJQFhNf6NpJAUrd0GgQomk
         CL3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:sender:dkim-signature:user-agent:in-reply-to
         :content-disposition:mime-version:references:message-id:subject:cc
         :to:from:date;
        bh=BDA6Ko/d05MsugX1KCmdhakB/UEuWEqecuuA/CKUBo8=;
        b=Hry1d2cMD+TzbDud2Ig8sMxDzbhuteelCZWqjtX1SHnKmvgvWlbnO9qPDgtrXUe0LE
         fGQ8FFSTmyHvFG/q2Bu1Tddu75oSy/I5EiydDZrkHoP0ZZv5yxM/NgTmW+P5F41tG7+R
         DANHL7Df/Ugu8kUiLRmh9E9W6ok+FVXRRC7fSmfe4sEWna6kyMJZaPUBIoQB7wJoDEq3
         8i5v2gkSi2rzBxVbAxfojqAP1+CIvraC1BVW67cC1gjJhymPP+0ZpDJCMPbft4VLNXu8
         rhKA71VZ//WEZ1e8cs+p2sWwKGAwcQb8lo7BGccCG5U1pTkgB+Yv5cRHzH+rIubzRqa0
         REFA==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=SB7gFcf4;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67])
        by mx.google.com with ESMTP id z13si3336333pgv.508.2019.03.07.00.45.29;
        Thu, 07 Mar 2019 00:45:44 -0800 (PST)
Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=SB7gFcf4;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1726477AbfCGIo6 (ORCPT <rfc822;epopevad@gmail.com> + 99 others);
        Thu, 7 Mar 2019 03:44:58 -0500
Received: from hqemgate16.nvidia.com ([216.228.121.65]:3703 "EHLO
        hqemgate16.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1726101AbfCGIo5 (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 7 Mar 2019 03:44:57 -0500
Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate16.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA)
        id <B5c80da070000>; Thu, 07 Mar 2019 00:44:55 -0800
Received: from hqmail.nvidia.com ([172.20.161.6])
  by hqpgpgate101.nvidia.com (PGP Universal service);
  Thu, 07 Mar 2019 00:44:55 -0800
X-PGP-Universal: processed;
        by hqpgpgate101.nvidia.com on Thu, 07 Mar 2019 00:44:55 -0800
Received: from nvidia.com (172.20.13.39) by HQMAIL101.nvidia.com
 (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 7 Mar
 2019 08:44:55 +0000
Date:   Thu, 7 Mar 2019 00:44:54 -0800
From:   Neo Jia <cjia@nvidia.com>
To:     Lu Baolu <baolu.lu@linux.intel.com>
CC:     Joerg Roedel <joro@8bytes.org>,
        David Woodhouse <dwmw2@infradead.org>,
        Alex Williamson <alex.williamson@redhat.com>,
        Kirti Wankhede <kwankhede@nvidia.com>, <ashok.raj@intel.com>,
        <sanjay.k.kumar@intel.com>, <jacob.jun.pan@intel.com>,
        <kevin.tian@intel.com>,
        Jean-Philippe Brucker <jean-philippe.brucker@arm.com>,
        <yi.l.liu@intel.com>, <yi.y.sun@intel.com>, <peterx@redhat.com>,
        <tiwei.bie@intel.com>, <xin.zeng@intel.com>,
        <iommu@lists.linux-foundation.org>, <kvm@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>
Subject: Re: [PATCH v7 9/9] vfio/type1: Handle different mdev isolation type
Message-ID: <20190307084453.GA9288@nvidia.com>
References: <20190222021927.13132-1-baolu.lu@linux.intel.com>
 <20190222021927.13132-10-baolu.lu@linux.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20190222021927.13132-10-baolu.lu@linux.intel.com>
X-NVConfidentiality: public
User-Agent: Mutt/1.6.2 (2016-07-01)
X-Originating-IP: [172.20.13.39]
X-ClientProxiedBy: HQMAIL103.nvidia.com (172.20.187.11) To
 HQMAIL101.nvidia.com (172.20.187.10)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1551948295; bh=BDA6Ko/d05MsugX1KCmdhakB/UEuWEqecuuA/CKUBo8=;
        h=X-PGP-Universal:Date:From:To:CC:Subject:Message-ID:References:
         MIME-Version:Content-Type:Content-Disposition:In-Reply-To:
         X-NVConfidentiality:User-Agent:X-Originating-IP:X-ClientProxiedBy;
        b=SB7gFcf48r457Z/cn9qMSWRqc7D6n6z2ytMSePgUKvdBE8XbN91M4b5TSKT4FzzHn
         Yo7mBalPWNkd4oio7PZYgyurudkQvi7dYMV/jDFkI05v4TrqhnjSIcnFeu9vXA5tkW
         OynRvEsxqNG1X6w5HRJ99Sams9o1Ly4ZtdbPUjTZi9VrOdrDCj7bzttEiWmHt4h/Vq
         Fr0/g2PNvX05THEFs9YK4UR0DS8stIuDcgY710nzDOuQsM3BeyIO7J64aNO2f9Tt2U
         +mjCNoqn6ow5I45K4mU4gvk6rzFzNH9aUsYhrnVfdaxeVmKcTpinA10ZC8mxpHeYXo
         zDgZVZanBHvIQ==
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Feb 22, 2019 at 10:19:27AM +0800, Lu Baolu wrote:
> This adds the support to determine the isolation type
> of a mediated device group by checking whether it has
> an iommu device. If an iommu device exists, an iommu
> domain will be allocated and then attached to the iommu
> device. Otherwise, keep the same behavior as it is.
> 
> Cc: Ashok Raj <ashok.raj@intel.com>
> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
> Signed-off-by: Liu Yi L <yi.l.liu@intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> Reviewed-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
> ---
>  drivers/vfio/vfio_iommu_type1.c | 48 ++++++++++++++++++++++++++++-----
>  1 file changed, 41 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index ccc4165474aa..f1392c582a3c 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -1368,13 +1368,40 @@ static void vfio_iommu_detach_group(struct vfio_domain *domain,
>  		iommu_detach_group(domain->domain, group->iommu_group);
>  }
>  

Hi Baolu,

To allow IOMMU-awared mdev, I think you need to modify the
vfio_iommu_type1_pin_pages and vfio_iommu_type1_unpin_pages to remove the
iommu->external_domain check.

Could you please include that in your patch? If not, I can send out a separate
patch to address that issue.

Thanks,
Neo

> +static bool vfio_bus_is_mdev(struct bus_type *bus)
> +{
> +	struct bus_type *mdev_bus;
> +	bool ret = false;
> +
> +	mdev_bus = symbol_get(mdev_bus_type);
> +	if (mdev_bus) {
> +		ret = (bus == mdev_bus);
> +		symbol_put(mdev_bus_type);
> +	}
> +
> +	return ret;
> +}
> +
> +static int vfio_mdev_iommu_device(struct device *dev, void *data)
> +{
> +	struct device **old = data, *new;
> +
> +	new = vfio_mdev_get_iommu_device(dev);
> +	if (!new || (*old && *old != new))
> +		return -EINVAL;
> +
> +	*old = new;
> +
> +	return 0;
> +}
> +
>  static int vfio_iommu_type1_attach_group(void *iommu_data,
>  					 struct iommu_group *iommu_group)
>  {
>  	struct vfio_iommu *iommu = iommu_data;
>  	struct vfio_group *group;
>  	struct vfio_domain *domain, *d;
> -	struct bus_type *bus = NULL, *mdev_bus;
> +	struct bus_type *bus = NULL;
>  	int ret;
>  	bool resv_msi, msi_remap;
>  	phys_addr_t resv_msi_base;
> @@ -1409,23 +1436,30 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
>  	if (ret)
>  		goto out_free;
>  
> -	mdev_bus = symbol_get(mdev_bus_type);
> +	if (vfio_bus_is_mdev(bus)) {
> +		struct device *iommu_device = NULL;
>  
> -	if (mdev_bus) {
> -		if ((bus == mdev_bus) && !iommu_present(bus)) {
> -			symbol_put(mdev_bus_type);
> +		group->mdev_group = true;
> +
> +		/* Determine the isolation type */
> +		ret = iommu_group_for_each_dev(iommu_group, &iommu_device,
> +					       vfio_mdev_iommu_device);
> +		if (ret || !iommu_device) {
>  			if (!iommu->external_domain) {
>  				INIT_LIST_HEAD(&domain->group_list);
>  				iommu->external_domain = domain;
> -			} else
> +			} else {
>  				kfree(domain);
> +			}
>  
>  			list_add(&group->next,
>  				 &iommu->external_domain->group_list);
>  			mutex_unlock(&iommu->lock);
> +
>  			return 0;
>  		}
> -		symbol_put(mdev_bus_type);
> +
> +		bus = iommu_device->bus;
>  	}
>  
>  	domain->domain = iommu_domain_alloc(bus);
> -- 
> 2.17.1
>