Received: by 10.192.165.148 with SMTP id m20csp3577437imm; Mon, 23 Apr 2018 08:49:25 -0700 (PDT) X-Google-Smtp-Source: AIpwx49GOSR1twoosM54+BvSf7kW9rM8NEsuoOV4a0lnS7Hi1xLmVDVKs0JgXIXToXTlHH/JTi4f X-Received: by 10.98.163.21 with SMTP id s21mr20641575pfe.168.1524498565035; Mon, 23 Apr 2018 08:49:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1524498564; cv=none; d=google.com; s=arc-20160816; b=ujOwXdwr/qeCondNtXrrpgsWMX+R0KXA9KL9y7aX7mU9rBLJYh6XKZK4VV0MX1oJQe nIBihrr1ClS5AewdgdPskBAkuPhw59zkg5nH6Att8eUlhXt1WvtV3x0/ER7t3qVzfZ7m GJ2okCbVPQVo4iIDnph8i7/fPTN55rZePYfofkKTsf3yr6wRfdM7/IUsSyfsPq276OFP UNwXXnnABfEGzkLfAUTnRnEc5BkhyVXuT/YYWwuvqY+MKbKWuGrgg6Z8yfVeGh7DCUDt gKClhoQrVKIZ5lRtDEiELUk8oF/6vR38kswqL47W0Xnpf5+TJ95l19eHzRQjep56unFd VlcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:dkim-signature :arc-authentication-results; bh=SDwStLkFnsnxng/X3bLjPL8fhP6o/2T3zGV8weLcX4g=; b=mmXYkoP71f21oViRLznbQueIa9Qgjzip4UA4EaBlZYJQfYALFTx+kdsoNjIdtZ1KIp 5ly7SuZEOCXjTv3DEDt7ylvMumjkDjI1vqNybCLLUh/zIWBFK2dKcjcpy4CgnVCVDqco HpBBL0JI2zYpzjGadcAAkEI7muMwfkSdhpSG5S4kxI54sHYMH0hzE3MWIGBGnJ6jFnZf 874PDY63SJDGzt6w28PUEzV9qGlT4SFpZzS2U5joRblNOf1DgPsb3yY4D0XSyiHPQLzh PtcsWNVfY+ko5hXjHbWb/odFVzJgP9LtRKuTxcGdtdZyjS+IMaSmLeMESThthwugUU8X gfdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gePFVbVl; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 76si9806233pge.492.2018.04.23.08.49.09; Mon, 23 Apr 2018 08:49:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gePFVbVl; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932109AbeDWPrs (ORCPT + 99 others); Mon, 23 Apr 2018 11:47:48 -0400 Received: from mail-oi0-f68.google.com ([209.85.218.68]:42590 "EHLO mail-oi0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754992AbeDWPrl (ORCPT ); Mon, 23 Apr 2018 11:47:41 -0400 Received: by mail-oi0-f68.google.com with SMTP id t27-v6so14704659oij.9; Mon, 23 Apr 2018 08:47:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=SDwStLkFnsnxng/X3bLjPL8fhP6o/2T3zGV8weLcX4g=; b=gePFVbVl9b8Rm7fgbm40/pVuWPfbTYtPi05yXXfqxPJlth+070duN3lwImLDh4wh+B 3JGRyFSEAp3UfAfWo+bgCCPVy1ifJ7E+mhG8b66+z6J8hczolOvunYqNORqwzcF3d3UY ndkE379bB+V5uw+Y7DgSCOEAHcvyaZYONfOaagtVItLN4UdCd2yWy3oKn9sNH7ErsfLX uLcQIkJgUitlneUY84oNCk/VKv2ygqutDsGqu0rxgk7UwcWQpxEQWiUTk8I/9ILqTGgj mps7P+/OCxAxMnpCaj8lvdBb0D/HFPwX4ECDYDqPvBduC0lqanB+PopJNRk9Qg/H/6uX B6QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=SDwStLkFnsnxng/X3bLjPL8fhP6o/2T3zGV8weLcX4g=; b=enfDui/LLdr4k5/lpScpAuOkhsIy5RSe2daDzD485g1YhyzYde158Etf1C4Uyhg2Me 2XX9cAFFBl5Wf4w6bU73wxkwDlGXi+tQ0u4d20LkUFp8kGmErWdV0MTqCAGAhanYzskR hFsRn9l/HqIKrn//lmmLkz5bwDx87hXbp5o4bLuWLPtctYMHIRqNB7uRgxN/5sVOl+IT kaz5VyLe3UDe5huVw/mRxOsLnsDxvxt4B/gjI8SCKLbHOz7gXp2uSyIovkKuOOMTpnrW Nqo2lrO+Z31aI5Q9giW4GQpFdcesQVY93p8Q8RNQTAgaWfBV2nZt3LQyh8pGZFdNhIVD EwQQ== X-Gm-Message-State: ALQs6tB6208Tz34Hjt6Hy+bNfDojaigKaU9yEXBxBMDo2+67bj6q+LSg 3dwjH79ACv5ZUSKaBijpRSq4WsAszV9KMbjkhAE= X-Received: by 2002:aca:3441:: with SMTP id b62-v6mr12156852oia.129.1524498460323; Mon, 23 Apr 2018 08:47:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.201.52.1 with HTTP; Mon, 23 Apr 2018 08:47:39 -0700 (PDT) In-Reply-To: <84adb7c2-bcf7-5e17-a9d0-482416e2d26c@redhat.com> References: <20180420162633.46077.49012.stgit@ahduyck-green-test.jf.intel.com> <20180421203437.GW28657@bhelgaas-glaptop.roam.corp.google.com> <84adb7c2-bcf7-5e17-a9d0-482416e2d26c@redhat.com> From: Alexander Duyck Date: Mon, 23 Apr 2018 08:47:39 -0700 Message-ID: Subject: Re: [pci PATCH v8 0/4] Add support for unmanaged SR-IOV To: Don Dutile Cc: Bjorn Helgaas , Alexander Duyck , Bjorn Helgaas , linux-pci@vger.kernel.org, virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, Netdev , "Daly, Dan" , LKML , linux-nvme@lists.infradead.org, Keith Busch , netanel@amazon.com, Maximilian Heyne , "Wang, Liang-min" , "Rustad, Mark D" , David Woodhouse , Christoph Hellwig , dwmw@amazon.co.uk Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Apr 23, 2018 at 8:21 AM, Don Dutile wrote: > On 04/21/2018 04:34 PM, Bjorn Helgaas wrote: >> >> On Fri, Apr 20, 2018 at 12:28:08PM -0400, Alexander Duyck wrote: >>> >>> This series is meant to add support for SR-IOV on devices when the VFs >>> are >>> not managed by the kernel. Examples of recent patches attempting to do >>> this >>> include: >>> virto - https://patchwork.kernel.org/patch/10241225/ >>> pci-stub - https://patchwork.kernel.org/patch/10109935/ >>> vfio - https://patchwork.kernel.org/patch/10103353/ >>> uio - https://patchwork.kernel.org/patch/9974031/ >>> >>> Since this is quickly blowing up into a multi-driver problem it is >>> probably >>> best to implement this solution as generically as possible. >>> >>> This series is an attempt to do that. What we do with this patch set is >>> provide a generic framework to enable SR-IOV in the case that the PF >>> driver >>> doesn't support managing the VFs itself. >>> >>> I based my patch set originally on the patch by Mark Rustad but there >>> isn't >>> much left after going through and cleaning out the bits that were no >>> longer >>> needed, and after incorporating the feedback from David Miller. At this >>> point >>> the only items to be fully reused was his patch description which is now >>> present in patch 3 of the set. >>> >>> This solution is limited in scope to just adding support for devices that >>> provide no functionality for SR-IOV other than allocating the VFs by >>> calling pci_enable_sriov. Previous sets had included patches for VFIO, >>> but >>> for now I am dropping that as the scope of that work is larger then I >>> think I can take on at this time. >>> >>> v2: Reduced scope back to just virtio_pci and vfio-pci >>> Broke into 3 patch set from single patch >>> Changed autoprobe behavior to always set when num_vfs is set >>> non-zero >>> v3: Updated Documentation to clarify when sriov_unmanaged_autoprobe is >>> used >>> Wrapped vfio_pci_sriov_configure to fix build errors w/o SR-IOV in >>> kernel >>> v4: Dropped vfio-pci patch >>> Added ena and nvme to drivers now using >>> pci_sriov_configure_unmanaged >>> Dropped pci_disable_sriov call in virtio_pci to be consistent with >>> ena >>> v5: Dropped sriov_unmanaged_autoprobe and pci_sriov_conifgure_unmanaged >>> Added new patch that enables pci_sriov_configure_simple >>> Updated drivers to use pci_sriov_configure_simple >>> v6: Defined pci_sriov_configure_simple as NULL when SR-IOV is not enabled >>> Updated drivers to drop "#ifdef" checks for IOV >>> Added pci-pf-stub as place for PF-only drivers to add support >>> v7: Dropped pci_id table explanation from pci-pf-stub driver >>> Updated pci_sriov_configure_simple to drop need for err value >>> Fixed comment explaining why pci_sriov_configure_simple is NULL >>> v8: Dropped virtio from the set, support to be added later after TC >>> approval >>> >>> Cc: Mark Rustad >>> Cc: Maximilian Heyne >>> Cc: Liang-Min Wang >>> Cc: David Woodhouse >>> >>> --- >>> >>> Alexander Duyck (4): >>> pci: Add pci_sriov_configure_simple for PFs that don't manage VF >>> resources >>> ena: Migrate over to unmanaged SR-IOV support >>> nvme: Migrate over to unmanaged SR-IOV support >>> pci-pf-stub: Add PF driver stub for PFs that function only to >>> enable VFs >>> >>> >>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 28 ------------- >>> drivers/nvme/host/pci.c | 20 ---------- >>> drivers/pci/Kconfig | 12 ++++++ >>> drivers/pci/Makefile | 2 + >>> drivers/pci/iov.c | 31 +++++++++++++++ >>> drivers/pci/pci-pf-stub.c | 54 >>> ++++++++++++++++++++++++++ >>> include/linux/pci.h | 3 + >>> include/linux/pci_ids.h | 2 + >>> 8 files changed, 106 insertions(+), 46 deletions(-) >>> create mode 100644 drivers/pci/pci-pf-stub.c >> >> >> I tentatively applied these to pci/virtualization-review. >> >> The code changes look fine, but I want to flesh out the changelogs a >> little bit before merging them. >> >> For example, I'm not sure what you mean by "devices where the PF is >> not capable of managing VF resources." >> > I agree w/Bjorn's assessment of the changelog. > The VF's are (minimally) assigned via the pf-stub driver, so they are > 'managed by the kernel'. > The security model is the same as the existing one, which was the issue we > resolved in the previous set(s) of patches. > > I am hoping that something like vfio will be used to deal with the VF > ownership > and the reset mechanisms during assignement & de-assignment to 'guests' > (qemu-kvm, DPDK, or whatever user-process), > so the known, existing security model(s) is(are) maintained as well. > If so, it'd be good to add such verbage somewhere (as 0/n is not kept in > anything but possibly Bjorn's patchwork, or whatever patch mgmt tool he > uses, and future reference would be good to have) say, an update to > Documentation/PCI/pci-iov-howto.txt. > > So... the 'unmanaged SR-IOV' Subject, IMO, is not a valid Subject for the > patch series any longer. > > No objections to the patch series, as Bjorn noted, just the commit > log(s)/nomenclature of what is really being done. > The expectation of VF enablement via the PF was born out of the fairly > complicated, and unique PF vs VF drivers of the first implementations, which > AlexD knows so well. This "VFs act just like PFs without SRIOV > capabilities" support is what this patch set enables with a much lighter > configuration mechanism. > So, maybe the patch set ought to be 'lightweight SRIOV enablement'. > > --dd I'd be good with this being referred to as "lightweight SRIOV enablement". The only reason why I was referring to it as "unmanaged" was because I am used to drivers that use the PF MMIO registers to manage VF resources and that doesn't exist in this model. Obviously this is all still managed via the extended PCIe configuration though so there is still some management taking place by the PCI subsystem in the kernel. Thanks. - Alex