Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 3 May 2022 10:59:44 +0000
From:   Wei Liu <wei.liu@kernel.org>
To:     Dexuan Cui <decui@microsoft.com>
Cc:     wei.liu@kernel.org, kys@microsoft.com, haiyangz@microsoft.com,
        sthemmin@microsoft.com, lorenzo.pieralisi@arm.com,
        bhelgaas@google.com, linux-hyperv@vger.kernel.org,
        linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
        mikelley@microsoft.com, robh@kernel.org, kw@linux.com,
        helgaas@kernel.org, alex.williamson@redhat.com,
        boqun.feng@gmail.com, Boqun.Feng@microsoft.com, jakeo@microsoft.com
Subject: Re: [PATCH v2] PCI: hv: Do not set PCI_COMMAND_MEMORY to reduce VM
 boot time
Message-ID: <20220503105944.nezlg26jfxv4fqha@liuwe-devbox-debian-v2>
References: <20220502074255.16901-1-decui@microsoft.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220502074255.16901-1-decui@microsoft.com>
Precedence: bulk

On Mon, May 02, 2022 at 12:42:55AM -0700, Dexuan Cui wrote:
> Currently when the pci-hyperv driver finishes probing and initializing the
> PCI device, it sets the PCI_COMMAND_MEMORY bit; later when the PCI device
> is registered to the core PCI subsystem, the core PCI driver's BAR detection
> and initialization code toggles the bit multiple times, and each toggling of
> the bit causes the hypervisor to unmap/map the virtual BARs from/to the
> physical BARs, which can be slow if the BAR sizes are huge, e.g., a Linux VM
> with 14 GPU devices has to spend more than 3 minutes on BAR detection and
> initialization, causing a long boot time.
> 
> Reduce the boot time by not setting the PCI_COMMAND_MEMORY bit when we
> register the PCI device (there is no need to have it set in the first place).
> The bit stays off till the PCI device driver calls pci_enable_device().
> With this change, the boot time of such a 14-GPU VM is reduced by almost
> 3 minutes.
> 
> Link: https://lore.kernel.org/lkml/20220419220007.26550-1-decui@microsoft.com/
> Tested-by: Boqun Feng (Microsoft) <boqun.feng@gmail.com>
> Signed-off-by: Dexuan Cui <decui@microsoft.com>
> Reviewed-by: Michael Kelley <mikelley@microsoft.com>
> Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Jake Oshins <jakeo@microsoft.com>

Applied to hyperv-next. Thanks.