Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <3c80a21f-865a-83a1-1abe-dd7180002d55@gmail.com>
Date:   Wed, 25 May 2022 23:46:03 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Subject: Re: [RESEND PATCH v5] PCI: Make sure the bus bridge powered on when
 scanning bus
To:     Bjorn Helgaas <helgaas@kernel.org>
Cc:     bhelgaas@google.com, linux-pci@vger.kernel.org,
        linux-kernel@vger.kernel.org,
        "Rafael J . Wysocki" <rafael@kernel.org>,
        Mika Westerberg <mika.westerberg@linux.intel.com>,
        "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
        Yicong Yang <yangyicong@hisilicon.com>
References: <20220524205842.GA269611@bhelgaas>
From:   Yicong Yang <yangyccccc@gmail.com>
In-Reply-To: <20220524205842.GA269611@bhelgaas>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

在 2022/5/25 4:58, Bjorn Helgaas 写道:
> On Tue, May 17, 2022 at 08:43:19PM +0800, Yicong Yang wrote:
>> When the bus bridge is runtime suspended, we'll fail to rescan
>> the devices through sysfs as we cannot access the configuration
>> space correctly when the bridge is in D3hot.
> Is the "D3hot" above a typo?  I think devices are supposed to respond
> to config accesses when in D3hot.  PCIe r6.0, sec 5.3.1.4.1:
>
>   Configuration and Message requests are the only TLPs accepted by a
>   Function in the D3Hot state. ...


It's right that we can access the bridge's configuration space
when it's in D3hot. But we need to access the configuration space of
downstream devices in a rescan, which is inaccessible in the upstream
bridge's D3hot state. So the "D3hot" state in the commit is qualified
to the *bridge*.


>
>   Functions that are in D3Hot are permitted to be transitioned by
>   software (writing to their PMCSR PowerState field) to the D0active
>   state or the D0uninitialized state. Functions that are in D3Hot must
>   respond to Configuration Space accesses as long as power and clock
>   are supplied so that they can be returned to D0 by software.
>
>> It can be reproduced like:
>>
>> $ echo 1 > /sys/bus/pci/devices/0000:80:00.0/0000:81:00.1/remove
>> $ echo 1 > /sys/bus/pci/devices/0000:80:00.0/pci_bus/0000:81/rescan
>>
>> 0000:80:00.0 is a Root Port and it is runtime-suspended, so
>> 0000:81:00.1 is unreachable after a rescan.
>>
>> Power up the bridge when scanning the child bus and allow it to
>> suspend again by adding pm_runtime_get_sync()/pm_runtime_put()
>> in pci_scan_child_bus_extend().
>>
>> Cc: Rafael J. Wysocki <rafael@kernel.org>
>> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
>> Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
>> ---
>>  drivers/pci/probe.c | 12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
>> index 17a969942d37..b108e72b6586 100644
>> --- a/drivers/pci/probe.c
>> +++ b/drivers/pci/probe.c
>> @@ -2859,11 +2859,20 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus,
>>  	unsigned int used_buses, normal_bridges = 0, hotplug_bridges = 0;
>>  	unsigned int start = bus->busn_res.start;
>>  	unsigned int devfn, fn, cmax, max = start;
>> +	struct pci_dev *bridge = bus->self;
>>  	struct pci_dev *dev;
>>  	int nr_devs;
>>  
>>  	dev_dbg(&bus->dev, "scanning bus\n");
>>  
>> +	/*
>> +	 * Make sure the bus bridge is powered on, otherwise we may not be
>> +	 * able to scan the devices as we may fail to access the configuration
>> +	 * space of subordinates.
>> +	 */
>> +	if (bridge)
>> +		pm_runtime_get_sync(&bridge->dev);
>> +
>>  	/* Go find them, Rover! */
>>  	for (devfn = 0; devfn < 256; devfn += 8) {
>>  		nr_devs = pci_scan_slot(bus, devfn);
>> @@ -2976,6 +2985,9 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus,
>>  		}
>>  	}
>>  
>> +	if (bridge)
>> +		pm_runtime_put(&bridge->dev);
>> +
>>  	/*
>>  	 * We've scanned the bus and so we know all about what's on
>>  	 * the other side of any bridges that may be on this bus plus
>> -- 
>> 2.24.0
>>