Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender
  authenticity information available from domain of
  igor.druzhinin@citrix.com) identity=pra;
  client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com;
  envelope-from="igor.druzhinin@citrix.com";
  x-sender="igor.druzhinin@citrix.com";
  x-conformance=sidf_compatible
Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of
  igor.druzhinin@citrix.com designates 162.221.158.21 as
  permitted sender) identity=mailfrom;
  client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com;
  envelope-from="igor.druzhinin@citrix.com";
  x-sender="igor.druzhinin@citrix.com";
  x-conformance=sidf_compatible; x-record-type="v=spf1";
  x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133
  ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4
  ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88
  ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all"
Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender
  authenticity information available from domain of
  postmaster@mail.citrix.com) identity=helo;
  client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com;
  envelope-from="igor.druzhinin@citrix.com";
  x-sender="postmaster@mail.citrix.com";
  x-conformance=sidf_compatible
IronPort-SDR: T/9zojaCofQTVlg9JytA1HFg6m2EOIp6isp71QNFOWodT3M/pxCZDFS6Tp7lRml6yh9gRo2kio
 k4m0NoQDKq4gwgl/vh+1lZsUVEdxbh0ykvTE3sxXrqKw8IeRa4vj4u+eAuPXXOqnOd03HFyjme
 oMU975HElEZIHLolgJex75ZNWkzp3vEu7c6F6MWHfuf8EvN3yzZL2fw+bqaBQuuOUXRimGOswx
 omBipzSNypUdsO2QvhRuh8Ky9w7+glJ3gnAXM8VxjTLQ0iyvFQraFIUR4PtigIttZLRI4ee31Q
 6qM=
Subject: Re: [PATCH] xen/pci: try to reserve MCFG areas earlier
To:     Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        <xen-devel@lists.xenproject.org>, <linux-kernel@vger.kernel.org>
CC:     <jgross@suse.com>
References: <1567556431-9809-1-git-send-email-igor.druzhinin@citrix.com>
 <5054ad91-5b87-652c-873a-b31758948bd7@oracle.com>
From:   Igor Druzhinin <igor.druzhinin@citrix.com>
Message-ID: <e3114d56-51cd-b973-1ada-f6a60a7e99cc@citrix.com>
Date:   Sat, 7 Sep 2019 00:00:24 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.8.0
MIME-Version: 1.0
In-Reply-To: <5054ad91-5b87-652c-873a-b31758948bd7@oracle.com>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 06/09/2019 23:30, Boris Ostrovsky wrote:
> On 9/3/19 8:20 PM, Igor Druzhinin wrote:
>> If MCFG area is not reserved in E820, Xen by default will defer its usage
>> until Dom0 registers it explicitly after ACPI parser recognizes it as
>> a reserved resource in DSDT. Having it reserved in E820 is not
>> mandatory according to "PCI Firmware Specification, rev 3.2" (par. 4.1.2)
>> and firmware is free to keep a hole E820 in that place. Xen doesn't know
>> what exactly is inside this hole since it lacks full ACPI view of the
>> platform therefore it's potentially harmful to access MCFG region
>> without additional checks as some machines are known to provide
>> inconsistent information on the size of the region.
>>
>> Now xen_mcfg_late() runs after acpi_init() which is too late as some basic
>> PCI enumeration starts exactly there. Trying to register a device prior
>> to MCFG reservation causes multiple problems with PCIe extended
>> capability initializations in Xen (e.g. SR-IOV VF BAR sizing). There are
>> no convenient hooks for us to subscribe to so try to register MCFG
>> areas earlier upon the first invocation of xen_add_device(). 
> 
> 
> Where is MCFG parsed? pci_arch_init()?

It happens twice:
1) first time early one in pci_arch_init() that is arch_initcall - that
time pci_mmcfg_list will be freed immediately there because MCFG area is
not reserved in E820;
2) second time late one in acpi_init() which is subsystem_initcall right
before where PCI enumeration starts - this time ACPI tables will be
checked for a reserved resource and pci_mmcfg_list will be finally
populated.

The problem is that on a system that doesn't have MCFG area reserved in
E820 pci_mmcfg_list is empty before acpi_init() and our PCI hooks are
called in the same place. So MCFG is still not in use by Xen at this
point since we haven't reached our xen_mcfg_late().

Igor