Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <ca35a14d-501d-265e-b196-a87e1e994cd0@amd.com>
Date:   Tue, 13 Sep 2022 10:10:04 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH RFC 0/2] Generate device tree node for pci devices
Content-Language: en-US
To:     Frank Rowand <frowand.list@gmail.com>, <linux-pci@vger.kernel.org>,
        <devicetree@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <robh@kernel.org>, <helgaas@kernel.org>
CC:     <clement.leger@bootlin.com>, <max.zhen@amd.com>,
        <sonal.santan@amd.com>, <larry.liu@amd.com>, <brian.xu@amd.com>,
        <stefano.stabellini@xilinx.com>, <trix@redhat.com>
References: <1661809417-11370-1-git-send-email-lizhi.hou@amd.com>
 <1d9faa2e-e3fc-d104-c85f-4035233848d6@gmail.com>
From:   Lizhi Hou <lizhi.hou@amd.com>
In-Reply-To: <1d9faa2e-e3fc-d104-c85f-4035233848d6@gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2022 17:10:06.4021
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ed65e13-4920-4f1d-9128-08da95aace4f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT056.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7583
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 9/13/22 00:00, Frank Rowand wrote:
> On 8/29/22 16:43, Lizhi Hou wrote:
>> This patch series introduces OF overlay support for PCI devices which
>> primarily addresses two use cases. First, it provides a data driven method
>> to describe hardware peripherals that are present in a PCI endpoint and
>> hence can be accessed by the PCI host. An example device is Xilinx/AMD
>> Alveo PCIe accelerators. Second, it allows reuse of a OF compatible
>> driver -- often used in SoC platforms -- in a PCI host based system. An
>> example device is Microchip LAN9662 Ethernet Controller.
>>
>> This patch series consolidates previous efforts to define such an
>> infrastructure:
>> https://lore.kernel.org/lkml/20220305052304.726050-1-lizhi.hou@xilinx.com/
>> https://lore.kernel.org/lkml/20220427094502.456111-1-clement.leger@bootlin.com/
>>
>> Normally, the PCI core discovers PCI devices and their BARs using the
>> PCI enumeration process. However, the process does not provide a way to
>> discover the hardware peripherals that are present in a PCI device, and
>> which can be accessed through the PCI BARs. Also, the enumeration process
>> does not provide a way to associate MSI-X vectors of a PCI device with the
>> hardware peripherals that are present in the device. PCI device drivers
>> often use header files to describe the hardware peripherals and their
>> resources as there is no standard data driven way to do so. This patch
>> series proposes to use flattened device tree blob to describe the
>> peripherals in a data driven way. Based on previous discussion, using
>> device tree overlay is the best way to unflatten the blob and populate
>> platform devices. To use device tree overlay, there are three obvious
>> problems that need to be resolved.
>>
>> First, we need to create a base tree for non-DT system such as x86_64. A
>> patch series has been submitted for this:
>> https://lore.kernel.org/lkml/20220624034327.2542112-1-frowand.list@gmail.com/
>> https://lore.kernel.org/lkml/20220216050056.311496-1-lizhi.hou@xilinx.com/
>>
>> Second, a device tree node corresponding to the PCI endpoint is required
>> for overlaying the flattened device tree blob for that PCI endpoint.
>> Because PCI is a self-discoverable bus, a device tree node is usually not
>> created for PCI devices. This series adds support to generate a device
>> tree node for a PCI device which advertises itself using PCI quirks
>> infrastructure.
>>
>> Third, we need to generate device tree nodes for PCI bridges since a child
>> PCI endpoint may choose to have a device tree node created.
>>
>> This patch series is made up of two patches.
>>
>> The first patch is adding OF interface to allocate an OF node. It is copied
>> from:
>> https://lore.kernel.org/lkml/20220620104123.341054-5-clement.leger@bootlin.com/
>>
>> The second patch introduces a kernel option, CONFIG_PCI_OF. When the option
>> is turned on, the kernel will generate device tree nodes for all PCI
>> bridges unconditionally. The patch also shows how to use the PCI quirks
>> infrastructure, DECLARE_PCI_FIXUP_FINAL to generate a device tree node for
>> a device. Specifically, the patch generates a device tree node for Xilinx
>> Alveo U50 PCIe accelerator device. The generated device tree nodes do not
>> have any property. Future patches will add the necessary properties.
>>
>> Clément Léger (1):
>>    of: dynamic: add of_node_alloc()
>>
>> Lizhi Hou (1):
>>    pci: create device tree node for selected devices
>>
>>   drivers/of/dynamic.c        |  50 +++++++++++++----
>>   drivers/pci/Kconfig         |  11 ++++
>>   drivers/pci/bus.c           |   2 +
>>   drivers/pci/msi/irqdomain.c |   6 +-
>>   drivers/pci/of.c            | 106 ++++++++++++++++++++++++++++++++++++
>>   drivers/pci/pci-driver.c    |   3 +-
>>   drivers/pci/pci.h           |  16 ++++++
>>   drivers/pci/quirks.c        |  11 ++++
>>   drivers/pci/remove.c        |   1 +
>>   include/linux/of.h          |   7 +++
>>   10 files changed, 200 insertions(+), 13 deletions(-)
>>
> The patch description leaves out the most important piece of information.
>
> The device located at the PCI endpoint is implemented via FPGA
>     - which is programmed after Linux boots (or somewhere late in the boot process)
>        - (A) and thus can not be described by static data available pre-boot because
>              it is dynamic (and the FPGA program will often change while the Linux
>              kernel is already booted
>        - (B) can be described by static data available pre-boot because the FPGA
>              program will always be the same for this device on this system
>
> I am not positive what part of what I wrote above is correct and would appreciate
> some confirmation of what is correct or incorrect.

There are 2 series devices rely on this patch:

     1) Xilinx Alveo Accelerator cards (FPGA based device)

     2) lan9662 PCIe card

           please see: 
https://lore.kernel.org/lkml/20220427094502.456111-1-clement.leger@bootlin.com/

For Xilinx Alveo device, it is (A). The FPGA partitions can be 
programmed dynamically after boot.


Thanks,

Lzhi

>
> -Frank