Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        LKML <linux-kernel@vger.kernel.org>
Cc:     "x86@kernel.org" <x86@kernel.org>, Joerg Roedel <joro@8bytes.org>,
        Will Deacon <will@kernel.org>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Marc Zyngier <maz@kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jason Gunthorpe <jgg@mellanox.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        "Williams, Dan J" <dan.j.williams@intel.com>,
        Logan Gunthorpe <logang@deltatee.com>,
        "Raj, Ashok" <ashok.raj@intel.com>, Jon Mason <jdmason@kudzu.us>,
        Allen Hubbe <allenbh@gmail.com>,
        "Ahmed S. Darwish" <darwi@linutronix.de>
Subject: RE: [patch V2 12/33] PCI/MSI: Add support for per device MSI[X]
 domains
In-Reply-To: <87zgchew4u.ffs@tglx>
References: <20221121083657.157152924@linutronix.de>
 <20221121091327.163146917@linutronix.de>
 <BN9PR11MB5276E9AD0F86A27A23ED8CFB8C0C9@BN9PR11MB5276.namprd11.prod.outlook.com>
 <87zgchew4u.ffs@tglx>
Date:   Wed, 23 Nov 2022 22:50:11 +0100
Message-ID: <8735a9e3xo.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On Wed, Nov 23 2022 at 12:41, Thomas Gleixner wrote:
> On Wed, Nov 23 2022 at 08:08, Kevin Tian wrote:
>>> +bool pci_setup_msi_device_domain(struct pci_dev *pdev)
>>> +{
>>> +	if (WARN_ON_ONCE(pdev->msix_enabled))
>>> +		return false;
>>
>> the check already exists in __pci_enable_msi_range()
>>
>>> +bool pci_setup_msix_device_domain(struct pci_dev *pdev, unsigned int
>>> hwsize)
>>> +{
>>> +	if (WARN_ON_ONCE(pdev->msix_enabled))
>>> +		return false;
>>
>> ditto.
>>
>> btw according to the comment this should check pdev->msi_enabled.
>
> Yeah, those are probably redundant.

I fixed the MSIX check and kept them for paranoia reasons, so changes in
the calling code get caught.

>> This is general PCI MSI logic. So an open related to my rely to patch02,
>> is it correct for PCI core to assume that the real parent imposes all the
>> restrictions and there is no need to further go down the hierarchy?
>
> That was my working assumption and it turned out to be correct with both
> x86 and ARM.

As a follow up, I went through some of the other architectures,
especially the places which have extra limitations and it turns out that
the restriction comes always from the direct parent.

If that ever changes then we need a callback which lets us evaluate the
resulting capabilities through the hierarchy. That's nothing which can
be evaluated directly.

Just look at the x86 hierarchy with IR. IR allows multi PCI-MSI, but the
vector domain does not. Who is right? That's a decision which is made in
the particular hierarchy.

For now it's valid that the direct MSI parent has the proper set
available.

Thanks,

        tglx