Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Jason Gunthorpe <jgg@nvidia.com>
Cc:     LKML <linux-kernel@vger.kernel.org>, x86@kernel.org,
        Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
        linux-pci@vger.kernel.org, Bjorn Helgaas <bhelgaas@google.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Marc Zyngier <maz@kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Dave Jiang <dave.jiang@intel.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Kevin Tian <kevin.tian@intel.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Logan Gunthorpe <logang@deltatee.com>,
        Ashok Raj <ashok.raj@intel.com>, Jon Mason <jdmason@kudzu.us>,
        Allen Hubbe <allenbh@gmail.com>,
        "Ahmed S. Darwish" <darwi@linutronix.de>,
        Reinette Chatre <reinette.chatre@intel.com>
Subject: Re: [patch 12/33] PCI/MSI: Add support for per device MSI[X] domains
In-Reply-To: <Y3U2QFeoDB17kbIC@nvidia.com>
References: <20221111133158.196269823@linutronix.de>
 <20221111135205.951710169@linutronix.de> <Y3U2QFeoDB17kbIC@nvidia.com>
Date:   Wed, 16 Nov 2022 23:38:52 +0100
Message-ID: <87tu2yo77n.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On Wed, Nov 16 2022 at 15:13, Jason Gunthorpe wrote:
> On Fri, Nov 11, 2022 at 02:58:30PM +0100, Thomas Gleixner wrote:
>> +	.info = {
>> +		.flags			= MSI_COMMON_FLAGS | MSI_FLAG_PCI_MSIX,
>> +		.bus_token		= DOMAIN_BUS_PCI_DEVICE_MSIX,
>> +	},
>> +};
>
> I like this splitting alot, it makes the whole thing make so much more
> sense.

:)

>> +bool pci_setup_msi_device_domain(struct pci_dev *pdev)
>> +{
>> +	if (WARN_ON_ONCE(pdev->msix_enabled))
>> +		return false;
>> +
>> +	if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSI))
>> +		return true;
>> +	if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSIX))
>> +		msi_remove_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN);
>> +
>> +	return pci_create_device_domain(pdev, &pci_msi_template, 1);
>
> Hardwired to one 1? What about multi-msi?

MSI has exactly ONE descriptor whether it's single or multi-MSI.

Multi-MSI can have several interrupts hanging off the same descriptor,
but that's not how MSI looks at it because you write ONE message and the
hardware does the substitution of the low bits depending on which vector
is raised.

I pondered to change that, but that would have required to create yet
another code path for the 20years legacy and to adjust every single
implementation of PCI/MSI domains or the underlying parents to handle
this new world order. About 5 years later we might talk about per device
domains then.

Thanks,

        tglx