Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Dexuan Cui <decui@microsoft.com>, Ming Lei <ming.lei@redhat.com>,
        Christoph Hellwig <hch@lst.de>,
        Christian Borntraeger <borntraeger@de.ibm.com>,
        Stefan Haberland <sth@linux.vnet.ibm.com>,
        Jens Axboe <axboe@kernel.dk>,
        Marc Zyngier <marc.zyngier@arm.com>,
        "linux-pci\@vger.kernel.org" <linux-pci@vger.kernel.org>,
        "linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc:     Long Li <longli@microsoft.com>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        Michael Kelley <mikelley@microsoft.com>
Subject: Re: irq_build_affinity_masks() allocates improper affinity if num_possible_cpus() > num_present_cpus()?
In-Reply-To: <KU1P153MB0120D20BC6ED8CF54168EEE6BF0D0@KU1P153MB0120.APCP153.PROD.OUTLOOK.COM>
References: <KU1P153MB0120D20BC6ED8CF54168EEE6BF0D0@KU1P153MB0120.APCP153.PROD.OUTLOOK.COM>
Date:   Tue, 06 Oct 2020 20:57:39 +0200
Message-ID: <87lfgj6v30.fsf@nanos.tec.linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On Tue, Oct 06 2020 at 06:47, Dexuan Cui wrote:
> I'm running a single-CPU Linux VM on Hyper-V. The Linux kernel is v5.9-rc7
> and I have CONFIG_NR_CPUS=256.
>
> The Hyper-V Host (Version 17763-10.0-1-0.1457) provides a guest firmware,
> which always reports 128 Local APIC entries in the ACPI MADT table. Here
> only the first Local APIC entry's "Processor Enabled" is 1 since this
> Linux VM is configured to have only 1 CPU. This means: in the Linux kernel,
> the "cpu_present_mask" and " cpu_online_mask " have only 1 CPU (i.e. CPU0),
> while the "cpu_possible_mask" has 128 CPUs, and the "nr_cpu_ids" is 128.
>
> I pass through an MSI-X-capable PCI device to the Linux VM (which has
> only 1 virtual CPU), and the below code does *not* report any error
> (i.e. pci_alloc_irq_vectors_affinity() returns 2, and request_irq()
> returns 0), but the code does not work: the second MSI-X interrupt is not
> happening while the first interrupt does work fine.
>
> int nr_irqs = 2;
> int i, nvec, irq;
>
> nvec = pci_alloc_irq_vectors_affinity(pdev, nr_irqs, nr_irqs,
>                 PCI_IRQ_MSIX | PCI_IRQ_AFFINITY, NULL);

Why should it return an error?

> for (i = 0; i < nvec; i++) {
>         irq = pci_irq_vector(pdev, i);
>         err = request_irq(irq, test_intr, 0, "test_intr", &intr_cxt[i]);
> }

And why do you expect that the second interrupt works?

This is about managed interrupts and the spreading code has two vectors
to which it can spread the interrupts. One is assigned to one half of
the possible CPUs and the other one to the other half. Now you have only
one CPU online so only the interrupt with has the online CPU in the
assigned affinity mask is started up.

That's how managed interrupts work. If you don't want managed interrupts
then don't use them.

Thanks,

        tglx