Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date:   Tue, 29 Sep 2020 18:25:37 +0100
From:   Marc Zyngier <maz@kernel.org>
To:     Jon Hunter <jonathanh@nvidia.com>
Cc:     Jisheng Zhang <Jisheng.Zhang@synaptics.com>,
        Kunihiko Hayashi <hayashi.kunihiko@socionext.com>,
        Neil Armstrong <narmstrong@baylibre.com>,
        linux-pci@vger.kernel.org,
        Binghui Wang <wangbinghui@hisilicon.com>,
        Bjorn Andersson <bjorn.andersson@linaro.org>,
        Masahiro Yamada <yamada.masahiro@socionext.com>,
        Thierry Reding <thierry.reding@gmail.com>,
        linux-arm-kernel@axis.com, Vidya Sagar <vidyas@nvidia.com>,
        Fabio Estevam <festevam@gmail.com>,
        Jerome Brunet <jbrunet@baylibre.com>,
        Rob Herring <robh@kernel.org>,
        Jesper Nilsson <jesper.nilsson@axis.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Kevin Hilman <khilman@baylibre.com>,
        Pratyush Anand <pratyush.anand@gmail.com>,
        linux-tegra@vger.kernel.org, Krzysztof Kozlowski <krzk@kernel.org>,
        Kishon Vijay Abraham I <kishon@ti.com>,
        Kukjin Kim <kgene@kernel.org>,
        NXP Linux Team <linux-imx@nxp.com>,
        Xiaowei Song <songxiaowei@hisilicon.com>,
        Richard Zhu <hongxing.zhu@nxp.com>,
        Martin Blumenstingl <martin.blumenstingl@googlemail.com>,
        linux-arm-msm@vger.kernel.org,
        Sascha Hauer <s.hauer@pengutronix.de>,
        Yue Wang <yue.wang@amlogic.com>,
        linux-samsung-soc@vger.kernel.org,
        Bjorn Helgaas <bhelgaas@google.com>,
        linux-amlogic@lists.infradead.org, linux-omap@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org,
        Jingoo Han <jingoohan1@gmail.com>,
        Andy Gross <agross@kernel.org>, linux-kernel@vger.kernel.org,
        Stanimir Varbanov <svarbanov@mm-sol.com>,
        Pengutronix Kernel Team <kernel@pengutronix.de>,
        Gustavo Pimentel <gustavo.pimentel@synopsys.com>,
        Shawn Guo <shawnguo@kernel.org>,
        Lucas Stach <l.stach@pengutronix.de>
Subject: Re: [PATCH v2 0/5] PCI: dwc: improve msi handling
In-Reply-To: <8e06a370-a37a-5f33-b43b-2830adb31b3e@nvidia.com>
References: <20200924190421.549cb8fc@xhacker.debian>
 <de4d9294-4f6d-c7d1-efc7-c8ef6570bd64@nvidia.com>
 <20200929184851.22682ff1@xhacker.debian>
 <8e06a370-a37a-5f33-b43b-2830adb31b3e@nvidia.com>
User-Agent: Roundcube Webmail/1.4.8
Message-ID: <d4a6eea3c5e33a3a4056885419df95a7@kernel.org>
Precedence: bulk

On 2020-09-29 14:22, Jon Hunter wrote:
> Hi Jisheng,
> 
> On 29/09/2020 11:48, Jisheng Zhang wrote:
>> Hi Jon,
>> 
>> On Fri, 25 Sep 2020 09:53:45 +0100 Jon Hunter wrote:
>> 
>>> 
>>> On 24/09/2020 12:05, Jisheng Zhang wrote:
>>>> Improve the msi code:
>>>> 1. Add proper error handling.
>>>> 2. Move dw_pcie_msi_init() from each users to designware host to 
>>>> solve
>>>> msi page leakage in resume path.
>>> 
>>> Apologies if this is slightly off topic, but I have been meaning to 
>>> ask
>>> about MSIs and PCI. On Tegra194 which uses the DWC PCI driver, 
>>> whenever we
>>> hotplug CPUs we see the following warnings ...
>>> 
>>>  [      79.068351] WARNING KERN IRQ70: set affinity failed(-22).
>>>  [      79.068362] WARNING KERN IRQ71: set affinity failed(-22).
>>> 
>> 
>> I tried to reproduce this issue on Synaptics SoC, but can't reproduce 
>> it.
>> Per my understanding of the code in kernel/irq/cpuhotplug.c, this 
>> warning
>> happened when we migrate irqs away from the offline cpu, this 
>> implicitly
>> implies that before this point the irq has bind to the offline cpu, 
>> but how
>> could this happen given current dw_pci_msi_set_affinity() 
>> implementation
>> always return -EINVAL
> 
> By default the smp_affinity should be set so that all CPUs can be
> interrupted ...
> 
> $ cat /proc/irq/70/smp_affinity
> 0xff
> 
> In my case there are 8 CPUs and so 0xff implies that the interrupt can
> be triggered on any of the 8 CPUs.
> 
> Do you see the set_affinity callback being called for the DWC irqchip 
> in
> migrate_one_irq()?

The problem is common to all MSI implementations that end up muxing
all the end-point MSIs into a single interrupt. With these systems,
you cannot set the affinity of individual MSIs (they don't target a
CPU, they target another interrupt... braindead). Only the mux
interrupt can have its affinity changed.

So returning -EINVAL is the right thing to do.

          M.
-- 
Jazz is not dead. It just smells funny...