Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <33a546ea17fe62ca9fc763041fe340ad5e4945fe.camel@redhat.com>
Subject: Re: [PATCH v2] dmaengine: idxd: avoid deadlock in
 process_misc_interrupts()
From:   Jerry Snitselaar <jsnitsel@redhat.com>
To:     "Yu, Fenghua" <fenghua.yu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Zhu, Tony" <tony.zhu@intel.com>
Cc:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Vinod Koul <vkoul@kernel.org>,
        "dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>
Date:   Wed, 24 Aug 2022 14:34:00 -0700
In-Reply-To: <IA1PR11MB6097988CC9C2C1C713E3D8339B739@IA1PR11MB6097.namprd11.prod.outlook.com>
References: <20220823162435.2099389-1-jsnitsel@redhat.com>
         <20220823163709.2102468-1-jsnitsel@redhat.com>
         <905d3feb-f75b-e91c-f3de-b69718aa5c69@intel.com>
         <20220824005435.jyexxvjxj3z7tc2f@cantor>
         <223e5a43-95a5-da54-0ff7-c2e088a072e3@intel.com>
         <38e416b47bb30fa161e52f24ecbcf95015480fed.camel@redhat.com>
         <IA1PR11MB6097988CC9C2C1C713E3D8339B739@IA1PR11MB6097.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4 (3.44.4-1.fc36) 
MIME-Version: 1.0
Precedence: bulk

On Wed, 2022-08-24 at 21:11 +0000, Yu, Fenghua wrote:
> Hi, Jerry,
>=20
> > I see another potential issue. If a software reset is attempted
> > idxd_device_reinit()
> > will be called which walks the wqs, and if a wq has the state
> > IDXD_WQ_ENABLED it calls idxd_wq_enable(), but the first thing
> > idxd_wq_enable() does is see that the state is IDXD_WQ_ENABLED and
> > returns 0.
> > Without the wq enable command being sent, it will not be re-
> > enabled, yes?
>=20
> Could you please describe how to reproduce the issues and test case?

Nothing special, just running dsa_user_test_runner.sh from idxd-config
(3.4.6.3) with intel_iommu=3Don,sm_on (default is lazy dma domain) on an
Intel supplied SPR system. During the run there is a dmar fault, halts
the device with it needing an flr reset. That is where the deadlock was
noticed. Then after the dsa_user_runner.sh fails, if you do modprobe -r
idxd, you hit the warning in devm_iounmap().

The idxd_device_reinit() case I haven't reproduced, I'm just looking at
the code.

I should note that I've only run into an issue with that model
of system. The other models of SPR systems don't hit an issue. This is
with a 6.0-rc1 kernel, and our kernel. I'm guessing there is an issue
with that model of system/fw. I've seen both fault reasons 0x50 and
0x58 when it faults.

Regards,
Jerry

>=20
> Thanks.
>=20
> -Fenghua