Received-SPF: pass (google.com: domain of linux-kernel+bounces-96575-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>
CC: "kvm@vger.kernel.org" <kvm@vger.kernel.org>, "eric.auger@redhat.com"
	<eric.auger@redhat.com>, "clg@redhat.com" <clg@redhat.com>, "Chatre,
 Reinette" <reinette.chatre@intel.com>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>
Subject: RE: [PATCH 1/7] vfio/pci: Disable auto-enable of exclusive INTx IRQ
Thread-Topic: [PATCH 1/7] vfio/pci: Disable auto-enable of exclusive INTx IRQ
Thread-Index: AQHacAtu1xn5Hoku+0au9dqEK9o17bEr8fnggADG1QCAALkOQA==
Date: Fri, 8 Mar 2024 07:20:49 +0000
Message-ID: <BN9PR11MB5276AA639CD6C3D23C6C9CE48C272@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20240306211445.1856768-1-alex.williamson@redhat.com>
	<20240306211445.1856768-2-alex.williamson@redhat.com>
	<BL1PR11MB52711AF96C93A7C2B70FE12D8C202@BL1PR11MB5271.namprd11.prod.outlook.com>
 <20240307131716.5feda507.alex.williamson@redhat.com>
In-Reply-To: <20240307131716.5feda507.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?IFTxka29BS7wWAGPgRCaUi37IEScBl15ViiRViK4Xc4peBq7f6/5jr2hqF9I?=
 =?us-ascii?Q?Tr3VXQrKecrrDr1AXUVuZSh2N68hfFDh8F0UW3CMShigOKETvAI5ZE3Lcyzg?=
 =?us-ascii?Q?EkGWyN9xvRFzHiuvTj0ihQtMKBeD5usl30WsQvyGnaaSBhBOamjM7UU5L5lX?=
 =?us-ascii?Q?jI6NmiIZHR9jMb7QEOn1zZ97ILQu2t8OPgTmb0dXOxCKL1WGAwKpSKzkXvJ8?=
 =?us-ascii?Q?wYjv6mlPFyHIZ7O4Dj3+lXjV7PBtucw1cft7JI+xqeP23EGTrOMM7qBLnOdH?=
 =?us-ascii?Q?+OQcq5a7z59EGsltS7ieZeSFD50u4iE1f4vGO5yBp3lMB27pn1Fin27Qtq4a?=
 =?us-ascii?Q?oa/kJssSnKpKXjCBBRAvILnYjMu2OEu9SvYnizywEd1+SDFH/K6MWLo8GmeN?=
 =?us-ascii?Q?yuBeiM+zeVKU6czm8E3506uVm5JQN0BKZwBkU0yZq9n0Cn9haAdsArsS8Z0e?=
 =?us-ascii?Q?UwP9jebTVWfYGJXdzPMx8kWygUblomlGeA2wGcpYl7T6gYzFWjEYpW5J+G4L?=
 =?us-ascii?Q?NPgMO5NXArOVioEEC09O4dns+Lq6wFQWBulgtiMdFNXmiYdYYtr4OSXESgmA?=
 =?us-ascii?Q?zIq5Cvp9NgNYY3IcyJpnCKc0dEegeUojAvaIqZ9GRdEOaHtVz2G4Y84vX2cG?=
 =?us-ascii?Q?Of//GS/BW1L2UzN/lLzRATN7kL/59VtNvGeEdSrAkccHq665sY6Re+t3EZk6?=
 =?us-ascii?Q?dUI2+l+sX+vz2pYmHDvSoxeXd8RHBmngpB4UBDfy7Y0HTqXG9LMyyr+PQA3E?=
 =?us-ascii?Q?4Hm6EPbGT0z3NFVYjRtZkhJYu7685UqGrUHvdyT/sXZ8pPGal6WOosXbMVGd?=
 =?us-ascii?Q?awt0yeJKk42zcH9iuXbVn2zQ8v2amVGChWDSs+vrnawWC/k3dzOjYNGp7E3Q?=
 =?us-ascii?Q?z2AR66QKU6JuN0FMBCWXCrZOQxtxb203cqilEG9EHf0yD23e0oW5FSMl/ipt?=
 =?us-ascii?Q?wCtFUqMIx4ImqEr3VM0B26l7md7YdLQxqBILiBETX+8/qn7TvE0JM2oL9Z4P?=
 =?us-ascii?Q?dDGhrrbjKx9BD7mdUvft9mDgIc9vgPq7u97QCf/f0sxnQNUMgOmtulfp6lrc?=
 =?us-ascii?Q?5Ei0Ed2Om0NOo9dASCvxEMHs2qSJByVzCHFKo0O/GUp3YEhH/sN3kqit/szP?=
 =?us-ascii?Q?85hzHY/7BJtT/igABaPysLaq1neH9IV81FpOOLpqWsULW6ERWl6jhbzCLg1N?=
 =?us-ascii?Q?WqodmJfSkf7DcsPVv0jOkmuZq3g8XzEMV9oGLP4AtwW2Wuq0JIfIvbO3qszg?=
 =?us-ascii?Q?lHFTfNLvq0lqGfGqHoKYQf1Stf0W0vc3nZoyzoZyjkz8vgU6UgPQGVSx2Z4B?=
 =?us-ascii?Q?vtHoyoKAfZwqrHpKVhdnHO6YG/jyHH4ncwOSZfkYpzhyv9VfFKhwIxCJKx/c?=
 =?us-ascii?Q?D/W6JIh66hMcWgONtMIvXg0AvQMeC0U2xUoN6f5DqX6cloeMbNCso0Gu8nkR?=
 =?us-ascii?Q?VBHg7KKToBvuwtXuB1QZ3ayfHD9lpPrCdjWBMLopLJ+siq0if6Av7y74Nyew?=
 =?us-ascii?Q?lGLHfwurpu4sNt7xQt+wYDVQRtdSLdCwQKv7JIIML7u9LLHx/yUythZ6amf2?=
 =?us-ascii?Q?vF6ag68LF1Q/cRqvu8Zl9a/zToN5nvM64+pAFsXk?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 36394d81-4468-4b66-89dd-08dc3f404818
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2024 07:20:49.9255
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rVZ3umjAxmNW37ivn6fZiu1juI/q8O4AK5PjlxlNbP0sLPcRLrW7wtECcHPF5s5uEnFJv+CovI8oMlBNublpfA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5274
X-OriginatorOrg: intel.com

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Friday, March 8, 2024 4:17 AM
>=20
> On Thu, 7 Mar 2024 08:28:45 +0000
> "Tian, Kevin" <kevin.tian@intel.com> wrote:
>=20
> > > From: Alex Williamson <alex.williamson@redhat.com>
> > > Sent: Thursday, March 7, 2024 5:15 AM
> > >
> > > Currently for devices requiring masking at the irqchip for INTx, ie.
> > > devices without DisINTx support, the IRQ is enabled in request_irq()
> > > and subsequently disabled as necessary to align with the masked statu=
s
> > > flag.  This presents a window where the interrupt could fire between
> > > these events, resulting in the IRQ incrementing the disable depth twi=
ce.
> >
> > Can you elaborate the last point about disable depth?
>=20
> Each irq_desc maintains a depth field, a disable increments the depth,
> an enable decrements.  On the disable transition from 0 to 1 the IRQ
> chip is disabled, on the enable transition from 1 to 0 the IRQ chip is
> enabled.
>=20
> Therefore if an interrupt fires between request_irq() and
> disable_irq(), vfio_intx_handler() will disable the IRQ (depth 0->1).
> Note that masked is not tested here, the interrupt is expected to be
> exclusive for non-pci_2_3 devices.  @masked would be redundantly set to
> true.  The setup call path would increment depth to 2.  The order these
> happen is not important so long as the interrupt is in-flight before
> the setup path disables the IRQ.
>=20
> > > This would be unrecoverable for a user since the masked flag prevents
> > > nested enables through vfio.
> >
> > What is 'nested enables'?
>=20
> In the case above we have masked true and disable depth 2.  If the user
> now unmasks the interrupt then depth is reduced to 1, the IRQ is still
> disabled, and masked is false.  The masked value is now out of sync
> with the IRQ line and prevents the user from unmasking again.  The
> disable depth is stuck at 1.
>=20
> Nested enables would be if we allowed the user to unmask a line that we
> think is already unmasked.

Thanks! clear to me now.

>=20
> > > Instead, invert the logic using IRQF_NO_AUTOEN such that exclusive IN=
Tx
> > > is never auto-enabled, then unmask as required.
> > >
> > > Fixes: 89e1f7d4c66d ("vfio: Add PCI device driver")
> > > Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> >
> > But this patch looks good to me:
> >
> > Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> >
> > with one nit...
> >
> > >
> > > +	/*
> > > +	 * Devices without DisINTx support require an exclusive interrupt,
> > > +	 * IRQ masking is performed at the IRQ chip.  The masked status is
> >
> > "exclusive interrupt, with IRQ masking performed at..."
>=20
> TBH, the difference is too subtle for me.  With my version above you
> could replace the comma with a period, I think it has the same meaning.
> However, "...exclusive interrupt, with IRQ masking performed at..."
> almost suggests that we need a specific type of exclusive interrupt
> with this property.  There's nothing unique about the exclusive
> interrupt, we could arbitrarily decide we want an exclusive interrupt
> for DisINTx masking if we wanted to frustrate a lot of users.
>=20
> Performing masking at the IRQ chip is actually what necessitates the
> exclusive interrupt here.  Thanks,
>=20

make sense. and I saw you replaced the commaon with a period in patch4.