Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
Feedback-ID: i1568416f:Fastmail
Date:   Wed, 27 Sep 2023 12:33:05 +0200
From:   Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= 
        <marmarek@invisiblethingslab.com>
To:     linux-kernel@vger.kernel.org
Cc:     Juergen Gross <jgross@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
        Jan Beulich <jbeulich@suse.com>,
        "moderated list:XEN HYPERVISOR INTERFACE" 
        <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3] xen-pciback: Consider INTx disabled when MSI/MSI-X is
 enabled
Message-ID: <ZRQE4jt/a1RJ0wUw@mail-itl>
References: <20221118154931.1928298-1-marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
        protocol="application/pgp-signature"; boundary="EOjJ45zJghDRhlN7"
Content-Disposition: inline
In-Reply-To: <20221118154931.1928298-1-marmarek@invisiblethingslab.com>
Precedence: bulk


--EOjJ45zJghDRhlN7
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 27 Sep 2023 12:33:05 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	"moderated list:XEN HYPERVISOR INTERFACE" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3] xen-pciback: Consider INTx disabled when MSI/MSI-X is
 enabled

On Fri, Nov 18, 2022 at 04:49:23PM +0100, Marek Marczykowski-G=C3=B3recki w=
rote:
> Linux enables MSI-X before disabling INTx, but keeps MSI-X masked until
> the table is filled. Then it disables INTx just before clearing MASKALL
> bit. Currently this approach is rejected by xen-pciback.
> According to the PCIe spec, device cannot use INTx when MSI/MSI-X is
> enabled (in other words: enabling MSI/MSI-X implicitly disables INTx).
>=20
> Change the logic to consider INTx disabled if MSI/MSI-X is enabled. This
> applies to three places:
>  - checking currently enabled interrupts type,
>  - transition to MSI/MSI-X - where INTx would be implicitly disabled,
>  - clearing INTx disable bit - which can be allowed even if MSI/MSI-X is
>    enabled, as device should consider INTx disabled anyway in that case
>=20
> Fixes: 5e29500eba2a ("xen-pciback: Allow setting PCI_MSIX_FLAGS_MASKALL t=
oo")
> Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingsl=
ab.com>

Ping?

The issue pointed out by Jason was fixed in Xen:
https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Dcommit;h=3D913751d7af6e78d6=
5c1e2adf4887193c827f0c5e

> ---
> Changes in v3:
>  - allow clearing INTx regardless of MSI/MSI-X state, to be consistent
>    with enabling MSI/MSI-X
> Changes in v2:
>  - restructure the patch to consider not only MASKALL bit, but enabling
>    MSI/MSI-X generally, without explicitly disabling INTx first
> ---
>  drivers/xen/xen-pciback/conf_space.c          | 19 +++++++++++------
>  .../xen/xen-pciback/conf_space_capability.c   |  3 ++-
>  drivers/xen/xen-pciback/conf_space_header.c   | 21 +++----------------
>  3 files changed, 18 insertions(+), 25 deletions(-)
>=20
> diff --git a/drivers/xen/xen-pciback/conf_space.c b/drivers/xen/xen-pciba=
ck/conf_space.c
> index 059de92aea7d..d47eee6c5143 100644
> --- a/drivers/xen/xen-pciback/conf_space.c
> +++ b/drivers/xen/xen-pciback/conf_space.c
> @@ -288,12 +288,6 @@ int xen_pcibk_get_interrupt_type(struct pci_dev *dev)
>  	u16 val;
>  	int ret =3D 0;
> =20
> -	err =3D pci_read_config_word(dev, PCI_COMMAND, &val);
> -	if (err)
> -		return err;
> -	if (!(val & PCI_COMMAND_INTX_DISABLE))
> -		ret |=3D INTERRUPT_TYPE_INTX;
> -
>  	/*
>  	 * Do not trust dev->msi(x)_enabled here, as enabling could be done
>  	 * bypassing the pci_*msi* functions, by the qemu.
> @@ -316,6 +310,19 @@ int xen_pcibk_get_interrupt_type(struct pci_dev *dev)
>  		if (val & PCI_MSIX_FLAGS_ENABLE)
>  			ret |=3D INTERRUPT_TYPE_MSIX;
>  	}
> +
> +	/*
> +	 * PCIe spec says device cannot use INTx if MSI/MSI-X is enabled,
> +	 * so check for INTx only when both are disabled.
> +	 */
> +	if (!ret) {
> +		err =3D pci_read_config_word(dev, PCI_COMMAND, &val);
> +		if (err)
> +			return err;
> +		if (!(val & PCI_COMMAND_INTX_DISABLE))
> +			ret |=3D INTERRUPT_TYPE_INTX;
> +	}
> +
>  	return ret ?: INTERRUPT_TYPE_NONE;
>  }
> =20
> diff --git a/drivers/xen/xen-pciback/conf_space_capability.c b/drivers/xe=
n/xen-pciback/conf_space_capability.c
> index 097316a74126..eb4c1af44f5c 100644
> --- a/drivers/xen/xen-pciback/conf_space_capability.c
> +++ b/drivers/xen/xen-pciback/conf_space_capability.c
> @@ -236,10 +236,11 @@ static int msi_msix_flags_write(struct pci_dev *dev=
, int offset, u16 new_value,
>  		return PCIBIOS_SET_FAILED;
> =20
>  	if (new_value & field_config->enable_bit) {
> -		/* don't allow enabling together with other interrupt types */
> +		/* don't allow enabling together with other interrupt type */
>  		int int_type =3D xen_pcibk_get_interrupt_type(dev);
> =20
>  		if (int_type =3D=3D INTERRUPT_TYPE_NONE ||
> +		    int_type =3D=3D INTERRUPT_TYPE_INTX ||
>  		    int_type =3D=3D field_config->int_type)
>  			goto write;
>  		return PCIBIOS_SET_FAILED;
> diff --git a/drivers/xen/xen-pciback/conf_space_header.c b/drivers/xen/xe=
n-pciback/conf_space_header.c
> index 981435103af1..fc0332645966 100644
> --- a/drivers/xen/xen-pciback/conf_space_header.c
> +++ b/drivers/xen/xen-pciback/conf_space_header.c
> @@ -104,24 +104,9 @@ static int command_write(struct pci_dev *dev, int of=
fset, u16 value, void *data)
>  		pci_clear_mwi(dev);
>  	}
> =20
> -	if (dev_data && dev_data->allow_interrupt_control) {
> -		if ((cmd->val ^ value) & PCI_COMMAND_INTX_DISABLE) {
> -			if (value & PCI_COMMAND_INTX_DISABLE) {
> -				pci_intx(dev, 0);
> -			} else {
> -				/* Do not allow enabling INTx together with MSI or MSI-X. */
> -				switch (xen_pcibk_get_interrupt_type(dev)) {
> -				case INTERRUPT_TYPE_NONE:
> -					pci_intx(dev, 1);
> -					break;
> -				case INTERRUPT_TYPE_INTX:
> -					break;
> -				default:
> -					return PCIBIOS_SET_FAILED;
> -				}
> -			}
> -		}
> -	}
> +	if (dev_data && dev_data->allow_interrupt_control &&
> +	    ((cmd->val ^ value) & PCI_COMMAND_INTX_DISABLE))
> +		pci_intx(dev, !(value & PCI_COMMAND_INTX_DISABLE));
> =20
>  	cmd->val =3D value;
> =20
> --=20
> 2.37.3
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--EOjJ45zJghDRhlN7
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmUUBOEACgkQ24/THMrX
1yzBAQf/SHf0RTHfqbq+msnyxOU14bDfbEM8oyJHi5kRnq7CbBGWTQxj2SzluNcZ
rWnV06yAeD4uolUPHsgK86pKR3JDOQ89YhVj9aCMOe/n5cwVeV7V6S7PGrnOrzTt
i6ctMcafEA7gJtSPXnPI2q6t5rqQkS1YU15U5JxKB46RA9ALJx1/gnYdkzuiLgof
L4MX81mVKmniiTot3G5krhABVsVjAJXWP+Wuh8gVApaSqrPa76hGGpxmdmTsQ3KU
HuLtIO03tkjbogleWkMjH6rR8GP317Eq8ivwjna9qovV2Usn62x/HVO+ctl3+2/7
G9/IcYUCwe0HFI0AZucdc9Om/eqpCg==
=WCCk
-----END PGP SIGNATURE-----

--EOjJ45zJghDRhlN7--