Date:   Mon, 20 Mar 2023 13:12:27 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Alex Williamson <alex.williamson@redhat.com>
Cc:     Reinette Chatre <reinette.chatre@intel.com>, yishaih@nvidia.com,
        shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com,
        tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org,
        dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com,
        fenghua.yu@intel.com, tom.zanussi@linux.intel.com,
        linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 8/8] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
Message-ID: <ZBiF6/AkznbXHWbx@nvidia.com>
References: <cover.1678911529.git.reinette.chatre@intel.com>
 <549e6300c0ea011cdce9a2712d49de4efd3a06b7.1678911529.git.reinette.chatre@intel.com>
 <20230317150554.6bf92337.alex.williamson@redhat.com>
 <61296e93-6268-05cd-e876-680e07645a16@intel.com>
 <20230317170149.2be79d5a.alex.williamson@redhat.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230317170149.2be79d5a.alex.williamson@redhat.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4sOYf7Q4d5L1p0oFhfONRUPRMxDZeVWukSSl0XhtwVZg0BDJpBetveTge96l?=
 =?us-ascii?Q?Lm0vtlPSQ8FGU3YB38m9m8Y95wssbE8ghagPH1H0GG87cMYuI4A2BhQQ8vb/?=
 =?us-ascii?Q?p0VoQ+u+egkoYb5wY847IhePPvvV0DbY2JxoIot3DH1i8Tq4CeZGDsPTsImd?=
 =?us-ascii?Q?y1YD31fzUrhWa93bTyDD86TZ9MC22Lscg/GdfKwhM9vWetMseCfVWkrs/grV?=
 =?us-ascii?Q?boXS91EmaPRzlnF5bTCiZxtZRrdmJq3M2J4/jtzPlPwV64VkW6x0RW4dEJe/?=
 =?us-ascii?Q?sALex0pP9v42heutEUTcOH3F+/jyGyoqYLLwghxNPP93GYxCAwc35jlO7Bux?=
 =?us-ascii?Q?E7pCvYxUQ0MAmwqkXG4VR1CR2007XpQ/jolR9fwj+9dd0gTG0W9gyUDZmWP1?=
 =?us-ascii?Q?r2qygFH5opYZ2R5shFvzzYh3OYXIDb43eBV6Wrf224/BAJEuQv5HGivjwAWO?=
 =?us-ascii?Q?zPw5MKZ9nMk7PhMcMJB+aTNNMZHoG17vX2S195QkLS7kQGtzkCGqPQ+aBQkf?=
 =?us-ascii?Q?kZwNe0WwlS792Jsgch0/HobNfXRxIVOmpn0d+0uOv67UC9lu/0DfteHZYKdq?=
 =?us-ascii?Q?CXrdhL2QfeX1ULoSdgGRcuG0cyJoXwbhtgNZcVZTXeslJsUapAMLS+aF4T4i?=
 =?us-ascii?Q?SQUQ+Nbg9J5xCuYNh05rFpcDm3tB1TNS0tVXilTg5TS3aIn1jfgBI4+GPdpd?=
 =?us-ascii?Q?hGEXn3PAMoMytmQE/saqQxwO6Oys9eBOrWlJkFazLKUTVKYm4zRkfQV8vIyk?=
 =?us-ascii?Q?2O+VyZr/vVO9Pn5r9pTzXFqkGo2vQAG+d91Vy0V/tAiVyCJDaY50sJmdDTfV?=
 =?us-ascii?Q?t62A3LphYO3DEmBmR81HMOxTwKSjMEj8Xnpq7avBI7quBT6z5Vb/eCaEfY9y?=
 =?us-ascii?Q?5KqfxrQbmT7A7cpI0zjq2/0g/bmjJZFEDqJmeN4DrcNzo2/2z9PokOOsVFMz?=
 =?us-ascii?Q?8MtbsTDZSAGcYZ9PufRmdVd0gGizUxcPeMO0ggUkgN1mDKl0QCLDAJxyKy/B?=
 =?us-ascii?Q?I+bB2DjIDT++SSmTx+khm7Qe0wjsGnrITN7/SZvna3CRX1eU+5rEY92lxf77?=
 =?us-ascii?Q?PYRusP2wgeo91dauRZA6K0TnJxxFPm/NetQqby/Lezo5aQq4lu5Q2EQCjCjs?=
 =?us-ascii?Q?bc4CFt6ufnsMDNMx3DFD+zwY6BTloz9U75ubszHdYMJrWzRGXCW6mMKQkH9z?=
 =?us-ascii?Q?h8sQjB5wSSIVr0SUa+22LQemD7fAq0qBf7JC+CCN5lt9Ct2ZPcwRN3A7ByAN?=
 =?us-ascii?Q?bHn3SWaLxLf0bHKW8ZFNrupn0XefVxpfUaCja5+4QPr3FG4UVhRc1fEmDO/r?=
 =?us-ascii?Q?AHcaKTCf4EI+vZDvWeyL6+4OWzv2hLJq5vJFqzhVHLCxIWhBh5Rxdzg34+Ud?=
 =?us-ascii?Q?ePH4MUU74zCSRKgoD0OXuhTFvTgUji01ZRChmCESJJwgsQUIK4Q2F9k0CBKM?=
 =?us-ascii?Q?rXo7DP06VxLdM1WK3n5cNRuz+GtwGirJqOmyY/V8a/19m5C2DZoT1YtzneFS?=
 =?us-ascii?Q?sxjoPpguis3f9NUIsMv+DqMokOjCrIuV0Gsn0nGn6k51UEOqdCHtNjLtWY+J?=
 =?us-ascii?Q?yu3+36RtliReQkM3R3EVaSeKokGA+URuYp6ug2+8?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b30fafaf-4419-4a20-5022-08db295de6b2
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2023 16:12:28.3918
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 582u3DKq7ex2lYh54JR+DlkNwvxv7+cGkZ1RQuBOlkX8jFw8nUsWPAFPGnKrdLUD
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8989
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Mar 17, 2023 at 05:01:49PM -0600, Alex Williamson wrote:

> pci_setup_msix_device_domain() says it returns true if:
> 
>  *  True when:
>  *      - The device does not have a MSI parent irq domain associated,
>  *        which keeps the legacy architecture specific and the global
>  *        PCI/MSI domain models working
>  *      - The MSI-X domain exists already
>  *      - The MSI-X domain was successfully allocated
> 
> That first one seems concerning, dynamic allocation only works on irq
> domain configurations.  What does that exclude and do we care about any
> of them for vfio-pci?  

Several archs and other weird things override the MSI-X programming. Eg
by turning it into a hypervisor call or something. These were not
converted to dynamic mode.

So at the VFIO level you can get end up with MSI support but done
through legacy paths that don't support dynamic allocation. Eg on
POWER, xen, etc.

Jason