Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Wed, 27 Apr 2022 12:01:38 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Matthew Rosato <mjrosato@linux.ibm.com>
Cc:     linux-s390@vger.kernel.org, alex.williamson@redhat.com,
        cohuck@redhat.com, schnelle@linux.ibm.com, farman@linux.ibm.com,
        pmorel@linux.ibm.com, borntraeger@linux.ibm.com, hca@linux.ibm.com,
        gor@linux.ibm.com, gerald.schaefer@linux.ibm.com,
        agordeev@linux.ibm.com, svens@linux.ibm.com, frankja@linux.ibm.com,
        david@redhat.com, imbrenda@linux.ibm.com, vneethv@linux.ibm.com,
        oberpar@linux.ibm.com, freude@linux.ibm.com, thuth@redhat.com,
        pasic@linux.ibm.com, pbonzini@redhat.com, corbet@lwn.net,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-doc@vger.kernel.org
Subject: Re: [PATCH v6 16/21] vfio-pci/zdev: add open/close device hooks
Message-ID: <20220427150138.GA2512703@nvidia.com>
References: <20220426200842.98655-1-mjrosato@linux.ibm.com>
 <20220426200842.98655-17-mjrosato@linux.ibm.com>
 <20220427140410.GX2125828@nvidia.com>
 <f6c78792-9cf7-0cde-f760-76166f9b7eb7@linux.ibm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <f6c78792-9cf7-0cde-f760-76166f9b7eb7@linux.ibm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RSZNyvliBmRf9F95iZCtH/YzZMcJbuWlahPZddLEk5qyE1BAReYBmt732Dbg?=
 =?us-ascii?Q?ekOWaxeU8dYTq+1hNcGG8KzEX4ppzlnoy927LCnakt6e60+jkiyWoHRc9L3C?=
 =?us-ascii?Q?XqWz3Zv8LPbmAzQfV6sxOdAIy0pYo34dtIA8idl1uuT/TtYE2bealSD6vmkW?=
 =?us-ascii?Q?8iJbXZsbZZuYfgV3/KMRi4DehfexA/mBK2l6pm1Jr9JueJyd6ZFqke/vMAIB?=
 =?us-ascii?Q?eRDUdU0FRXuj2LZx6qPJj291LHwlWLUtFZcQObdorDN1tNxcU559sEzKwU9w?=
 =?us-ascii?Q?b2/qxDvP2ePh7g/+Z3xG5lj8Z3FvBty0x4phTzuYWwiecgGOUpfNdkG1rRcO?=
 =?us-ascii?Q?fnr6GLz1SQYp5glHu6/vid3b88a/MzvQVY8qVCwoGvDyqjPMIM11BuDe9MpZ?=
 =?us-ascii?Q?Xrpbd+s+aeAFcUBUzwMY64RwUz6q4BXmEwzo2iNUJXK/hrvhGniwLQSQGTZ3?=
 =?us-ascii?Q?Kd3nlYjP3rGmEA8fhcLsv+pk1FgWPxeIF1pRe3YPmkQd6KWswGSPiaq5kRhu?=
 =?us-ascii?Q?wFxg50Ss7FqNIXvbCuP7rmTYgBXnsFjQkR1K1vRH/jC8myiKGB6hl/ljEm8y?=
 =?us-ascii?Q?WGVXNfz3eZYpHBjdWArc73L0uje52oyHuEarN3QFIX+Y8nKGvaNiMUleKsLZ?=
 =?us-ascii?Q?nV47mcSYAEtC06IsoZUcTdyXQ32qkUrCbW0mhHnLyypczYwP58lcYiuki8EK?=
 =?us-ascii?Q?ljFQGZ6QvQ1PyPIFqa1tuPbIkN3BH3QpnraEiXs5+n9iZCAwYlZ2a87nYV7Q?=
 =?us-ascii?Q?FufOMPfe7QUkAaQZPdbSBeRj56FF82LXbI3NyeL7N533EAMh4j99UWZQjGw2?=
 =?us-ascii?Q?GXnPLekiC6DSIBkbgVUnm6h5oFPKxPPww4nnTiudAubzi9FxCgKxjLUGDOlv?=
 =?us-ascii?Q?9ND6HF7S1EdRuKjq9hjH7DiXvQRWvPBdBTUypp1jgXXnFDN091vGNL63v0Ds?=
 =?us-ascii?Q?hEcQ1GaWw06kY2oEqOBu4J4Y89DRB0NyYyWpyLJK1QVGfZoH0OYv+9cXHJOg?=
 =?us-ascii?Q?EU9aM7H2CFwYkrodxQ3K2NCsOPM0EeeDPbmFa7iDZ1MqRlvHJLFVzDO+3evg?=
 =?us-ascii?Q?Hz70+jOPzAz1FMzNrD1BvWConK16O8ou+zCG7BQKHTm4vNVaE6ahE8LCrkVd?=
 =?us-ascii?Q?1hnUEOt7oYo52jF9yYmn3HVeqzXKpNp4OypvHnAfpxunsssI0K6QbC0FvByJ?=
 =?us-ascii?Q?MLJ6UI9/tItHmwBeDL/KvtXJRteApfNK/bZEoSsahtF4nMiXLSrU88Ra9p26?=
 =?us-ascii?Q?DizAKFUC0hn9r/Lo9yiE5h7SC7MpFzKnpzHEzOnyV6Whndb8o0TlR7lIYLgN?=
 =?us-ascii?Q?QZkm3U9xmSF/5cTTT2js1P9eVSahbeZWdEiSOPoKblx5rNRzByhz5TK19zDM?=
 =?us-ascii?Q?5TALQ/Xc2OPvsPvAw0AdBSU2JygEOTWWjJsOQIN2XSGjY/onohtFpnhphAOc?=
 =?us-ascii?Q?woa7/Bk2L63ogwgrY0Jlj3ZGgVP2oCPZ3gf5rqpdtFl4raGsm6cAyKtcqnIZ?=
 =?us-ascii?Q?b54rifAt/KKrdRr+5+roI2fRqSLxC1gQotikWckDEK6PMuZGoWgnAbu+1sti?=
 =?us-ascii?Q?PE4j5R6+U1t9vWUeJg/0V4NfBJGM/4u3FjT8+FbvY/noeGaAnvqXIKs65xNl?=
 =?us-ascii?Q?Hls9aFuu+8OsD0tgvwz8XxDb0nzKa5htC07XtWnkZGVo9G1zImQ1kgeilSSc?=
 =?us-ascii?Q?oUAMCaQd4xeHyZC4sh0w/5YAlCTK6kNCy5VbdWGXRkgwlPkB0tGhgHIYbyoc?=
 =?us-ascii?Q?Tx4fhXgvDg=3D=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 73efaabd-bbae-47bf-d8d5-08da285ed507
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2022 15:01:39.4163
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HLPONy7iws3SHaSZl2AEr+4hyJke4qE50Vg5eRdPH6vJ0Q7ZLYupWr44Prb7Nr12
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6135
X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Apr 27, 2022 at 10:42:07AM -0400, Matthew Rosato wrote:
> On 4/27/22 10:04 AM, Jason Gunthorpe wrote:
> > On Tue, Apr 26, 2022 at 04:08:37PM -0400, Matthew Rosato wrote:
> > 
> > > +static int vfio_pci_zdev_group_notifier(struct notifier_block *nb,
> > > +					unsigned long action, void *data)
> > > +{
> > > +	struct zpci_dev *zdev = container_of(nb, struct zpci_dev, nb);
> > > +	int (*fn)(struct zpci_dev *zdev, struct kvm *kvm);
> > > +	int rc = NOTIFY_OK;
> > > +
> > > +	if (action == VFIO_GROUP_NOTIFY_SET_KVM) {
> > > +		if (!zdev)
> > > +			return NOTIFY_DONE;
> > > +
> > > +		fn = symbol_get(kvm_s390_pci_register_kvm);
> > > +		if (!fn)
> > > +			return NOTIFY_DONE;
> > > +
> > > +		if (fn(zdev, (struct kvm *)data))
> > > +			rc = NOTIFY_BAD;
> > > +
> > > +		symbol_put(kvm_s390_pci_register_kvm);
> > 
> > Is it possible this function can be in statically linked arch code?
> > 
> > Or, actually, is zPCI useful anyhow without kvm ie can you just have a
> > direct dependency here?
> > 
> 
> zPCI devices (zpci_dev) exist regardless of whether kvm is configured or
> not, and you can e.g. bind the associated PCI device to vfio-pci when KVM is
> not configured (or module not loaded) and get the existing vfio-pci-zdev
> extensions for that device (extra VFIO_DEVICE_INFO response data).  Making a
> direct dependency on KVM would remove that; this was discussed in a prior
> version because this extra info is not used today outside of a KVM usecase
> are not specific to kvm that need vfio-pci-zdev).

I'm a bit confused, what is the drawback of just having a direct
symbol dependency here? It means vfio loads a little extra kernel
module code, but is that really a big worry given almost all vfio
users on s390 will be using it with kvm?

Or is there some technical blocker? (circular dep or something?)

Jason