Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 16 May 2022 15:35:58 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Matthew Rosato <mjrosato@linux.ibm.com>
Cc:     alex.williamson@redhat.com, linux-s390@vger.kernel.org,
        cohuck@redhat.com, schnelle@linux.ibm.com, farman@linux.ibm.com,
        pmorel@linux.ibm.com, borntraeger@linux.ibm.com, hca@linux.ibm.com,
        gor@linux.ibm.com, gerald.schaefer@linux.ibm.com,
        agordeev@linux.ibm.com, svens@linux.ibm.com, frankja@linux.ibm.com,
        david@redhat.com, imbrenda@linux.ibm.com, vneethv@linux.ibm.com,
        oberpar@linux.ibm.com, freude@linux.ibm.com, thuth@redhat.com,
        pasic@linux.ibm.com, pbonzini@redhat.com, corbet@lwn.net,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-doc@vger.kernel.org
Subject: Re: [PATCH v7 17/22] vfio-pci/zdev: add open/close device hooks
Message-ID: <20220516183558.GN1343366@nvidia.com>
References: <20220513191509.272897-1-mjrosato@linux.ibm.com>
 <20220513191509.272897-18-mjrosato@linux.ibm.com>
 <20220516172734.GE1343366@nvidia.com>
 <7a31ec36-ceaf-dcef-8bd0-2b4732050aed@linux.ibm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <7a31ec36-ceaf-dcef-8bd0-2b4732050aed@linux.ibm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?pKl0JzoeP7NQ9Ks5YKtslzPhbPnmqGQiP/Zt0ptrT0xGDV8tjkQLlSMHH8hA?=
 =?us-ascii?Q?bV/mfNT4vM5pJIN2L/g5WD5Z2NIl2aQ1Av13y17GhWxHcmj+k1q3eMNwcWJQ?=
 =?us-ascii?Q?QuKUP2PmfEoLgBQGcl+jioAgBOIvTBI7V54JIgp1aKWOiL5GK+QlaruRpGLf?=
 =?us-ascii?Q?O50aPGR8ai5rXRj7wp789N0zsBRi2L73SeZZry6byVr++sOTDPnfSgryMtS/?=
 =?us-ascii?Q?FHTGyHB1eth8xL8BB5YIQN7QI/HP9DMk+rH6Ney0E/usifCeZiWIaYinxMg8?=
 =?us-ascii?Q?P1sJWf65ZrasfcSC9pc2Sl0lQYaWsffrpLRQQqdOIUTFLqsZ5AepEMJXfgZ0?=
 =?us-ascii?Q?u7uUB9x6e8XNCAR/1glHzFgt3D/aZt7GVEypvJJ1hLM04v1bG7M1zpJ6GgZm?=
 =?us-ascii?Q?keHcWG2sX2s0KfDp5ArvbrccAJFDS9wWIYXGWYux9TX94RcpADB3bP/1+x2n?=
 =?us-ascii?Q?q7dDn179j7PybR3yawzaygk/VmaBoP1Q65IwBK1F8qo6pTGLhVfwqHX3Ky8s?=
 =?us-ascii?Q?GmpTWmhB9XZDBR6lNzReqaA9WQPFGg1gF8ts6GKQDnGyk3OTCW1rh4UAKmmR?=
 =?us-ascii?Q?RzqbzegGCAaI65HK9TXCJTMKth9M4Iu8k8K5cb5T43qs8qCiVNmK9JZldE5l?=
 =?us-ascii?Q?lU3Y0ZuTu0pWKCsCriMYJMnN9oKkG9Hz3IAgBmXwXd+0WwDW11+gjDBPORBN?=
 =?us-ascii?Q?joIhDZ4uw+YEtR7v7nXSNitD0qtj0plbxnqMnh7tZg6zFHwoAIJL6iRI7JdB?=
 =?us-ascii?Q?RD6j4NkaWkH8eo5u6LSeM5RudvXPg4pjzFZ1C6m+GsOXZo+MwT3Yuju9qbFF?=
 =?us-ascii?Q?CvXl/S2hDRUHgSIixa6i65Vddd7IFF0Jk9GAZIdAlT4ARsSvpDTuumCoPj2M?=
 =?us-ascii?Q?2Z5+9ulLh+UewP9+DRMl7638Y0/WSf2HkJYKjfhOKr/HTm1aldZB2lEhSTYt?=
 =?us-ascii?Q?7vPgV70o6209VnKaF7PmcqnopAxvy+oIqXl1LfAS6tr4ItaejQga4h3IXuFk?=
 =?us-ascii?Q?loDAowI08BungmwtaosyLQWGhN63HzOGhqPB720nH0+jt+3SlEF/RL+OKsbG?=
 =?us-ascii?Q?m34uyFD13/ilCfA/WkXSXo/Vl6Dytq+9j1m0UEw+DsJ5d1W7k8X8eBe6xd3J?=
 =?us-ascii?Q?72PMk3GgdvrzB+wIZWt4VBPEKRqXYIccA7uyahrAecvRkoKI59UQJAhZI8ut?=
 =?us-ascii?Q?qAnV4GWSDlQ+ZiZowj0R//ZlwVrEFaEuomB6M1RwVM6OU8lh2Zho+qo/otAF?=
 =?us-ascii?Q?cPJPaK4SgpT8jYVQzhha85lY3rkjaBomOwJr7PVJKd3a0QCbf/ayb5In1zOV?=
 =?us-ascii?Q?GSkYMHho6FJua1blUrxFR0WpNDAdVORwigaTlnw4Ks8gIuknTR9Cs8BzOw91?=
 =?us-ascii?Q?mnrwtgLq0Mz7IT7tYRjBcu3XV5Ek2nCmDI5nrgkFEVoemfnw6ZFZF+G1LvbC?=
 =?us-ascii?Q?G1ZSOzk+K+G0KvbKwKOzmn0Vd62vRMi4tbORneJOvujEuU4NHfhORfXVfsSW?=
 =?us-ascii?Q?7lQZ/c1oZTkGid+zWS7f7lHaCJpW/aPsiM1FLMJhhqLGnLxGJ1lPa7klotdP?=
 =?us-ascii?Q?oAb5rmFxZvnKTwo3OimhKAaWvbxauyk3hnSLrTINZFffJ6+gnewp2ixpgaju?=
 =?us-ascii?Q?gy0rer7B9BnFC81A4JN4uDeytjdPFHRcFpMygfV8/+ZaITl2CKbeN8/rQ9Ho?=
 =?us-ascii?Q?D908FpKwc1BUdj/5VEuchHlVOCRQc0/jJy0Z73+GbDcUnwBGptsMAiguCdJy?=
 =?us-ascii?Q?iBuarBciFQ=3D=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 235b7883-4e06-44ed-f263-08da376aec14
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2022 18:35:59.5546
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 57XHv/Kb2WaM09eoyGik7ssfRuhi6c8eQHPFtweM8LK4I1QwIL8xuA+hAhuzYtz1
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3275
X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, May 16, 2022 at 02:30:46PM -0400, Matthew Rosato wrote:

> Conceptually I think this would work for QEMU anyway (it always sets the kvm
> before we open the device).  I tried to test the idea quickly but couldn't
> get the following to apply on vfio-next or your vfio_group_locking -- but I
> understand what you're trying to do so I'll re-work and try it out.

I created it on 8c9350e9bf43de1ebab3cc8a80703671e6495ab4 which is the
vfio_group_locking.. I can send you a github if it helps
https://github.com/jgunthorpe/linux/commits/vfio_group_lockin

> @Alex can you think of any usecase/reason why we would need to be able to
> set the KVM sometime after the device was opened?  Doing something like
> below would break that, as this introduces the assumption that the group is
> associated with the KVM before the device is opened (and if it's not, the
> device open fails).

Keep in mind that GVT already hard requires this ordering to even
allow open_device to work - so it already sets a floor for what
userspace can do..

Jason