Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 27 Sep 2021 12:01:19 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>,
        Leon Romanovsky <leonro@nvidia.com>
Cc:     "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "mgurtovoy@nvidia.com" <mgurtovoy@nvidia.com>,
        liulongfang <liulongfang@huawei.com>,
        "Zengtao (B)" <prime.zeng@hisilicon.com>,
        Jonathan Cameron <jonathan.cameron@huawei.com>,
        "Wangzhou (B)" <wangzhou1@hisilicon.com>
Subject: Re: [PATCH v3 6/6] hisi_acc_vfio_pci: Add support for VFIO live
 migration
Message-ID: <20210927150119.GB964074@nvidia.com>
References: <20210915095037.1149-1-shameerali.kolothum.thodi@huawei.com>
 <20210915095037.1149-7-shameerali.kolothum.thodi@huawei.com>
 <20210915130742.GJ4065468@nvidia.com>
 <fe5d6659e28244da82b7028b403e11ae@huawei.com>
 <20210916135833.GB327412@nvidia.com>
 <a440256250c14182b9eefc77d5d399b8@huawei.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <a440256250c14182b9eefc77d5d399b8@huawei.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?xACYfh3VLm+Yin/0UxqVjr6CAkf5nWukw+yEQPFdF7b2/RSY2itPJeDBkqsV?=
 =?us-ascii?Q?Cbm0gKp19Ai5dBjWL2mlJK0SM65+JO3SQWZnAKHOuMi5WXsfLjoBP9SQ2wyu?=
 =?us-ascii?Q?LECPsZ0HumGS7bEqVQeVpBoz8ZLVBud1oy/KHFwKf1Qp3uCEhhFgjymd1wQJ?=
 =?us-ascii?Q?MSx054aHyGhQo9VpePDf0zOZiFXwtRTv1nWHX1u28EusosV5BHWMOafxKR8S?=
 =?us-ascii?Q?c+KRfhX7833casUUrBHGDYYsJOzrwJvl12UwZOgJTkoGVLRAuuiJXkOmo1kJ?=
 =?us-ascii?Q?3F/Sek7tkuetoN4I6wYUNbkcjYBZEhH3x5k4Vf9eSIeuYewt+oja+xj5Col/?=
 =?us-ascii?Q?uYDvQPOWbpSlzIN6uV9K3q/oiAxCeg40MkposLJOJtfil1Up9hTws3QnoqdX?=
 =?us-ascii?Q?HCn2PG607QjCHOHUVZRySGUb2lUMsNs0g70MBA4mbk9Bx8sUU32pFVc7SO11?=
 =?us-ascii?Q?nvavv3V8YE+DgL2TMTC1gg8Bq2ehZam7g4+XQG9QhXwfwqcgqsKx1wFaWfmA?=
 =?us-ascii?Q?IDAWXZonOxvjFgpDvbBizUC0PdoBWDbf0JmZ0jSMkElyWYQmHdkGipkErrx/?=
 =?us-ascii?Q?nkxTT6zVw897YvKij9aQIm92w9AZmvFXdfNgAiXj4eEV7nC1tZ5E+cU4VtVH?=
 =?us-ascii?Q?qzHsaNm5B+A1MsuJvl6cOzHC6cPYEtPoFqKoF0xMlMPr3jLSX2fI7WWoSYQJ?=
 =?us-ascii?Q?iXiXZGq5yXnTu8+Ch1v799sUlbb9xIdYR2AV+SXG3Z4Zo48Fz3nRLXFjtCsF?=
 =?us-ascii?Q?UAC69i5/hiV4lXVFdUVs5pwURQpqc5H8vpbqyzhEeDcUmvMs63937dU9QBoo?=
 =?us-ascii?Q?WYqjDc0qksrJ5VpHj+AL2bn1GwtEZ85dDH6VcOJQ8EFbf42qudpZ6U8rZAvo?=
 =?us-ascii?Q?fdVsSIqxmZtrB/gHr0prQx3j+BlsiS015iHi0ltGD8p/dxqtsh2B+rEhXwp8?=
 =?us-ascii?Q?2uLf1ZgLjLdkmwYEQ+m5ID5QvuEoN72WBP4b7Smtk6sRL9SmGJTPM/kdSSYm?=
 =?us-ascii?Q?k4b8imUs4OcNJHs6rl+bYJhpeER/iW2NukMJJEtVtwJc01SRRAwBreyj6DF0?=
 =?us-ascii?Q?k9/2Uj9wSYRjZrBPTcsbNH5wRZ5+LxdRNEFiALbEoOA6ZREhOf96sTsbOFfH?=
 =?us-ascii?Q?+gvmUpyG+SNCdbkGJEPz3KHUOKVlV0Anis3121tnd3qRMrPeexpAASWjU/T5?=
 =?us-ascii?Q?RXjSHIECARt6krN2L87IhAh/otfCEb0F7Nq2/JK7pYav20duL/RDidinm3WZ?=
 =?us-ascii?Q?3XeCjQuiEtcsRLZNWMRfSwA3ME+mJZPoHkrAhDkEiI03Q6ik+/RFsKk02Uon?=
 =?us-ascii?Q?PEccffFhSxqHLE3mH+xzczsJ?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 598f8267-2adc-41e3-77af-08d981c7aa63
X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB5506.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 15:01:20.8608
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eZv0ReOAmNKwhdELNsCpPiBk0E3ZYi+al6hrvqvmIxSRX+Y3mGZ5/WiAfRA3UHOu
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5304
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On Mon, Sep 27, 2021 at 01:46:31PM +0000, Shameerali Kolothum Thodi wrote:

> > > > Nope, this is locked wrong and has no lifetime management.
> > >
> > > Ok. Holding the device_lock() sufficient here?
> > 
> > You can't hold a hisi_qm pointer with some kind of lifecycle
> > management of that pointer. device_lock/etc is necessary to call
> > pci_get_drvdata()
> 
> Since this migration driver only supports VF devices and the PF
> driver will not be removed until all the VF devices gets removed,
> is the locking necessary here?

Oh.. That is really busted up. pci_sriov_disable() is called under the
device_lock(pf) and obtains the device_lock(vf).

This means a VF driver can never use the device_lock(pf), otherwise it
can deadlock itself if PF removal triggers VF removal.

But you can't access these members without using the device_lock(), as
there really are no safety guarentees..

The mlx5 patches have this same sketchy problem.

We may need a new special function 'pci_get_sriov_pf_devdata()' that
confirms the vf/pf relationship and explicitly interlocks with the
pci_sriov_enable/disable instead of using device_lock()

Leon, what do you think?

Jason