Received: by 2002:a05:6a11:4021:0:0:0:0 with SMTP id ky33csp4196486pxb; Mon, 27 Sep 2021 11:22:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJznkvq37w9V7DPAbEQDYLKRqXB95FqL32YDgCQFIEARjX7bWCs4R1FZpZnHk8QuwF8TMJec X-Received: by 2002:a50:9552:: with SMTP id v18mr1701816eda.395.1632766975106; Mon, 27 Sep 2021 11:22:55 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1632766975; cv=pass; d=google.com; s=arc-20160816; b=IIvhvvSXoVJ9TpJxy1KTgC1qiY/PmIfYr2NSNe8UTiyGx4aSltZ2/FCN+EvwX7OBRn +suy9WhXNrmg0nvtZlB+nQfIiVSRKj/VEcL08m0+Hji2/lNC498P5bGICFmXY3VCiu9g yhtN82PKKpkq0tKNwRuegHaLBxLnuKQyz+t2qDvmdIMmLK+RbXWyweBnjMtN1JcwX3v/ tKM+EaaDFMWASBfvZboplbJNaetEcKxkxwIRJ2d3aZzfVrk8DbXfBlWJE0OorccojDcz 3M0/rBj7KJkB7l1FWyq4dunrzPGTZkbGOUQElZKLyjTgggDxc0l5sLWi0RSUrY3MOFRS hytA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:in-reply-to:content-disposition :references:message-id:subject:cc:to:from:date:dkim-signature; bh=hsLxpVSTdaFaYkAq34E2ZyJbTFSAmUvzR4Z8ie+Ewz4=; b=Pfh/nYSovt8lWv5vmWmnfOMfc/T1PGubrHsz9rmXzSTQGirOvorg8IaXe6fSXV8lVX 9c1ODI+i1Ci9VGjlEMnUBNHW+bxN9HFhjjydafzqRG1AeSGB63RmaiEHxMAwjry77NRY 52d+3WKvOKXRpPZwpIm1COrtMW9ZLpBTeXEkKAhoz5dA7h7NbfxkGvDgAvaxmel4W645 xNeLEBsmPcf1EOjk32TogfmkxfLOeou0Ka+NVwENOn5EQamPoL37PZ+ow5FEyRfk3cAO ZJJaMS9CaMjo7l5RRcEZBXMV35zVxNOOeTOVcPuY0D1kOC9t4MKf0fVt8fIfiYs+l4Zy TzZA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=lPDQjTJZ; arc=pass (i=1 spf=pass spfdomain=nvidia.com dkim=pass dkdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=nvidia.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id p27si3332136edi.289.2021.09.27.11.22.30; Mon, 27 Sep 2021 11:22:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=lPDQjTJZ; arc=pass (i=1 spf=pass spfdomain=nvidia.com dkim=pass dkdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235997AbhI0SYG (ORCPT + 99 others); Mon, 27 Sep 2021 14:24:06 -0400 Received: from mail-bn1nam07on2063.outbound.protection.outlook.com ([40.107.212.63]:22247 "EHLO NAM02-BN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235950AbhI0SYF (ORCPT ); Mon, 27 Sep 2021 14:24:05 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MaojQz/kodTcTKaErn0XmEIHCdVkxMhmdKhfDWpbbduXkyrYoUSdcDb5bZlebpsoBVKUQlVf2gTeO2yChz814kXW9V+gECU0ZTpSqrlXcuG3ClEjF0ofxPnrMskN8it046uptiDEq0EXZ+OVtZLbOv2z46RcSWbtZ9bov90GI6GcM9qkhb3uCP+eJ/6rkwrQYxH6WovG2mvBbdL3aVpE+FlfBmHjezGJlWR0Rg4VV6jE0bIsGV97rL3FijqupDOViGGKLb4Yiq/4mKxBve/gnbB3Q7DLMafC/YrKCXjgHxvx+ulwK4Lsn2VtgGpQEyD7x16rIQAg/g0E07yBWmTmvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hsLxpVSTdaFaYkAq34E2ZyJbTFSAmUvzR4Z8ie+Ewz4=; b=lIoUgK62pvrnWxrEMeZBbjbTmjRXo+eFCbdRudDhxMNtWgTdMHvBJhtppWwnuoUIYrY5eu3lUhGbrB9sUq48hRfsu2aEEimMlPYm0J9zy2AKHab7NtdmcL+md/LYuEP69IpZ5UOfckg3+dA8o3g+OUa3Ao/ShB7S2/DtAQaYe4gMKCaPUKW6Wk5M4tQXNmtzY7NTS6fiWEOfjnrvBM53aOo4K6K7B4qVp38yIM14sc6SUZXgfFdFHe05FrN3bWHayXMUIoqS3OBsV/pji5yFxG8FVO4S8jGZuv+XQMqdT0Y+ftHedptbvaAKpMqY77xj/pktrOvjL7v07sYuZB81Og== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hsLxpVSTdaFaYkAq34E2ZyJbTFSAmUvzR4Z8ie+Ewz4=; b=lPDQjTJZDTEUmV36rusO4fG549e1zgZiRVZmy/W6u/jK2g6GFGVGiJjbNpPYnkoDJbfbN+e3Vin44EN353FgzzfRZwmlo4YJqRO8h3f72Z4Lbnd655KgE5qSZUjkbICO9kBeNobXFe0z9D4SGqOjCkS7Ge1/KswnXHiVXx3FeKKWt4mpuN+fUGTXULzx3zh1QSFIV8bc1lNV/LG1vj9X4IXlx7zqsYKnL7vREzRyGxjXqGbCqbmDGPjSKlzfsvcKmm6KmOcPAy4Al2Q5jq9CymW6x66Ydffc/OITdSYtja6LOgaUn0RJbj5oiHAUxQI9VNk0TKZpEWmKY+fKrblpTw== Authentication-Results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; Received: from BL0PR12MB5506.namprd12.prod.outlook.com (2603:10b6:208:1cb::22) by BL1PR12MB5288.namprd12.prod.outlook.com (2603:10b6:208:314::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Mon, 27 Sep 2021 18:22:26 +0000 Received: from BL0PR12MB5506.namprd12.prod.outlook.com ([fe80::e8af:232:915e:2f95]) by BL0PR12MB5506.namprd12.prod.outlook.com ([fe80::e8af:232:915e:2f95%8]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021 18:22:26 +0000 Date: Mon, 27 Sep 2021 15:22:24 -0300 From: Jason Gunthorpe To: Leon Romanovsky Cc: Shameerali Kolothum Thodi , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-crypto@vger.kernel.org" , "alex.williamson@redhat.com" , "mgurtovoy@nvidia.com" , liulongfang , "Zengtao (B)" , Jonathan Cameron , "Wangzhou (B)" Subject: Re: [PATCH v3 6/6] hisi_acc_vfio_pci: Add support for VFIO live migration Message-ID: <20210927182224.GA1575668@nvidia.com> References: <20210915095037.1149-1-shameerali.kolothum.thodi@huawei.com> <20210915095037.1149-7-shameerali.kolothum.thodi@huawei.com> <20210915130742.GJ4065468@nvidia.com> <20210916135833.GB327412@nvidia.com> <20210927150119.GB964074@nvidia.com> <20210927160627.GC964074@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BLAPR03CA0019.namprd03.prod.outlook.com (2603:10b6:208:32b::24) To BL0PR12MB5506.namprd12.prod.outlook.com (2603:10b6:208:1cb::22) MIME-Version: 1.0 Received: from mlx.ziepe.ca (142.162.113.129) by BLAPR03CA0019.namprd03.prod.outlook.com (2603:10b6:208:32b::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend Transport; Mon, 27 Sep 2021 18:22:26 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1mUvGi-006bwc-99; Mon, 27 Sep 2021 15:22:24 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2b44ad60-9cf9-4f85-e839-08d981e3c204 X-MS-TrafficTypeDiagnostic: BL1PR12MB5288: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2NKa7emTU1RqrHzCkX/JXdis6X7YwtQUWJPVEEbAsgZCSAQH4LPKRTWACakGfcZVl2hlGU4hTgbDnUuM92EeXkHJqxnH3MAZVCiOVfolk3jrOhnxVl1F0w8F6YvGmHwgVNR8KuxrtfKzjqfb1zttc/2pzOPsOhjl/aSoXV7TWbFN8AHnedvyllz7Rhwleg9kdpgPpZR5g/RmlrQiA0mJ/Z+6rQX4Qmw0xzyO+n8R5WEa1m08wEkgoR18k0oZb2dgyJ63Y/3jtcGnxg8ESMxusmquT2KZhNGzpjijkbvgk8nbz4EcwRl8BrEXGAnWUL33oDV/3mQTUmC3w1fsyMXVu5m9aqcmaCfEsyVjG1MUulkxprOQZRqjR7e+OpIXtgeOy32dXSm6h6F7dyTXo3j368XnAeeYBDQw4jPRx6KiXpewYo6QRNG+uUnLGiCbeHLgPPuST2QinfZTMmQBxi369mV0ZG9JmkT9WWQ80Alpq/pN1TQBeWm7etSrLZBZjOVWswq9nJkl3ZwXF76iithPfzIyxofpqmduDdbbyIyyf2+EJ0X6qHoEuQZY2H5e+EC5D2J9R+KMHy1xWRtMaC719uY2nq7MF3Cf2SSQz+MY/5OeT7Ip5uI+URAuwBYoJhNHJw5pIleldw+n9+Vyt/TRMQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR12MB5506.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(38100700002)(66946007)(66556008)(426003)(66476007)(508600001)(2616005)(26005)(33656002)(4326008)(186003)(36756003)(83380400001)(6636002)(6862004)(86362001)(54906003)(4744005)(316002)(37006003)(9746002)(9786002)(8676002)(5660300002)(8936002)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Gvkt+RSR0kpchVFscfv4I1inblvDbuspfZc/t8AG+45tlbiDhA9c+bpowwqa?= =?us-ascii?Q?5ZT2lQfVJa7pciDY4e6m6O3ZoHXiBfyH6XhIWdXm4q+sw/4Bk1ZZnq3UQUGk?= =?us-ascii?Q?PFUgiz8AByK3IOxY0y92tzh9aTaJPSR5rWidPikHpQ8HULlW9eUskaaf0bf5?= =?us-ascii?Q?o3BVcnJNWctAD2LZqhleVb4veitv+5MlekJj+psCQak7t7ovdFgoBzU4tTmE?= =?us-ascii?Q?IbDcbEJRvQ0ZGyyMFk3H1ve7EJ5m4/BXowkoWo9eqND6cmFmdfbPee6CI+X5?= =?us-ascii?Q?GjJSMmc6kylnDtgkmbr2YxGommFtwWGNcAFISH5QAcLMzY1ql07A0y8zcNWo?= =?us-ascii?Q?7kEK13t4HuSGt0h+oH0RZehTgJdemQxaHYGRXv8s9jNGWMyLUCLN7lphS7qY?= =?us-ascii?Q?6+gBfQDXlcnaWFQrsxAHOeMB5V8zZk511zF16UppLNTivYxGo9b8LRVISFtE?= =?us-ascii?Q?koaeXnMm6RP3St0H4Mn938oFG4r7MhOueUElkypNLzBgObaImZzGXNfLjDZ1?= =?us-ascii?Q?cfnzqxQk3oEgaRorn7FFdsMDRFpOK4CfQ3VQRCvtfd5uccoercxooyLYnobd?= =?us-ascii?Q?UpbpLPxSeQ4osHKY/nFjJ511hne272cg0kMuOdV2kcBMT1BmEH9lnU60Ol4k?= =?us-ascii?Q?xQteR1OkU4/9SvryEQKE+bYQ5JpZfjLTI9xruTZtU1Ql+dWi+UPkolBAdj/K?= =?us-ascii?Q?ZC9rjaaV1t9fefyBSLCW3H2fRcwbNrwpz3roiuZeTR7jm8yZfzvup3d7JiOC?= =?us-ascii?Q?TeN7SYdvrAp6V5zipyyqvfWh60b+oCCc20x4ud0lfw6SgK3v6qp84+gZQ8HS?= =?us-ascii?Q?zf0BH+wSW/v87ak9MtQtGbNo+xPKgTuf5fAK6Glq1pc3970Ua8q3qm15vX35?= =?us-ascii?Q?rkY9j81MAlTtAwXeWn7rJ4EvTaF/ciFJ6v/vw18tQyMQIu7Rd3Eh7e2dgDPK?= =?us-ascii?Q?FUT/g4zBv7soplMf6/cIcRyjR9A8LEVf46m3TFjD8VrhOkEYwMNVGg+O0FJN?= =?us-ascii?Q?+bV2RgeTO7F3krcGCtvSFUDyKEPG8+uwUzUUFn4NYnDU2w3YnxL5ujzqg/6C?= =?us-ascii?Q?H/KHYp5NbMK1cAmZ4JDr5W6f1y5e8sfTPXdujodEY+ALneFNB2SZjuGEHdKo?= =?us-ascii?Q?ae063GQdU9iTRPO4jAdhN9po0/63sW0ptr5Mnj43EPjeLZU7eak2IKzNcm/p?= =?us-ascii?Q?VXUd/fraP/ZLIsws8iUWv2YE/PmUcW0zNwZiJSo9xwz4UgjPuuI0awgsEJmV?= =?us-ascii?Q?grctSZyGz7xyCCAYUbc7s/hZSB6dOHXe7DFBdAomFiRcpNmT+QmrUquTD2TE?= =?us-ascii?Q?z0gM2u96Rh5RPCXavmTWi3/H?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2b44ad60-9cf9-4f85-e839-08d981e3c204 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB5506.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 18:22:26.3899 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eUgn8YUMnDagTp7JtDebh7e+Q1FxRmF0M9T/+FqdEEk6Owq7eDyUsFyHxR4+MQhK X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5288 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org On Mon, Sep 27, 2021 at 09:17:19PM +0300, Leon Romanovsky wrote: > > The point is to all out a different locking regime that relies on the > > sriov enable/disable removing the VF struct devices > > You can't avoid trylock, because this pci_get_sriov_pf_devdata() will be > called in VF where it already holds lock, so attempt to take PF lock > will cause to deadlock. My whole point is we cannot use the device_lock *at all* and a pci_get_sriov_pf_devdata() would not have it. Instead it would have some test to confirm that the 'current' struct device is a VF of the 'target' struct device and thus the drvdata must be valid so long as the 'current' struct device hasn't completed remove. It is a completely different locking scheme than device lock. It also relies on the PF driver placing the sriov enable/disable 'locks' in the correct place relative to their drvdata's. Jason