Received: by 2002:a05:7412:8d10:b0:f3:1519:9f41 with SMTP id bj16csp1189482rdb; Wed, 6 Dec 2023 10:57:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IE2mWPdaK/55UzpZv/hrh9EpCgz4n46gF6LkuVoLn0GTT8h1o+NpnEtl523+20VcP+prR/6 X-Received: by 2002:a05:6a20:a39c:b0:18f:c92c:86a8 with SMTP id w28-20020a056a20a39c00b0018fc92c86a8mr948704pzk.15.1701889035648; Wed, 06 Dec 2023 10:57:15 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1701889035; cv=pass; d=google.com; s=arc-20160816; b=Q9z1S9dR3RvEkRh5nMff1BF0VhsqdIcEkn87kBzPYOGhr+r4T+xVfTQnCTwprKFXVV yBRPubMe6mqK1kXSQqTL2dcBDq//bDmYuzq4eqca8H5+RYR8I0FSo3I/HNqRixk6gVM0 9e8ZDt7TVO19gRPV/QhWp4q15cdvXOK7kzBDgtufCH4iED1AA9Z43O7EmWNvdftLNh7x YBOs+70pliheSLM26jieOTaHeaOuqAHvfLd46B+chA2EejHSho5ByJOmhF0pfW2jxhNX 1fklCwO7FfjyGwrI8wei9VT4e2yaKM5V8RuuTy52FOnh/POFCCNF2hS3JbqsQRM2xMPY HF2Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:in-reply-to:content-disposition :references:message-id:subject:cc:to:from:date:dkim-signature; bh=tQmo138YafK38DlduIoaP0BZK8uIclfvmQmYz3eyR1s=; fh=k5doefdr4QcVQGZUqMnaKlCK1MFVWRK7THwtnyoqv5s=; b=sMDlI8lPOnFlzicIEtIPn+2HMVwKNwaZbLmqlxWlNVm7JQ8qB32N5kU7IspUFHE0k8 XMB0iiOxeSeZxtPN4A20ssbiC4qEVaxg+HUe/p/E+GtZj/gl+/iylrvyLmDVkrpA7yVu W6zSg9fb8BvF2C7uqzbQVws1WCXi6EFvgbJ4qpieEJj/su4S/TfVheq/18mHW63xKnyE zxft2L9amZsOZ/L9E1ARo22YthMjxNIydXcR5Ao38pSW1JlQn4Fb/VkYyvVxdaN4sTPK gwfa72wVs3+9lvTwIAFYI+zp6GFxrRc22VQdKsTk28sal1ykA06Afuzw80smfQU8eLP0 awTw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=ae2BqefS; arc=pass (i=1 spf=pass spfdomain=nvidia.com dkim=pass dkdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Return-Path: Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id g62-20020a636b41000000b005b92842d469si322995pgc.62.2023.12.06.10.57.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Dec 2023 10:57:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=ae2BqefS; arc=pass (i=1 spf=pass spfdomain=nvidia.com dkim=pass dkdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 3B0E480D04FD; Wed, 6 Dec 2023 10:57:12 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230010AbjLFS46 (ORCPT + 99 others); Wed, 6 Dec 2023 13:56:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229997AbjLFS44 (ORCPT ); Wed, 6 Dec 2023 13:56:56 -0500 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2083.outbound.protection.outlook.com [40.107.93.83]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE469135; Wed, 6 Dec 2023 10:57:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YDoP4ap8lbIrqSpixS5FncTXoyrYs7bxJNnfvrKEe5/58weEyJjanvfGgKf5UhHfqmlxBDv6KvBnPtWQTIQTnisBzI37xtaT8Zu2PaqHQVBQCa6aL0Jhi2XjFfCRrXqqblATTJJO7HEywzVwX6PcJsnJZoUepaT2zh7oyfUvA4pv28m/PdqB1fw7ltrgkifYRleGjb1nQZIgnc1AVyF2/5MnPWQ7kLfAg7G/vkhBiw343UXM6eGAvHYdxcI88ctxxm/8bWSP2GT+MrKRwgOKW2IBjvMt4jLTAlimQbTxKgUEOfYQm6EzeXOsBf34AyOQsScme05w0UlBWo/TMdCh1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tQmo138YafK38DlduIoaP0BZK8uIclfvmQmYz3eyR1s=; b=hFN9ry/izTYQybPygloniRH5u6RP6/1HXTKH8U0suxCwivFiMN7qRMFdapBY3Aglgzzv1DosfA3MQhXuxDNgT/z0H3olIP8r0N2ViIY+DQpj3g0iHZ+vDSB6L9Ls4RT/qO5Tfd6cGg6ToMR+OSX7y1ztUrikJ+OwlhFDgnOuBiklqoXNtFdqgIDxsygUmSXNf1Ou3+853w2Grt3jtFtQE4LKIunqNuLlPw7LFrejUgrT60cRXzomGU9Xc6tSRlXbHtBl2KrXBWPwvQ/kfcuSvgF6CcYi+XBJizLJ/THbdJRGQ4lzL2U0j4fB+yxmoSPAXnwFNTBRyHoq9uBq0TSPwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tQmo138YafK38DlduIoaP0BZK8uIclfvmQmYz3eyR1s=; b=ae2BqefSRt1o2nP7uaOC7om4O/miC0CIqH9GVhrBCH5Q++8lvg6iJziEiRvYMNiI7lk9J9MOB52LZSUVA09hZHuOyNZc1g9DKv0USCBDvpaiMoT+NWvfRID9ubjkgXksey+aPlNsvkw6AfAv2tQir7ScYW2jKzZq6Jj4MwHXW7StFMA9k2FaxX14eJ9k27GBbTmMhrDTWM7kU10UzogeDlSvUJEns2sOt7EehIQVewhyDg9cCnWpyqKC+zR447HHEUxmajybzHnAP24A8E9QJ1fpknn0/Dv/RIQr5G1WM0VKHJhZD4hcuVDi5V3XfLfKMJbEhpYTg7Ju2tsPh7snDA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by SJ2PR12MB9211.namprd12.prod.outlook.com (2603:10b6:a03:55e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Wed, 6 Dec 2023 18:56:59 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93%4]) with mapi id 15.20.7046.034; Wed, 6 Dec 2023 18:56:59 +0000 Date: Wed, 6 Dec 2023 14:56:57 -0400 From: Jason Gunthorpe To: Yi Liu Cc: joro@8bytes.org, alex.williamson@redhat.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com, cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com Subject: Re: [PATCH v7 3/3] iommu/vt-d: Add iotlb flush for nested domain Message-ID: <20231206185657.GC2692119@nvidia.com> References: <20231117131816.24359-1-yi.l.liu@intel.com> <20231117131816.24359-4-yi.l.liu@intel.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231117131816.24359-4-yi.l.liu@intel.com> X-ClientProxiedBy: BL1PR13CA0436.namprd13.prod.outlook.com (2603:10b6:208:2c3::21) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|SJ2PR12MB9211:EE_ X-MS-Office365-Filtering-Correlation-Id: 21327926-6b38-4993-d729-08dbf68d1fca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rFvIhzn9BLWCQC7hjFk6aWH6j1di0kuA2s4uzWZVkVkgA6faWY2gpYI0ep6Qnhn3+45sLQth4tSqFrkrhHdyM9zhSwAig+CyvzbQjrljV7VzMoH1akTJz+JYsZN1WsQbyXL0h78zyAHbXBmJ/8cblNSUVibyqswSpXsRUbp1EvkdYC6PFhH4T7SSlmD6HExJ5VzOI7fF5Wa9CumipFT0/yHGAuX6M8jvodDfN6PkLMCS+UogLsX1QQ3zfi+jRhTNnLOgEdV/fTyFJU8vPzAp1QiqJ7tp1HJt6fXzIcjmh++S2fxqX2OIRkswcauUCYhnqM+nc//nl842XqQJID6iwEFN1Y2T26wSSLTpCNq8k9hkL3OEZvcpDgtyGVzvWL5KIbbG5PsXwKmYuL9bRLWeD/o/e6bdow8jgS5MsMm3sT27YtpKQoKlz1JOXe2RfrAOUqDi42oY7Vf3tE7C1xIjmrpVeL2Dwm1hjH7PbG6fmsnuDFQ3RK1Mth4uyTDgPV0HuBoBFGBpb+ApKmyUVE22Nb9n4/FV8K9xY0ViryDWwXo9EVP6dMeGyKwRMDJMpJRk X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(376002)(396003)(346002)(366004)(136003)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(7416002)(4744005)(2906002)(33656002)(316002)(66556008)(6916009)(36756003)(66476007)(66946007)(8676002)(86362001)(4326008)(8936002)(41300700001)(38100700002)(5660300002)(83380400001)(478600001)(6486002)(26005)(1076003)(6512007)(2616005)(6506007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?OzZakPdjdbJRFEK1PxwrN2pGyLuRNKhDdKUnAd6isVoXAOteuRo90kZ6pt6W?= =?us-ascii?Q?xJigNud9RDX8TtrPPRN08XF1AdEVQleZ1SUlznulMxLTEUhBuc7bJKI+TSsY?= =?us-ascii?Q?hToDxmEAUay63jtg0VsTFZ/jVn26c/KWQTnYayci/MPuejd5UgGSEFqYtniO?= =?us-ascii?Q?mkx37BXSw221GIB1eD+5IHD34yhbR/lNC3cHWQEBDxnuf0IArlSQK5G9J6wE?= =?us-ascii?Q?A3z8cQ4U8mQFnu5IQ4iLz+/FU+NUNIRpF0HoI0RmSx+RtPlWliRYBQDoUIda?= =?us-ascii?Q?lSyBJLPQe7ItZjrerD+jc+nbQTWK1cl8/XggnEAl47XGy5AdEzHgf7EG5y+t?= =?us-ascii?Q?muIuTjYFr19tKMNXijCn2yWiUD/G3c9KhcLndVSNjx9Z8pEKu1mqfMQWRPaY?= =?us-ascii?Q?K7fDvrTHzFqMNLt08qFSvlLNlByhTWni1kJtSAxEz/+EWC4WIqw/WLJEnyKR?= =?us-ascii?Q?gto7qXvky5xO389OCXZ1KiR6/iS5c58qalA0kPpNv98x03w2LAQPflOYXjBO?= =?us-ascii?Q?GgKi4si1wGn92rTpStGBWlr1BP/RxHLaW4EDLPeU3HQ2xuiGuD3vFvO40HNj?= =?us-ascii?Q?MdpiM6471rn4q2vUePMtY43wluHuG9gt3mOKue5IpsbFspbyrX3cbNfniLxO?= =?us-ascii?Q?KQMRrDrNuJFmD+/FjJBgpAcxHF+3yqEzRIFEchWSWWYIaDFAsc/r+ucasKVx?= =?us-ascii?Q?Sevlf/UgC4UZQ/sKfyuEDzDkeiCZGR1v2yIM06BNCFR6LBiGRjZZgYAW946I?= =?us-ascii?Q?3lVj4U5oRpFmyJX04NA7zPMKhJjIE4gLrIkrLmzf8CGDxJgtcV8zaeylaI7R?= =?us-ascii?Q?QhlAuKUsP58WUK8+RIuB1a3VD4nscciFDdnU1HEfxvZiZsslGwHcnNokU2GG?= =?us-ascii?Q?5KtBgIzUVbzOcrQ5lBOABeGZK5iiwgeHZ38IvE8KQwrdGpF0d8s6UIDpwEdS?= =?us-ascii?Q?NfedUCviZ7cvavxkqF7daUOHLFvE0dH3UBcWWDSVYv0y66s3Z4ez4t+NCDDo?= =?us-ascii?Q?PHmb4PRYZ/j1UzmTH8L7nqQPFSYXqgJP8LQjUHwyoXHDTLQZFapBJ+SpY+CI?= =?us-ascii?Q?meI4wtbwz1fQ8c6UdLwdGDwjYHyNWQJfHOwNpqUoidMjk7+3hPsiQTuNY4oC?= =?us-ascii?Q?fUqwPK9tk97spEW7ZUEFcDaRzD4kL1qQKhXPJ7VBuplXpLZUdV63Ufz3Afue?= =?us-ascii?Q?FyO2sH6KNtEXLllQ+/dGKWV4CSXA82u7ls+Y4rJ/Nn5xICoFnAn5Vn6VwNID?= =?us-ascii?Q?XMAqQ+X+bc1c578OIQDiQnqCaPb7TViXtWXYqpsP74/8Ltt4IZ5qYaoYcS3f?= =?us-ascii?Q?AGgI599wTTEIBuloIWwohFNm4fvEZ7AC8XR/y8ARtcrcj3UQrVahQYKdBCUp?= =?us-ascii?Q?gwL9SldejFfX2n6vJ0Y+R2qSGjaYfzMBDfg8EQqTCx1CC6ILIbYz8jmBSDDb?= =?us-ascii?Q?5pUHU9vjr0+Seu1qA9SeXlaitZdpi2oBU+2AlBKTBXmGOQQKd0MleEhpOicu?= =?us-ascii?Q?i/syOHqXPUDqSTRHaLbRJ9WeX4tNKdvyeIDhxJFrPXkbc2UOdPcsTEFmoyHG?= =?us-ascii?Q?Kc+R7AiV1fBmoc1G9cfG8QEkqx9nEF40c2yJe1cy?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 21327926-6b38-4993-d729-08dbf68d1fca X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2023 18:56:58.9676 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JmDbYFupTItlLaCKLQgc3cLl5etBV3nAl16clfUno8jduPkWsj3pEuFVpf3dIQkj X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9211 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 06 Dec 2023 10:57:12 -0800 (PST) On Fri, Nov 17, 2023 at 05:18:16AM -0800, Yi Liu wrote: > +static int intel_nested_cache_invalidate_user(struct iommu_domain *domain, > + struct iommu_user_data_array *array, > + u32 *cerror_idx) > +{ > + struct dmar_domain *dmar_domain = to_dmar_domain(domain); > + struct iommu_hwpt_vtd_s1_invalidate inv_info; > + u32 index; > + int ret; > + > + /* REVISIT: > + * VT-d has defined ITE, ICE, IQE for invalidation failure per hardware, > + * but no error code yet, so just set the error code to be 0. > + */ > + *cerror_idx = 0; > + > + for (index = 0; index < array->entry_num; index++) { > + ret = iommu_copy_struct_from_user_array(&inv_info, array, > + IOMMU_HWPT_DATA_VTD_S1, > + index, __reserved); > + if (ret) { > + pr_err_ratelimited("Failed to fetch invalidation request\n"); > + break; No error prints on ioctls! > + if (inv_info.addr == 0 && inv_info.npages == -1) > + intel_flush_iotlb_all(domain); -1 is clearer written as U64_MAX - same remark for the comment documenting it. Jason