Received: by 2002:a05:6a10:a0d1:0:0:0:0 with SMTP id j17csp2460838pxa; Fri, 7 Aug 2020 11:40:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwJn6yh4BfU5HbBePIgr3DnCgn06iPqkEvKpMfxtJPqWIuja2h0LITt2KCgN93aRfcngLlV X-Received: by 2002:aa7:cb45:: with SMTP id w5mr10132493edt.77.1596825630678; Fri, 07 Aug 2020 11:40:30 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id v19si6130187ejc.260.2020.08.07.11.40.05; Fri, 07 Aug 2020 11:40:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=R4kYYWeZ; arc=fail (body hash mismatch); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726212AbgHGSje (ORCPT + 99 others); Fri, 7 Aug 2020 14:39:34 -0400 Received: from hqnvemgate25.nvidia.com ([216.228.121.64]:14304 "EHLO hqnvemgate25.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725893AbgHGSjc (ORCPT ); Fri, 7 Aug 2020 14:39:32 -0400 Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Fri, 07 Aug 2020 11:38:41 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Fri, 07 Aug 2020 11:39:32 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Fri, 07 Aug 2020 11:39:32 -0700 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 7 Aug 2020 18:39:30 +0000 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.56) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Fri, 7 Aug 2020 18:39:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RukYmoJ8Lo85ZtlgoY42H85gbJXZ8A/AyhRbkr79Dj7t09DwSah5a1HCTuWXMLK7OTopgi85JIEEKv8ZE04GvJsZaRJe2vi5lSRHW4s+aXMIsstoxwnf7Qp0NVauTMrqteB6txzwDdXpcBK/LoSTDRbSCgpwwcc4UdY4cQIbk22mK2aYzm0vYGspOPzv0sXO1+v+iidB8Cqveh//GXUOd/Yegkv5NajwxZS+XAQVM+RvdNsiLYnsoCdYGsgaCdIPvWQL8T2UP0pTw9Wb6bH3JRXZc5YTdLx0TP7IByvLNlhhdc4rGcYVXj85YCiuqNw0h6+8MhbMsES4g4EbMXpoVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LSedvlXCbDmzOblX/9NK3RwxARgxvJi8yTM9k8MLHb0=; b=lTVi4lkhDS+Hjpi7JsqDvjkj9ip2/h3pV0xjNhk8EDFbyRj/wjq/ZL7UDVbDAD08xXYeVeRqAepPuQubR//Re75bhIFn1KBMbQdSWi1zMWNoUrNlfd/NDUjua+aaT//kIN52mbVv5y/x8Eb5c9orLabM4kxNCnwtdtMmyKE3JyUvzSv2vfbAY5ag5zooQjsjdd1lglKuaoN6ZoblWhAa6dsBQp3aOJE81IP3MrwuhSmIGW85ZijzEMca28fMafZtEbGD4wpLu6wxbiWq4d6bDus5+1pLKvLaovWxhbl9OyGT20z36elLbqnlL5BBmMnMBGlZULJISuc9pEWh6amTdA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM5PR12MB2583.namprd12.prod.outlook.com (2603:10b6:4:b3::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3261.18; Fri, 7 Aug 2020 18:39:29 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::2d79:7f96:6406:6c76]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::2d79:7f96:6406:6c76%3]) with mapi id 15.20.3239.024; Fri, 7 Aug 2020 18:39:28 +0000 Date: Fri, 7 Aug 2020 15:39:27 -0300 From: Jason Gunthorpe To: "Dey, Megha" CC: Thomas Gleixner , "gregkh@linuxfoundation.org" , Marc Zyngier , "Jiang, Dave" , "vkoul@kernel.org" , "bhelgaas@google.com" , "rafael@kernel.org" , "hpa@zytor.com" , "alex.williamson@redhat.com" , "Pan, Jacob jun" , "Raj, Ashok" , "Liu, Yi L" , "Lu, Baolu" , "Tian, Kevin" , "Kumar, Sanjay K" , "Luck, Tony" , "Lin, Jing" , "Williams, Dan J" , "kwankhede@nvidia.com" , "eric.auger@redhat.com" , "parav@mellanox.com" , "Hansen, Dave" , "netanelg@mellanox.com" , "shahafs@mellanox.com" , "yan.y.zhao@linux.intel.com" , "pbonzini@redhat.com" , "Ortiz, Samuel" , "Hossain, Mona" , "dmaengine@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "x86@kernel.org" , "linux-pci@vger.kernel.org" , "kvm@vger.kernel.org" Subject: Re: [PATCH RFC v2 02/18] irq/dev-msi: Add support for a new DEV_MSI irq domain Message-ID: <20200807183927.GY16789@nvidia.com> References: <20200806001927.GM19097@mellanox.com> <87tuxfhf9u.fsf@nanos.tec.linutronix.de> <014ffe59-38d3-b770-e065-dfa2d589adc6@intel.com> <87h7tfh6fc.fsf@nanos.tec.linutronix.de> <20200807120650.GR16789@nvidia.com> <20200807123831.GA645281@kroah.com> <20200807133428.GT16789@nvidia.com> <87v9hufln7.fsf@nanos.tec.linutronix.de> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: X-ClientProxiedBy: MN2PR05CA0039.namprd05.prod.outlook.com (2603:10b6:208:236::8) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (156.34.48.30) by MN2PR05CA0039.namprd05.prod.outlook.com (2603:10b6:208:236::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3283.5 via Frontend Transport; Fri, 7 Aug 2020 18:39:28 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1k47H5-004oPA-6I; Fri, 07 Aug 2020 15:39:27 -0300 X-Originating-IP: [156.34.48.30] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1de33126-ae2d-46f0-97fa-08d83b013784 X-MS-TrafficTypeDiagnostic: DM5PR12MB2583: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bqJkOZLIGTQPDdOa/Wae7QbHVKidL28w9W/onH3vEpeFd89luV61ZFp3ebNz5jp1Coywq+SB719AQavae4K19mahNHIOfGnqfsExRpnBfg+7InZhp98oRgK3ST/wBwuH/tWmivCt0AH4wYvhJTbULM23uk1cur4e+wEMY5olXkX/1RVlywlaGSbSsV17gW/K0OV9u3JdvcymxrXMsrgP/YB+JwuvXo4HeMOZkOedug0O1me0gNtu+pF/Hntwa469vUg3Dnhftkrh3tprTcF9s/WxufTtDqOzgPkWd4rbdfChH40PNIfiYIU2Ov4yUBh2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(396003)(376002)(366004)(346002)(136003)(9746002)(9786002)(54906003)(66556008)(8936002)(478600001)(66476007)(4326008)(66946007)(2906002)(36756003)(1076003)(33656002)(316002)(8676002)(2616005)(6916009)(426003)(7406005)(7416002)(4744005)(86362001)(5660300002)(26005)(186003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: QdgxuRgzxrQKFRtoD2QrnkOFnZTeCn++wsw9Vf64lQcV0pA9uzFO3SLqXTRo3W4QhDb2r3s70mCHbvBCI6Le55aAZJqwBd3rIRNMs1+5iI2/QcRestHazqJmZHYrJ6849d9kX8ib8Pu0Q84aVgdPYuM8o9IbDv1G6Ob5ujlezsDXsZQQGTf7O58WKF6c/ITWc4xN/y3cUiaUYQMBbGDYN0Dn4V0CyNUWzU/PvpKi9ilkqmHBt+nUKcIjW692rl76Ff2z4LNfpoyD5qpvQ5yl+6xmUwFV/+Rsb8styG9QjRmP3PiT5/wvT2vZzaxHIJ8XtrFCHm2dhJnSA4+kwM4ChYb2BZB98MTlurXKU/UYE6SMdemHs6UcvHvJixfBjFKXYcxjYyysQ+vpVXuGozzdE+JsmuoyihGuSgVi4f00JUNQyKO7qvsOu80CVha4Vp22LB98AjUSWUiVRMY1XM9y3MTYNsC8sIMVxn2lQByYjaG+GXRukuMnq4r+FJn2CGygaQfcebRR6GhZQCuzJ9vYaNrtpwTN3e1zZ7n+RNuKHeUeDmGQHXzjWAIikilonOET/6LU4NVuwFgCz6Z/YgXKw+Yce+cD+4uEvgkJ0vPen5nxCDvpWM/xOMhC3UXMV3gax/TqP5aYdXsW9Fp/8k/n9w== X-MS-Exchange-CrossTenant-Network-Message-Id: 1de33126-ae2d-46f0-97fa-08d83b013784 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Aug 2020 18:39:28.8925 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LCIxTAbkdDckG9Iu5G4p6sT5n89I/SkkwXkbr8AaM72dYK84UOstqxF0oPenE9/K X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2583 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1596825521; bh=CFpBs/oDEPdjTRVHVoJyZycTlbnk0Z4tOWvHDqCnsNw=; h=X-PGP-Universal:ARC-Seal:ARC-Message-Signature: ARC-Authentication-Results:Authentication-Results:Date:From:To:CC: Subject:Message-ID:References:Content-Type:Content-Disposition: Content-Transfer-Encoding:In-Reply-To:X-ClientProxiedBy: MIME-Version:X-MS-Exchange-MessageSentRepresentingType: X-Originating-IP:X-MS-PublicTrafficType: X-MS-Office365-Filtering-Correlation-Id:X-MS-TrafficTypeDiagnostic: X-LD-Processed:X-MS-Exchange-Transport-Forked: X-Microsoft-Antispam-PRVS:X-MS-Oob-TLC-OOBClassifiers: X-MS-Exchange-SenderADCheck:X-Microsoft-Antispam: X-Microsoft-Antispam-Message-Info:X-Forefront-Antispam-Report: X-MS-Exchange-AntiSpam-MessageData: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-OriginalArrivalTime: X-MS-Exchange-CrossTenant-FromEntityHeader: X-MS-Exchange-CrossTenant-Id:X-MS-Exchange-CrossTenant-MailboxType: X-MS-Exchange-CrossTenant-UserPrincipalName: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=R4kYYWeZe0zvR0BuXjAHCRUbn64jTSAtTWBiNNkg0ee2gwYdG0VuoBR+Fd1dJDbJS lYnMdqnkfCsY3aWV3Yd+48j6rzIYGWtfbHPDAAOFL7Y6/oSIa1SLi+ZiIz+BplQYKf 1Y9LEB5omXjqP8WSUk/bpdXkxYfmCnUk5pPFrNTIBBLCvNoC8U88z4qUKAux8sTOGr lx1XGpto02Wn8z7szo2LJi1P3I6Hav/5V5GkR78CdlfZDCW7XLKdp7TOjHEKj4M5GG PkYDX8Oszd/DbAx7e/KmCZsoNH6mysy3zisYpz8cFc1o3ZAjlXYd8wCr4P+6MF8Go7 DpQ9DjNxENJdg== Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Aug 07, 2020 at 10:54:51AM -0700, Dey, Megha wrote: > So from the hierarchical domain standpoint, we will have: > - For DSA device: vector->intel-IR->IDXD > - For Jason's device: root domain-> domain A-> Jason's device's IRQ domai= n > - For any other intel IMS device in the future which > =C2=A0=C2=A0=C2=A0 does not require interrupt remapping: vector->new devi= ce IRQ domain > =C2=A0=C2=A0=C2=A0 requires interrupt remapping: vector->intel-IR->new de= vice IRQ domain I think you need a better classification than Jason's device or Intel's device :) Shouldn't the two cases be either you take the parent domain from the IOMMU or you take the parent domain from the pci device? What other choices could a PCI driver make? Jason