Date:   Tue, 31 Jan 2023 22:36:27 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Bjorn Helgaas <helgaas@kernel.org>
Cc:     Baolu Lu <baolu.lu@linux.intel.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Joerg Roedel <jroedel@suse.de>,
        Matt Fagnani <matt.fagnani@bell.net>,
        Christian =?utf-8?B?S8O2bmln?= <christian.koenig@amd.com>,
        Kevin Tian <kevin.tian@intel.com>,
        Vasant Hegde <vasant.hegde@amd.com>,
        Tony Zhu <tony.zhu@intel.com>, linux-pci@vger.kernel.org,
        iommu@lists.linux.dev, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 1/1] PCI: Add translated request only flag for
 pci_enable_pasid()
Message-ID: <Y9nQK9P3HOxEeZ4U@nvidia.com>
References: <030e66e0-fb54-b77d-5094-4786684ba97d@linux.intel.com>
 <20230201001419.GA1776086@bhelgaas>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230201001419.GA1776086@bhelgaas>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RLMlnAYIi5+7CMYEXp8f2WbwBtceZTeWm5OtgjhOvc4GYPGAVTOh+cMkQsp9?=
 =?us-ascii?Q?RFpdvZs90xwb7ll5nGfjCHh3je3EEUF6LJdR3SuKlPVbQ37IE1RKFCgU9V0y?=
 =?us-ascii?Q?5doaQAqjX4oaxaRf75GUDizNd0eS3CAUEEEkcyc/pmECvbQXP5eIS5Byty9z?=
 =?us-ascii?Q?dx40MZviU95xVJ+l6CjZnB7P6bKXCQIegW7RXdrRT6PZt9H8Fk8sinJrAAbK?=
 =?us-ascii?Q?WmLF1N9BxiVY5n7J7/UpQ00IzdEWVC+28Il0IXHGvzgGTaL7G7WSORXDqQXB?=
 =?us-ascii?Q?WNyC0fj+/FkuSoom9ueCNdNsHw4+B4piGD61eQxtCv9WPKNdbhCEA8xgyd4y?=
 =?us-ascii?Q?t/fyHXcDwcQ7cZd6doP7Tb/8e74szqTTbB9F6S8ZDlhh4rcLMGqPGFYb2fOz?=
 =?us-ascii?Q?Z2IoATjJIj6I+Dm+rZX2GjEn8IDMZwrWZrabo9jwyOYB1qM/cjOKJIcYSGy/?=
 =?us-ascii?Q?LECpK++KEpKTVGFU7s0ILGjeJx3+MW5qYhkd2I9pBTagd26YpJnqR8FFTjcx?=
 =?us-ascii?Q?QqUi3aFW/0iSHhTu2UJ0h2myAJEVvbmnKYueHNBNXlXQZ99sPtHjUwC+wzLp?=
 =?us-ascii?Q?FpfemrEqm7KuxNws15+tEuGroMgjTNyJCK2OEwZkWYumWcPBXDOhxkwZwJuS?=
 =?us-ascii?Q?r+62x3KTq+UuNrz6Hy7wAIPUCMAD8g7KpG8PmlEwu0T7FUoBHF0LleahuZoC?=
 =?us-ascii?Q?rD3ZOlW0aHe9NJggdAcQBmqwSMtwSXLLE8jwj26ew7cgHmTNrPBqDAo7vl16?=
 =?us-ascii?Q?qyRYLl74W3K8CyzqZ/FjBxH9wnJKMXH4ll8+sUFB66cmX7UOGAv05wqV4/mj?=
 =?us-ascii?Q?H/1NRHbYUw9HmNuIkX3dASVG4xbMkTokj9YiHelbTZv6NCJbBcZdS+smp1U6?=
 =?us-ascii?Q?KvjcEDf6zdfNWS8IUgdQg62gu06OaiEVW/CuCCn6pUpm/6JVc1wS2Ti6hekX?=
 =?us-ascii?Q?WwsP1TcgLR7dfdbWExMC3OhRfG/8ZlgC+5ANbBOYsk4hRbZUuzshAUWumDhz?=
 =?us-ascii?Q?VO2TSVL61JPpyHHpg+SOXGrRvNDGx8C8jCpEzPIPxF9R5M70oacrKGnzakOY?=
 =?us-ascii?Q?0MX1ZeyTYoWg3A8FdDdvHpkLj2cOALJQAiFBCdjFVL956z9TgCy/W3PESAPN?=
 =?us-ascii?Q?Iogloxhrgf34pwWQwoSQ+PPOXVbBdeY+7IKtZa1GX+f2TEb4OPpQ0R5N6GYb?=
 =?us-ascii?Q?gDBABv6dfEu/kqSwF2d3EPZG6NbqIMN5852mFZ4iUoSA7K4vHBv/fwM8E6Bd?=
 =?us-ascii?Q?Bkje8uG/aOCvcpDtRTqNfgKTc6RcZo1Jv3Bjokhn/RyEAtLJ5Oqc+wdqYKmR?=
 =?us-ascii?Q?/DsOvQtCFclw/F3OCAdPAdMCoj+0c2MQbhy8spx68wGqg/R63mHOn1wq1wFg?=
 =?us-ascii?Q?dMiyRnL5+6pLvAj/BeV86q/pl1VKYmV6Rw9jN6vHE3UhsxvdyzPmiWb58hlU?=
 =?us-ascii?Q?imRQrTiDEiY631Q+di5m8tCRfM+43CBpeGZP7W8Xkfd0z5FpkHW7OH8bwusJ?=
 =?us-ascii?Q?HVd2koadspJxia/eh+98b16MDxsq8cTpVZFrzJQrVLIfHuDsQnKOINwcYMbf?=
 =?us-ascii?Q?R+XqmrDlf5GmA8Ejw5IRiNivBHhMkK4i3u15xciq?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 029bf56e-708a-4a27-8804-08db03fd1ed8
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2023 02:36:28.3266
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: h/Ha38uTltMgaHfVNamAMcR8hUPUj9Ob+1WNEDnMZjzwSZB+9tfalFWiuD1G0vDQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4548
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Jan 31, 2023 at 06:14:19PM -0600, Bjorn Helgaas wrote:

> > AMD GPU is one of those devices.
> 
> I guess you mean the AMD GPU has ATS, PRI, and PASID Capabilities?
> And furthermore, that the GPU *always* uses Translated addresses with
> PASID?

I'm not versed in the spec lingo, but the GPU issues MemRd/Wrs with
the translated bit set and no PASID header - which is the correct form
for an address that was translated by ATS.

To get to that it issues ATS requests, and only the ATS related
requests will carry the PASID.

ATS related requests always route to the root port, which is why it is
functionally equivalent to ACS RR/UF in these cases.

Translated requests always route where they are supposed to go, even
with P2P and things.

> And this applies even if there is no ACS or ACS doesn't support
> PCI_ACS_RR and PCI_ACS_UF.
> 
> The black screen happens because ... ?

AMD GPU driver bugs blow up if it cannot setup PASID.

> I couldn't figure out the NULL pointer dereference.  I expected it to
> be from a BUG() or similar in report_iommu_fault(), but I don't see
> that.

IIRC it is a buggy error unwind handling in the AMD GPU driver.
 
Jason