Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e3146447-6af8-9049-e951-915e2770988b@linux.intel.com>
Date:   Tue, 11 Jul 2023 10:52:23 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Cc:     baolu.lu@linux.intel.com,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/5] iommu/vt-d: debugfs: Enhancements to IOMMU debugfs
Content-Language: en-US
To:     "Liu, Jingqi" <jingqi.liu@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>
References: <20230625150442.42197-1-Jingqi.liu@intel.com>
 <BN9PR11MB5276885D6A142BC94D80C2618C29A@BN9PR11MB5276.namprd11.prod.outlook.com>
 <4f18a7ca-3a57-0097-51f0-490f876831dc@intel.com>
 <BN9PR11MB5276ECF766C4F89F5083992F8C2EA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <f2f23ca1-0881-4601-7ea5-c5e42ab83f62@intel.com>
From:   Baolu Lu <baolu.lu@linux.intel.com>
In-Reply-To: <f2f23ca1-0881-4601-7ea5-c5e42ab83f62@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 2023/7/11 9:40, Liu, Jingqi wrote:
> On 7/4/2023 3:54 PM, Tian, Kevin wrote:
>>> From: Liu, Jingqi <jingqi.liu@intel.com>
>>> Sent: Monday, July 3, 2023 10:37 PM
>>>
>>> On 7/3/2023 3:15 PM, Tian, Kevin wrote:
>>>>> From: Liu, Jingqi <jingqi.liu@intel.com>
>>>>> Sent: Sunday, June 25, 2023 11:05 PM
>>>>>
>>>>> The original debugfs only dumps all IOMMU page tables without pasid
>>>>> supported. It traverses all devices on the pci bus, then dumps all 
>>>>> page
>>>>> tables based on device domains. This traversal is from software
>>>>> perspective.
>>>>>
>>>>> This series dumps page tables by traversing root tables, context 
>>>>> tables,
>>>>> pasid directories and pasid tables from hardware perspective. By
>>>>> specifying source identifier and PASID, it supports dumping specified
>>>>> page table or all page tables in legacy mode or scalable mode.
>>>>>
>>>>> For a device that only supports legacy mode, specify the source
>>>>> identifier, and search the root table and context table to dump its
>>>>> page table. It does not support to specify PASID.
>>>>>
>>>>> For a device that supports scalable mode, specify a
>>>>> {source identifier, PASID} pair and search the root table, context 
>>>>> table
>>>>> and pasid table to dump its page table.  If the pasid is not 
>>>>> specified,
>>>>> it is set to RID_PASID.
>>>>>
>>>>> Switch to dump all page tables by specifying "auto".
>>>>>
>>>>> Examples are as follows:
>>>>> 1) Dump the page table of device "00:1f.0" that only supports legacy
>>>>> mode.
>>>>>
>>>>> $ sudo echo 00:1f.0 >
>>>>> /sys/kernel/debug/iommu/intel/domain_translation_struct
>>>>> $ sudo cat /sys/kernel/debug/iommu/intel/domain_translation_struct
>>>>> Device 0000:00:1f.0 @0x105407000
>>>>> IOVA_PFN                PML5E                   PML4E
>>>>> 0x0000000000000 |       0x0000000000000000      0x0000000105408003
>>>>> 0x0000000000001 |       0x0000000000000000      0x0000000105408003
>>>>> 0x0000000000002 |       0x0000000000000000      0x0000000105408003
>>>>> 0x0000000000003 |       0x0000000000000000      0x0000000105408003
>>>>>
>>>>> PDPE                    PDE                     PTE
>>>>> 0x0000000105409003      0x000000010540a003      0x0000000000000003
>>>>> 0x0000000105409003      0x000000010540a003      0x0000000000001003
>>>>> 0x0000000105409003      0x000000010540a003      0x0000000000002003
>>>>> 0x0000000105409003      0x000000010540a003      0x0000000000003003
>>>>>
>>>>> [...]
>>>>>
>>>>> 2) Dump the page table of device "00:0a.0" with pasid "2".
>>>>>
>>>>> $ sudo echo 00:0a.0,2 >
>>>>> /sys/kernel/debug/iommu/intel/domain_translation_struct
>>>>> $ sudo cat /sys/kernel/debug/iommu/intel/domain_translation_struct
>>>> What about creating a directory layout per {dev, pasid} so the user can
>>>> easily figure out and dump?
>>>>
>>>> e.g.
>>>>
>>>> /sys/kernel/debug/iommu/intel/00:0a.0/0/domain_translation_struct
>>>> /sys/kernel/debug/iommu/intel/00:0a.0/2/domain_translation_struct
>>> Thanks.
>>>
>>> Do you mean create a directory for each device, whether it supports
>>> PASID or not ?
>> every device has PASID#0 valid, i.e. RID2PASID.
> Sorry for the late response.
> Got it. Thanks.
>>> Seems the PASID can be assigned at runtime.
>>> So it needs to support creating debugfs file at runtime in IOMMU driver.
>>> Looks like this requires modifying IOMMU driver.
>>>
>> Isn't this patch trying to modify the driver?
> I just tried not to modify the driver except debugfs.
> I'll try this implementation.

I'd second Kevin's suggestion.

If you check how usb xhci dumps its contexts for devices, you can see
the similar scheme.

# ls /sys/kernel/debug/usb/xhci/0000:00:14.0/devices
01  02  03  04  05

In our case, pasid 0 is special which denotes the domain attached to the
RID.

Best regards,
baolu