Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <cf8b0163-56c9-0540-c635-b541132a77cd@intel.com>
Date:   Mon, 3 Jul 2023 22:48:06 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: [PATCH] iommu/vt-d: debugfs: Increment the reference count of
 page table page
Content-Language: en-US
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        David Woodhouse <dwmw2@infradead.org>,
        Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20230625022808.42942-1-Jingqi.liu@intel.com>
 <BN9PR11MB527672460238BCBA63CD3CA58C29A@BN9PR11MB5276.namprd11.prod.outlook.com>
From:   "Liu, Jingqi" <jingqi.liu@intel.com>
In-Reply-To: <BN9PR11MB527672460238BCBA63CD3CA58C29A@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SWlTMWplNFltUDNXeGhlbmlkSWNjdlZqUS8wejhlTG9IWFExbjVPbkYyRXVD?=
 =?utf-8?B?K0R6cEFDMjVnVG11ZkJwcWxBTHRzQmtVQnlOcVNUZXhrckluTmRKWVRndDg3?=
 =?utf-8?B?MXZ1K2F4eUZqejdjZXkwM3IrZTJiMkZ1T29RSlhFS2Y5YjM2Z21Vd0VOZnVR?=
 =?utf-8?B?MFN1YndwSVVvYXR4NDF4OTBvR1B6NXRxU2lzcFArb1FueHBwY2ExRFAySUR3?=
 =?utf-8?B?TFVLaC90SHc3ZGRQbVZ3M2Fkai9PTWtwRTRkMkhaVEVYb1h5Sm5IZjRxL1F6?=
 =?utf-8?B?azZWNDVRQzRhdWgwMmVtclprYWNKZHJyZkVBVVB0M0NFWWlLaW9wWXlxSkh2?=
 =?utf-8?B?RnVJYitJazEvM0RUajR6NHFERGc4OEJkTEpVUS9jZlA0SlFXbng0VldROHJz?=
 =?utf-8?B?RjBOMlJnQjZuYjJtZFMya3UzUWFwV1FUNGpnQ1YxM0tJR2U3L2dMdWlYS0pX?=
 =?utf-8?B?ZWthWGp3ay9iNkVJNjM5SUJ3Q2Zrc1NOUVRhdDE3VUNHcVBhOFNNSC9tUm1P?=
 =?utf-8?B?Q2NyUlZMNnd4bXdBcUc5Z2pJaGU1MnVqMXd4V2lxNmtJNDVuN0lDWFdjZ2d3?=
 =?utf-8?B?cnFzeDBzOTBjQ05RU1hIUlJwSGhhSDFEUG0xRGhRUmhLeU5ST0szeVZjZUtK?=
 =?utf-8?B?SitNNVBqa1pvUG5tcmhubW9OalN4M1JtOUowcUJHeXg2VitiSHlPK2FsenV4?=
 =?utf-8?B?RHl3QmJiL25PVWltVlpmd1ovc256UG5HT2pBZ1lzYmJsa0tmSG8rUTBDNGJJ?=
 =?utf-8?B?b1p6WjU5cENpTllYTXZ2c2lJc0xaUURlSDFZVXkvS2l4WXJZOGNqVVd3a21P?=
 =?utf-8?B?OXdyZ25halh5RVNCVDZjdlFBNTdyNHFHZFRaOUVQL0JtdUd4djU1cHVEblAw?=
 =?utf-8?B?TW1hc0RocXJuVENOTXdWQzJZV0tzeEx0SWFEWGRyaUlQRU5XTlBibzR6S0NO?=
 =?utf-8?B?cUpoZDNySHUwMkYxRTU1NXdweWNlZE5uWEREdzZoU1daZ0Y2bk1uNnMwQUJo?=
 =?utf-8?B?elBSNVViSFJSMVh3eDJha0djSE5MWVZFOHhlQWhzQlFzc2RSU2lrMHFwSjNI?=
 =?utf-8?B?ZzY0Sk5DbHJtbGVjL0M0VW9haEpZQ0hjOFhUSlZza1FGU1RTWjhUUGtOM1FS?=
 =?utf-8?B?NElNTXN0WUZGckR6T3VRZk1tMzNGOE1HVXo4S0RJdXV2MHdmY0d2aFdhQkt0?=
 =?utf-8?B?NW1ldU5taTVYaXdVaTY2R3FjQ0xBQllEOWR2ai9jakg3eU81RmsvVjk4Sjdh?=
 =?utf-8?B?Nnp2WTMyVU53ZmNDWGE2TmRTNEtuRVhRNWxVbHhZZFpwNE1kampLaFFBWnR2?=
 =?utf-8?B?dDNlMGEvZ3F2dFVLb1Z4cUE5TnBjdXdFOXZXNWZ6YzlTcTVqenZMRTlSa0ND?=
 =?utf-8?B?ZStQbU1NdkRkRGxTNjZGTlVZeTJTTWZhbWZIMkM2WnVsWDVndURLM1J5NURm?=
 =?utf-8?B?SW1jL0JzaXFvdHRDYnQzSDg4KzAvUzFPanYwUENGek4ycGVyb0VrSHpCcVdU?=
 =?utf-8?B?aHZiM3k2NXQzUjcydTdRMzFTdDhGdXYyUUtCS1JTTWxPa0RQTzdTQUIxNUsw?=
 =?utf-8?B?bUhqU2hDNElkRGpGZ0N3SWtvUU0rSEdYem4wbS9vcHo2UjBMRGhqUXZMMno4?=
 =?utf-8?B?OG5IdlFqUUxFdHdyUjErNnlndGloMmMwVUdmQ2xSTlJQbVFUZ0VsSVF2L211?=
 =?utf-8?B?UHlQS0NUaUtUUmsybVJBK0ZjNFNyNUVOSGlLOHI5VHNrUnhwd1lIMnp6NWRD?=
 =?utf-8?B?ZERJS2oxbVhPcStBeDJva08yeUR6a3NlZkVvQVBLMlpodEtWSzYvRUk1ZElo?=
 =?utf-8?B?UGE3UzVMNzhEN3Z1Tk9YamRXbzUrbDNnM0xlaDZIVTlXUnd6ckczR3FaYmhx?=
 =?utf-8?B?U01Tb0VLUEtaZjFldHFvNkZtcVR0OGx3VW4vU3kyNmVKakNXME5JQjN1bnBV?=
 =?utf-8?B?anJMKzN6T1pVQjM1aWdteU1ocTZWbGpwRzQvUk8xWE1yMUYwQmRabGp3cjNH?=
 =?utf-8?B?Vms0ZW9LOVJER3JSVjlUUy9HdW0vUnQ3cXBUYWU4UklobTBzYmxuMWFxVDJw?=
 =?utf-8?B?ZUc2MVJSbnVPMC83NXJVQUsyaHViTmRIQXBhaTh2bm1QR21qR2VaM0hoKzly?=
 =?utf-8?Q?U+4Rz2aEEEH2jQGbfD5k4ED/B?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 24565dbf-aa35-4b95-19ea-08db7bd489b2
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5469.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jul 2023 14:48:17.9661
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ER/DdmGEjC3uE9nOTO+GSU1n0vqZqLSDqaZTpTcyvGq2LnTo58P5DZjIuR7Na0h/iC+zHaa5XN2Flj1YaCj5KQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB8417
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 7/3/2023 3:00 PM, Tian, Kevin wrote:
>> From: Liu, Jingqi <jingqi.liu@intel.com>
>> Sent: Sunday, June 25, 2023 10:28 AM
>>
>> There may be a race with iommu_unmap() interface when traversing a page
>> table.
>>
>> When debugfs traverses an IOMMU page table, iommu_unmap() may clear
>> entries and free the page table pages pointed to by the entries.
>> So debugfs may read invalid or freed pages.
>>
>> To avoid this, increment the refcount of a page table page before
>> traversing the page, and decrement its refcount after traversing it.
> I'm not sure how this race can be fully avoided w/o cooperation in the
> unmap path...
Thanks.
Indeed, in order to fully avoid this, need to cooperate in the unmap 
path. :)
>> Signed-off-by: Jingqi Liu <Jingqi.liu@intel.com>
>> ---
>>   drivers/iommu/intel/debugfs.c | 36 +++++++++++++++++++++++++++++++++-
>> -
>>   1 file changed, 34 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/iommu/intel/debugfs.c b/drivers/iommu/intel/debugfs.c
>> index 1f925285104e..d228e1580aec 100644
>> --- a/drivers/iommu/intel/debugfs.c
>> +++ b/drivers/iommu/intel/debugfs.c
>> @@ -333,9 +333,41 @@ static void pgtable_walk_level(struct seq_file *m,
>> struct dma_pte *pde,
>>   		path[level] = pde->val;
>>   		if (dma_pte_superpage(pde) || level == 1)
>>   			dump_page_info(m, start, path);
>> -		else
>> -			pgtable_walk_level(m,
>> phys_to_virt(dma_pte_addr(pde)),
>> +		else {
>> +			struct page *pg;
>> +			u64 pte_addr;
>> +
>> +			/*
>> +			 * The entry references a Page-Directory Table
>> +			 * or a Page Table.
>> +			 */
>> +retry:
>> +			pte_addr = dma_pte_addr(pde);
>> +			pg = pfn_to_page(pte_addr >> PAGE_SHIFT);
>> +			if (!get_page_unless_zero(pg))
>> +				/*
>> +				 * If this page has a refcount of zero,
>> +				 * it has been freed, or will be freed.
>> +				 */
>> +				continue;
>> +
>> +			/* Check if the value of the entry is changed. */
>> +			if (pde->val != path[level]) {
>> +				put_page(pg);
>> +
>> +				if (!dma_pte_present(pde))
>> +					/* The entry is invalid. Skip it. */
>> +					continue;
>> +
>> +				/* The entry has been updated. */
>> +				path[level] = pde->val;
>> +				goto retry;
>> +			}
>> +
>> +			pgtable_walk_level(m, phys_to_virt(pte_addr),
>>   					   level - 1, start, path);
> What about pde->val getting cleared after phys_to_virt(pte_addr) leading
> to all the levels below 'pg' being freed? In that case this code still walks
> the stale 'pg' content which however all point to invalid pages then.
There are 2 cases for the page pointed to by the PTE below 'pg'.
1) the page has been freed.
      It will be skipped after the following check:
                         if (!get_page_unless_zero(pg))
                                 /*
                                  * If this page has a refcount of zero,
                                  * it has been freed, or will be freed.
                                  */
                                 continue;
      Debugfs won't walk further.

2) The page has not been freed.
      The content of this page is stale.
      Dumping these stale content seems to be acceptable for debugfs.

      If all the PTEs below 'pg' can be cleared before being freed in 
the unmap path,
      the following check can avoid to walk these stale pages.
                 if (!dma_pte_present(pde))
                         continue;
> It's probably simpler to just check the format of each PTE (e.g. whether
> any reserved bit is set) and if abnormal then break the current level of
> walk.
Good point.
It seems the bit used to check must be differentiated between valid and 
stale.
But the PTE may not change whether it is valid or stale.

Thanks,
Jingqi
>> +			put_page(pg);
>> +		}
>>   		path[level] = 0;
>>   	}
>>   }
>> --
>> 2.21.3