Received-SPF: pass (google.com: domain of linux-kernel+bounces-121472-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <51f493a9-08e7-44d8-ae4a-58b2994ea276@oracle.com>
Date: Wed, 27 Mar 2024 14:59:44 +0000
Subject: Re: [PATCH] selftests: iommu: add config needed for iommufd_fail_nth
To: Jason Gunthorpe <jgg@ziepe.ca>,
        Muhammad Usama Anjum <usama.anjum@collabora.com>,
        Nicolin Chen <nicolinc@nvidia.com>
Cc: kernel@collabora.com, iommu@lists.linux.dev,
        linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org,
        Kevin Tian <kevin.tian@intel.com>, Shuah Khan <shuah@kernel.org>
References: <20240325090048.1423908-1-usama.anjum@collabora.com>
 <31fcc276-acd6-4277-bd6c-4a871c7fb28a@collabora.com>
 <20240326150340.GE8419@ziepe.ca>
 <56cc8b9e-c1cf-4520-ba45-b1237e8b7b64@collabora.com>
 <20240327114958.GG8419@ziepe.ca>
Content-Language: en-US
From: Joao Martins <joao.m.martins@oracle.com>
In-Reply-To: <20240327114958.GG8419@ziepe.ca>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?bHdUTEZXZjNkN20vemNibWZaV2pOR3ZPM2l6Zks1bFVGbnR4bWQyMWJVVXNO?=
 =?utf-8?B?OEt5UllxVkhSNmhFcC9yOGFLNXlLZHRoN1RDUStia1dmTlZ2Y3U5RDNqbkNV?=
 =?utf-8?B?QTQ4UlI3dzlrUU5MYWFTcXJFUkdsYm1FeUR1WXNhTDNnY1J1RVdjdUFUNkNO?=
 =?utf-8?B?RFlDOVpPNlljcExEZkZXdml3blFSdGowTWlRRVVRMEpGK0paa2VHek9lRWNk?=
 =?utf-8?B?TERZRlozVlhFNmFhMlZBZlZIN3ZOTjhtZzdUU1pBMitEeFBYRkZVYUNnVDZ2?=
 =?utf-8?B?aElMR2c4czdnbGxlWTBZcTk2ZlFnajlvTGp2bHJsbUQrQm1vMFVudFdsLzJB?=
 =?utf-8?B?NHhzYUVPem5zL1lZTWxGQXZVZ0N1MVA3eW9SYXMrOWg0OGNTUW9neXNSeUNk?=
 =?utf-8?B?SERMNzdGUEd4Y2ZicTZhM0xOTGc2NDZtK2hLQjZSRU5FNGdVeHk0MFpvdnFx?=
 =?utf-8?B?Ylc2bGFJRGtIUFo0QlhPRkt2OE5KaVdGOHE5ZENjYjVDdzZQaXBnVjFZZ0R4?=
 =?utf-8?B?YVQ2anllV2c4UlpSTTE5dFU1NkJTbDV5T3l2b2tTWFl2TUxKS3krRERsRk56?=
 =?utf-8?B?MWp4YUF0V1lRazRTS05XWUlIRjExeXZZMWxQTkZBWi9ZV2RlMmpLUzJlQnUz?=
 =?utf-8?B?ZDFoWUdqVWFGRWVYamdSL3FaQWVUeVZlR3VRUTN6amdwdVdWUGhQTHBNaGMw?=
 =?utf-8?B?emdodVFML3hjOGllVmMrVlRxQyswT29KaWR5WEgzZnF6ZSs3MCszZkQ4RTJ5?=
 =?utf-8?B?cXd6R1pDWG9oU2tLaUlMK0VGNjdzTXVWQjNRNk5UZlhNakF0dWV2bTAwVXEy?=
 =?utf-8?B?bWxlbm9WdGxWaHBqOUZUbFJDUFJJYW1waFpDbXRsVDMzUkZqUDRFUjJqMWJp?=
 =?utf-8?B?VVNwWENjSzd2SmRQSW1saWt2aXl0TWxSSjZoTHdMN3dLOGt1bkE3bHlIbVVP?=
 =?utf-8?B?c3gwUmtkL3dWbzI1VDZtREh3TUVqelZPN211WGU3czZ5UFYyR1hNeGQ2SUYx?=
 =?utf-8?B?ZW5DQnBDclZEUmpYT0UzWE9rWUlIeW5SVVlEUVhWenR3STJ2RlpzZGlPbFNW?=
 =?utf-8?B?Ym5ObEd0ZlE1NkZEbEJpRWJ0SThQMEZYdVZsVFY5ZkRCUDhoWGNZZ0h2dlRL?=
 =?utf-8?B?aWJ1ZnpwS2RoejJYWFlWMmE3UDRxQWdobTQwNnFkZ3haWUo4VE9FejgwaWdC?=
 =?utf-8?B?OWUySURBcFlXSWo0aUFsREhXS2JWZVZMU0EvUHo3VXF1RkRRaXVmRUdZTnhx?=
 =?utf-8?B?aFNWMU9EcG9sNFZMNGIxTzJOdnBCeGN4VFFVRTRnZmVWMENQTWk1bFJDMzFw?=
 =?utf-8?B?TXdialdKRUdLU2I2Vk5KTFlNcTJSVkNCNEdZTHI0eHBNVWgvdzlEelo0cEZM?=
 =?utf-8?B?NXpSODNyVkhPUjZzYTY3Z3VDMHcxV2k3blJWN1ViYU9mVzkyQms3cTVUbFJP?=
 =?utf-8?B?ZWJtcUpEM0x3R3hrNEV5ZjVrWm9ramdNUk9yYzk0R2x4QzNOU2pTWlZxQWla?=
 =?utf-8?B?dVlUV2ovWXkrU2RYbkhwNHMvaWE2VUVhZnNNa2lpSTQ1RkhJOXc5TktwN2FD?=
 =?utf-8?B?akRXWmxiU1B3dTljQnBvUFI1TG1yRllEQUVsdS8yTWNSY1QxTzhBQ0Z3eEV1?=
 =?utf-8?B?SldTNWdyRmd0N2JxYzU4bldybGJvOVNtL3ovclY1NFBuZWhzTnBBb29Qckk1?=
 =?utf-8?B?QmhFR3hLUkdrazgzejF2cFI5WGM2amdtbWwrSUd0dU92UzJ1anJyWDlRYzdq?=
 =?utf-8?B?K0JGcXczKzZ6UHdaNGZLRSt0WEp5ZWE4M1dKd1ZiVGpycjNMR3FYYTlXZmEz?=
 =?utf-8?B?SnFPR2tSZlBiZWg1ZXFPQWhESDNpaGtNeGNRN1lUaFpacDZLRjZ4ZU9LRkZm?=
 =?utf-8?B?bVlYVndjUlY5VjQwMDlPSk1IbVNRWEdLRFB5VVRYdjUzekU4dDNYYmphVjRY?=
 =?utf-8?B?Wi9ER21DWTFHNEZqQkdHYnE2T0hnK0gweUpmYWpBL1NwVnljR2FEejdyWEFm?=
 =?utf-8?B?WDhIL0ZDeGl1SUVyL2lhU3BoblU3MjdHcTkyNVBadElKMEtLdE81RmVCeGF0?=
 =?utf-8?B?QVdWRGdyVmNZaEI2R21ZN1FYYzlYNnJjcnJ5TStsUU55akltd1pPRmdxMGV6?=
 =?utf-8?B?TkpqcXQxUUhQbVlZQlA1SlVNOWNobFVuWmRMSUxFRzNEeHkzdmJQMWQxS2Qw?=
 =?utf-8?B?UVE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	hQVA1Y8hg9Z47jIZVFvMfZpE4Pnazk9FZuNinJDLrhw4ItqFxP5CLKntXJBKT6cPIdte/E9jBEr1zg6Tps/rWoA46sTZ4HcJSjjoVWNnmmb/XEumh5ab6lcW+Qp0SYnUnggnjfq2EEhD3oUtVNf8N50nrXVvkM0vLjU+x7LVCMbolyuga8nAWPxhgn7xrTZnTXn1H4VF8vc6u0oY7vhpIYLuixtH3CIpoo1+tTJi6374NagdzIUhL7YJVVNPYWG14614q6UY0r6khCAORncbNa2AAlDAKzYw1DBmAToGtrsyY80hu0JuwBjKc+afAxbqg8HarFaLwhXwyz8uv9aP4yTzkzjK8GKJIe7WNTEoNlpHpzFiaNGnTzaGDHe7t+UxQxSxtyY/QctIoUt7D0nK27IwUsZF08yqrcd4LTt8yIKyHRXw0uvoLf2v5e3A2gyjF7g+PMhLADQTBHAstruitbQGJZuv1kN+sa/Bqwe8s4PKoCPNG7MUNxzXifPtp/X2WerL2pBWEsEWJR9ghcsuNBv6LV/tOIhIldqMtqqjYQmMbDLtGUpRrHbBQ6MsRz3qp9LwnHNFQqCtMHAxSnhEmR0l2Q9gWYKBA0u71/JPVJM=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f109ccfd-9762-48b7-6ae7-08dc4e6e8d12
X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5893.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2024 14:59:50.2242
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wSPCQGBNcZxRRAQFgOKiy/NBkNf/HbomxyzmDUS11B3xHWx2ozZBy3IXcV4sUvJGdcaGINukZHW5ELdQBX1OZYljM1q1NbbEZFCb+TmY0QA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4724
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-03-27_12,2024-03-27_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 suspectscore=0
 mlxlogscore=900 bulkscore=0 spamscore=0 adultscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2403210000
 definitions=main-2403270102
X-Proofpoint-ORIG-GUID: 485qmAULmT2aNToLU_6nA_Pk75kfIPwl
X-Proofpoint-GUID: 485qmAULmT2aNToLU_6nA_Pk75kfIPwl

On 27/03/2024 11:49, Jason Gunthorpe wrote:
> On Wed, Mar 27, 2024 at 03:14:25PM +0500, Muhammad Usama Anjum wrote:
>> On 3/26/24 8:03 PM, Jason Gunthorpe wrote:
>>> On Tue, Mar 26, 2024 at 06:09:34PM +0500, Muhammad Usama Anjum wrote:
>>>> Even after applying this config patch and following snippet (which doesn't
>>>> terminate the program if mmap doesn't allocate exactly as the hint), I'm
>>>> finding failed tests.
>>>>
>>>> @@ -1746,7 +1748,7 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
>>>>         assert((uintptr_t)self->buffer % HUGEPAGE_SIZE == 0);
>>>>         vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
>>>>                    mmap_flags, -1, 0);
>>>> -       assert(vrc == self->buffer);
>>>> +       assert(vrc == self->buffer);// ???
>>>>
>>>> On x86:
>>>> # Totals: pass:176 fail:4 xfail:0 xpass:0 skip:0 error:0
>>>> On ARM64:
>>>> # Totals: pass:166 fail:14 xfail:0 xpass:0 skip:0 error:0
>>>>
>>>> The log files are attached.
>>>
>>> You probably don't have enough transparent huge pages available to the process
>>>
>>>       echo 1024 > /proc/sys/vm/nr_hugepages
>> After making huge pages available, the iommufd test always passed on x86.
>> But there are still failures on arm64. I'm looking into the failures.
> 
> Oh that is really strange. Joao? Nicolin?
> 
Definitely strange, I'll have a look.

So it set the expected number of dirty bits as that assert doesn't fail, but it
is failing when we check that even bits are set but not odd ones. Like it's
hasn't set those bits.

For mock tests there should be no difference between x86 and ARM assuming the
typical 4K page-size. Maybe this is 64k base pages in ARM? That's the only thing
that I can think of that affected mock domain.

Muhammad, could you paste your kconfig?

>> #  RUN           iommufd_dirty_tracking.domain_dirty128k.get_dirty_bitmap ...
>> # iommufd_utils.h:374:get_dirty_bitmap:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty128k.get_dirty_bitmap
>> not ok 139 iommufd_dirty_tracking.domain_dirty128k.get_dirty_bitmap
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty128k.get_dirty_bitmap_no_clear ...
>> # iommufd_utils.h:374:get_dirty_bitmap_no_clear:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap_no_clear: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty128k.get_dirty_bitmap_no_clear
>> not ok 140 iommufd_dirty_tracking.domain_dirty128k.get_dirty_bitmap_no_clear
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty256k.get_dirty_bitmap ...
>> # iommufd_utils.h:374:get_dirty_bitmap:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty256k.get_dirty_bitmap
>> not ok 144 iommufd_dirty_tracking.domain_dirty256k.get_dirty_bitmap
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty256k.get_dirty_bitmap_no_clear ...
>> # iommufd_utils.h:374:get_dirty_bitmap_no_clear:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap_no_clear: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty256k.get_dirty_bitmap_no_clear
>> not ok 145 iommufd_dirty_tracking.domain_dirty256k.get_dirty_bitmap_no_clear
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty640k.get_dirty_bitmap ...
>> # iommufd_utils.h:374:get_dirty_bitmap:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty640k.get_dirty_bitmap
>> not ok 149 iommufd_dirty_tracking.domain_dirty640k.get_dirty_bitmap
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty640k.get_dirty_bitmap_no_clear ...
>> # iommufd_utils.h:374:get_dirty_bitmap_no_clear:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap_no_clear: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty640k.get_dirty_bitmap_no_clear
>> not ok 150 iommufd_dirty_tracking.domain_dirty640k.get_dirty_bitmap_no_clear
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap ...
>> # iommufd_utils.h:374:get_dirty_bitmap:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap
>> not ok 159 iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear ...
>> # iommufd_utils.h:374:get_dirty_bitmap_no_clear:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap_no_clear: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear
>> not ok 160 iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty256M.get_dirty_bitmap ...
>> # iommufd_utils.h:374:get_dirty_bitmap:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty256M.get_dirty_bitmap
>> not ok 164 iommufd_dirty_tracking.domain_dirty256M.get_dirty_bitmap
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty256M.get_dirty_bitmap_no_clear ...
>> # iommufd_utils.h:374:get_dirty_bitmap_no_clear:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap_no_clear: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty256M.get_dirty_bitmap_no_clear
>> not ok 165 iommufd_dirty_tracking.domain_dirty256M.get_dirty_bitmap_no_clear
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty256M_huge.get_dirty_bitmap ...
>> # iommufd_utils.h:374:get_dirty_bitmap:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty256M_huge.get_dirty_bitmap
>> not ok 169 iommufd_dirty_tracking.domain_dirty256M_huge.get_dirty_bitmap
> 
>> #  RUN           iommufd_dirty_tracking.domain_dirty256M_huge.get_dirty_bitmap_no_clear ...
>> # iommufd_utils.h:374:get_dirty_bitmap_no_clear:Expected j < npte (1) == test_bit(i + j, (unsigned long *)bitmap) (0)
>> # get_dirty_bitmap_no_clear: Test terminated by assertion
>> #          FAIL  iommufd_dirty_tracking.domain_dirty256M_huge.get_dirty_bitmap_no_clear
>> not ok 170 iommufd_dirty_tracking.domain_dirty256M_huge.get_dirty_bitmap_no_clear
>