Received-SPF: pass (google.com: domain of linux-kernel+bounces-209320-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <95005e7c-3705-48c5-8ee2-3d9b0688fcbc@nvidia.com>
Date: Mon, 10 Jun 2024 23:25:38 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 0/5] cleanups, fixes, and progress towards avoiding "make
 headers"
To: Jeff Xu <jeffxu@chromium.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, Shuah Khan <shuah@kernel.org>,
 Andrei Vagin <avagin@google.com>, Axel Rasmussen <axelrasmussen@google.com>,
 Christian Brauner <brauner@kernel.org>, David Hildenbrand
 <david@redhat.com>, Kees Cook <kees@kernel.org>,
 Kent Overstreet <kent.overstreet@linux.dev>,
 "Liam R . Howlett" <Liam.Howlett@oracle.com>,
 Muhammad Usama Anjum <usama.anjum@collabora.com>,
 Peter Xu <peterx@redhat.com>, Rich Felker <dalias@libc.org>,
 linux-mm@kvack.org, linux-kselftest@vger.kernel.org,
 LKML <linux-kernel@vger.kernel.org>
References: <20240608021023.176027-1-jhubbard@nvidia.com>
 <CABi2SkVoNyXLrfU71gnv1qVUAADpUmFXiDoKKPc54MLb5JpB+Q@mail.gmail.com>
 <a8f9d4e2-8a12-4e5d-bd22-9c92955135f4@nvidia.com>
 <CABi2SkU8=tjWhD-e=OdiVr+YeU+BZZLB_vMfkNb-VWpbP2xcng@mail.gmail.com>
Content-Language: en-US
From: John Hubbard <jhubbard@nvidia.com>
In-Reply-To: <CABi2SkU8=tjWhD-e=OdiVr+YeU+BZZLB_vMfkNb-VWpbP2xcng@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?c0s2R3pzZU1oMHVqNHB3eXBoaTNLTXZwRjVFekdWeWZpVmgrMmZkRWFjL3BD?=
 =?utf-8?B?RWQ0ckc0dWthSkVVZU5hanBJRlFEa1ZmbmgzbVhxaTlMZFBabjRpNExwa21O?=
 =?utf-8?B?V2pzM2tPaXN1bVZvcm1yWm1rTXdwNTVmK2hQS04za2lVZkpYMHkybmlHNld4?=
 =?utf-8?B?SzVKY2tzcEdhTG1DRndsZnRHYkljREdNQzkxb3hhZVkxeGVwZy92a1FNMG5R?=
 =?utf-8?B?dmNoSHhCMFdRWmlub2JraTgrWWFHSGg4NG5LWUExWkE0TlI3cC9Zd3FIcGpy?=
 =?utf-8?B?RW5ybU4zSHdxeTcwR20vQ3UycVJObEhoVFphYzhhQW1EbEFrb0ZnSE95dU5S?=
 =?utf-8?B?aWpMNC9HTnBqNlk0cUFvbFNSbkRTVnZnTWRBbDNNVHdMVUFvVDdsUFo1U2w2?=
 =?utf-8?B?dCt2Wk9iSEhBMHNjTmNGMlAzRXZlbWd3Vkx4ZXpZUGRDQzViWkFYN0NYZ1d1?=
 =?utf-8?B?cU0rYVF1YlQ2YTZOYnVXeE9kVzVUSElwK2JhcS9hMnVsZitDLzVXakE4cThL?=
 =?utf-8?B?a0ZQSGtDNHFaVEdRVDFCbm1RV0ZHUk5Sa1dwaHpodW1SMU1LZjc2djdURHcw?=
 =?utf-8?B?K3dNUE41Zm94RGY4SmdlOWl2a1ozSnFPN1dpUWlNaVpHSC81NjVabmk0U1NI?=
 =?utf-8?B?dXFSWjU0bnZmNWlRZVU0R1hueHdTTGpsckxwWnp2TmRsTWhoL3pQMUhHMHVH?=
 =?utf-8?B?OTZDZkwrd3N5N3ByR1Q5SVJZRjN0NG45VXBJV3lXaFhTVnNSZlpBTVI4bVJH?=
 =?utf-8?B?U1VERTBCVnU3N2k3NFRpNUE4dUV5d0ZuQVZqVGlocXdlVkNNdDhEaDdYY2da?=
 =?utf-8?B?Qi9tVFV4VXQwalJJZ0NqWEpDTFVsR0ZUdWhXU2ZEaVNQTUpoVlhOekZXWERJ?=
 =?utf-8?B?MGdkN1U1MUdpajFkcW1hc1NKR2VjZkkzaXE3MzhSUDlIbFVJR250ZTlpNk03?=
 =?utf-8?B?NTlpaEVqcEdSck1WWHFTbXA2ZVlRdWlvUERDUnVZc2Z6VmQyNWNIcnBRYkNO?=
 =?utf-8?B?ajVWUUY1QzBsTjlMcktCbzBqcDFGUTJvTldtYXJiTk9CWmg0NzVkT1BRZnMv?=
 =?utf-8?B?Mk9FZHdncEJFVEZCVDhzVThnVUdyd1g1K0ZsZ3BMRkM0N2lHVjU3Vit1cHhy?=
 =?utf-8?B?VWFPMUorQWhUSFQySTNwdEVlcWw2UkdTSmtQTlQzZzBNcFdJVk5BSVE0emxt?=
 =?utf-8?B?di85K1pPVXA1UWxOWXA3aVJ0SDNCUFVFZ3N5WjNJVWFzcjFUeDlRdlZQTHdL?=
 =?utf-8?B?NkdHcXJ2eE9rVU96TUpXZi84b0NXYktXcEhzcVVzQXVEL1BYK0ZmalgwU0lv?=
 =?utf-8?B?T2tPVEtQZWJVeW0xUm5iRGNOYTFLcmtCeEVKQkp0UHR2N2tVOGdjbDlIT0xm?=
 =?utf-8?B?dDZwZlpTMXlHd3RNR0JrcWFwNmY3Wk5FcXM2anNQWC9PK25NTm4xb3Q4VHFY?=
 =?utf-8?B?ZGlaamtIc2ZBNFhkM1dhTlBDZEliV0loTDZzM3IzUDBib1BEdGpuQ3lwTXV0?=
 =?utf-8?B?MTE1THRiQ1RlTk9HMFh0ZjhabHpBeXNoOWg1b0loYy9KRklWbUo0OWZ0M3Jh?=
 =?utf-8?B?eEVPa2sxUlRNeU9nNldsV1BONzNnd0VSMkJXZ2hLbUd0Tmxpbnp5VjhJUEc4?=
 =?utf-8?B?S0U1bjVwRjlMRGlVYmF4ekxRYTRhVFNTbUdPSWg1dFE1Z25XTkh3eDM1MFh5?=
 =?utf-8?B?bER0czArM1ZLd1ZMZGp6bUIyOGxLVE9qSmh0bFMzZmxhVG50eFN4WTllNU5S?=
 =?utf-8?B?L21FeklVdDRLNklNMGlJOTg2WW5SN2d6Um0wa1dpWXByTHVVdjhnUkpmM1Ra?=
 =?utf-8?B?bkZ3U0huMU1KTWNTeGlVWmdJZldMUitkb0h0dy9KL3JFV2VhZFYxclZwUDJY?=
 =?utf-8?B?dFVpV0V6dllpeUxLc0k4OUtQTTNNek9zSysvMXNSSnI5WUE4blBiUFE2c0xo?=
 =?utf-8?B?anN0Mys2MFF0UytqaDdSOEQzb3I4NEFOcDZrLzFjL2l3MG1MTzdBNWN3NmpU?=
 =?utf-8?B?MkV5QXFKSHNpVGxOV1lpQ29FVER3eWxIbUZGS2VieFZLWFJjeDBIQlM2alpC?=
 =?utf-8?B?NXVWOHo3cnFEd3k2K1NuaG10eVlPMG1VMk9jT0xrZTI5NzVsalRibU1KRDBF?=
 =?utf-8?B?Mm5wcS9OWllhS1U0Nm13cGZmOFBuRndhMW9HS3VmVUZOQTllQXl4dlVzMXI2?=
 =?utf-8?B?MVE9PQ==?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a10cef8e-ebb3-4788-74ac-08dc89df6280
X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB4130.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2024 06:26:10.3887
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OswYvcYkM5mHMpLzGO9xi+pO8216rS+xG0GcbKGM1SqkvCQo3+V5Alfwcp5K4xqIbq1VQv/3tKW4d7BfjisalA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9119

On 6/10/24 9:45 PM, Jeff Xu wrote:
> On Mon, Jun 10, 2024 at 9:34 PM John Hubbard <jhubbard@nvidia.com> wrote:
>> On 6/10/24 9:21 PM, Jeff Xu wrote:
>>> Hi
>>>
>>>
>>> On Fri, Jun 7, 2024 at 7:10 PM John Hubbard <jhubbard@nvidia.com> wrote:
>>>>
>>>> Eventually, once the build succeeds on a sufficiently old distro, the
>>>> idea is to delete $(KHDR_INCLUDES) from the selftests/mm build, and then
>>>> after that, from selftests/lib.mk and all of the other selftest builds.
>>>>
>>>> For now, this series merely achieves a clean build of selftests/mm on a
>>>> not-so-old distro: Ubuntu 23.04:
>>>>
>>>> 1. Add __NR_mseal.
>>>>
>>>> 2. Add fs.h, taken as usual from a snapshot of ./usr/include/linux/fs.h
>>>> after running "make headers". This is how we have agreed to do this sort
>>>> of thing, see [1].
>>>>
>>> What is the "official" way to build selftests/mm ?
>>
>>   From Documentation/dev-tools/kselftest.rst, it is:
>>
>>     $ make headers
>>     $ make -C tools/testing/selftests
>>
>>> I tried a few ways, but it never worked, i.e. due to head missing.
>>
>> You are correct. Today's rules require "make headers" first. But
>> I'm working on getting rid of that requirement, because it causes
>> problems for some people and situations.
>>
>> (Even worse is the follow-up rule, in today's documentation,
>> that tells us to *run* the selftests from within Make! This
>> is just madness.
> 
> That is hilarious! :)

:)

> 
>>   Because the tests need to run as root in
>> many cases. And Make will try to rebuild if necessary...thus
>> filling your tree full of root-owned files...but that's for
>> another time.)
>>
>>>
>>> 1>
>>> cd tools/testing/selftests/mm
>>> make
>>>
>>> migration.c:10:10: fatal error: numa.h: No such file or directory
>>>      10 | #include <numa.h>
>>>         |          ^~~~~~~~
>>> compilation terminated.
>>>
>>> 2>
>>> make headers
>>> make -C tools/testing/selftests
>>>
>>> make[1]: Entering directory
>>> '/usr/local/google/home/jeffxu/mm/tools/testing/selftests/mm'
>>>     CC       migration
>>> migration.c:10:10: fatal error: numa.h: No such file or directory
>>>      10 | #include <numa.h>
>>>
>>
>> Well, actually, for these, one should install libnuma-dev and
>> numactl (those are Ubuntu package names. Arch Linux would be:
>> numactl).
>>
>> I think. The idea is: use system headers if they are there, and
>> local kernel tree header files if the items are so new that they
>> haven't made it to $OLDEST_DISTO_REASONABLE.
>>
>> Something like that.
>>
> But I don't want to install random packages if possible.

Well...keep in mind that it's not really random. If a test program
requires numa.h, it's typically because it also links against libnuma,
which *must* be supplied by the distro if you want to build. Because
it doesn't come with the kernel, of course.

So what you're really saying is that you'd like to build and run
whatever you can at the moment--the build should soldier on past
failures as much as possible.

> 
> Can makefile rule continue to the next target in case of failure though ?

That could be done, in general. The question is if that's really what
we want, or should want. Maybe...

> right now it stopped  at migration.c , if it continues to the next target, then
> I don't  need to use gcc to manually build mseal_test.

Let me take a peek at it in the morning.


thanks,
-- 
John Hubbard
NVIDIA