Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.117.160 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C
Message-ID: <8f3bb4a8-80cd-547e-df65-2d17827362f0@nvidia.com>
Date:   Wed, 14 Jun 2023 16:27:18 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: [PATCH] selftests: lib.mk: fix out-of-tree builds
Content-Language: en-US
To:     Anders Roxell <anders.roxell@linaro.org>
CC:     <shuah@kernel.org>, <linux-kselftest@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <usama.anjum@collabora.com>
References: <20230613074931.666966-1-anders.roxell@linaro.org>
 <928ad485-2dc2-2e02-98a2-fa203441f463@nvidia.com>
 <CADYN=9KtYe6jo+MDm+3NVwfNFzsOgoRRKxu_hSJeEPMuLRGgeg@mail.gmail.com>
From:   John Hubbard <jhubbard@nvidia.com>
In-Reply-To: <CADYN=9KtYe6jo+MDm+3NVwfNFzsOgoRRKxu_hSJeEPMuLRGgeg@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2023 23:27:34.3657
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c22d91b0-fa10-4599-3627-08db6d2eeec3
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D5.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7176
X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 6/14/23 14:16, Anders Roxell wrote:
>> (It's too bad that we have all this duplication between the Makefile
>> and lib.mk.)
> 
> I agree, also the duplication of get_sys_includes in bpf, hid and net Makefile's
> Do you have any idea how we can remove the duplication?
> 

Well, since you asked... :)

Yes. Given that the goal is to support two similar, related activities:
	a) make -C tools/testing/selftests
	b) make -C tools/testing/selftests/<subtest>

, we could move common functionality into common_vars.mk and
common_targets.mk, and include those from both the top level
tools/testing/selftests/Makefile, and the subtest Makefiles.

This allows deduplication in a fairly safe sort of way, because it also
simultaneously moves everything to a non-recursive Make design. That's
especially nice, because then dependencies work reliably, and you also
don't have to worry about propagating Make variables (via Make's export
directive).

I haven't done this, because a few years ago I somehow got the vague
impression that it would not be well-received. But maybe that's
pessimistic and/or obsolete--times change, after all!

***************************************************************
Contents of tools/testing/selftests/Makefile:

include common_vars.mk

include alsa/alsa.mk
include amd-pstate/amd-pstate.mk
include arm64/arm64.mk
...etc, there are 100+ subdirs

include common_targets.mk

(This is instead of doing "TARGETS += xxx" in the Makefile. Including
the subtest *.mk fragments allows each subtest to add features and
quirks as necessary, while also enabling full dependency checking via a
true non-recursive Make.)

***************************************************************
Contents of tools/testing/selftests/common_targets.mk:

all: kernel_headers
     ...

Other rules/targets here.

***************************************************************
Contents of tools/testing/selftests/common_vars.mk:

Common variables, some of which may be overridden by a Makefile.

***************************************************************

Using alsa as an example of a subtest:

***************************************************************
Contents of tools/testing/selftests/alsa/alsa.mk:
TARGETS += alsa

TEST* items

Plus: alsa-specific items, such as:
CFLAGS += $(shell pkg-config --cflags alsa)

***************************************************************
Contents of tools/testing/selftests/alsa/Makefile:

include ../common_vars.mk
include alsa.mk

...maybe very little else needs to go here.

include ../common_targets.mk


thanks,
-- 
John Hubbard
NVIDIA