Received: by 2002:a05:6358:7058:b0:131:369:b2a3 with SMTP id 24csp7144162rwp; Tue, 18 Jul 2023 10:39:08 -0700 (PDT) X-Google-Smtp-Source: APBJJlGO7YMmXhNjytjmtCHKC/qzljJMR+k1voY4Uc3A8tf3uX/TLfsTktn8LV42vkXM/HT0Xe+1 X-Received: by 2002:a05:6a20:8e28:b0:133:6e3d:68d0 with SMTP id y40-20020a056a208e2800b001336e3d68d0mr19354pzj.4.1689701948093; Tue, 18 Jul 2023 10:39:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689701948; cv=none; d=google.com; s=arc-20160816; b=OB9cG3DtsdregkaiUryntbDDjkCJetZWdaD1XcRIQFZ9Us+VFZyL+UielbUBVhoZlw 7zWfhxlYWsyBKIidwmvUHBfKcyLi/R+5KA3UeBdANiwQsBLzOHUFvIB7NNTSVgAOUSO2 GIXbUvzHrZxBLP5K17toHeOtgYTJSO15Ugs7XLPOgdDHiYKUz7sXcO0cpkzuqWVxwWGv fELIyaI6PmD+byWGZbaodaJybvUItSDWGDbGLaGG3yTl0LdpI32g1x19ZVvGCSq95jdh q78qbzt1Vwi/p+5Ic2hH7wwhwv1xH5jrxleamO4oKnrRK89it6A+CWhFBK2pn8IToJON EUcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-transfer-encoding :content-disposition:mime-version:references:message-id:subject:cc :to:from:date; bh=64/Uc9ruzOH0UyhDpciHP9j3b1UFP48s0HTTf506FKU=; fh=/H3TOV2Vqd8suSSLIVInxGoUdD8EPsrSGxPKrmKw31w=; b=ABZZHcs/6vNOFZkyB7BPjkUb1y77zTKtzuAkMKQN+fKi2Q7Bf3JPuXMixknur4YQ4r T+O2d5a4x/jngWWkI1IYJbS+3pZ3KD2G/Yp6zqHnDWoeuXoNoMzygS3megMWRjLCD1h6 wfidqZUOjpVViZQQni9icLwXClSzyxZJWrwdY32wsViHF9QzfK9zb1eRd5IlXOjSaDx2 cD0EFA+P6hMIKaiwlC9qeLXgIZXuJxpYbc5AM9r5H9TYrstQx7S0iUZVYy+Gfs1VeLeZ MdZtncm3ponAwtjZKwbsFTjNKyGizr4An44o9MSWfl/8NuYH0LIfZsM+nbNpXK+bQTOa XVBA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g130-20020a636b88000000b0055c96379d01si1912857pgc.705.2023.07.18.10.38.54; Tue, 18 Jul 2023 10:39:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232351AbjGRROj (ORCPT + 99 others); Tue, 18 Jul 2023 13:14:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231582AbjGRROh (ORCPT ); Tue, 18 Jul 2023 13:14:37 -0400 Received: from 1wt.eu (ded1.1wt.eu [163.172.96.212]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id D6F3C1710; Tue, 18 Jul 2023 10:14:31 -0700 (PDT) Received: (from willy@localhost) by mail.home.local (8.17.1/8.17.1/Submit) id 36IHENeq014396; Tue, 18 Jul 2023 19:14:23 +0200 Date: Tue, 18 Jul 2023 19:14:23 +0200 From: Willy Tarreau To: Thomas =?iso-8859-1?Q?Wei=DFschuh?= Cc: Zhangjin Wu , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: Re: nolibc: KTAP output and test reports Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_PASS, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Thomas, On Tue, Jul 18, 2023 at 09:31:06AM +0200, Thomas Wei?schuh wrote: > Hi Willy, > > On 2023-06-08 00:15:27+0200, Thomas Wei?schuh wrote: > > after your recent discussions about the test output and report I > > wondered if it would make sense to switch nolibc-test to KTAP output > > format [0]. > > > > With this it would be possible to have a wrapper script run each > > architecture test as its own test subcomponent. > > A (K)TAP parser/runner could then directly recognize and report failing > > testcases, making it easier to validate. > > > > Also maybe we can hook it up into the regular kselftests setup and have > > the bots run it as part of that. > > > > The kernel even includes a header-only library to implement the format [1]. > > It also should be fairly easy to emit the format without a library. > > Did you have a chance to look at this? > > If you are not categorically opposed I would create a proof of concept > for further discussion. I remember I had a quick look but had no opinion about it: it's not very clear to me how this will be consumed because I don't know the tools around and cannot invest time learning them. If you think it can bring some value without complicating the usage, maintenance and contribution of what we currently have, maybe let's give it a try. But I would like to be sure we are careful about preserving the current ease of use, as I'd hate to have to go through the makefile to figure how to get back the simple output format that we can trivially read or grep without any extra tools. That might for example mean that we'd need an option to change the output format (and I think it's possible because most of the outputs are done inside wrappers). But in any case, feel free to explore, experiment and share your findings. Hoping this helps, Willy