Received: by 2002:a6b:500f:0:0:0:0:0 with SMTP id e15csp960434iob; Fri, 13 May 2022 17:47:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxtKnG+lMrofOkp9Yu9yPsq+Q/D5rZ91rQsJQLjsaNAjqDcFguhsVkD1sNWlvdSSddbaTcy X-Received: by 2002:a05:600c:4e06:b0:394:56be:19c with SMTP id b6-20020a05600c4e0600b0039456be019cmr16932034wmq.168.1652489255115; Fri, 13 May 2022 17:47:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1652489255; cv=none; d=google.com; s=arc-20160816; b=WaToa4YftihEjIGplxoEe2hz7JjdSjOGaODN+YpKHZwb92nwE7fWj2qEixC63a/VCC zYDBhEPjkvoTsMas3+y7obewiruqp4k1GKKlLou8BdW+GAb9jnORVYtZq+KtGtQmy4pt KWlW1QIj+/s8Iigo3LouV40S46rHxmkRa1FSTnUiD+6uNT3q8ObCGPc8lzd7vW5DpfYA w6XMAUyhvlruaGKaemawJWIeCeGmRebElf7Giz8R18LkNTq0mRyM5tFpnMrz7WViHGHk e8P1qZ5nbOSHyhZkTLsLY57ZZjKVVRcwW0fe67FbUEKnO4/AoNHleWJxk+pTu6XJQtoz 2IIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=rF5TwgxqcKOTnH6e3JjjcZnpXw1SrE4T3AtfEU9WAmA=; b=joJH1u3agkwA3xk8C0GCKLg8dWRP2/SIuIOVi27yPX6zW+zTMjumX/yeOlHFha9kjD zSPWZ6+E4BJ8KyZ95bQBOQrr//V2JXnBJQubpr28lD6iC8snCmG8bexr3tlCCe9loHxG JSsd93/2hSuiH4EH8oLkgOMAJzoc5iDLPMjllTwj+Ukb+5Jiv29eb+v0RjSog9IdGklq yCbCwwi63XnM+q8kmObZ+I9wfjJg0cBM8N9UFIj3u5X2mVIi9hevjFLu73L/L5AoZ7jf Rtb+DMKXvhHqxjZ8dq/tsXMYH2OHNkEoF8wlwOKXUm+nOodaPvOMGrUP4EMGjJ7ioIAg JJrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=P4dnIDAH; spf=softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [23.128.96.19]) by mx.google.com with ESMTPS id a10-20020a056000188a00b002060d0fe9acsi4300448wri.526.2022.05.13.17.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 May 2022 17:47:35 -0700 (PDT) Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=P4dnIDAH; spf=softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5521C337577; Fri, 13 May 2022 16:26:47 -0700 (PDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1357145AbiELRxo (ORCPT + 99 others); Thu, 12 May 2022 13:53:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344469AbiELRxm (ORCPT ); Thu, 12 May 2022 13:53:42 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABEB424D617 for ; Thu, 12 May 2022 10:53:41 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id m20so11684660ejj.10 for ; Thu, 12 May 2022 10:53:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rF5TwgxqcKOTnH6e3JjjcZnpXw1SrE4T3AtfEU9WAmA=; b=P4dnIDAHRA//qYjQW/VtuRT1f+AWqvasomN7glq/CP85sbLSclvxORqzKSRJM1nU2n YuEZ0fA5uFsuc8M0MyjCWNI16kSGNp1Bn0JaPlMXkXq9uJhFaAqfW9+1XVsuBt9B1FZD FkscgFHN5nMwNd8rQfi8GxrRbWQTg0pReDu0V+PLnpBTT9hNN1edF876C646DmZv+KkA zC4D3lga+dg51lx5QdUOr2seNJYbzMVa4dGQTr3qy1zY6MmkzUn6meRL0VV3P7P4m1EG 0adXneAez5QIHnDMyZORoVOqznwj4GMwROiJnTlAdfC3EVjNrHAL/GQAWinypgQeeSPU Q8fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rF5TwgxqcKOTnH6e3JjjcZnpXw1SrE4T3AtfEU9WAmA=; b=kgpWI+42vexhj2ehUBu/Yi5Gd6RsLaw/AGa0UHJ9gMd1J4beIYqfvtbGfhsx//kXdd ObpepkG9ipavybKxYujVB5JL42dWX3iD7OWxaGNMISkDszyW2eU4HAKJqQpr1T4LwRJ4 Omkk3tN/nz9cAdCeduPU75xbxUTimxujLKkQKr+qlaMahcuDrSy1TAcHYApN7nSujyiK lHzAaj+BrsWsW/wgcF0Ji4MIF3D1RynKVswTzedhfK7L07iJP9BtYs9d2TO75KULhaa/ iwUAY9Wgx+Yuohq+SBnVr8Lmya3Mi4hzRNSiTXk0/gAVeFoHYkgbacpmjPNIMzPZlhMH YqSg== X-Gm-Message-State: AOAM5320CQFaMpntoRMDlN30uV+xElxaMsm2acJ2AkEbOkoH4rPBVeFD OyscAzZ0kWfv0pa8qWSNjdOiGzfBnMOnD3TmAylDMA== X-Received: by 2002:a17:907:8a24:b0:6f4:82c9:9adc with SMTP id sc36-20020a1709078a2400b006f482c99adcmr978237ejc.223.1652378020059; Thu, 12 May 2022 10:53:40 -0700 (PDT) MIME-Version: 1.0 References: <20220426173334.3871399-1-dlatypov@google.com> <20220426173334.3871399-2-dlatypov@google.com> In-Reply-To: <20220426173334.3871399-2-dlatypov@google.com> From: Brendan Higgins Date: Thu, 12 May 2022 13:53:28 -0400 Message-ID: Subject: Re: [PATCH 2/3] kunit: tool: make parser stop overwriting status of suites w/ no_tests To: Daniel Latypov Cc: davidgow@google.com, linux-kernel@vger.kernel.org, kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org, skhan@linuxfoundation.org Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.5 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Apr 26, 2022 at 1:33 PM Daniel Latypov wrote: > > Consider this invocation > $ ./tools/testing/kunit/kunit.py parse < TAP version 14 > 1..2 > ok 1 - suite > # Subtest: no_tests_suite > # catastrophic error! > not ok 1 - no_tests_suite > EOF > > It will have a 0 exit code even though there's a "not ok". > > Consider this one: > $ ./tools/testing/kunit/kunit.py parse < TAP version 14 > 1..2 > ok 1 - suite > not ok 1 - no_tests_suite > EOF > > It will a non-zero exit code. > > Why? > We have this line in the kunit_parser.py > > parent_test = parse_test_header(lines, test) > where we have special handling when we see "# Subtest" and we ignore the > explicit reported "not ok 1" status! > > Also, NO_TESTS at a suite-level only results in a non-zero status code > where then there's only one suite atm. > > This change is the minimal one to make sure we don't overwrite it. > > Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins