Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 11 Sep 2023 15:29:00 +0100
From:   Mark Brown <broonie@kernel.org>
To:     =?iso-8859-1?Q?N=EDcolas_F=2E_R=2E_A=2E?= Prado 
        <nfraprado@collabora.com>
Cc:     Takashi Iwai <tiwai@suse.com>, kernel@collabora.com,
        Jaroslav Kysela <perex@perex.cz>,
        Shuah Khan <shuah@kernel.org>, alsa-devel@alsa-project.org,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org
Subject: Re: [PATCH] kselftest/alsa: Mark test plan as skipped when no cards
 are available
Message-ID: <ZP8kLM5ln2YBCHap@finisterre.sirena.org.uk>
References: <20230908181242.95714-1-nfraprado@collabora.com>
 <0fa0901e-d271-438d-bc2b-11399ad3b07c@sirena.org.uk>
 <868a8f08-ba39-4550-8e7b-0572ea03c4e7@notapiano>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
        protocol="application/pgp-signature"; boundary="upLhFwPGUAS0CWZq"
Content-Disposition: inline
In-Reply-To: <868a8f08-ba39-4550-8e7b-0572ea03c4e7@notapiano>
Precedence: bulk


--upLhFwPGUAS0CWZq
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Sep 11, 2023 at 08:35:37AM -0400, N=EDcolas F. R. A. Prado wrote:
> On Sat, Sep 09, 2023 at 12:08:22AM +0100, Mark Brown wrote:

> > Why?

> To better reflect the actual test plan status. If 0 tests were run, it do=
esn't
> really make sense to say that the test plan passed, rather it was skipped=
 since
> nothing was run. So with this change, if there's a regression that preven=
ts the
> soundcard driver from even probing, the result won't be "pass", but "skip=
", and
> the reason 'No soundcard available' will be in the logs.

So, I would interpret the overall result for the suite as being "No
errors were found in any of the cards discovered" if there is no
configuration file specified which enumerates the set of cards that are
expected (if there is a config file that's a different matter, we know
what we're expecting).  I'm not sure that the different behaviour for 0
cards is super useful.

--upLhFwPGUAS0CWZq
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAmT/JCsACgkQJNaLcl1U
h9BPVAf/RSFHSOlUvU0zkBcxBW+G0eqxamHxmKQmBzKhecWQfBvToFYzyPMTI8hj
1nkjmK70vMjHhjLZZri+srrnWwBLzVknDtJbaMFXkzFQibRN3P/60ChXGtg9WC1X
5YzCyo+Na071zvmceag5yoyC55RoWZiMknKj89Y4DjL6cnExT7MEHuBGlwioUjjP
1NApQduQGNWL8TNDcUkbDRa+ZPAkB6LspNVY4J7+S+R8jCqIINtX/YlKK+1oz60a
50FFhWVNgjGigFqy/yGMaAocWDhSkTrf5xqHb5CjlaleQlhSW1Lh/TVaClZNPI9E
0F0Aj163sUBtchAoz9QUWSmzw0qWDg==
=bHB7
-----END PGP SIGNATURE-----

--upLhFwPGUAS0CWZq--