Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
MIME-Version: 1.0
References: <20220510070140.45407-1-tomeu.vizoso@collabora.com>
 <20220510141329.54414-1-tomeu.vizoso@collabora.com> <CAPM=9tzLR-wsLhg2ikGjoK06s-ju5XWa1rtPPiUpN=pwD1vgtA@mail.gmail.com>
 <CAHk-=wg8YgH1h3wrm9CtXff7rSewa+NE0Z5upb1GOE8XiTL9HA@mail.gmail.com>
 <CAF6AEGusO9XAqHNatJLgV+wpVoyyLg1vHtUsnSkAxJeV7n3WNg@mail.gmail.com>
 <CAHk-=wjbE0f2AGroB1Hy=fx2fh7cRpS0wNdB46Ybk14Mb0b5Jw@mail.gmail.com> <CAHk-=wjy4DY_ya8TBs9W2wLWHibBiHMQW2T43DQR1SGRkqD=gw@mail.gmail.com>
In-Reply-To: <CAHk-=wjy4DY_ya8TBs9W2wLWHibBiHMQW2T43DQR1SGRkqD=gw@mail.gmail.com>
From:   Rob Clark <robdclark@gmail.com>
Date:   Wed, 11 May 2022 13:14:24 -0700
Message-ID: <CAF6AEGtWCkOv4xpWz+ds9fSiB4_W4CV9exYyus1G_-crD2YFGg@mail.gmail.com>
Subject: Re: [Freedreno] Adding CI results to the kernel tree was Re: [RFC v2]
 drm/msm: Add initial ci/ subdirectory
To:     Linus Torvalds <torvalds@linux-foundation.org>
Cc:     Sean Paul <sean@poorly.run>,
        freedreno <freedreno@lists.freedesktop.org>,
        Daniel Vetter <daniel@ffwll.ch>,
        Tomeu Vizoso <tomeu.vizoso@collabora.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
        Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
        Abhinav Kumar <quic_abhinavk@quicinc.com>,
        Maxime Ripard <mripard@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        dri-devel <dri-devel@lists.freedesktop.org>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        linux-arm-msm <linux-arm-msm@vger.kernel.org>,
        Dave Airlie <airlied@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, May 11, 2022 at 12:12 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Wed, May 11, 2022 at 12:08 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > The kernel tree might have just the expected *failures* listed, if
> > there are any. Presumably the ci tree has to have the expected results
> > anyway, so what's the advantage of listing non-failures?
>
> .. put another way: I think a list of "we are aware that these
> currently fail" is quite reasonable for a development tree, maybe even
> with a comment in the commit that created them about why they
> currently fail.
>
> That also ends up being very nice if you fix a problem, and the fix
> commit might then remove the failure for the list, and that all makes
> perfect sense.
>
> But having just the raw output of "these are the expected CI results"
> that is being done and specified by some other tree entirely - that
> seems pointless and just noise to me. There's no actual reason to have
> that kind of noise - and update that kind of noise - that I really
> see.

Yeah, the only reason we have full results is that the current tool to
check for pass/fail of the entire CI job is 'diff' ;-)

It has the nice benefit of generating a patch for you to squash into
whatever commit to update the expectation files, I suppose.  But we
have something more clever on the mesa-ci side of things where we list
skips/flakes/expected-fails but not expected-passes.  To be fair, the
# of tests on the mesa side is something on the order of 750,000, I
don't expect to ever get close to that # on the kernel side.

BR,
-R

>
>                 Linus