Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31;
Date:   Mon, 11 Dec 2023 14:07:45 +0100
From:   Greg KH <gregkh@linuxfoundation.org>
To:     Guillaume Tucker <gtucker@gtucker.io>
Cc:     Gustavo Padovan <gustavo.padovan@collabora.com>,
        stable@vger.kernel.org,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Shreeya Patel <shreeya.patel@collabora.com>,
        "kernelci@lists.linux.dev" <kernelci@lists.linux.dev>
Subject: Re: stable/LTS test report from KernelCI (2023-12-08)
Message-ID: <2023121131-delirious-roster-e729@gregkh>
References: <738c6c87-527e-a1c2-671f-eed6a1dbaef3@collabora.com>
 <2023120846-taste-saga-c4a9@gregkh>
 <1ca05280-a03c-66c0-cd67-87c58c8f3929@gtucker.io>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1ca05280-a03c-66c0-cd67-87c58c8f3929@gtucker.io>
Precedence: bulk

On Mon, Dec 11, 2023 at 11:14:03AM +0100, Guillaume Tucker wrote:
> On a related topic, it was once mentioned that since stable
> releases occur once a week and they are used as the basis for
> many distros and products, it would make sense to have
> long-running tests after the release has been declared.  So we
> could have say, 48h of testing with extended coverage from LTP,
> fstests, benchmarks etc.  That would be a reply to the email with
> the release tag, not the patch review.

What tests take longer than 48 hours?

> I've mentioned before the concept of finding "2nd derivatives" in
> the rest results, basically the first delta gives you all the
> regressions and then you do a delta of the regressions to find
> the new ones.  Maintainer trees would be typically comparing
> against mainline or say, the -rc2 tag where they based their
> branch.  In the case of stable, it would be between the stable-rc
> branch being tested and the base stable branch with the last
> tagged release.

Yes, that is going to be required for this to be useful.

> One last thing, I see there's a change in KernelCI now to
> actually stop sending the current (suboptimal) automated reports
> to the stable mailing list:
> 
>   https://github.com/kernelci/kernelci-jenkins/pull/136
> 
> Is this actually what people here want?

If these reports are currently for me, I'm just deleting them as they
provide no value anymore.  So yes, let's stop this until we can get
something that actually works for us please.

thanks,

greg k-h