Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 20 Dec 2022 15:46:20 +0000
From:   Jonathan Cameron <Jonathan.Cameron@Huawei.com>
To:     Luis Chamberlain <mcgrof@kernel.org>
CC:     Dan Williams <dan.j.williams@intel.com>,
        <alison.schofield@intel.com>, <vishal.l.verma@intel.com>,
        <ira.weiny@intel.com>, <bwidawsk@kernel.org>, <dave@stgolabs.net>,
        <a.manzanares@samsung.com>, <linux-cxl@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [RFC] cxl_test: upgrade as a first class citizen selftests
 capable driver
Message-ID: <20221220154620.000003b2@Huawei.com>
In-Reply-To: <Y6Crh5DiGPPzKoYp@bombadil.infradead.org>
References: <20221217034947.1174795-1-mcgrof@kernel.org>
        <639d4bb71bada_b41e329452@dwillia2-xfh.jf.intel.com.notmuch>
        <20221218160824.0000583d@Huawei.com>
        <Y6Crh5DiGPPzKoYp@bombadil.infradead.org>
Organization: Huawei Technologies Research and Development (UK) Ltd.
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Mon, 19 Dec 2022 10:20:55 -0800
Luis Chamberlain <mcgrof@kernel.org> wrote:

> On Sun, Dec 18, 2022 at 04:08:24PM +0000, Jonathan Cameron wrote:
> > QEMU based CI should go two ways:
> > 1) QEMU CI would typically pin particular kernel version and verify that
> >    QEMU changed don't break that. If we need new features for a new test,
> >    we move that kernel version used.  Existing tests should never break
> >    against a fixed kernel version as that's a regression in QEMU (or
> >    maybe a bug elsewhere) Ultimately we should have this running in the
> >    normal QEMU gitlab CI.  
> 
> Sounds sensible.
> 
> > 2) Kernel CI against QEMU would typically pin particular QEMU version
> >    and check that kernel changes don't break.  This will have rough edges
> >    for a while yet as we are still adding mandatory features to the QEMU
> >    emulation (e.g. events support).  Again, as we add new features / tests
> >    may need to move the QEMU version forwards to support them.  
> 
> Sure - but for this today other than ensuring a kernel does not crash upon
> bootup we also have cxl_test, but not much else.
> 
> We'll want to exand a set of target tests on CXL enabled nodes, without
> cxl_test. Other than verifying the topology matches, we'll want to start
> mimicking actual use cases / performance stuff.

Definitely good to mimic usecases, but in using the QEMU emulation,
performance is probably not sensible as there are some horrible slow
paths in the emulation such as how we do interleaving.  Could be
improved with some caching of look ups, but the result wouldn't look
much like real devices.

Doing it on a limited set of hardware is viable, but it'll be a while
before we have all the fun options available.

> 
> > I don't think we much care about backwards compatibility so once we've
> > moved the pinned element forwards in the above, we won't care about the
> > old version.   
> 
> Making tests simply skip if the feature is not available doens't take
> much effort but forward thinking.
> 
> > The aim here isn't really to ensure no regressions when
> > running on QEMU (though that CI is nice to have), but more that we have
> > no problems in kernel side of things.  
> 
> Sure.
> 
> > This is a way off yet.  Not seeing this as being part of linux.git.
> > The QEMU CI stuff will be in the qemu.git and Kernel CI stuff probably
> > sit out of tree - there shouldn't be a tight coupling beyond new tests
> > wanting to check available features etc. I might ask a friendly
> > CI project to add this to their normal runs.  
> 
> OK in case it helps, cxl-enabled qemu building bringup / ndctl building
> and install is all now automated and integratead as part of kdevops so
> patches welcomed to expand that coverage.

Excellent.  I'll take a look at kdevops sometime next year.

> 
> > I don't have strong feelings on cxl_test. Tend not to use it myself
> > and haven't yet contributed to it.  
> 
> Thanks, this is useful information.

Different approaches. I can appreciate cxl_tests usecase, but I've mostly been hacking
on the corners that are tricky to test with a mocking driver and
it's easy to hack stuff into the QEMU emulation (for me - now :)

Jonathan

> 
>   Luis