Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220510141329.54414-1-tomeu.vizoso@collabora.com> <20220511061533.56881-1-tomeu.vizoso@collabora.com>
In-Reply-To: <20220511061533.56881-1-tomeu.vizoso@collabora.com>
From:   Rob Clark <robdclark@gmail.com>
Date:   Wed, 11 May 2022 06:20:36 -0700
Message-ID: <CAF6AEGts6cKOrmRruo8uSXSW=Kq58VZxdPuS13Q7jVajbvga-g@mail.gmail.com>
Subject: Re: [RFC v3] drm/msm: Add initial ci/ subdirectory
To:     Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc:     Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
        Maxime Ripard <mripard@kernel.org>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        Jonathan Corbet <corbet@lwn.net>, Sean Paul <sean@poorly.run>,
        Abhinav Kumar <quic_abhinavk@quicinc.com>,
        dri-devel <dri-devel@lists.freedesktop.org>,
        linux-doc@vger.kernel.org,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-arm-msm <linux-arm-msm@vger.kernel.org>,
        freedreno <freedreno@lists.freedesktop.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, May 10, 2022 at 11:15 PM Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
>
> And use it to store expectations about what the drm/msm driver is
> supposed to pass in the IGT test suite.
>
> Also include a configuration file that points to the out-of-tree CI
> scripts.
>
> By storing the test expectations along the code we can make sure both
> stay in sync with each other, and so we can know when a code change
> breaks those expectations.
>
> This will allow all contributors to drm/msm to reuse the infrastructure
> already in gitlab.freedesktop.org to test the driver on several
> generations of the hardware.
>
> v2:
>   - Fix names of result expectation files to match SoC
>   - Don't execute tests that are going to skip on all boards
>
> v3:
>   - Remove tracking of dmesg output during test execution
>
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> ---
>  Documentation/gpu/msm_automated_testing.rst   |  70 +++++++++
>  drivers/gpu/drm/msm/ci/gitlab-ci.yml          |  11 ++
>  drivers/gpu/drm/msm/ci/msm.testlist           | 144 ++++++++++++++++++
>  .../gpu/drm/msm/ci/msm_apq8016_results.txt    | 141 +++++++++++++++++
>  .../gpu/drm/msm/ci/msm_apq8096_results.txt    | 141 +++++++++++++++++
>  drivers/gpu/drm/msm/ci/msm_sc7180_results.txt | 142 +++++++++++++++++
>  drivers/gpu/drm/msm/ci/msm_sdm845_results.txt | 142 +++++++++++++++++
>  7 files changed, 791 insertions(+)
>  create mode 100644 Documentation/gpu/msm_automated_testing.rst
>  create mode 100644 drivers/gpu/drm/msm/ci/gitlab-ci.yml
>  create mode 100644 drivers/gpu/drm/msm/ci/msm.testlist
>  create mode 100644 drivers/gpu/drm/msm/ci/msm_apq8016_results.txt
>  create mode 100644 drivers/gpu/drm/msm/ci/msm_apq8096_results.txt
>  create mode 100644 drivers/gpu/drm/msm/ci/msm_sc7180_results.txt
>  create mode 100644 drivers/gpu/drm/msm/ci/msm_sdm845_results.txt
>

[snip]

> diff --git a/drivers/gpu/drm/msm/ci/gitlab-ci.yml b/drivers/gpu/drm/msm/ci/gitlab-ci.yml
> new file mode 100644
> index 000000000000..9b7caa7fcab2
> --- /dev/null
> +++ b/drivers/gpu/drm/msm/ci/gitlab-ci.yml
> @@ -0,0 +1,11 @@
> +variables:
> +  # Change this to use your fork of drm-ci
> +  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
> +  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha msm

btw, shouldn't we list an exact commit sha (or tag) for drm-ci, or am
I overlooking something?  We want to be able to uprev the drm-ci
version (and therefore igt version) in sync with any necessary updates
to the expectations

BR,
-R

> +
> +include:
> +  - project: *drm-ci-project-path
> +    ref: *drm-ci-commit-sha
> +    file:
> +      - '.msm-gitlab-ci.yml'
> +      - '.gitlab-ci.yml'