Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [Freedreno] [RFC v2] drm/msm: Add initial ci/ subdirectory
To:     Jessica Zhang <quic_jesszhan@quicinc.com>
Cc:     linux-doc@vger.kernel.org, linux-arm-msm@vger.kernel.org,
        Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
        Maxime Ripard <mripard@kernel.org>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        Jonathan Corbet <corbet@lwn.net>,
        Rob Clark <robdclark@gmail.com>, Sean Paul <sean@poorly.run>,
        Abhinav Kumar <quic_abhinavk@quicinc.com>,
        linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org,
        freedreno@lists.freedesktop.org
References: <20220510070140.45407-1-tomeu.vizoso@collabora.com>
 <20220510141329.54414-1-tomeu.vizoso@collabora.com>
 <e4e03cd8-3ebc-e5e1-e7d0-6bdc038049b5@quicinc.com>
From:   Tomeu Vizoso <tomeu.vizoso@collabora.com>
Message-ID: <33ab308f-f0a1-b5b9-1bcb-b4387f5a4152@collabora.com>
Date:   Wed, 11 May 2022 06:25:05 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <e4e03cd8-3ebc-e5e1-e7d0-6bdc038049b5@quicinc.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 5/10/22 9:39 PM, Jessica Zhang wrote:
> 
> 
> On 5/10/2022 7:13 AM, Tomeu Vizoso wrote:
>> +igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
> 
> I see that there are some KMS subtests/tests are missing such as 
> kms_atomic_transition -- are they unstable across all boards?
> 
>  From what I've seen, kms_atomic_transition is unstable only on Trogdor, 
> but has stable results on other boards.

I don't know of a good reason for not having the tests in this list. 
Developers can follow the instructions at "How to expand coverage" to 
add tests if the driver can consistently pass (or fail) them.

There is ongoing work to have a test runner that will better handle 
flaky tests, which will make it possible to expand the list without 
having to resort to per-soc test lists.

>> +igt@kms_addfb_basic@size-max,pass
>> +igt@kms_addfb_basic@too-wide,pass
>> +igt@kms_addfb_basic@too-high,dmesg-warn
> 
> For test results on Trogdor, is is possible to have them be 
> success/fail/skip only?
> 
> Results such as dmesg-warn/dmesg-fail are igt_runner specific and 
> because there isn't support for igt_runner on ChromeOS, they will be 
> difficult to replicate and debug.

As Rob said later, it's probably better to drop that distinction. Maybe 
at some future point we will want to track expected kmsg output but I 
think there are lots of work to do before that makes sense.

Will do that in a v3.

>> +igt@kms_universal_plane@universal-plane-pipe-a-functional,skip
>> +igt@kms_universal_plane@disable-primary-vs-flip-pipe-a,pass
>> +igt@kms_universal_plane@disable-primary-vs-flip-pipe-b,fail
> 
> We could probably skip checking the results for kms_universal_plane on 
> Trogdor for now, since this is a test affected by the hack regression. 
> There is an IGT patch in the works for fixing the 
> disable-primary-vs-flip-pipe-b failure, so it should be updated pretty 
> soon too.

So, how this should work is for the patch that fixes that bug to update 
also the expectation files (see "How to update test expectations" in the 
docs). In this case though, the fix is likely to land before this patch, 
so I will just update the expectations once I rebase and the fix is 
brought into my branch.

Thanks for the feedback!

Tomeu