On 09.06.23 15:01, Janusz Krzysztofik wrote:
> Visible glitches have been observed when running graphics applications on
> Linux under Xen hypervisor. Those observations have been confirmed with
> failures from kms_pwrite_crc Intel GPU test that verifies data coherency
> of DRM frame buffer objects using hardware CRC checksums calculated by
> display controllers, exposed to userspace via debugfs. Affected
> processing paths have then been identified with new IGT test variants that
> mmap the objects using different methods and caching modes [1].
>
> When running as a Xen PV guest, Linux uses Xen provided PAT configuration
> which is different from its native one. In particular, Xen specific PTE
> encoding of write-combining caching, likely used by graphics applications,
> differs from the Linux default one found among statically defined minimal
> set of supported modes. Since Xen defines PTE encoding of the WC mode as
> _PAGE_PAT, it no longer belongs to the minimal set, depends on correct
> handling of _PAGE_PAT bit, and can be mismatched with write-back caching.
>
> When a user calls mmap() for a DRM buffer object, DRM device specific
> .mmap file operation, called from mmap_region(), takes care of setting PTE
> encoding bits in a vm_page_prot field of an associated virtual memory area
> structure. Unfortunately, _PAGE_PAT bit is not preserved when the vma's
> .vm_flags are then applied to .vm_page_prot via vm_set_page_prot(). Bits
> to be preserved are determined with _PAGE_CHG_MASK symbol that doesn't
> cover _PAGE_PAT. As a consequence, WB caching is requested instead of WC
> when running under Xen (also, WP is silently changed to WT, and UC
> downgraded to UC_MINUS). When running on bare metal, WC is not affected,
> but WP and WT extra modes are unintentionally replaced with WC and UC,
> respectively.
>
> WP and WT modes, encoded with _PAGE_PAT bit set, were introduced by commit
> 281d4078bec3 ("x86: Make page cache mode a real type"). Care was taken
> to extend _PAGE_CACHE_MASK symbol with that additional bit, but that
> symbol has never been used for identification of bits preserved when
> applying page protection flags. Support for all cache modes under Xen,
> including the problematic WC mode, was then introduced by commit
> 47591df50512 ("xen: Support Xen pv-domains using PAT").
>
> The issue needs to be fixed by including _PAGE_PAT bit into a bitmask used
> by pgprot_modify() for selecting bits to be preserved. We can do that
> either internally to pgprot_modify() (as initially proposed), or by making
> _PAGE_PAT a part of _PAGE_CHG_MASK. If we go for the latter then, since
> _PAGE_PAT is the same as _PAGE_PSE, we need to note that _HPAGE_CHG_MASK
> -- a huge pmds' counterpart of _PAGE_CHG_MASK, introduced by commit
> c489f1257b8c ("thp: add pmd_modify"), defined as (_PAGE_CHG_MASK |
> _PAGE_PSE) -- will no longer differ from _PAGE_CHG_MASK. If such
> modification of _PAGE_CHG_MASK was irrelevant to its users then one might
> wonder why that new _HPAGE_CHG_MASK symbol was introduced instead of
> reusing the existing one with that otherwise irrelevant bit (_PAGE_PSE in
> that case) added.
>
> Assume that adding _PAGE_PAT to _PAGE_CHG_MASK doesn't break pte_modify()
> and its users, and go for it. Also, add _PAGE_PAT_LARGE to
> _HPAGE_CHG_MASK for symmetry. For better clarity, split out common bits
> from both symbols to another one and use it together with specific bits
> when defining the masks.
>
> v3: Separate out common bits of _PAGE_CHG_MASK and _HPAGE_CHG_MASK into
> _COMMON_PAGE_CHG_MASK (Rick),
> - fix hard to parse wording of 'what' part of commit description (on
> Dave's request).
> v2: Keep pgprot_modify() untouched, make _PAGE_PAT part of _PAGE_CHG_MASK
> instead (Borislav),
> - also add _PAGE_PAT_LARGE to _HPAGE_CHG_MASK (Juergen).
>
> [1] https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/commit/0f0754413f14
>
> Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/7648
> Fixes: 281d4078bec3 ("x86: Make page cache mode a real type")
> Signed-off-by: Janusz Krzysztofik <[email protected]>
> Tested-by: Marek Marczykowski-Górecki <[email protected]>
> Reviewed-by: Andi Shyti <[email protected]>
> Acked-by: Juergen Gross <[email protected]> # v1
> Cc: Borislav Petkov <[email protected]>
> Cc: Dave Hansen <[email protected]>
> Cc: "Edgecombe, Rick P" <[email protected]>
> Cc: [email protected] # v3.19+
Reviewed-by: Juergen Gross <[email protected]>
Juergen