Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <a2e95587-418b-879f-2468-8699a6df4a6a@suse.com>
Date:   Fri, 20 May 2022 08:05:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.0
Subject: Re: [PATCH 2/2] x86/pat: add functions to query specific cache mode
 availability
Content-Language: en-US
To:     Chuck Zmudzinski <brchuckz@netscape.net>
CC:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Andy Lutomirski <luto@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        Joonas Lahtinen <joonas.lahtinen@linux.intel.com>,
        Rodrigo Vivi <rodrigo.vivi@intel.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        xen-devel@lists.xenproject.org, x86@kernel.org,
        linux-kernel@vger.kernel.org, intel-gfx@lists.freedesktop.org,
        dri-devel@lists.freedesktop.org, Juergen Gross <jgross@suse.com>
References: <20220503132207.17234-1-jgross@suse.com>
 <20220503132207.17234-3-jgross@suse.com>
 <1d86d8ff-6878-5488-e8c4-cbe8a5e8f624@suse.com>
 <0dcb05d0-108f-6252-e768-f75b393a7f5c@suse.com>
 <77255e5b-12bf-5390-6910-dafbaff18e96@netscape.net>
From:   Jan Beulich <jbeulich@suse.com>
In-Reply-To: <77255e5b-12bf-5390-6910-dafbaff18e96@netscape.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?g48/Tuk3XWFbnSqtRBTSYTbU318qy8GrOJwMMqgwMzGd3UHT/w5KtVf/bhQl?=
 =?us-ascii?Q?Roc2uIZZ0Inq+u5l9utd73cq1poi+7NU073o2JoxZuy6Wzny7r8XECMSUQC/?=
 =?us-ascii?Q?1xCBh6WBpZTP5noCw/GT2Bf2tJgn2CmohdU+lRKUKFoaHGuexHTUP/sZ8p3g?=
 =?us-ascii?Q?KqMhabJNfQ7t1AC9deo+7FY4oKby4iIccsU/MyWxgV8nGiTNAumvAFRyKV03?=
 =?us-ascii?Q?RcNPUvBhpAXWSXY22CP1T9GQC31VhDjfGdTshJwoNV2JxGSDV2jta3cFio8T?=
 =?us-ascii?Q?xW9MzERJPu4URRCUhPlVjt9Y8nqzDKh7qsdYrVRziZ3Z43aXPH9QMT21Zy3q?=
 =?us-ascii?Q?NNFAoWiYodIhD53oYZq7KT2p+5G/qXWgD93Fu66WuG0a3LkI48SXSdejrL6G?=
 =?us-ascii?Q?Ec+DcWEc46QJzuJ540YA0YFLOYG1fJPLF/g53pkhvtR2Kb1k738AtL6dEr9u?=
 =?us-ascii?Q?Eboed82wcxmyZppS9B9+757PqVrnGPDGUbgVQJ7FDi3U6fwbQJy6Ga4HFxlZ?=
 =?us-ascii?Q?W8BXnZDn0+rDP3WYzAsWwdxyiBQL2FSb6SCtPhyxB69HqDlX+ovHL9ku6o7F?=
 =?us-ascii?Q?u0ry6oXyLOwjFiZS0/Mwd3JZKo4mw3u3a/09sT1X6Zf3uLol8NoHiX76npSX?=
 =?us-ascii?Q?Jwc7qboNnYAq75pm0mXpV8NSDj7vGIr6yLXAJ5s30p4uS686tuoAWRMeYSfg?=
 =?us-ascii?Q?64+1ARwLr4Jg+azL+HnMxsl55mZuG9ehc9WAVeORabjGfXTLSRDoDyFG0v/0?=
 =?us-ascii?Q?+IXqIV5VU4ARu7mePLbgb1Oo3sv35KIehFY3Ynjmf1uqVDwqkiZ6bUtwPXnf?=
 =?us-ascii?Q?QWPmX7JKCjKeZUiJBQPysqTA5QGvFJuKvGPcg9CZb8dlrhhmILvyie/6yRRM?=
 =?us-ascii?Q?J0z0ejpLV/0J9sv7geO5adwFcOimZjF+T+SbfnXYw9b5ik1ar04xy5v5tdr0?=
 =?us-ascii?Q?juUNv1oOZ8p06auQdR8AEUGEFRmXmFbUOgczPvdJQyWIzkPhY58evLMnYSbf?=
 =?us-ascii?Q?Mhp5nXT3pxsU6kBH3nZzC71ajbaILW/JsjEwUOWIaSQ4XN1RQitUI+WmgF6j?=
 =?us-ascii?Q?nPOOH8rtk97akb2ZyTUbeiSfYtzOq52qQ27lkFKHe8TpImJ8IrQZwPuqu4Yk?=
 =?us-ascii?Q?m5DW5gQnr17VCXHh7QtkJU+b2Z7zzDbuUFEisL9PBb5GJtqjSaZ+MD0Edp9X?=
 =?us-ascii?Q?l46MsjzV603+C77Hg77qUZe430pXlTeVwQAnWiM0+JX5gBpK4Wn3DFqxNkcU?=
 =?us-ascii?Q?eBCpLxLMBAHshkT502SsmyFXs9yD9T8mCP/yXmrVNKQRieQuGC+e8GlFRxZt?=
 =?us-ascii?Q?PiEldgLAUukpXmPfT2pZGkDlV7jPxWttDPibZjxDYRS93LPmDX56IxgUhDDn?=
 =?us-ascii?Q?EPaNAelsbiL1rvpMBv3mqSys8+y91L5xJj8p5wnUo41VY3s3mizEU8qodtq7?=
 =?us-ascii?Q?IOoyURw3ifMXEHBeycXMr1qY5rzxjVahy2xjs47DIczyi8FKjK6Kc8iFeJuL?=
 =?us-ascii?Q?RMnQaqSYJCpwIyJXSrEcD2HkWq9A7Ah3GErNfrBplZ1YE6GCpEcpXHXSPhUS?=
 =?us-ascii?Q?zr20I6lVYGCLD4KULrF933NpizjBUQ19QHuiKQswA/U+2kQGt/GbZvOIdM9O?=
 =?us-ascii?Q?WdspQlp3M/F3K8JJBFV+jtOaclbWnaRfd30Rgv9Ts+sQphziPfh33WQQChpZ?=
 =?us-ascii?Q?bXlzfBkVZtUPFMmTMZ13CzOsuWJnlAveSbCysrsZkJh4AsHHJ3nMtSP45Wqj?=
 =?us-ascii?Q?dRDrbl8Dzw=3D=3D?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 89e43463-1c92-474b-376c-08da3a26baca
X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2022 06:05:24.5360
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: H2WUDTGiEHuZmmVuglqYS/UwXc6+bkKPBmjlCfHQOA4kSEEBTV5kD/OW7CEkFidsY/5r7LXByR3cn7+zAo/GAA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB9066
X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,
	T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 20.05.2022 06:43, Chuck Zmudzinski wrote:
> On 5/4/22 5:14 AM, Juergen Gross wrote:
>> On 04.05.22 10:31, Jan Beulich wrote:
>>> On 03.05.2022 15:22, Juergen Gross wrote:
>>>> Some drivers are using pat_enabled() in order to test availability of
>>>> special caching modes (WC and UC-). This will lead to false negatives
>>>> in case the system was booted e.g. with the "nopat" variant and the
>>>> BIOS did setup the PAT MSR supporting the queried mode, or if the
>>>> system is running as a Xen PV guest.
>>> ...
>>>> Add test functions for those caching modes instead and use them at the
>>>> appropriate places.
>>>>
>>>> Fixes: bdd8b6c98239 ("drm/i915: replace X86_FEATURE_PAT with=20
>>>> pat_enabled()")
>>>> Fixes: ae749c7ab475 ("PCI: Add arch_can_pci_mmap_wc() macro")
>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> ...
>>>
>>>> --- a/arch/x86/include/asm/pci.h
>>>> +++ b/arch/x86/include/asm/pci.h
>>>> @@ -94,7 +94,7 @@ int pcibios_set_irq_routing(struct pci_dev *dev,=20
>>>> int pin, int irq);
>>>> =C2=A0 =C2=A0 =C2=A0 #define HAVE_PCI_MMAP
>>>> -#define arch_can_pci_mmap_wc()=C2=A0=C2=A0=C2=A0 pat_enabled()
>>>> +#define arch_can_pci_mmap_wc()=C2=A0=C2=A0=C2=A0 x86_has_pat_wc()
>>>
>>> Besides this and ...
>>>
>>>> --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c
>>>> +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c
>>>> @@ -76,7 +76,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void=20
>>>> *data,
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (args->flags & ~(I915_MMAP_WC))
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -EINVAL;
>>>> =C2=A0 -=C2=A0=C2=A0=C2=A0 if (args->flags & I915_MMAP_WC && !pat_enab=
led())
>>>> +=C2=A0=C2=A0=C2=A0 if (args->flags & I915_MMAP_WC && !x86_has_pat_wc(=
))
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -ENODEV;
>>>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 obj =3D i915_gem_object_lookup(f=
ile, args->handle);
>>>> @@ -757,7 +757,7 @@ i915_gem_dumb_mmap_offset(struct drm_file *file,
>>>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (HAS_LMEM(to_i915(dev)))
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mmap_type =3D I=
915_MMAP_TYPE_FIXED;
>>>> -=C2=A0=C2=A0=C2=A0 else if (pat_enabled())
>>>> +=C2=A0=C2=A0=C2=A0 else if (x86_has_pat_wc())
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mmap_type =3D I=
915_MMAP_TYPE_WC;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else if (!i915_ggtt_has_aperture(to_gt(=
i915)->ggtt))
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -ENODEV;
>>>> @@ -813,7 +813,7 @@ i915_gem_mmap_offset_ioctl(struct drm_device=20
>>>> *dev, void *data,
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break;
>>>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 case I915_MMAP_OFFSET_WC:
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!pat_enabled())
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!x86_has_pat_wc())
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 return -ENODEV;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 type =3D I915_M=
MAP_TYPE_WC;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break;
>>>> @@ -823,7 +823,7 @@ i915_gem_mmap_offset_ioctl(struct drm_device=20
>>>> *dev, void *data,
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break;
>>>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 case I915_MMAP_OFFSET_UC:
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!pat_enabled())
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!x86_has_pat_uc_minus(=
))
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 return -ENODEV;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 type =3D I915_M=
MAP_TYPE_UC;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break;
>>>
>>> ... these uses there are several more. You say nothing on why those wan=
t
>>> leaving unaltered. When preparing my earlier patch I did inspect them
>>> and came to the conclusion that these all would also better observe the
>>> adjusted behavior (or else I couldn't have left pat_enabled() as the=20
>>> only
>>> predicate). In fact, as said in the description of my earlier patch, in
>>> my debugging I did find the use in i915_gem_object_pin_map() to be the
>>> problematic one, which you leave alone.
>>
>> Oh, I missed that one, sorry.
>=20
> That is why your patch would not fix my Haswell unless
> it also touches i915_gem_object_pin_map() in
> drivers/gpu/drm/i915/gem/i915_gem_pages.c
>=20
>>
>> I wanted to be rather defensive in my changes, but I agree at least the
>> case in arch_phys_wc_add() might want to be changed, too.
>=20
> I think your approach needs to be more aggressive so it will fix
> all the known false negatives introduced by bdd8b6c98239
> such as the one in i915_gem_object_pin_map().
>=20
> I looked at Jan's approach and I think it would fix the issue
> with my Haswell as long as I don't use the nopat option. I
> really don't have a strong opinion on that question, but I
> think the nopat option as a Linux kernel option, as opposed
> to a hypervisor option, should only affect the kernel, and
> if the hypervisor provides the pat feature, then the kernel
> should not override that,

Hmm, why would the kernel not be allowed to override that? Such
an override would affect only the single domain where the
kernel runs; other domains could take their own decisions.

Also, for the sake of completeness: "nopat" used when running on
bare metal has the same bad effect on system boot, so there
pretty clearly is an error cleanup issue in the i915 driver. But
that's orthogonal, and I expect the maintainers may not even care
(but tell us "don't do that then").

Jan

> but because of the confusion, maybe
> a warning could be printed with the nopat option when a
> hypervisor provides the feature so the user can at least have a
> knob to tweak if if does not behave the way the user intends.
> But I must admit, I don't know if the Xen hypervisor has an
> option also to disable pat. If not, then maybe Jan's more
> aggressive approach with nopat might be needed if for
> some reason pat really needs to be disabled in the Linux
> when Linux is running on Xen or another hypervisor, but I don't
> know of any cases when that would be needed.
>=20
> Chuck
>=20