Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <77255e5b-12bf-5390-6910-dafbaff18e96@netscape.net>
Date:   Fri, 20 May 2022 00:43:13 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.8.0
Subject: Re: [PATCH 2/2] x86/pat: add functions to query specific cache mode
 availability
Content-Language: en-US
To:     Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Andy Lutomirski <luto@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        Joonas Lahtinen <joonas.lahtinen@linux.intel.com>,
        Rodrigo Vivi <rodrigo.vivi@intel.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        xen-devel@lists.xenproject.org, x86@kernel.org,
        linux-kernel@vger.kernel.org, intel-gfx@lists.freedesktop.org,
        dri-devel@lists.freedesktop.org
References: <20220503132207.17234-1-jgross@suse.com>
 <20220503132207.17234-3-jgross@suse.com>
 <1d86d8ff-6878-5488-e8c4-cbe8a5e8f624@suse.com>
 <0dcb05d0-108f-6252-e768-f75b393a7f5c@suse.com>
From:   Chuck Zmudzinski <brchuckz@netscape.net>
In-Reply-To: <0dcb05d0-108f-6252-e768-f75b393a7f5c@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 5/4/22 5:14 AM, Juergen Gross wrote:
> On 04.05.22 10:31, Jan Beulich wrote:
>> On 03.05.2022 15:22, Juergen Gross wrote:
>>> Some drivers are using pat_enabled() in order to test availability of
>>> special caching modes (WC and UC-). This will lead to false negatives
>>> in case the system was booted e.g. with the "nopat" variant and the
>>> BIOS did setup the PAT MSR supporting the queried mode, or if the
>>> system is running as a Xen PV guest.
>> ...
>>> Add test functions for those caching modes instead and use them at the
>>> appropriate places.
>>>
>>> Fixes: bdd8b6c98239 ("drm/i915: replace X86_FEATURE_PAT with 
>>> pat_enabled()")
>>> Fixes: ae749c7ab475 ("PCI: Add arch_can_pci_mmap_wc() macro")
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ...
>>
>>> --- a/arch/x86/include/asm/pci.h
>>> +++ b/arch/x86/include/asm/pci.h
>>> @@ -94,7 +94,7 @@ int pcibios_set_irq_routing(struct pci_dev *dev, 
>>> int pin, int irq);
>>>       #define HAVE_PCI_MMAP
>>> -#define arch_can_pci_mmap_wc()    pat_enabled()
>>> +#define arch_can_pci_mmap_wc()    x86_has_pat_wc()
>>
>> Besides this and ...
>>
>>> --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c
>>> +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c
>>> @@ -76,7 +76,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void 
>>> *data,
>>>       if (args->flags & ~(I915_MMAP_WC))
>>>           return -EINVAL;
>>>   -    if (args->flags & I915_MMAP_WC && !pat_enabled())
>>> +    if (args->flags & I915_MMAP_WC && !x86_has_pat_wc())
>>>           return -ENODEV;
>>>         obj = i915_gem_object_lookup(file, args->handle);
>>> @@ -757,7 +757,7 @@ i915_gem_dumb_mmap_offset(struct drm_file *file,
>>>         if (HAS_LMEM(to_i915(dev)))
>>>           mmap_type = I915_MMAP_TYPE_FIXED;
>>> -    else if (pat_enabled())
>>> +    else if (x86_has_pat_wc())
>>>           mmap_type = I915_MMAP_TYPE_WC;
>>>       else if (!i915_ggtt_has_aperture(to_gt(i915)->ggtt))
>>>           return -ENODEV;
>>> @@ -813,7 +813,7 @@ i915_gem_mmap_offset_ioctl(struct drm_device 
>>> *dev, void *data,
>>>           break;
>>>         case I915_MMAP_OFFSET_WC:
>>> -        if (!pat_enabled())
>>> +        if (!x86_has_pat_wc())
>>>               return -ENODEV;
>>>           type = I915_MMAP_TYPE_WC;
>>>           break;
>>> @@ -823,7 +823,7 @@ i915_gem_mmap_offset_ioctl(struct drm_device 
>>> *dev, void *data,
>>>           break;
>>>         case I915_MMAP_OFFSET_UC:
>>> -        if (!pat_enabled())
>>> +        if (!x86_has_pat_uc_minus())
>>>               return -ENODEV;
>>>           type = I915_MMAP_TYPE_UC;
>>>           break;
>>
>> ... these uses there are several more. You say nothing on why those want
>> leaving unaltered. When preparing my earlier patch I did inspect them
>> and came to the conclusion that these all would also better observe the
>> adjusted behavior (or else I couldn't have left pat_enabled() as the 
>> only
>> predicate). In fact, as said in the description of my earlier patch, in
>> my debugging I did find the use in i915_gem_object_pin_map() to be the
>> problematic one, which you leave alone.
>
> Oh, I missed that one, sorry.

That is why your patch would not fix my Haswell unless
it also touches i915_gem_object_pin_map() in
drivers/gpu/drm/i915/gem/i915_gem_pages.c

>
> I wanted to be rather defensive in my changes, but I agree at least the
> case in arch_phys_wc_add() might want to be changed, too.

I think your approach needs to be more aggressive so it will fix
all the known false negatives introduced by bdd8b6c98239
such as the one in i915_gem_object_pin_map().

I looked at Jan's approach and I think it would fix the issue
with my Haswell as long as I don't use the nopat option. I
really don't have a strong opinion on that question, but I
think the nopat option as a Linux kernel option, as opposed
to a hypervisor option, should only affect the kernel, and
if the hypervisor provides the pat feature, then the kernel
should not override that, but because of the confusion, maybe
a warning could be printed with the nopat option when a
hypervisor provides the feature so the user can at least have a
knob to tweak if if does not behave the way the user intends.
But I must admit, I don't know if the Xen hypervisor has an
option also to disable pat. If not, then maybe Jan's more
aggressive approach with nopat might be needed if for
some reason pat really needs to be disabled in the Linux
when Linux is running on Xen or another hypervisor, but I don't
know of any cases when that would be needed.

Chuck