Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp2442169pxb; Fri, 29 Oct 2021 01:13:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyewEzOVszZ8pV71Gp6lX+jP/NsYfNZ2nlAW6rX6DllS9zqwPGFJcqyrf8geNf3fCV9TiWK X-Received: by 2002:a05:6a00:2311:b0:431:c19f:2a93 with SMTP id h17-20020a056a00231100b00431c19f2a93mr9567409pfh.11.1635495192139; Fri, 29 Oct 2021 01:13:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635495192; cv=none; d=google.com; s=arc-20160816; b=We9RGJn/+xnkFQIsWv/LbnK0lLTGNR2zXZ9q90ERoiemruL0fC7yfZL+2S568lc4Rx H4Ct7hmBJMbx6QCX0uEFPpdv5pFsdr4jkwtOR7GJJLdQMfMymzYtdY3ZQ3Q1sQ/TjkyI K40InkeYFqKGQDR5tIingfIBwWzDfChswnDvTfWauGYwoq2r4jg4w5l5GDOlkRAHevIS q8c4Z/pxGhw3ZWaQAwZBZ5dXHmM47T6z7dxdlxo/PNElqUa9bwuwpwVy2QjnIAuawuU2 UtKSOihCsp+gq2oaHZIsvuKDGgDOrp3yD+7wWB78PiLoioc61SLP5KhTFTpP6kKLOXGS ta4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:references :cc:to:from:content-language:subject:user-agent:mime-version:date :message-id:dkim-signature; bh=Xu96aStupZygtVSLHvKF+rN1vtm48eum6wSRQF0b9e4=; b=ZRODILzhB7yMErnPwNwNom7ZWL4EF/TkejHUqnw27pzu3ooxVHsbqRzoXyIPQeZuVH rEGFRlEnu5GflRC/3oWQdLXnv8OzaimMObQLdv4JrBX1hUt3JJFb7/v6zLu6ZdGAiRih WiDocZWOtn112xXttFfhAbSu80JTBabBX9SK0sovQzhB6BravxbpnUXMDxAHuIzxNpHq 02pXLZiVDgQ6vBwIVJnxBbhh8BDPBaYkRXf2vnq2bpA+0MAAv6wzLlX8jR+47B1iimmP wVgaET+GJRJUkxUgwXBeAHD9gbZ4jKo1UM12D5JYcchTQOT8L/5Ybjr0jYHdoYqNTCRh yQVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=KDTB7NtJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id d189si6854630pgc.86.2021.10.29.01.12.58; Fri, 29 Oct 2021 01:13:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=KDTB7NtJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232414AbhJ2IMk (ORCPT + 99 others); Fri, 29 Oct 2021 04:12:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56886 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232341AbhJ2IMi (ORCPT ); Fri, 29 Oct 2021 04:12:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635495010; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xu96aStupZygtVSLHvKF+rN1vtm48eum6wSRQF0b9e4=; b=KDTB7NtJvp7LvXn0CxD2NyvNyrZSfwLGv0d0+5+w1HAXk77rRWI2G4kWxCcUJz91qHvjkU c13J9U+6uRXYNr5ONhkuln7FlQ8oKtBppS3FZedhtqtCtGdEnhcerMemB/EeOJIEGOBzht tX5dtT576HPBu9QaUHiugGkeAbocM9A= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-296-Vm87dH2_NkOYdL67qlZ4bg-1; Fri, 29 Oct 2021 04:10:08 -0400 X-MC-Unique: Vm87dH2_NkOYdL67qlZ4bg-1 Received: by mail-ed1-f72.google.com with SMTP id z20-20020a05640240d400b003dce046ab51so8422420edb.14 for ; Fri, 29 Oct 2021 01:10:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:from:to:cc:references:in-reply-to :content-transfer-encoding; bh=Xu96aStupZygtVSLHvKF+rN1vtm48eum6wSRQF0b9e4=; b=Uy/WxXIUfdT49WBmUEdMZ3PRDJJc6Kml199BSU/p90BQfVRJLemLaKOV4ylXk9j1cm EARq+GA4DzYB0ETGBSAQpHR9gtdpNOBX+SLKjjSOkjLo6UX2StQZZxDeRA3Sz2Bd2NC8 mgMHT/11IHmpU30MokbRGZUalFdkfXMVrWdfqtJA+h9Aq9VDL+dtnuqL4hDrQ0HjpN+m p6ZiSENrfivY2pZvYK2mOpTpyhZX1CccPilXq0NkFqGXPiXcjoT/LVVynTy5U5h7gY9B 5my+g61q7YqMx00g3aQL3yLJWTtd6ExUMmiIMoV6x2I5rerifu2zOUTgboMU4z10ouGb XfzA== X-Gm-Message-State: AOAM530bGqWemUTKNbIo+lhmavyxeipYn/+EUfXsxMTP06UJSr2k++DN AdblXeUsfW2lg11rSq+hUh/cweLWGUB/ayKCHKQhrwmtv+9cI51BgXHy/qFvLt6ZOyKKi7n/i+K GAYds9Gju5Hc9jvUnn8QeysKC X-Received: by 2002:a17:907:a089:: with SMTP id hu9mr12035233ejc.70.1635495007380; Fri, 29 Oct 2021 01:10:07 -0700 (PDT) X-Received: by 2002:a17:907:a089:: with SMTP id hu9mr12035214ejc.70.1635495007158; Fri, 29 Oct 2021 01:10:07 -0700 (PDT) Received: from ?IPV6:2001:1c00:c1e:bf00:1054:9d19:e0f0:8214? (2001-1c00-0c1e-bf00-1054-9d19-e0f0-8214.cable.dynamic.v6.ziggo.nl. [2001:1c00:c1e:bf00:1054:9d19:e0f0:8214]) by smtp.gmail.com with ESMTPSA id ne2sm2519895ejc.44.2021.10.29.01.10.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 29 Oct 2021 01:10:06 -0700 (PDT) Message-ID: <82035130-d810-9f0b-259e-61280de1d81f@redhat.com> Date: Fri, 29 Oct 2021 10:10:05 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Subject: Re: [PATCH v5 1/2] x86/PCI: Ignore E820 reservations for bridge windows on newer systems Content-Language: en-US From: Hans de Goede To: Bjorn Helgaas Cc: "Rafael J . Wysocki" , Mika Westerberg , =?UTF-8?Q?Krzysztof_Wilczy=c5=84ski?= , Bjorn Helgaas , Myron Stowe , Juha-Pekka Heikkila , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H . Peter Anvin" , linux-acpi@vger.kernel.org, linux-pci@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, =?UTF-8?Q?Benoit_Gr=c3=a9goire?= , Hui Wang , stable@vger.kernel.org, "Rafael J . Wysocki" References: <20211022012034.GA2703195@bhelgaas> <75d1ef5a-13d9-9a67-0139-90b27b084c84@redhat.com> In-Reply-To: <75d1ef5a-13d9-9a67-0139-90b27b084c84@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Bjorn, On 10/22/21 11:53, Hans de Goede wrote: > Hi Bjorn, > > On 10/22/21 03:20, Bjorn Helgaas wrote: >> On Thu, Oct 21, 2021 at 07:15:57PM +0200, Hans de Goede wrote: >>> On 10/20/21 23:14, Bjorn Helgaas wrote: >>>> On Wed, Oct 20, 2021 at 12:23:26PM +0200, Hans de Goede wrote: >>>>> On 10/19/21 23:52, Bjorn Helgaas wrote: >>>>>> On Thu, Oct 14, 2021 at 08:39:42PM +0200, Hans de Goede wrote: >>>>>>> Some BIOS-es contain a bug where they add addresses which map to system >>>>>>> RAM in the PCI host bridge window returned by the ACPI _CRS method, see >>>>>>> commit 4dc2287c1805 ("x86: avoid E820 regions when allocating address >>>>>>> space"). >>>>>>> >>>>>>> To work around this bug Linux excludes E820 reserved addresses when >>>>>>> allocating addresses from the PCI host bridge window since 2010. >>>>>>> ... >>>> >>>>>> I haven't seen anybody else eager to merge this, so I guess I'll stick >>>>>> my neck out here. >>>>>> >>>>>> I applied this to my for-linus branch for v5.15. >>>>> >>>>> Thank you, and sorry about the build-errors which the lkp >>>>> kernel-test-robot found. >>>>> >>>>> I've just send out a patch which fixes these build-errors >>>>> (verified with both .config-s from the lkp reports). >>>>> Feel free to squash this into the original patch (or keep >>>>> them separate, whatever works for you). >>>> >>>> Thanks, I squashed the fix in. >>>> >>>> HOWEVER, I think it would be fairly risky to push this into v5.15. >>>> We would be relying on the assumption that current machines have all >>>> fixed the BIOS defect that 4dc2287c1805 addressed, and we have little >>>> evidence for that. >>> >>> It is a 10 year old BIOS defect, so hopefully anything from 2018 >>> or later will not have it. >> >> We can hope. AFAIK, Windows allocates space top-down, while Linux >> allocates bottom-up, so I think it's quite possible these defects >> would never be discovered or fixed. In any event, I don't think we >> have much evidence either way. > > Ack. > >>>> I'm not sure there's significant benefit to having this in v5.15. >>>> Yes, the mainline v5.15 kernel would work on the affected machines, >>>> but I suspect most people with those machines are running distro >>>> kernels, not mainline kernels. >>> >>> Fedora and Arch do follow mainline pretty closely and a lot of >>> users are affected by this (see the large number of BugLinks in >>> the commit). >>> >>> I completely understand why you are reluctant to push this out, but >>> your argument about most distros not running mainline kernels also >>> applies to chances of people where this may cause a regression >>> running mainline kernels also being quite small. >> >> True. >> >>>> This issue has been around a long time, so it's not like a regression >>>> that we just introduced. If we fixed these machines and regressed >>>> *other* machines, we'd be worse off than we are now. >>> >>> If we break one machine model and fix a whole bunch of other machines >>> then in my book that is a win. Ideally we would not break anything, >>> but we can only find out if we actually break anything if we ship >>> the change. >> >> I'm definitely not going to try the "fix many, break one" argument on >> Linus. Of course we want to fix systems, but IMO it's far better to >> leave a system broken than it is to break one that used to work. > > Right, what I meant to say with "a win" is a step in the right direction, > we definitely must address any regressions coming from this change as > soon as we learn about them. > >>>> In the meantime, here's another possibility for working around this. >>>> What if we discarded remove_e820_regions() completely, but aligned the >>>> problem _CRS windows a little more? The 4dc2287c1805 case was this: >>>> >>>> BIOS-e820: 00000000bfe4dc00 - 00000000c0000000 (reserved) >>>> pci_root PNP0A03:00: host bridge window [mem 0xbff00000-0xdfffffff] >>>> >>>> where the _CRS window was of size 0x20100000, i.e., 512M + 1M. At >>>> least in this particular case, we could avoid the problem by throwing >>>> away that first 1M and aligning the window to a nice 3G boundary. >>>> Maybe it would be worth giving up a small fraction (less than 0.2% in >>>> this case) of questionable windows like this? >>> >>> The PCI BAR allocation code tries to fall back to the BIOS assigned >>> resource if the allocation fails. That BIOS assigned resource might >>> fall outside of the host bridge window after we round the address. >>> >>> My initial gut instinct here is that this has a bigger chance >>> of breaking things then my change. >>> >>> In the beginning of the thread you said that ideally we would >>> completely stop using the E820 reservations for PCI host bridge >>> windows. Because in hindsight messing with the windows on all >>> machines just to work around a clear BIOS bug in some was not a >>> good idea. >>> >>> This address-rounding/-aligning you now suggest, is again >>> messing with the windows on all machines just to work around >>> a clear BIOS bug in some. At least that is how I see this. >> >> That's true. I assume Red Hat has a bunch of machines and hopefully >> an archive of dmesg logs from them. Those logs should contain good >> E820 and _CRS information, so with a little scripting, maybe we could >> get some idea of what's out there. > > We do have a (large-ish) test-lab, but that contains almost exclusively > servers, where as the original problem was on Dell Precision laptops. > > Also I'm not sure if I can get aggregate data from the lab's machines. > I can reserve time on any model we have to debug specific problems, > but that is targeting one specific model. I'll ask around about this. So I had another idea to get us a whole bunch of dmesg outputs and that is to use the database collected by linux-hardware.org . The dmesg were already individually accessible by selecting a specific model machine, but I asked them if they could do a dump and I just got an email that a dmesg dump is now available here: https://github.com/linuxhw/Dmesg Note be careful with the size of the repository - it will take ~3 gigabytes of network traffic and ~20 gigabytes of space on the drive to checkout it. So if you want dmesg outputs to grep through for e820 / host-bridge-window info, here you go. Regards, Hans