Received-SPF: pass (google.com: domain of linux-kernel+bounces-27461-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Tue, 16 Jan 2024 15:20:23 +0100
From: Niklas Cassel <cassel@kernel.org>
To: Lennert Buytenhek <kernel@wantstofly.org>
Cc: Damien Le Moal <dlemoal@kernel.org>, linux-ide@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: ASMedia ASM1062 (AHCI) hang after "ahci 0000:28:00.0: Using
 64-bit DMA addresses"
Message-ID: <ZaaQpiW3OOZTSyXw@x1-carbon>
References: <ZaZ2PIpEId-rl6jv@wantstofly.org>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZaZ2PIpEId-rl6jv@wantstofly.org>

Hello Lennert,

On Tue, Jan 16, 2024 at 02:27:40PM +0200, Lennert Buytenhek wrote:
> Hi,
> 
> On kernel 6.6.x, with an ASMedia ASM1062 (AHCI) controller, on an
> ASUSTeK Pro WS WRX80E-SAGE SE WIFI mainboard, PCI ID 1b21:0612 and
> subsystem ID 1043:858d, I got a total apparent controller hang,
> rendering the two attached SATA devices unavailable, that was
> immediately preceded by the following kernel messages:
> 
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: Using 64-bit DMA addresses
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00000 flags=0x0000]
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00300 flags=0x0000]
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00380 flags=0x0000]
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00400 flags=0x0000]
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00680 flags=0x0000]
> [Thu Jan  4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00700 flags=0x0000]
> 
> It seems as if the controller has problems with 64-bit DMA addresses,
> and the comments around the source of the message in
> drivers/iommu/dma-iommu.c seem to point into that same direction:
> 
>         /*
>          * Try to use all the 32-bit PCI addresses first. The original SAC vs.
>          * DAC reasoning loses relevance with PCIe, but enough hardware and
>          * firmware bugs are still lurking out there that it's safest not to
>          * venture into the 64-bit space until necessary.
>          *
>          * If your device goes wrong after seeing the notice then likely either
>          * its driver is not setting DMA masks accurately, the hardware has
>          * some inherent bug in handling >32-bit addresses, or not all the
>          * expected address bits are wired up between the device and the IOMMU.
>          */
>         if (dma_limit > DMA_BIT_MASK(32) && dev->iommu->pci_32bit_workaround) {
>                 iova = alloc_iova_fast(iovad, iova_len,
>                                        DMA_BIT_MASK(32) >> shift, false);
>                 if (iova)
>                         goto done;
> 
>                 dev->iommu->pci_32bit_workaround = false;
>                 dev_notice(dev, "Using %d-bit DMA addresses\n", bits_per(dma_limit));
>         }

The DMA mask is set here:
https://github.com/torvalds/linux/blob/v6.7/drivers/ata/ahci.c#L967

And should be called using:
hpriv->cap & HOST_CAP_64
https://github.com/torvalds/linux/blob/v6.7/drivers/ata/ahci.c#L1929

Where hpriv->cap is capabilities reported by the AHCI controller itself.
So it definitely seems like your controller supports 64-bit addressing.

I guess it could be some problem with your BIOS.
Have you tried updating your BIOS?


If that does not work, perhaps you could try this (completely untested) patch:
(You might need to modify the strings to match the exact strings reported by
your BIOS.)

If it works, we need to add a specific BIOS version too, see e.g.
https://github.com/torvalds/linux/blob/v6.7/drivers/ata/ahci.c#L1310


diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 3a5f3255f51b..35dead43142c 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -1034,6 +1034,30 @@ static void ahci_p5wdh_workaround(struct ata_host *host)
        }
 }
 
+static bool ahci_broken_64_bit(struct pci_dev *pdev)
+{
+       static const struct dmi_system_id sysids[] = {
+               {
+                       .ident = "ASUS Pro WS WRX80E-SAGE",
+                       .matches = {
+                               DMI_MATCH(DMI_BOARD_VENDOR,
+                                         "ASUSTeK Computer INC."),
+                               DMI_MATCH(DMI_BOARD_NAME, "Pro WS WRX80E-SAGE"),
+                       },
+               },
+               { }
+       };
+       const struct dmi_system_id *dmi = dmi_first_match(sysids);
+
+       if (!dmi)
+               return false;
+
+       dev_warn(&pdev->dev, "%s: forcing 32bit DMA, update BIOS\n",
+                dmi->ident);
+
+       return true;
+}
+
 /*
  * Macbook7,1 firmware forcibly disables MCP89 AHCI and changes PCI ID when
  * booting in BIOS compatibility mode.  We restore the registers but not ID.
@@ -1799,6 +1823,10 @@ static int ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
        if (ahci_broken_devslp(pdev))
                hpriv->flags |= AHCI_HFLAG_NO_DEVSLP;
 
+       /* must set flag prior to save config in order to take effect */
+       if (ahci_broken_64_bit(pdev))
+               hpriv->flags |= AHCI_HFLAG_32BIT_ONLY;
+
 #ifdef CONFIG_ARM64
        if (pdev->vendor == PCI_VENDOR_ID_HUAWEI &&
            pdev->device == 0xa235 &&