Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: Failure to shutdown/reboot with intel_iommu=on
To:     Joerg Roedel <jroedel@suse.de>
Cc:     =?UTF-8?Q?Uwe_Kleine-K=c3=b6nig?= <uwe@kleine-koenig.org>,
        linux-kernel@vger.kernel.org, rafael.j.wysocki@intel.com,
        tglx@linutronix.de, x86@kernel.org,
        Lu Baolu <baolu.lu@linux.intel.com>
References: <20200506144558.GA4019@taurus.defre.kleine-koenig.org>
 <20200508150734.GP8135@suse.de>
 <ff5eb4e2-89a3-57a5-c6d6-42bf743e5d16@redhat.com>
 <20200512133429.GZ8135@suse.de>
From:   Lenny Szubowicz <lszubowi@redhat.com>
Message-ID: <26b328dc-2161-fdaa-f533-bf0027c13efc@redhat.com>
Date:   Tue, 12 May 2020 16:00:26 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.8.0
MIME-Version: 1.0
In-Reply-To: <20200512133429.GZ8135@suse.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 5/12/20 9:34 AM, Joerg Roedel wrote:
> On Mon, May 11, 2020 at 09:43:11AM -0400, Lenny Szubowicz wrote:
>> I suspect that you have TPM 2.x functionality enabled in the BIOS/firmware.
>>
>> Unless you are actually using the TPM, try setting it to TPM 1.2 mode.
>> I've seen an incompatiblity on other Lenovo laptops between using the
>> IOMMU, TPM 2.x implementation in firmware, and shutdown/suspend.
> 
> Interesting, has this been debugged further into the TPM code?
> 
> 
> 	Joerg
> 

I believe the problem is in the Lenovo firmware and not in the kernel.

There are essentially two problems:
  1. TPM 2.0 doesn't work when the IOMMU is enabled
  2. Suspend/shutdown hangs when problem 1 is encountered on boot

Lenovo's firmware implementation of TPM 2.0 functionality on some of their
laptops uses DMA. When you ask the kernel to enable the IOMMU, this DMA
access is correctly blocked by the IOMMU hardware. If you look at your
dmesg log from when you have TPM 2.0 and the IOMMU enabled, there are
TPM timeout messages that indicate the inability to initialize and use
the TPM capability.

The hang on shutdown or S3 suspend appears to be in firmware, i.e.
after the kernel has transferred control back to the firmware.
It makes no difference if the kernel actively shuts down the IOMMU
before transferring control to the firmware on a suspend or shutdown.
The hang still occurs.

My guess is that the firmware wants to do some TPM related processing
on shutdown and suspend and can't handle the TPM state that exists
due to the startup failure. But that's just a guess. I don't know
what the firmware is actually doing.

Some Lenovo laptops provide an ACPI DMAR RMRR that identifies the memory
range that the kernel should open up for permissable DMA access
for this purpose. Unfortunately, the PCI device that performs these
DMA operations is hidden from the kernel by the BIOS. Given that the
associated PCI device is hidden, the Linux kernel does not act upon
the associated DMAR RMRR.

                        -Lenny.