Received-SPF: pass (google.com: domain of linux-kernel+bounces-179502-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <b12ae551-e7a4-435b-b7ff-368d6c1ae7a1@molgen.mpg.de>
Date: Wed, 15 May 2024 08:02:55 +0200
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: DMAR-IR: IRQ remapping was enabled on dmar6 but we are not in
 kdump mode
To: Baolu Lu <baolu.lu@linux.intel.com>
Cc: =?UTF-8?B?SsO2cmcgUsO2ZGVs?= <joro@8bytes.org>,
 Will Deacon <will@kernel.org>, iommu@lists.linux.dev,
 linux-kernel@vger.kernel.org
References: <5517f76a-94ad-452c-bae6-34ecc0ec4831@molgen.mpg.de>
 <433452d0-589a-49c8-8044-dcc93d5be90a@linux.intel.com>
 <24bf9a11-6abd-4ccf-9ca1-3cf75c45d374@molgen.mpg.de>
 <42b53bff-4027-4cb6-a457-e26fd62895e5@linux.intel.com>
 <61ce93c7-e89c-4217-8095-dde9fb01763c@molgen.mpg.de>
 <7eb01b85-9233-4f21-865e-6d128f39fb46@linux.intel.com>
Content-Language: en-US
From: Paul Menzel <pmenzel@molgen.mpg.de>
In-Reply-To: <7eb01b85-9233-4f21-865e-6d128f39fb46@linux.intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Dear Baolu,


Am 15.05.24 um 04:13 schrieb Baolu Lu:
> On 5/15/24 3:46 AM, Paul Menzel wrote:
>> Am 23.01.24 um 01:55 schrieb Baolu Lu:
>>> On 2024/1/22 22:53, Paul Menzel wrote:
>>>> Am 22.01.24 um 13:38 schrieb Baolu Lu:
>>>>> On 2024/1/19 22:45, Paul Menzel wrote:
>>>>>>
>>>>>> On a Dell PowerEdge T640, Linux 5.9 and 6.6.12 warn about kdump:
>>>>>>
>>>>>>      [    2.728445] DMAR-IR: IRQ remapping was enabled on dmar6 but we are not in kdump mode
>>>>>>      [    2.736544] DMAR-IR: IRQ remapping was enabled on dmar5 but we are not in kdump mode
>>>>>>      [    2.744620] DMAR-IR: IRQ remapping was enabled on dmar4 but we are not in kdump mode
>>>>>>      [    2.752695] DMAR-IR: IRQ remapping was enabled on dmar3 but we are not in kdump mode
>>>>>>      [    2.760774] DMAR-IR: IRQ remapping was enabled on dmar2 but we are not in kdump mode
>>>>>>      [    2.768847] DMAR-IR: IRQ remapping was enabled on dmar1 but we are not in kdump mode
>>>>>>      [    2.776922] DMAR-IR: IRQ remapping was enabled on dmar0 but we are not in kdump mode
>>>>>>      [    2.784999] DMAR-IR: IRQ remapping was enabled on dmar7 but we are not in kdump mode
>>>>>>
>>>>>> Looking through the logs, this only happens when using kexec to 
>>>>>> restart the system.
>>>>>
>>>>> The code that warned this is,
>>>>>
>>>>>   599         if (ir_pre_enabled(iommu)) {
>>>>>   600                 if (!is_kdump_kernel()) {
>>>>>   601                         pr_warn("IRQ remapping was enabled on %s but we are not in kdump mode\n",
>>>>>   602                                 iommu->name);
>>>>>   603                         clear_ir_pre_enabled(iommu);
>>>>>   604                         iommu_disable_irq_remapping(iommu);
>>>>>   605                 }
>>>>>
>>>>> The VT-d interrupt remapping is enabled during boot, but this is not a
>>>>> kdump kernel.
>>>>>
>>>>> Do you mind checking whether the disable interrupt remapping callback
>>>>> was called during kexec reboot?
>>>>>
>>>>> 1121 struct irq_remap_ops intel_irq_remap_ops = {
>>>>> 1122         .prepare                = intel_prepare_irq_remapping,
>>>>> 1123         .enable                 = intel_enable_irq_remapping,
>>>>> 1124         .disable                = disable_irq_remapping,
>>>>> 1125         .reenable               = reenable_irq_remapping,
>>>>> 1126         .enable_faulting        = enable_drhd_fault_handling,
>>>>> 1127 };
>>>>
>>>> Is there a way to check this without rebuilding the Linux kernel?
>>>
>>> I am not sure, but you can check whether any messages are dumped in the
>>> path of .disable callback? or try to use ftrace?
>>
>> With
>>
>> ```
>> diff --git a/drivers/iommu/intel/irq_remapping.c 
>> b/drivers/iommu/intel/irq_remapping.c
>> index 712ebfc9870c6..146f19ae5b5f1 100644
>> --- a/drivers/iommu/intel/irq_remapping.c
>> +++ b/drivers/iommu/intel/irq_remapping.c
>> @@ -1030,6 +1030,7 @@ static void disable_irq_remapping(void)
>>       struct dmar_drhd_unit *drhd;
>>       struct intel_iommu *iommu = NULL;
>>
>> +     pr_warn("XXX: Called %s\n", __func__);
>>       /*
>>        * Disable Interrupt-remapping for all the DRHD's now.
>>        */
>> ```
>>
>> I can’t see anything in the logs, so it does not seem to be called.
>>
>> Can you reproduce the issue?
> 
> How did you reproduce this?

On a “server” (with Intel Xeon?), in my case Dell PowerEdge T640 and 
Dell PowerEdge R930 (Intel E7-8891 v3), run

     kexec /boot/bzImage --initrd=/boot/grub/initramfs.igz --reuse-cmdline


Kind regards,

Paul