Received-SPF: pass (google.com: domain of linux-kernel+bounces-207269-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <d0fcc26e-1e2f-4783-8a17-5f09c9729aa2@linux.intel.com>
Date: Sun, 9 Jun 2024 15:30:53 +0800
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Cc: baolu.lu@linux.intel.com, =?UTF-8?B?SsO2cmcgUsO2ZGVs?= <joro@8bytes.org>,
 Will Deacon <will@kernel.org>, iommu@lists.linux.dev,
 linux-kernel@vger.kernel.org
Subject: Re: DMAR-IR: IRQ remapping was enabled on dmar6 but we are not in
 kdump mode
To: Paul Menzel <pmenzel@molgen.mpg.de>
References: <5517f76a-94ad-452c-bae6-34ecc0ec4831@molgen.mpg.de>
 <433452d0-589a-49c8-8044-dcc93d5be90a@linux.intel.com>
 <24bf9a11-6abd-4ccf-9ca1-3cf75c45d374@molgen.mpg.de>
 <42b53bff-4027-4cb6-a457-e26fd62895e5@linux.intel.com>
 <61ce93c7-e89c-4217-8095-dde9fb01763c@molgen.mpg.de>
 <7eb01b85-9233-4f21-865e-6d128f39fb46@linux.intel.com>
 <b12ae551-e7a4-435b-b7ff-368d6c1ae7a1@molgen.mpg.de>
 <754664f9-ac5b-406e-99bd-1b179ea8333b@molgen.mpg.de>
Content-Language: en-US
From: Baolu Lu <baolu.lu@linux.intel.com>
In-Reply-To: <754664f9-ac5b-406e-99bd-1b179ea8333b@molgen.mpg.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 6/8/24 7:07 PM, Paul Menzel wrote:
> Dear Linux folks,
> 
> 
> Am 15.05.24 um 08:02 schrieb Paul Menzel:
> 
>> Am 15.05.24 um 04:13 schrieb Baolu Lu:
>>> On 5/15/24 3:46 AM, Paul Menzel wrote:
>>>> Am 23.01.24 um 01:55 schrieb Baolu Lu:
>>>>> On 2024/1/22 22:53, Paul Menzel wrote:
>>>>>> Am 22.01.24 um 13:38 schrieb Baolu Lu:
>>>>>>> On 2024/1/19 22:45, Paul Menzel wrote:
>>>>>>>>
>>>>>>>> On a Dell PowerEdge T640, Linux 5.9 and 6.6.12 warn about kdump:
>>>>>>>>
>>>>>>>>      [    2.728445] DMAR-IR: IRQ remapping was enabled on dmar6 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.736544] DMAR-IR: IRQ remapping was enabled on dmar5 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.744620] DMAR-IR: IRQ remapping was enabled on dmar4 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.752695] DMAR-IR: IRQ remapping was enabled on dmar3 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.760774] DMAR-IR: IRQ remapping was enabled on dmar2 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.768847] DMAR-IR: IRQ remapping was enabled on dmar1 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.776922] DMAR-IR: IRQ remapping was enabled on dmar0 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.784999] DMAR-IR: IRQ remapping was enabled on dmar7 
>>>>>>>> but we are not in kdump mode
>>>>>>>>
>>>>>>>> Looking through the logs, this only happens when using kexec to 
>>>>>>>> restart the system.
>>>>>>>
>>>>>>> The code that warned this is,
>>>>>>>
>>>>>>>   599         if (ir_pre_enabled(iommu)) {
>>>>>>>   600                 if (!is_kdump_kernel()) {
>>>>>>>   601                         pr_warn("IRQ remapping was enabled 
>>>>>>> on %s but we are not in kdump mode\n",
>>>>>>>   602                                 iommu->name);
>>>>>>>   603                         clear_ir_pre_enabled(iommu);
>>>>>>>   604                         iommu_disable_irq_remapping(iommu);
>>>>>>>   605                 }
>>>>>>>
>>>>>>> The VT-d interrupt remapping is enabled during boot, but this is 
>>>>>>> not a
>>>>>>> kdump kernel.
>>>>>>>
>>>>>>> Do you mind checking whether the disable interrupt remapping 
>>>>>>> callback
>>>>>>> was called during kexec reboot?
>>>>>>>
>>>>>>> 1121 struct irq_remap_ops intel_irq_remap_ops = {
>>>>>>> 1122         .prepare                = intel_prepare_irq_remapping,
>>>>>>> 1123         .enable                 = intel_enable_irq_remapping,
>>>>>>> 1124         .disable                = disable_irq_remapping,
>>>>>>> 1125         .reenable               = reenable_irq_remapping,
>>>>>>> 1126         .enable_faulting        = enable_drhd_fault_handling,
>>>>>>> 1127 };
>>>>>>
>>>>>> Is there a way to check this without rebuilding the Linux kernel?
>>>>>
>>>>> I am not sure, but you can check whether any messages are dumped in 
>>>>> the
>>>>> path of .disable callback? or try to use ftrace?
>>>>
>>>> With
>>>>
>>>> ```
>>>> diff --git a/drivers/iommu/intel/irq_remapping.c 
>>>> b/drivers/iommu/intel/irq_remapping.c
>>>> index 712ebfc9870c6..146f19ae5b5f1 100644
>>>> --- a/drivers/iommu/intel/irq_remapping.c
>>>> +++ b/drivers/iommu/intel/irq_remapping.c
>>>> @@ -1030,6 +1030,7 @@ static void disable_irq_remapping(void)
>>>>       struct dmar_drhd_unit *drhd;
>>>>       struct intel_iommu *iommu = NULL;
>>>>
>>>> +     pr_warn("XXX: Called %s\n", __func__);
>>>>       /*
>>>>        * Disable Interrupt-remapping for all the DRHD's now.
>>>>        */
>>>> ```
>>>>
>>>> I can’t see anything in the logs, so it does not seem to be called.
>>>>
>>>> Can you reproduce the issue?
>>>
>>> How did you reproduce this?
>>
>> On a “server” (with Intel Xeon?), in my case Dell PowerEdge T640 and 
>> Dell PowerEdge R930 (Intel E7-8891 v3), run
>>
>>      kexec /boot/bzImage --initrd=/boot/grub/initramfs.igz 
>> --reuse-cmdline
> 
> Were you able to fit some cycles into reproducing/analyzing this issue?

Yeah! I can reproduce this issue with my local development machine. But
I haven't had time to analyze it yet. Perhaps we can remove this
message, or make sure the .disable callback should be called?

Best regards,
baolu