Received-SPF: pass (google.com: domain of linux-kernel+bounces-131524-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <fe40498a-3bb2-43c6-b3e2-1e4e10205db1@gmail.com>
Date: Thu, 4 Apr 2024 21:45:05 +0800
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 00/15] Coalesced Interrupt Delivery with posted MSI
Content-Language: en-US
To: Jacob Pan <jacob.jun.pan@linux.intel.com>,
 LKML <linux-kernel@vger.kernel.org>, X86 Kernel <x86@kernel.org>,
 Peter Zijlstra <peterz@infradead.org>, iommu@lists.linux.dev,
 Thomas Gleixner <tglx@linutronix.de>, Lu Baolu <baolu.lu@linux.intel.com>,
 kvm@vger.kernel.org, Dave Hansen <dave.hansen@intel.com>,
 Joerg Roedel <joro@8bytes.org>, "H. Peter Anvin" <hpa@zytor.com>,
 Borislav Petkov <bp@alien8.de>, Ingo Molnar <mingo@redhat.com>
Cc: Paul Luse <paul.e.luse@intel.com>, Dan Williams
 <dan.j.williams@intel.com>, Jens Axboe <axboe@kernel.dk>,
 Raj Ashok <ashok.raj@intel.com>, "Tian, Kevin" <kevin.tian@intel.com>,
 maz@kernel.org, seanjc@google.com, Robin Murphy <robin.murphy@arm.com>
References: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com>
From: Robert Hoo <robert.hoo.linux@gmail.com>
In-Reply-To: <20240126234237.547278-1-jacob.jun.pan@linux.intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 1/27/2024 7:42 AM, Jacob Pan wrote:
> Hi Thomas and all,
> 
> This patch set is aimed to improve IRQ throughput on Intel Xeon by making use of
> posted interrupts.
> 
> There is a session at LPC2023 IOMMU/VFIO/PCI MC where I have presented this
> topic.
> 
> https://lpc.events/event/17/sessions/172/#20231115
> 
> Background
> ==========
> On modern x86 server SoCs, interrupt remapping (IR) is required and turned
> on by default to support X2APIC. Two interrupt remapping modes can be supported
> by IOMMU/VT-d:
> 
> - Remappable 	(host)
> - Posted	(guest only so far)
> 
> With remappable mode, the device MSI to CPU process is a HW flow without system
> software touch points, it roughly goes as follows:
> 
> 1.	Devices issue interrupt requests with writes to 0xFEEx_xxxx
> 2.	The system agent accepts and remaps/translates the IRQ
> 3.	Upon receiving the translation response, the system agent notifies the
> 	destination CPU with the translated MSI
> 4.	CPU's local APIC accepts interrupts into its IRR/ISR registers
> 5.	Interrupt delivered through IDT (MSI vector)
> 
> The above process can be inefficient under high IRQ rates. The notifications in
> step #3 are often unnecessary when the destination CPU is already overwhelmed
> with handling bursts of IRQs. On some architectures, such as Intel Xeon, step #3
> is also expensive and requires strong ordering w.r.t DMA. 

Can you tell more on this "step #3 requires strong ordering w.r.t. DMA"?

> As a result, slower
> IRQ rates can become a limiting factor for DMA I/O performance.
>