Date: Mon, 3 Apr 2017 12:19:28 -0700
From: Shaohua Li <shli@fb.com>
To: Joerg Roedel <jroedel@suse.de>
CC: <linux-kernel@vger.kernel.org>, <gang.wei@intel.com>,
        <hpa@linux.intel.com>, <mingo@kernel.org>, <kernel-team@fb.com>,
        <ning.sun@intel.com>, <srihan@fb.com>, <alex.eydelberg@intel.com>
Subject: Re: [RFC] x86/tboot: add an option to disable iommu force on
Message-ID: <20170403191927.GA35817@MacBook-Pro.local>
References: <b93d9e540a7f0e34fbf622602c433a583130ec88.1490120859.git.shli@fb.com>
 <20170322104900.GE8329@suse.de>
 <20170322115055.GA35752@dhcp-172-20-162-56.dhcp.thefacebook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20170322115055.GA35752@dhcp-172-20-162-56.dhcp.thefacebook.com>
User-Agent: Mutt/1.8.0 (2017-02-23)
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2017 19:19:46.5024 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR15MB1636
X-OriginatorOrg: fb.com
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-03_16:,,
 signatures=0
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 1248
Lines: 29

On Wed, Mar 22, 2017 at 07:50:55AM -0400, Shaohua Li wrote:
> On Wed, Mar 22, 2017 at 11:49:00AM +0100, Joerg Roedel wrote:
> > Hi Shaohua,
> > 
> > On Tue, Mar 21, 2017 at 11:37:51AM -0700, Shaohua Li wrote:
> > > IOMMU harms performance signficantly when we run very fast networking
> > > workloads. This is a limitation in hardware based on our observation, so
> > > we'd like to disable the IOMMU force on, but we do want to use TBOOT and
> > > we can sacrifice the DMA security bought by IOMMU. I must admit I know
> > > nothing about TBOOT, but TBOOT guys (cc-ed) think not eabling IOMMU is
> > > totally ok.
> > 
> > Can you elaborate a bit more on the setup where the IOMMU still harms
> > network performance? With the recent scalability improvements I measured
> > only a minimal impact on 10GBit networking.
> Hi,
> 
> It's 40GB networking doing XDP test. Software overhead is almost unaware, but
> it's the IOTLB miss (based on our analysis) which kills the performance. We
> observed the same performance issue even with software passthrough (identity
> mapping), only the hardware passthrough survives. The pps with iommu (with
> software passthrough) is only about ~30% of that without it.

Hi,

Any update on this?

Thanks,
Shaohua