Date: Mon, 24 Apr 2017 09:50:59 -0700
From: Shaohua Li <shli@fb.com>
To: "Sun, Ning" <ning.sun@intel.com>
CC: Joerg Roedel <jroedel@suse.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Wei, Gang" <gang.wei@intel.com>,
        "hpa@linux.intel.com" <hpa@linux.intel.com>,
        "mingo@kernel.org" <mingo@kernel.org>,
        "kernel-team@fb.com" <kernel-team@fb.com>,
        "srihan@fb.com" <srihan@fb.com>,
        "Eydelberg, Alex" <alex.eydelberg@intel.com>
Subject: Re: [RFC] x86/tboot: add an option to disable iommu force on
Message-ID: <20170424165059.GA65052@MacBook-Pro.local>
References: <b93d9e540a7f0e34fbf622602c433a583130ec88.1490120859.git.shli@fb.com>
 <20170322104900.GE8329@suse.de>
 <20170322115055.GA35752@dhcp-172-20-162-56.dhcp.thefacebook.com>
 <20170403191927.GA35817@MacBook-Pro.local>
 <20170407100840.GB23944@suse.de>
 <83BE7A8C0111FD48951B68B36BEFDFD40CF39F36@ORSMSX107.amr.corp.intel.com>
 <20170410043106.GA90090@MacBook-Pro-63.local.dhcp.thefacebook.com>
 <83BE7A8C0111FD48951B68B36BEFDFD40CF3C8DE@ORSMSX107.amr.corp.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <83BE7A8C0111FD48951B68B36BEFDFD40CF3C8DE@ORSMSX107.amr.corp.intel.com>
User-Agent: Mutt/1.8.0 (2017-02-23)
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2017 16:51:07.5964 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1647
X-OriginatorOrg: fb.com
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-24_13:,,
 signatures=0
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 3283
Lines: 76

Hi Joerg,

Is Ning's answer sufficient to justify merging the patch?

Thanks,
Shaohua


On Mon, Apr 10, 2017 at 09:28:46PM +0000, Sun, Ning wrote:
> From tboot perspective, it is ok to add the option "tboot_noforce" to Linux kernel Intel_iommu parameter for those performance hungry tboot users, so long as the users are aware of the security implication behind of this option.
>  
> Thanks,
> -ning
> 
> -----Original Message-----
> From: Shaohua Li [mailto:shli@fb.com] 
> Sent: Sunday, April 09, 2017 9:31 PM
> To: Sun, Ning <ning.sun@intel.com>
> Cc: Joerg Roedel <jroedel@suse.de>; linux-kernel@vger.kernel.org; Wei, Gang <gang.wei@intel.com>; hpa@linux.intel.com; mingo@kernel.org; kernel-team@fb.com; srihan@fb.com; Eydelberg, Alex <alex.eydelberg@intel.com>
> Subject: Re: [RFC] x86/tboot: add an option to disable iommu force on
> 
> On Fri, Apr 07, 2017 at 09:49:52PM +0000, Sun, Ning wrote:
> > Hi Shaohua,
> > 
> > One question, did you still see the network performance penalty when Linux kernel cmdline intel_iommu was set to off ( intel_iommu=off) ?
> 
> the boot parameter has no effect, it runs very early and set dmar_disable=1.
> The tboot code (tboot_force_iommu) runs later and force dmar_disabled = 0.
> 
> Thanks,
> Shaohua
>  
> > Thanks,
> > -ning
> > 
> > -----Original Message-----
> > From: Joerg Roedel [mailto:jroedel@suse.de]
> > Sent: Friday, April 07, 2017 3:09 AM
> > To: Shaohua Li <shli@fb.com>
> > Cc: linux-kernel@vger.kernel.org; Wei, Gang <gang.wei@intel.com>; 
> > hpa@linux.intel.com; mingo@kernel.org; kernel-team@fb.com; Sun, Ning 
> > <ning.sun@intel.com>; srihan@fb.com; Eydelberg, Alex 
> > <alex.eydelberg@intel.com>
> > Subject: Re: [RFC] x86/tboot: add an option to disable iommu force on
> > 
> > On Mon, Apr 03, 2017 at 12:19:28PM -0700, Shaohua Li wrote:
> > > On Wed, Mar 22, 2017 at 07:50:55AM -0400, Shaohua Li wrote:
> > > > On Wed, Mar 22, 2017 at 11:49:00AM +0100, Joerg Roedel wrote:
> > > > > Hi Shaohua,
> > > > > 
> > > > > On Tue, Mar 21, 2017 at 11:37:51AM -0700, Shaohua Li wrote:
> > > > > > IOMMU harms performance signficantly when we run very fast 
> > > > > > networking workloads. This is a limitation in hardware based 
> > > > > > on our observation, so we'd like to disable the IOMMU force 
> > > > > > on, but we do want to use TBOOT and we can sacrifice the DMA 
> > > > > > security bought by IOMMU. I must admit I know nothing about 
> > > > > > TBOOT, but TBOOT guys (cc-ed) think not eabling IOMMU is totally ok.
> > > > > 
> > > > > Can you elaborate a bit more on the setup where the IOMMU still 
> > > > > harms network performance? With the recent scalability 
> > > > > improvements I measured only a minimal impact on 10GBit networking.
> > > > Hi,
> > > > 
> > > > It's 40GB networking doing XDP test. Software overhead is almost 
> > > > unaware, but it's the IOTLB miss (based on our analysis) which 
> > > > kills the performance. We observed the same performance issue even 
> > > > with software passthrough (identity mapping), only the hardware 
> > > > passthrough survives. The pps with iommu (with software passthrough) is only about ~30% of that without it.
> > > 
> > > Any update on this?
> > 
> > An explicit Ack from the tboot guys would be good to have.
> > 
> > 
> > 	Joerg
> >