Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933969AbdDGVuI convert rfc822-to-8bit (ORCPT ); Fri, 7 Apr 2017 17:50:08 -0400 Received: from mga05.intel.com ([192.55.52.43]:8046 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752103AbdDGVt7 (ORCPT ); Fri, 7 Apr 2017 17:49:59 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.37,168,1488873600"; d="scan'208";a="246025414" From: "Sun, Ning" To: Joerg Roedel , Shaohua Li CC: "linux-kernel@vger.kernel.org" , "Wei, Gang" , "hpa@linux.intel.com" , "mingo@kernel.org" , "kernel-team@fb.com" , "srihan@fb.com" , "Eydelberg, Alex" Subject: RE: [RFC] x86/tboot: add an option to disable iommu force on Thread-Topic: [RFC] x86/tboot: add an option to disable iommu force on Thread-Index: AQHSonJMrm8hFMBwmkeIjAg++Ce2AqGhJEcAgAARTgCAE1lMAIAFr28AgABMz6A= Date: Fri, 7 Apr 2017 21:49:52 +0000 Message-ID: <83BE7A8C0111FD48951B68B36BEFDFD40CF39F36@ORSMSX107.amr.corp.intel.com> References: <20170322104900.GE8329@suse.de> <20170322115055.GA35752@dhcp-172-20-162-56.dhcp.thefacebook.com> <20170403191927.GA35817@MacBook-Pro.local> <20170407100840.GB23944@suse.de> In-Reply-To: <20170407100840.GB23944@suse.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMjQyNjY3NzQtMmViOS00MjIzLTljZDctMTkzODAwMGIyOTc5IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IkFuSnJtenlMNVM5ZVFCRE5vWmRSXC9FQVwvM1V6MGFRK045Y24wdVpITnFVUT0ifQ== x-ctpclassification: CTP_IC dlp-product: dlpe-windows dlp-version: 10.0.102.7 dlp-reaction: no-action x-originating-ip: [10.22.254.140] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2005 Lines: 44 Hi Shaohua, One question, did you still see the network performance penalty when Linux kernel cmdline intel_iommu was set to off ( intel_iommu=off) ? Thanks, -ning -----Original Message----- From: Joerg Roedel [mailto:jroedel@suse.de] Sent: Friday, April 07, 2017 3:09 AM To: Shaohua Li Cc: linux-kernel@vger.kernel.org; Wei, Gang ; hpa@linux.intel.com; mingo@kernel.org; kernel-team@fb.com; Sun, Ning ; srihan@fb.com; Eydelberg, Alex Subject: Re: [RFC] x86/tboot: add an option to disable iommu force on On Mon, Apr 03, 2017 at 12:19:28PM -0700, Shaohua Li wrote: > On Wed, Mar 22, 2017 at 07:50:55AM -0400, Shaohua Li wrote: > > On Wed, Mar 22, 2017 at 11:49:00AM +0100, Joerg Roedel wrote: > > > Hi Shaohua, > > > > > > On Tue, Mar 21, 2017 at 11:37:51AM -0700, Shaohua Li wrote: > > > > IOMMU harms performance signficantly when we run very fast > > > > networking workloads. This is a limitation in hardware based on > > > > our observation, so we'd like to disable the IOMMU force on, but > > > > we do want to use TBOOT and we can sacrifice the DMA security > > > > bought by IOMMU. I must admit I know nothing about TBOOT, but > > > > TBOOT guys (cc-ed) think not eabling IOMMU is totally ok. > > > > > > Can you elaborate a bit more on the setup where the IOMMU still > > > harms network performance? With the recent scalability > > > improvements I measured only a minimal impact on 10GBit networking. > > Hi, > > > > It's 40GB networking doing XDP test. Software overhead is almost > > unaware, but it's the IOTLB miss (based on our analysis) which kills > > the performance. We observed the same performance issue even with > > software passthrough (identity mapping), only the hardware > > passthrough survives. The pps with iommu (with software passthrough) is only about ~30% of that without it. > > Any update on this? An explicit Ack from the tboot guys would be good to have. Joerg