Date: Wed, 26 Apr 2017 12:39:38 +0200
From: Joerg Roedel <jroedel@suse.de>
To: Shaohua Li <shli@fb.com>
Cc: linux-kernel@vger.kernel.org, gang.wei@intel.com, hpa@linux.intel.com,
        mingo@kernel.org, kernel-team@fb.com, ning.sun@intel.com,
        srihan@fb.com, alex.eydelberg@intel.com
Subject: Re: [PATCH] x86/tboot: add an option to disable iommu force on
Message-ID: <20170426103938.GS5077@suse.de>
References: <fb5bfaa0847c13334a3332ea2f5f53220c5ac3d6.1493137246.git.shli@fb.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <fb5bfaa0847c13334a3332ea2f5f53220c5ac3d6.1493137246.git.shli@fb.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2642
Lines: 57

On Tue, Apr 25, 2017 at 09:28:53AM -0700, Shaohua Li wrote:
> IOMMU harms performance signficantly when we run very fast networking
> workloads. It's 40GB networking doing XDP test. Software overhead is
> almost unaware, but it's the IOTLB miss (based on our analysis) which
> kills the performance. We observed the same performance issue even with
> software passthrough (identity mapping), only the hardware passthrough
> survives. The pps with iommu (with software passthrough) is only about
> ~30% of that without it. This is a limitation in hardware based on our
> observation, so we'd like to disable the IOMMU force on, but we do want
> to use TBOOT and we can sacrifice the DMA security bought by IOMMU. I
> must admit I know nothing about TBOOT, but TBOOT guys (cc-ed) think not
> eabling IOMMU is totally ok.
> 
> So introduce a new boot option to disable the force on. It's kind of
> silly we need to run into intel_iommu_init even without force on, but we
> need to disable TBOOT PMR registers. For system without the boot option,
> nothing is changed.
> 
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt |  5 +++++
>  arch/x86/kernel/tboot.c                         |  3 +++
>  drivers/iommu/intel-iommu.c                     | 21 ++++++++++++++++++++-
>  include/linux/dma_remapping.h                   |  1 +
>  4 files changed, 29 insertions(+), 1 deletion(-)

Patch does not apply to my x86/vt-d branch.

> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index facc20a..10c393b 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1578,6 +1578,11 @@
>  			extended tables themselves, and also PASID support. With
>  			this option set, extended tables will not be used even
>  			on hardware which claims to support them.
> +		tboot_noforce [Default Off]
> +			By default, tboot will force Intel IOMMU on, which
> +			could harm performance for some workloads even IOMMU
> +			identity mapping is enabled. This option will avoid
> +			the 'force on' for Intel IOMMU.

Also the wording here should be more clear. How about:

> +			Do not force the Intel IOMMU enabled under
> +			tboot.	
> +			By default, tboot will force Intel IOMMU on, which
> +			could harm performance of some high-throughput
> +			devices like 40GBit network cards, even if
> +			identity mapping is enabled.
> +			Note that using this option lowers the security
> +			provided by tboot because it makes the system
> +			vulnerable to DMA attacks.

Regards,

	Joerg