Date: Thu, 27 Apr 2017 08:51:42 +0200
From: Ingo Molnar <mingo@kernel.org>
To: Shaohua Li <shli@fb.com>
Cc: linux-kernel@vger.kernel.org, gang.wei@intel.com, jroedel@suse.de,
        hpa@linux.intel.com, kernel-team@fb.com, ning.sun@intel.com,
        srihan@fb.com, alex.eydelberg@intel.com
Subject: Re: [PATCH V2] x86/tboot: add an option to disable iommu force on
Message-ID: <20170427065142.lnsdegq7zwxacqo2@gmail.com>
References: <1c2cadcf5cd7d19cea93c56435610e61b551bd1e.1493223474.git.shli@fb.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1c2cadcf5cd7d19cea93c56435610e61b551bd1e.1493223474.git.shli@fb.com>
User-Agent: NeoMutt/20170113 (1.7.2)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2638
Lines: 58


* Shaohua Li <shli@fb.com> wrote:

> IOMMU harms performance signficantly when we run very fast networking
> workloads. It's 40GB networking doing XDP test. Software overhead is
> almost unaware, but it's the IOTLB miss (based on our analysis) which
> kills the performance. We observed the same performance issue even with
> software passthrough (identity mapping), only the hardware passthrough
> survives. The pps with iommu (with software passthrough) is only about
> ~30% of that without it. This is a limitation in hardware based on our
> observation, so we'd like to disable the IOMMU force on, but we do want
> to use TBOOT and we can sacrifice the DMA security bought by IOMMU. I
> must admit I know nothing about TBOOT, but TBOOT guys (cc-ed) think not
> eabling IOMMU is totally ok.
> 
> So introduce a new boot option to disable the force on. It's kind of
> silly we need to run into intel_iommu_init even without force on, but we
> need to disable TBOOT PMR registers. For system without the boot option,
> nothing is changed.
> 
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt |  9 +++++++++
>  arch/x86/kernel/tboot.c                         |  3 +++
>  drivers/iommu/intel-iommu.c                     | 18 ++++++++++++++++++
>  include/linux/dma_remapping.h                   |  1 +
>  4 files changed, 31 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 33a3b54..8a3fb0d 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1579,6 +1579,15 @@
>  			extended tables themselves, and also PASID support. With
>  			this option set, extended tables will not be used even
>  			on hardware which claims to support them.
> +		tboot_noforce [Default Off]
> +			Do not force the Intel IOMMU enabled under tboot.
> +			By default, tboot will force Intel IOMMU on, which
> +			could harm performance of some high-throughput
> +			devices like 40GBit network cards, even if identity
> +			mapping is enabled.
> +			Note that using this option lowers the security
> +			provided by tboot because it makes the system
> +			vulnerable to DMA attacks.

So what's the purpose of this kernel option?

It sure isn't the proper solution for correctly architectured hardware/firmware 
(which can just choose not to expose the IOMMU!), and for one-time hacks for 
special embedded systems or for debugging why not just add an iommu=off option to 
force it off?

This just increases complexity for no good reason.

Thanks,

	Ingo