Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sat, 6 Aug 2022 03:55:18 +0800
From:   Chao Gao <chao.gao@intel.com>
To:     <linux-kernel@vger.kernel.org>, <iommu@lists.linux.dev>
CC:     <dave.hansen@intel.com>, <len.brown@intel.com>,
        <tony.luck@intel.com>, <rafael.j.wysocki@intel.com>,
        <reinette.chatre@intel.com>, <dan.j.williams@intel.com>,
        <kirill.shutemov@linux.intel.com>,
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        <ilpo.jarvinen@linux.intel.com>, <ak@linux.intel.com>,
        <alexander.shishkin@linux.intel.com>
Subject: Re: [RFC v2 0/2] swiotlb performance optimizations
Message-ID: <Yu11pu7GwmV0Bf0V@gao-cwp>
References: <20220718012818.107051-1-chao.gao@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20220718012818.107051-1-chao.gao@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?hgETw2IWAXBHbEK2HUa/bZXaWrChyF9vg5T3gbdfnghUP5VGHf+vmboQhFI+?=
 =?us-ascii?Q?tqtlYREhXsu/M9B1DO6prXOPnb4g+IZjW4JyREoL+dLt6Q5wbXDzMQbJN0AG?=
 =?us-ascii?Q?HdaXebPeokbmNZYyy3jO7eQs6reziE1aWtjbxxIwaD9cjoTNtobts9BFjKhs?=
 =?us-ascii?Q?k4EaQHibePGgUKnotSUBbrHksHY4i5AmKUeLrgYLQqSCuxXc2xvgo2aNdqEv?=
 =?us-ascii?Q?J++C/bpOlLW1iE/2+ZHKmwihSfIAQlSfe2FaBL3YgmYApZH65dSs0hkTmXU1?=
 =?us-ascii?Q?OEyFzulCXtYQ3gl/pITdNfarVfmdeiMkhPsB9HOcK6B2CIem+CyEuet6GHlu?=
 =?us-ascii?Q?8UGK1xZ8AZ/7bLQ/vqyEjmZtfridlLoYh6mFyAgV9rCMMmyxIAumyXhv1pYY?=
 =?us-ascii?Q?DJ9eRFP+qERibSJgDIE1/rxEI6ksWpzXC8J2H58AJh6LavuG2pn+UqFmzIZo?=
 =?us-ascii?Q?+LbC314AgBsB1RY84aF46oHFSZsCAkGgX0LFfnq1NUc34RjwOH3vVzk2m72o?=
 =?us-ascii?Q?AQlgnhJQdkNihoZfgalmtEX2M6pkNLg5SuPnM+wcanNBFhibrb8BshnlILr0?=
 =?us-ascii?Q?8sb86a0zZnu2lQFhtFaF8AToBpM7Tzc7jHiQZzI0gI6pNJohetqfTt2vmoLS?=
 =?us-ascii?Q?zziIdPHaj0FR/7Hmkn6o6a18nCSG4pEGe2HBmT71B6dlEGyshv399EmM7qw+?=
 =?us-ascii?Q?Gzcq5tBMITTffLkymglsbUjyLFYKTpMunAWc/QPs+CRLjmS5VLY8SpMaqjjA?=
 =?us-ascii?Q?kQMN4Zl0YxqULC6diEk3I6hM/fxmfZ1Fb/yhT3ltO/pdZMzA8fa9Mp/mY2ql?=
 =?us-ascii?Q?96JZFkD3QGPAEFBlWVV7dfkmKVT+4nX0HSH5ZAxkPLXSsTSjk0Grrc1LzX4O?=
 =?us-ascii?Q?qRF4/Za+bPuUeCBaINkzyUYEzsit1x6KGHFxqMDeZTtLJUuWNnyT3e3j75+S?=
 =?us-ascii?Q?v02ZA/DDtdhT7OskbuZnaU32EjCCD0joZfhr7xZ6Z6XhMnjDeQp8szyg3UYm?=
 =?us-ascii?Q?s5O5unyumcoGTp999aMT6yU1VTB/wU6mPFC9DbEK8kjQSmmsUmLweifCp6hz?=
 =?us-ascii?Q?XDGRXKbHdBsSZK9Rxvq9M4omDsuV8xSlt0KmeIW5dB3h/H2/FkfWpbalyYH+?=
 =?us-ascii?Q?Esnei/21Ek2jftQT2SrogJ3amBZQWsF4cgVCENaYN2Cf42llaguFbiui2Gs4?=
 =?us-ascii?Q?phH9j7cRc9Ui7pFhEGL/eJv6y4p5iHlRTRs1YXmDNTb41NVabycJzz9TDgt/?=
 =?us-ascii?Q?0g7YfLxiuLzMJO1M2cHvINwU5YaVXRbl1fsCb903v/2Bd/HXfmzryvZcjv9z?=
 =?us-ascii?Q?WjEpuyTXTIgaP4ZEFdkxkwRqsgjOvI+hjqpQwiHqWCWHw0ATB4qYEBRtqCgF?=
 =?us-ascii?Q?7CDHujEkwaWCQWnQxr8XZgj0RT0hZU8KD4cpRBhJpur9xwTOumVTVSS+N37r?=
 =?us-ascii?Q?kYJjBhP0HbehO91PA2ywqf5woSrKyUCHMvchKcnsqWhVUw4gLEnuPEQMytW5?=
 =?us-ascii?Q?5fpmJsTUY+ATA9JndCl4MCee/qXkMZkhd1PBiophaJOf9fBMvhTvF28sWPM5?=
 =?us-ascii?Q?RZDlF0Mfvndy85teCIvV+M5wMP1eWBd7FTke0MMW?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d24120d2-ec23-450c-ddb6-08da76d94692
X-MS-Exchange-CrossTenant-AuthSource: CY4PR1101MB2214.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2022 11:54:39.2700
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mm3KgHf0C72oiWQfJHqFEfBfjOLin16vu+PfKs+gDeF6haIni6EFZlZ6mTxBoyE7RczAUgCVrsLmfrFNFKEpoA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2558
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00,DATE_IN_FUTURE_06_12,
        DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Ping. Intel reviewers, comments and suggestions are welcome.

On Mon, Jul 18, 2022 at 09:28:16AM +0800, Chao Gao wrote:
>Intent of this post:
> Seek reviews from Intel reviewers and anyone else in the list
> interested in IO performance in confidential VMs. Need some acked-by
> reviewed-by tags before I can add swiotlb maintainers to "to/cc" lists
> and ask for a review from them.
>
>Changes from v1 to v2:
>- rebase to the latest dma-mapping tree.
>- drop the duplicate patch for mitigating lock contention
>- re-collect perf data
>
>swiotlb is now widely used by confidential VMs. This series optimizes
>swiotlb to reduce cache misses and lock contention during bounce buffer
>allocation/free and memory bouncing to improve IO workload performance in
>confidential VMs.
>
>Here are some FIO tests we did to demonstrate the improvement.
>
>Test setup
>----------
>
>A normal VM with 8vCPU and 32G memory, swiotlb is enabled by swiotlb=force.
>FIO block size is 4K and iodepth is 256. Note that a normal VM is used so
>that others lack of necessary hardware to host confidential VMs can reproduce
>results below.
>
>Results
>-------
>
>1 FIO job	read/write	IOPS (k)
>vanilla		read		216 
>		write		251 
>optimized	read		250 
>		write		270 
>
>1-job FIO sequential read/write perf increase by 19% and 8% respectively.
>
>Chao Gao (2):
>  swiotlb: use bitmap to track free slots
>  swiotlb: Allocate memory in a cache-friendly way
>
> include/linux/swiotlb.h |   8 ++-
> kernel/dma/swiotlb.c    | 127 +++++++++++++++++-----------------------
> 2 files changed, 60 insertions(+), 75 deletions(-)
>
>-- 
>2.25.1
>