Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Daniel Jordan <daniel.m.jordan@oracle.com>
To:     Alex Williamson <alex.williamson@redhat.com>,
        Cornelia Huck <cohuck@redhat.com>
Cc:     Jason Gunthorpe <jgg@nvidia.com>,
        Matthew Wilcox <willy@infradead.org>,
        Pavel Tatashin <pasha.tatashin@soleen.com>,
        Steven Sistare <steven.sistare@oracle.com>,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        Daniel Jordan <daniel.m.jordan@oracle.com>
Subject: [PATCH v2 0/3] vfio/type1: Batch page pinning
Date:   Fri, 19 Feb 2021 11:13:02 -0500
Message-Id: <20210219161305.36522-1-daniel.m.jordan@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from localhost.localdomain (98.229.125.203) by BL0PR0102CA0052.prod.exchangelabs.com (2603:10b6:208:25::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3868.28 via Frontend Transport; Fri, 19 Feb 2021 16:13:18 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e11312b1-1146-42ea-070d-08d8d4f145a9
X-MS-TrafficTypeDiagnostic: MWHPR1001MB2157:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <MWHPR1001MB215781D75E9EE81DAD4F3041D9849@MWHPR1001MB2157.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: U3CttiRU4H6WM9KUzWCRZG4ljEz38tLIpavZ9WqCMjlJvL1GdLJoHBsZqNGn0OigXxkOW//wV/my4VsNuhZ8vae/gpVMwR1291tCT9C+nsOhW9crn44VY44vLQErkh/VNM32HHcCMRyL8ZTtmBnJsdxu5mf91GuE8AyAa2gJm/UPd6ruZsntvw0xkux2FUJpErVaaQoy0HzapwfpGpo57x/vG1wWVF/QLsNPO3T36nB1j9AQU1zBKOi9A4z1ZWsMi8CFuCpo2pNy8q8jIb0N1DThqwtYUBRWcvvHftXIBfZ9/jQW4S58ZDbuGqYqC7p06zVIerCdXhPvuf3yWkfkpRyyP1X3IQ9OG9DRBv2D/6q8Njt0WSZd3P+qp/EMTwaA35vV03zi/ysgWM20Fp5W0EMRBs1X46P/gWm2kDYts3nAEXq4zbvTWU6aZti06LaB7kkjPjz9aw0/dx8mFdEIoaHAmoB5+TmCUaaN0Nf1GpB8b96piFrxj57aliqEKXHPk57UQgZdFg+aRQU5bFJdtBoOXC2ejquswdB4uMOHnJ2AHKN5ERdXDh7LonqMthkR8MiTVDR210kB4FLTd+cRxg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR10MB1774.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(346002)(136003)(366004)(376002)(8936002)(107886003)(66946007)(1076003)(6486002)(6506007)(956004)(5660300002)(86362001)(83380400001)(8676002)(66556008)(54906003)(478600001)(103116003)(2616005)(110136005)(6666004)(69590400012)(2906002)(52116002)(26005)(186003)(36756003)(6512007)(66476007)(16526019)(316002)(4326008);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?7PrW7/G2xdlLBTUnbwTC+acsu/Dj4U4sAAu8gFE7A3Uj9s0pVj7h35H+jWrp?=
 =?us-ascii?Q?yn/Hyl3DJ4LA3ij6RJANnqEIFfnh2fSNNzNFkayPmicMdosZ5SNgP+ZKHwBf?=
 =?us-ascii?Q?Pf93HwkvOLA+ytbbdXhc9NViZP3GrxAEszKZl1zPgDhdhmsDygjyL8wY2ScH?=
 =?us-ascii?Q?wpqp+fZPR7U5f/PO7Yu/+Dm85Z9ejVO3AV6HrYRIm60vcjxkMt18jLvZFwg1?=
 =?us-ascii?Q?RJI1N6pXki3N6Xm6gE+b23T6a6hK1EUvmkgVuwshYEhzPst8Ev7/XnfsnoEf?=
 =?us-ascii?Q?BIq8LtY08cdN7nYemR5mnyt4s4bxesB0BW5LUHBpgQ6kXGXpDEFRelhK3p4+?=
 =?us-ascii?Q?fa5yBrtixOJAPBvl4wLVoVUlEP8Ica0VkywViWM8e5/J8YEuElHHrlm2Gq5n?=
 =?us-ascii?Q?wp+9UOEa0RisPApu2gLwhQbEKT53/fh2PJCF7mBv8Q/Ue1OuYhf7ARQ6++zl?=
 =?us-ascii?Q?idBk7+RxYwF0DXaYRcRZ3sBHO9l7dX4pp7KRY2PyOv/aur12aonbKl+S75OL?=
 =?us-ascii?Q?L1FhP+yf/DKZWoiW8UXdRNLhv9jcgo8gNuKucNaRR3QRtrjJyU850Dmeb5we?=
 =?us-ascii?Q?VYrkFfn+x/QnznWOeliIjMHEoffBui+mUyGTpuwMsPirhkPgpYjBP/8yAZwe?=
 =?us-ascii?Q?kQnVikFgUW1qymi4XMLnpIrgRPFhpFW3E35wyi/3gQSAOFH2r8Qm6F/dFBuD?=
 =?us-ascii?Q?/kV2shwiAvlDCQ8AfFOnOCzwnP+FYMhAwS1R1RxFJA092P9oEe5X7n/PIWR+?=
 =?us-ascii?Q?csi6KJRNvuDk9kihDCkXMKNgLRi67Z4zeIEt+nQGYqyQEl7yEcTbVMuaCGxu?=
 =?us-ascii?Q?1u2SLKITLrnC8wFq6rFDmxFe9DshCSVGbICgRU7jWfAYMgBSDpHeIdZ0iWhO?=
 =?us-ascii?Q?0Exvtqq+WyszoN3OrhW6P1Jh+OZ27tqlUIivEOOgNxnIQwNhlyFOS+v4DCkt?=
 =?us-ascii?Q?k3Bx5se+AKIKUoNrMd4nbacCHfjDVi3LLH+toj4lezaHKW9bUdnupwlf2LZW?=
 =?us-ascii?Q?E6rP7zekfXdMpPpj8teHDMdfGUh125Pd40gt2E73IHi/TOUbPlqBe3YTbsdC?=
 =?us-ascii?Q?dom0iH1yRFNF4IJNoKsD42hsOAhQ3gI4PVMypgaRDlikOIM1dUIVWffL3aG9?=
 =?us-ascii?Q?fxW1n1BhFU9G25vMBvCegWflnKgwiHY3Z3+Jxb0ucQnOYPULdkCux5hpKJNw?=
 =?us-ascii?Q?BeTecH1fadXRlBQbnnXxtjTjJhp8LrpXMXTK9b6gtvABvIvtQktonxkBsF2a?=
 =?us-ascii?Q?/HGOgl4+3V75OBP1XWqAU2HgCVIMWydsbUril7hfUR2JY0EtTJE+ls9l6ShS?=
 =?us-ascii?Q?BC2dtyzOzbUuhS2Q7BqzbfML?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e11312b1-1146-42ea-070d-08d8d4f145a9
X-MS-Exchange-CrossTenant-AuthSource: MWHPR10MB1774.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2021 16:13:19.5816
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5ned8SBIbiHAp76DLCTpDA/4sQ+8JM0bVMIHUD+u8BGZCjq+ulmlqjxslql2eTECPxBQNcMc/1/Lo8HqNY0iaIxR5Se/6cho7njSsHxhX1k=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2157
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9900 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxlogscore=999
 phishscore=0 adultscore=0 mlxscore=0 suspectscore=0 malwarescore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102190127
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9900 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 impostorscore=0
 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 bulkscore=0
 priorityscore=1501 malwarescore=0 suspectscore=0 adultscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102190127
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

v2:
 - Fixed missing error unwind in patch 3 (Alex).  After more thought,
   the ENODEV case is fine, so it stayed the same.

 - Rebased on linux-vfio.git/next (no conflicts).

---

The VFIO type1 driver is calling pin_user_pages_remote() once per 4k page, so
let's do it once per 512 4k pages to bring VFIO in line with other drivers such
as IB and vDPA.

qemu guests with at least 2G memory start about 8% faster on a Xeon server,
with more detailed results in the last changelog.

Thanks to Matthew, who first suggested the idea to me.

Daniel


Test Cases
----------

 1) qemu passthrough with IOMMU-capable PCI device

 2) standalone program to hit
        vfio_pin_map_dma() -> vfio_pin_pages_remote()

 3) standalone program to hit
        vfio_iommu_replay() -> vfio_pin_pages_remote()

Each was run...

 - with varying sizes
 - with/without disable_hugepages=1
 - with/without LOCKED_VM exceeded

I didn't test vfio_pin_page_external() because there was no readily available
hardware, but the changes there are pretty minimal.

Daniel Jordan (3):
  vfio/type1: Change success value of vaddr_get_pfn()
  vfio/type1: Prepare for batched pinning with struct vfio_batch
  vfio/type1: Batch page pinning

 drivers/vfio/vfio_iommu_type1.c | 215 +++++++++++++++++++++++---------
 1 file changed, 155 insertions(+), 60 deletions(-)

base-commit: 76adb20f924f8d27ed50d02cd29cadedb59fd88f
-- 
2.30.1