Received: by 2002:ac0:98c7:0:0:0:0:0 with SMTP id g7-v6csp2303780imd; Fri, 2 Nov 2018 09:08:17 -0700 (PDT) X-Google-Smtp-Source: AJdET5eSHS9yZzex/pDyVdx6RJol106RdQzwl3WCx5Zw4mmpskmBtBiHTeeBRsJD8SioX7lIjHcC X-Received: by 2002:a63:db48:: with SMTP id x8mr11091085pgi.365.1541174897841; Fri, 02 Nov 2018 09:08:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1541174897; cv=none; d=google.com; s=arc-20160816; b=NDREe+B42Zr+2gD530iCMKNvDnwtwO0t82S6YqzH+WHjuNj4FMc/iEFWpZRhtms2qw 43UwpKyX9o7RfKemezeuSZZmDXXEexl4FtyUZQ2ZB46dUc7fJt8ePpzbkf0BGnp7WAFd RTLHH13J8L5byWfXj5B+P/OVsV7RxsCI66dqFqHgQ+u/TbjXudkMm2EhJXnrqA+fy5KW sBdA5Ipweq3wrF+gQRLUfReMq0Z/edCvv3uP/gCyTbZ4AjzleezX39u8+ug4wWEAg92N eJx45Mcb3E2SZbMyiIIjkN85WykPXOA20tZgOAp8d2PbKnhy9jJi3aVj4lDdWIbYO3nm ZCRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature; bh=ZWTqfDdY/i3uglUqiNPkhJL7ZpYMnD30gkQpqwOUGk8=; b=EDwdFea45TiIc86Db+MdIY3upSEFU0RC0DmT58bVXOCBVvzbDfZGYjx/hJuljpVAGs d+n15oxNq592jmwrFNBZTWBPs8QkzFN7FXXxgjNiFZLLY7R1UCj9w0tdKZtwxbjcPd2w 0WPPCoU0jxpfS0nE2IAWmd9NBZmYL/T/jy6r+hyWpvSSkeqfV7nohNc/kvs+JJBsRTJa NJgYjk6ywGgJ59lxESIc4lX5EN3sulsRrzgSXIsM5easy+M3sYlre9IrgkJUWWJMEdSG yUes9BjdfyB1GB+Ka7KJIy02vYwNCSwuIsXdcsACBGoLMH6PbobLOr5YswQ8OBSKKctp RPgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=IdxZEOCD; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f62-v6si34564603plf.288.2018.11.02.09.08.03; Fri, 02 Nov 2018 09:08:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=IdxZEOCD; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727848AbeKCBO6 (ORCPT + 99 others); Fri, 2 Nov 2018 21:14:58 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:45291 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726085AbeKCBO5 (ORCPT ); Fri, 2 Nov 2018 21:14:57 -0400 Received: by mail-qk1-f194.google.com with SMTP id d135so3787359qkc.12; Fri, 02 Nov 2018 09:07:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=ZWTqfDdY/i3uglUqiNPkhJL7ZpYMnD30gkQpqwOUGk8=; b=IdxZEOCDbbX5eAu4et0jBaQtRLPJQoMrXnhl7LH0P/HHdvvSJbnpK7JGIs+ltYVi+l 08TA9SAhhdK+B+cPyo112b39fBUWPl78JKP415enz2rL96uLQKNSg2yNJbqo1Eq+E/fD 2AD84uuQZS227mu1BYyxe0qW5YhO2NTPNSee2kKcyR8ljP60TpxmmvWKCS9zKLkT29sk Rs7rCLoDRyo7VAvbRU0zdPVAOJB13wKs5cgjj7WGiaqWXhHP2wJLRb6WAftd0kll08yJ xBxF63fdinNglxHrK1uT8l+mETcyh4c6mofj5G/rJ9mVsnhLS0OVOSTeXFFwLfsM621J zCGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ZWTqfDdY/i3uglUqiNPkhJL7ZpYMnD30gkQpqwOUGk8=; b=XW6X3uAHE8GRL5GHnBUGOwOpvBdj2N6VEUsQ2c5bPgGYgnbpgOcKPP7Y0Lv7WI9137 Y6Z9bu7wP3Z/RHkSosn8j9a3yE9rvYGiZlabPkHqYXGRoLlelOz+ssyhRl4FnXC+wcrK 8Y+QMcpWZW15XvGhlMqYyOb4SiWO2SKpUHGpkNZPsZKi6g4lmga9O2eECQYf9lx+yv7Y X9p8B99L8n7AMGWAzXLmzA9LncC3nDArd6wjJmc8wybsBibomPqLRzB+SLYQ7+fiobLx ntnQG75797VQgmCrJbTN9J3lHyeMUdpM7jKh83WjiDZYUMWZnKxQiOLTYFWEGXhoLMdO hc4w== X-Gm-Message-State: AGRZ1gJksQqgtfR0KZ+OT+IITEvvesiqhB26u4g7+y5UyxVb+c3KIp64 XgVh6pUcweLWet8NwrakoN4= X-Received: by 2002:aed:3024:: with SMTP id 33-v6mr11562409qte.29.1541174841134; Fri, 02 Nov 2018 09:07:21 -0700 (PDT) Received: from u1804-ini.default ([76.224.89.164]) by smtp.googlemail.com with ESMTPSA id j51-v6sm15507254qtj.26.2018.11.02.09.07.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Nov 2018 09:07:20 -0700 (PDT) From: Vitaly Mayatskikh To: "Michael S . Tsirkin" Cc: Jason Wang , kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Vitaly Mayatskikh Subject: [PATCH 0/1] vhost: parallel virtqueue handling Date: Fri, 2 Nov 2018 16:07:09 +0000 Message-Id: <20181102160710.3741-1-v.mayatskih@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, I stumbled across poor performance of virtio-blk while working on a high-performance network storage protocol. Moving virtio-blk's host side to kernel did increase single queue IOPS, but multiqueue disk still was not scaling well. It turned out that vhost handles events from all virtio queues in one helper thread, and that's pretty much a big serialization point. The following patch enables events handling in per-queue thread and increases IO concurrency, see IOPS numbers: # num-queues # bare metal # virtio-blk # vhost-blk 1 171k 148k 195k 2 328k 249k 349k 3 479k 179k 501k 4 622k 143k 620k 5 755k 136k 737k 6 887k 131k 830k 7 1004k 126k 926k 8 1099k 117k 1001k 9 1194k 115k 1055k 10 1278k 109k 1130k 11 1345k 110k 1119k 12 1411k 104k 1201k 13 1466k 106k 1260k 14 1517k 103k 1296k 15 1552k 102k 1322k 16 1480k 101k 1346k Vitaly Mayatskikh (1): vhost: add per-vq worker thread drivers/vhost/vhost.c | 123 +++++++++++++++++++++++++++++++----------- drivers/vhost/vhost.h | 11 +++- 2 files changed, 100 insertions(+), 34 deletions(-) -- 2.17.1