Received: by 2002:a25:b323:0:0:0:0:0 with SMTP id l35csp2916134ybj; Mon, 23 Sep 2019 11:30:01 -0700 (PDT) X-Google-Smtp-Source: APXvYqzP00UvhAQ+6G7sFThiS9UXxDJDi82u0p5gFNRpyRSjTduGQ5aIFIdDWwfb5AGvEZGPwE3P X-Received: by 2002:a17:906:5948:: with SMTP id g8mr1077148ejr.246.1569263401603; Mon, 23 Sep 2019 11:30:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569263401; cv=none; d=google.com; s=arc-20160816; b=Mk62cM+cJHFMiVP2nF4hng5pvog11FodR5QBD469JrqdSwAV9bScC0P10xd6Oyenu6 R5WGClfdZqKeY3Ltnf3+LOemKT3xLVl78o2w3DmHziblNXl2M2HRwwfNmpt+geC8yjxX BKbDPrSZ/m+n4kHpaqjoIAS7wztYzAPFkqDw0P7slWd5OeajDJTiC2mPq6AVf2CTsU8f 4LViAyCSbvrFJAcDJBBbXYyKrTuwBAdZwq88DIDHGiBE1/QfHiSGefpCIZ7r+w7+y635 hu7DZqQEdO7pVqZeSKl9gB9tA6nXkIknt9rfvytEU1znQ/AXMVxKNYKFCSiqwNzXhqav ne9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=1Wf5z+Rm13hw8aXOmRiiJRXlrz0DWVsNJTijjp3sDN8=; b=z2OCRWkVW2gQmghLw+JSsp5t75Z/CHcoHNAn3fI9Jm9ZbSiysBQ7cXMoOhrHUJRZVf wVzGgv1qs+iPadfYGIf0xu6GuCiXeSzJk50VJSfv5coJV1PIZcpJsnQfLRIfpeysNBKi 87V3wZMMyVQK/ubLL/JGsLRuouMgx1bA6FlX0ZD3K4q4NXeLCCUW6RRxMrm92sR5/OOB 4s3neFZg3quKJdoarRxCPgW4CCcAfiZjmJQkKj/e/fklLYFcNNXPpyDR5GfWLU04TyRg F5c6SqKs/S8cjW6qm8H5ubdJrs0leGa73rgBjyjGyzLh9+E+Uh6HGT2o5MnQD/4Tprg3 tlVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="K/xOoDlf"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r24si6673639edy.417.2019.09.23.11.29.38; Mon, 23 Sep 2019 11:30:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="K/xOoDlf"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727803AbfIVIJR (ORCPT + 99 others); Sun, 22 Sep 2019 04:09:17 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:40451 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727695AbfIVIJQ (ORCPT ); Sun, 22 Sep 2019 04:09:16 -0400 Received: by mail-wm1-f68.google.com with SMTP id b24so6029203wmj.5; Sun, 22 Sep 2019 01:09:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=1Wf5z+Rm13hw8aXOmRiiJRXlrz0DWVsNJTijjp3sDN8=; b=K/xOoDlfMbe+qlFn2KYQhsbEoGG46RQ4dcXfRr3fKJlQLbtL9EPdgxt+/T/8g6b53J URfnwyKh7t/nkt+brA1+jpiisnkFlxo21TjKGVTfKktPPZYFWMrunNTXNygQrgnaEf3D EKYQmmXyd759EJqVsRg77FRcqAYWi2Bz+UQceN4f2hgT86Fi00OOcDO+V4QA4VbNHBI8 RaujcDlCWcbtHdLsYGqRku+a6lR2lDxChO5sT9C855wwGajB/c66C8AZ7h1e0fIrbLyh bsdEF3y92lqbMmsT3zgpsHBEfINZv07ZoLfe6e0G5t9mRoL+JHwm2qH29e6dtTQNnm2T JmbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=1Wf5z+Rm13hw8aXOmRiiJRXlrz0DWVsNJTijjp3sDN8=; b=Ht8jBnci40uqQkUOiAXbkNF9DtRpyegQCsobzT+5m3d2E37KhQl9uAR6K0QPZD79a+ hGXmUcraEuNFKRfN0Neb//gbRnN+kDotBuMrmslt8YnhbXEqsr9+mmYEyqFVmFnCY9cc U4j+zjVEppvvkdGSka8ecMscBUKi5+MLn4KuDmA+6T8InLv58Yayu9CoHskxFQNNHqr6 DUSxKAQZavEuBT7bZl0PONWdXR/5ZIqsTfvZ3/GVORD4BKnL1EsyYWIIzOwCNJYeDZ0A IeGAA2jgel5lUtic2qqWZAaLnX+v2DtKWDSJr7PqUcm0t6xk/KtNY2Xcf5oNZxElplW5 nO7A== X-Gm-Message-State: APjAAAWMHDNty5hQS/EDJK0q1GjXFBGuv3GNR/wycVnkx6YU936ViX0W vRZKw7gSn1wrSzgMiHM93s0= X-Received: by 2002:a1c:a796:: with SMTP id q144mr9656814wme.15.1569139754446; Sun, 22 Sep 2019 01:09:14 -0700 (PDT) Received: from localhost.localdomain ([109.126.147.119]) by smtp.gmail.com with ESMTPSA id x5sm7726983wrt.75.2019.09.22.01.09.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Sep 2019 01:09:13 -0700 (PDT) From: "Pavel Begunkov (Silence)" To: Jens Axboe , Ingo Molnar , Peter Zijlstra , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Pavel Begunkov Subject: [PATCH v2 0/2] Optimise io_uring completion waiting Date: Sun, 22 Sep 2019 11:08:49 +0300 Message-Id: X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Pavel Begunkov There could be a lot of overhead within generic wait_event_*() used for waiting for large number of completions. The patchset removes much of it by using custom wait event (wait_threshold). Synthetic test showed ~40% performance boost. (see patch 2) v2: rebase Pavel Begunkov (2): sched/wait: Add wait_threshold io_uring: Optimise cq waiting with wait_threshold fs/io_uring.c | 35 ++++++++++++------ include/linux/wait_threshold.h | 67 ++++++++++++++++++++++++++++++++++ kernel/sched/Makefile | 2 +- kernel/sched/wait_threshold.c | 26 +++++++++++++ 4 files changed, 118 insertions(+), 12 deletions(-) create mode 100644 include/linux/wait_threshold.h create mode 100644 kernel/sched/wait_threshold.c -- 2.23.0