Received: by 2002:a25:683:0:0:0:0:0 with SMTP id 125csp204673ybg; Mon, 8 Jun 2020 21:09:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVegp/JSZR1I4qzfAtKbSuQv8owoSm2zvirCVZMVxoCQ/8xZlnpVRVMUWHKB2WFo2nLExV X-Received: by 2002:a17:906:edb6:: with SMTP id sa22mr24620935ejb.393.1591675789983; Mon, 08 Jun 2020 21:09:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591675789; cv=none; d=google.com; s=arc-20160816; b=DfQmvE6cTL622lEQ6ROrJtLIADwb8bD8lGr0T3I4YYRUg4FDvWjBCOIaVLsZIY6A/C E+WWetCxhnyTAWvUfGVsBU5p4VHCye+dNabMJxPtwJtjVLPUsdPD+WIe0DW6nn+jtD0L VkyGST2SjDRz6elcOi0OEpBvEdRLXml+GYgSfC1jQMZ1n3BphpRMKjbT287X0MZEDkUa rV+vCChC3ngBYAySLn3um4IXzGy1ZfLUPH4qHpllcMQcn0fQQJf12Dp3g76w0KvGc9tj l0JsYKfkL3Dx9eiT4abjAdECcZOfXfLXawZSFr3rMbK3vvUjJeZA/HlGAL3Td50GwJLl irfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=kcejXHVr0uPuufny+fpt/7B1mpNCm88I0z6cAzFfi+8=; b=dhwM3mSUKu6f6y8S+0sdGbwN0/zFKN2F+NFx86z92xo6dFVGbl70olivlVhtnvEAU0 Au04Nqbyi0n4KIaxM7NsJL82GaVyxWt4dDIjSQseoIR36s9/hgCSK778/Mi698BkmwOh wmveCYuXqtb3JP/zGh6kDwDxRUgPA6mGjgyMeh37pP8oyUsI4x5rJ9Dg+vsyyswWxdN8 E5HtV/1/xh1SEm89+VGB3JS5PnT3y0EdkSBPYepSXtqXunO0II8KtyfkPKnKEEnuxhdv HP4gMBFB2LAdYrsDArKEPS+h5R/Grh0cMvWcni4kFZhN1ub4BeXxLOMKCb1AJFq4w9UY XMAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@netflix.com header.s=google header.b=o4u7oQMF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=netflix.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id c2si10742020edm.499.2020.06.08.21.09.26; Mon, 08 Jun 2020 21:09:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@netflix.com header.s=google header.b=o4u7oQMF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=netflix.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726871AbgFIEHn (ORCPT + 99 others); Tue, 9 Jun 2020 00:07:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725772AbgFIEHm (ORCPT ); Tue, 9 Jun 2020 00:07:42 -0400 Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com [IPv6:2607:f8b0:4864:20::232]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CDC94C03E969 for ; Mon, 8 Jun 2020 21:07:41 -0700 (PDT) Received: by mail-oi1-x232.google.com with SMTP id j189so17472166oih.10 for ; Mon, 08 Jun 2020 21:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=kcejXHVr0uPuufny+fpt/7B1mpNCm88I0z6cAzFfi+8=; b=o4u7oQMFJn1d0+UfvVvq8De/R/SOF//a+JKt6mz0YUEPNPfNrtOqpEgF6BoweSG+hL 1/N0PYtVrNv017WCtDaNVWcnFTeo310ag/8h/MB5UrFOydL8IYJvSkZ0PVSFdBK1FK+j SWCNhLOS+krU+vGMEQkoK0D7DiA2amAHGSfWw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=kcejXHVr0uPuufny+fpt/7B1mpNCm88I0z6cAzFfi+8=; b=nH6IFDWIoMlHwuxkPiM7k1LwuaC4TwxY2knNBa73oyY9Xttf3CJCbSPk8y+H9DMVxt gBY9uvww+jYisbp1q+OrZxW9xt6rY/wYoKHZZZK+R24JtXDuoqJYob3OMWlbkDZYTemq 5buCKO9/pEhWMW3IhbMotU08ISBf/QvPXIOD0uZWAQnDLTbb9djgq0xGXRe6cpBJx4mo g2cz8BVlpkJHpseyoxtZs+feYoCq8ML17/qPUxV6Y6p1DVaZ5YNzCHOfaEDRIRLbEmE5 9LLni6oY8Iq3XggZhmYRBL7ieny9hmiEdMrJwgeeoAFHZymKp9Bf+FmjFYcOHbdbEUOv MRhg== X-Gm-Message-State: AOAM532FNxmTAXpQT/ReXm+ytR9IRBa/gc2VDeHWn4Nd0Mq8LTCXQThP yqIgdOKnMcCytsz9h7vlgSamaw== X-Received: by 2002:aca:4d13:: with SMTP id a19mr2020978oib.158.1591675660976; Mon, 08 Jun 2020 21:07:40 -0700 (PDT) Received: from mezcal.netflix.com ([2600:1700:3ec3:2450:25ca:3996:acb2:84a6]) by smtp.gmail.com with ESMTPSA id b3sm2846415ooq.36.2020.06.08.21.07.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 21:07:40 -0700 (PDT) From: Josh Snyder To: Jens Axboe , Mikulas Patocka , Mike Snitzer Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Josh Snyder Subject: [RFC 0/2] Increase accuracy and precision of sampled io_ticks Date: Mon, 8 Jun 2020 21:07:22 -0700 Message-Id: <20200609040724.448519-1-joshs@netflix.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit 5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting") introduces a sampling technique for calculating io_ticks. The sampling algorithm introduces bias in the calculation of I/O utilization. In my production system, this bias means that a workload which previously reported 10% I/O utilization now reports 80%. Patch 1 of this series eliminates the bias. The sampling technique is also subject to statistical noise. Because it infers io_ticks based on only 100 samples per second, io_ticks becomes imprecise, and subject to swings when measuring both random and deterministic workloads. Patch 2 of this series provides increased precision by raising the sampling rate.