Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756036AbcKCNjM (ORCPT ); Thu, 3 Nov 2016 09:39:12 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:53978 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750898AbcKCNjJ (ORCPT ); Thu, 3 Nov 2016 09:39:09 -0400 Subject: Re: [PATCH 1/4] block: add scalable completion tracking of requests To: Ming Lei References: <1478034325-28232-1-git-send-email-axboe@fb.com> <1478034325-28232-2-git-send-email-axboe@fb.com> CC: Jens Axboe , Linux Kernel Mailing List , linux-block , Christoph Hellwig From: Jens Axboe Message-ID: Date: Thu, 3 Nov 2016 07:38:55 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [64.134.26.173] X-ClientProxiedBy: SN1PR12CA0008.namprd12.prod.outlook.com (10.162.96.146) To MWHPR15MB1198.namprd15.prod.outlook.com (10.175.2.140) X-MS-Office365-Filtering-Correlation-Id: c4884595-64f1-49cc-0dde-08d403eec62f X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1198;2:yQFlPa8jByFUSBhdeUaOcGS3DsKTRKFv9J/dQf7Nni5xi9Zfmqdlemjc8Mou72avb2ufSWNKGmAKGIfjfmY583AcH86m/jRSK0Ho7Jlg9PzPvRA3unXtPVMiCmSRwSWApGCkB3sdeCsEc1sLmAxftUyQ+0YdmCnKDcU0j3rrR00Ztivd+ydZsDr0fSelZBTvb+hZEhOOyo9NtMuV0DPVNw==;3:Lu/sccv9QotdV8WrZb3Pn8ChsRQLlvynfc0tG1WMw0r2y5JT2Ts+jdCR4WAzuK6oiAhLR4/Dv4SY2jbb43a9Kq1FcQ3R5xsTiu5bU8AOkCCeKsXNcLAf7XbCL3l2d6Q732Me9T2bkYQG29B5+Rd/UQ== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1198; X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1198;25:GUhFoUgvjHnq2mQsFMlP0rmPGK69MoLESxipk8mSFQp/BK12VnuImVlS8Bwzz0u1z7yGg+xAGxuzeIB5iPc4HCdDudqEZBU4poUG0+TJyajG5wkZ/qXB4gPQmUS2yDGW5HBsNTNCs6lddBakt3QZw/WJA5MjGG/goZ7IrLYoWkX7vZFEoe7hU5Q+peQtIBGXPht4OSfOyPxEshfXtrVNYAmfgxZ5MWqiwcY3bJDIa2P54VZK8E9WafBTX0+V1aPyWM2o6xDzyKc9kVNrPoLF72gHVdymwqqAV7RnfP1W1UBQXZ8Ak8N+QMHTi29y70Km/OiOPcRx2VB6Y7+bzZvXLKUy6m+dEpdb8YzixW1TBM4oklG7OoTWIwRGMZjDLH87gKD03arzHpDa0x55I4eFSDai53AoZU9QTsS1XRVp3lzVHTJLXeuuMkVDV48HjPgVZgu/9YGcu7yhzvtSzC0/EhJC+gAp0/hMXPn+QQDPO2AaQkx/cWkRMIHnWs2+70GchRzj/2cNf6IdnZIU+bOtw0ItUKWLInPftNymGulX3pTByN2Ope8YX4yrI7lJxOxJQ3arp6Xk1jBvViUh4MsYeWHJdy534CtgBtWQB5PF+Y3il7ibZxs4Tb6onaRw6E4AWe1Tqhpns3NX+IKudbR8STMXuo8Yh4xbPsYBCWsEEv1hN2WQrWUMi7PldRWaj8mBoNFYTWbrR9vFrYcSaqKoJp8p+HDPDfNSJrcjgaYE9KY= X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1198;31:WwlSGxIsGMoSlU4afZL5lrgQ43AgzF0vbJEb1VkhELX9EgTNp9rjs77iVoOsetnJjSKxgC0zuS56IrszgZsr978a4n+syj6B4QP4ARIJHJhbrKJ1JCHMpBSpVgGO6BeQw1RiMXXiyNUpn+8tq2G1jEe+gIdfXhiHVRxHAg6hds3Ldm1cajGrM6/MNZaI49kbhFkjUdq5RQhBYF00v4aW/LKpd8KSLNLzLXEJB/gOGlj8NlNZ5wM56pZsDrHP7zMB;20:l9pRWVCEL/YpzPVv2u/9rS+eBabEZIzw8BfLDO5PRqTtxR8IERdm1BLUoJx/wMXYRUdP450iJpDFz12qh98JHzucXDFXWxXwmIaGi4NGJyT1mKq1RIKqqR2zs04zUwayNkkmVc98iaXD9TE1M4T2G0z+CzoxSwAS3XiGfJkRO/c=;4:mx4oiGS8K7xNqgZfinwyt7HINiF9nYAp8gbZABREowOJBQ1emW1zy/62iZFlw2Q6219Nk0fsIm7RTR+pJdFUmuXP5HmdX5Awd4eUs+d3LYkIXk/VIKbIXudgYBg+2dJjnC+ozYqs2iNSdzfsewIqUl5q+c8qlwmdRkWiubELaYNwY09ANEHOJ+GWSxYxlBwKqgRPWNmjFdmGFw2Pa7bEDyV3gdJkmJ+PUI4ADbo/Hpaz/Gdg7gKfqM3wL23Jqq4Q1jpL6Xn7jH4YbC4XUfng5dsG6bOoOw2Ro84AKvBP+Adl+Ery7pxVvh/E10eIZnPryP4/EFBAwrymw2zJqqIu3kdyBpSLc5WMkCf581xMXP+stHqmI0QEBccQlKLeQXQqcLl4zFdrYTuhl6enmIjs1w== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:MWHPR15MB1198;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1198; X-Forefront-PRVS: 011579F31F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(377454003)(24454002)(189002)(199003)(5660300001)(42186005)(105586002)(106356001)(92566002)(36756003)(23676002)(68736007)(305945005)(77096005)(33646002)(83506001)(31686004)(50466002)(230700001)(4001350100001)(86362001)(76176999)(586003)(31696002)(2906002)(4326007)(6916009)(2950100002)(81156014)(8676002)(65826007)(101416001)(81166006)(54356999)(50986999)(64126003)(110136003)(117156001)(97736004)(7846002)(6666003)(7736002)(65806001)(65956001)(66066001)(6116002)(3846002)(47776003)(189998001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR15MB1198;H:[192.168.6.194];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtNV0hQUjE1TUIxMTk4OzIzOlJrSlhtbW9kQStRbE1lWk5xbm83eldEMnoz?= =?utf-8?B?VGgvS1RDODR2Ry9OS0EyVjVMeEZtK3V5UjNBcTJEZFJUUjlpQkZETnN2aE5B?= =?utf-8?B?NWpCTVNoUXZKRk9ReXFBREkxWEcrekJXMjBhSm1hcTRHWEEvN0ZIWWZiZHlk?= =?utf-8?B?ZkpRc0R2Wjg1MXhHY2VnY0VwQlBYV2kyTWVqdUtnQ0poMExHN2x2a3BPVENF?= =?utf-8?B?eHRzYnF3anh1MWVsNWNkV3ZLY0ZHSnFOTXErUzF0cG1LNDdBRXJlankweW1r?= =?utf-8?B?bk9xV2pOUDZZVEMwY2cwaHZwMmVUckVxM1JxaGR5Y05vcUdvUXUyOGhXTkpN?= =?utf-8?B?cmQyMVRuNnJmdWF2TlpYdU4rd3lXNThoMDUzem5pY0tWU3ZHNGUybmdic2c0?= =?utf-8?B?bFc5YmFuRXYrK1dXS0c2R044MVhSMFVTZzg3blRRVjQzdVdUNjFYTEFkbFVm?= =?utf-8?B?SGgxQTRQMVZWOGFZbzVkUlB2aTQrL1QxaHRuUys1SHVTQ3dNLzBRZ1BCeU11?= =?utf-8?B?YVNFYk9tN3ZBYUFQUHY1YklDWlBnVzhiNi95bUd5VnVaUEVPY0dyTVZ0TFZt?= =?utf-8?B?WlpjVzVXT0VPQ2JwdWZEakpPTzNWYUIxbWpEMmhZd2xsVkZYRW9wVkdzK1N0?= =?utf-8?B?QnBTS1crUnZ3U05VTXdkS0VKMjZiWWdtVTMvOE1FamJPZWMxS3VCWXV5eldl?= =?utf-8?B?YXhISUxHdTl0TnhTWGtzWHVxU0lxSTFqbGdQUmxUVk5Gd0JCY0IvTHVnMDZL?= =?utf-8?B?OGpSSTA1UjloS2pMbnFEWENtOWRkRG1ZN0RlNkFDRzAzUHJJSU0yendFZ1VD?= =?utf-8?B?b1B2eVdibFBNdFRqdDRPb2JsR3FTdHZWc2pnQTNneTJzanhKSUkwSG82QUw0?= =?utf-8?B?OFB1c0UvS0lqYjcwV1BNbEd3dHgzQlVKQkZXd3F1ei91WTA2Y3l2ODFLd0x5?= =?utf-8?B?OVV5MnFVeHB0N0J4c2t0dDhHNktZRWhhbHRyTVNtd1lDL1hRZW9iZzF4QW5J?= =?utf-8?B?ZSsreGcySlFhM01NejN4a2tRNzdGbXQ5RFd0UGZwYnMrY3c3WStEWmZhWTAv?= =?utf-8?B?QTJOT3NFaFNERzl5ZUFjbzZEeUNsVkcwTTRsNDI5ZFVnd1M3cGhhNFF2Ykp2?= =?utf-8?B?ZjhHLzdiUWNkNk16WlJ6dGwrbmp6bHl2ZTJFWkRSSE1RbitBOVM3TTFYU21U?= =?utf-8?B?eUt5N3p6TlVuUWwvUkplRklGb0VBQ1dqbVVKSWpxclppVmM3cmpYU1ZZNkR1?= =?utf-8?B?UU9FN0ZsQ3Y1QTQvQmRRMHVWQWw2T29oaldyL3VhVGlEM3VRTkl2YzJQeGtF?= =?utf-8?B?c21ra3RxLzZQYzRRdE9TZHRnTGE5clhOaWc0ck5DY1FhdU1RYnBNeTczc1J1?= =?utf-8?B?OFM1Yng2MTJqRHRjYXdkSGZjK0hYWkJvN2xIMzhvbHcxOEIrWUlrOWJ5ZzV3?= =?utf-8?B?QTRsNzhnVkdGVXlFTFZ1alhWV1hhZGU5K2REckpMZG9aUEl5cGFHdEhyK0VI?= =?utf-8?B?ck93eUxBc2VLNkVoWUt0cWM5bko0K3c0a1puS3FMYTBBQVdkS3NTcFRub3Fn?= =?utf-8?B?d2M2aHp1QnhST1dqTFZROTBxUGtLcFg0VDZWdXhhb3hYbnBXNXRzV0pzNVpr?= =?utf-8?B?WmZOb3FsUXUyM1h5bnFKTXRPQnVJbm1ZRHJJZUlEeGU4SDJBRUtqakJLZnR4?= =?utf-8?B?a0xoUXQ4b01hb3RvLzIwd2JVdHM1cDlNZHp3cHdmSVRoUzd0clRCRjVFV1Jo?= =?utf-8?B?S0N0L0hXSDRYWk5ZYWw5UT09?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1198;6:I44QrFtTNiAyuIh21hiyfx2JtX8+uHX1PbP2q3oNpi5SVZ69/FonlC0mk3voEJqsk8IgfMM1MLhQQNYuoQ91lKkF85MAW7CrZ6nWjDBSPFP3Kqu6mRzxJgbW6w4SGGyvNDc1/OwnUPbZJQI1LlBCD2qcMXK6FWmt/W0tQa6v6FDoSOAXmPlXrJIoOWII97L4zMYjQBwlsWDC8B1BRfXoTBHwO4Yao9qaya5+luQ6de97EI3WjuBvQtuH0t+j2DbCwXvJo6e5dLiuVjaKbteOSs1Tsxd6wQ4hNWSmk3TGNVhu5DxAU8aWCxqXlF8QxpBi;5:cv3WfZUZw6oV+k/kDeQ3QJHKAW1055GnJ+FaCxoS6POLmtSf2do95MaGfdQbqOWjDa4zbEdft9xC7kFXzIU3vE92z5da3tfcY0BA0iD5zhAW2F6wEKvJXo9nwiwIeTOSd8OIl1u/mG/jJ9krlI7FIKUP75rjbPWrYs4+a/Cm9lo=;24:Wgu8DwQCURLOBIdvwrnQnbRds6V1+WLHOU5xKfsqyjwYCCNCfFh71km2atrSFzZKnYxNH2cHZuZ5VdzPxzEGnILV1cqOFA3QJohJsAz5f04= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1198;7:dZpY7/wLWiDzZcusA7jgEp+cVqTcYlhoOsAi38dl3ADtpHGDR/HbXfR2qmhVM7NpFUYSGKCSux6ZP3vOqTYBshvj8wtV0BPUhLqOSlsg4tqwNC1SzHOki0E4ZMkQtPL8Rj4OG627AAqtQm7Liiou7onBVcTvs9XNK5BaOAlCTV4kowbGN9QO9E8Ik9h+IbmVNlEvubq5/uJJN1RIL0YRfPPJzhQ2Tzsf5rSTPyGwj+v8HXT9hOK+lo9cY4hBZSuFuh6hYJp4fXfMEm89EfSChU/l0BosvJRRnbxpHWGL/fTcUY7hOs/MdewXwEEtdJjD7TnFv3P0+gFfEKm/Noldgtk8D4V0kXUnGXS652qhYdE=;20:uE2He17viY6qAgF3tNPV2FgyZRAK+24xEruRMLP4udMREwIr45K3FgUuD+b3DaolxCFbU6YpYhVEhj8qblPqU38vSgZouWiNmmoyKvZLyQ0VLdKIU9luiU9x9ZSRF442W9tLbUCVazXA5wuWsrHhtOcUjH0L4hFZdlt9C6+6eQA= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2016 13:39:01.4293 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1198 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-11-03_04:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1128 Lines: 30 On 11/03/2016 05:17 AM, Ming Lei wrote: >> diff --git a/block/blk-core.c b/block/blk-core.c >> index 0bfaa54d3e9f..ca77c725b4e5 100644 >> --- a/block/blk-core.c >> +++ b/block/blk-core.c >> @@ -2462,6 +2462,8 @@ void blk_start_request(struct request *req) >> { >> blk_dequeue_request(req); >> >> + blk_stat_set_issue_time(&req->issue_stat); >> + >> /* >> * We are now handing the request to the hardware, initialize >> * resid_len to full count and add the timeout handler. >> @@ -2529,6 +2531,8 @@ bool blk_update_request(struct request *req, int error, unsigned int nr_bytes) >> >> trace_block_rq_complete(req->q, req, nr_bytes); >> >> + blk_stat_add(&req->q->rq_stats[rq_data_dir(req)], req); > > blk_update_request() is often called lockless, so it isn't good to > do it here. It's not really a concern, not for the legacy path here nor the mq one where it is per sw context. The collisions are rare enough that it'll skew the latencies a bit for that short window, but then go away again. I'd much rather take that, than adding locking for this part. -- Jens Axboe