Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756683AbcKEU7w (ORCPT ); Sat, 5 Nov 2016 16:59:52 -0400 Received: from mail-pf0-f177.google.com ([209.85.192.177]:36050 "EHLO mail-pf0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754875AbcKEU7v (ORCPT ); Sat, 5 Nov 2016 16:59:51 -0400 Subject: Re: [PATCH 1/4] block: add scalable completion tracking of requests To: Ming Lei References: <1478034325-28232-1-git-send-email-axboe@fb.com> <1478034325-28232-2-git-send-email-axboe@fb.com> Cc: Jens Axboe , Linux Kernel Mailing List , linux-block , Christoph Hellwig From: Jens Axboe Message-ID: <5272bdb8-8a75-36ac-11d9-33d3719fa2ce@kernel.dk> Date: Sat, 5 Nov 2016 14:59:47 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1172 Lines: 26 On 11/04/2016 05:13 PM, Ming Lei wrote: >> Yes, that might be a good idea, since it doesn't cost us anything. >> For the mq case, I'm hard pressed to think of areas where we could >> complete IO in parallel on the same software queue. You'll never have >> a software queue mapped to multiple hardware queues. So we should >> essentially be serialized. > > For blk-mq, blk_mq_stat_add() is called in __blk_mq_complete_request() > which is often run from interrupt handler, and the CPU serving the > interrupt can be different with the submitting CPU for rq->mq_ctx. And > there can be several CPUs handling the interrupts originating from > same sw queue. BTW, one small improvement might be to call blk_mq_stat_add() on the curent ctx, in case it's different than rq->mq_ctx. That can happen if we have multiple CPUs per hardware queue. In reality, even for that case, a real race is rare. You'd have to rebalance interrupt masks basically, at least on x86 where multiple CPUs in the IRQ affinity mask still always trigger on the first one. So while we could just grab the current ctx instead, I don't think it's going to make a difference in practice. -- Jens Axboe