Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753053AbbF2Ob2 (ORCPT ); Mon, 29 Jun 2015 10:31:28 -0400 Received: from mail-vn0-f45.google.com ([209.85.216.45]:33890 "EHLO mail-vn0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753374AbbF2ObT (ORCPT ); Mon, 29 Jun 2015 10:31:19 -0400 MIME-Version: 1.0 In-Reply-To: References: <1434894751-6877-1-git-send-email-akinobu.mita@gmail.com> <1434894751-6877-5-git-send-email-akinobu.mita@gmail.com> Date: Mon, 29 Jun 2015 23:31:19 +0900 Message-ID: Subject: Re: [PATCH 4/4] blk-mq: fix mq_usage_counter race when switching to percpu mode From: Akinobu Mita To: Ming Lei Cc: Linux Kernel Mailing List , Jens Axboe Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1970 Lines: 41 2015-06-24 21:35 GMT+09:00 Ming Lei : > On Sun, Jun 21, 2015 at 9:52 PM, Akinobu Mita wrote: >> percpu_ref_switch_to_percpu() and percpu_ref_kill() must not be >> executed at the same time as the following scenario is possible: >> >> 1. q->mq_usage_counter is initialized in atomic mode. >> (atomic counter: 1) >> >> 2. After the disk registration, a process like systemd-udev starts >> accessing the disk, and successfully increases refcount successfully >> by percpu_ref_tryget_live() in blk_mq_queue_enter(). >> (atomic counter: 2) >> >> 3. In the final stage of initialization, q->mq_usage_counter is being >> switched to percpu mode by percpu_ref_switch_to_percpu() in >> blk_mq_finish_init(). But if CONFIG_PREEMPT_VOLUNTARY is enabled, >> the process is rescheduled in the middle of switching when calling >> wait_event() in __percpu_ref_switch_to_percpu(). >> (atomic counter: 2) > > This can only happen when freezing queue from CPU > hotplug happens before running wait_event() from blk_mq_finish_init(). > > So looks like we still may avoid the race by moving adding queue > into all_q_list to blk_mq_register_disk(), but the mapping need to > update at that time. As you suggested, we can avoid the problems described in patch 1/4, 2/4, and 4/4 by adding to all_q_list in blk_mq_register_disk() and removing from all_q_list in blk_mq_unregister_disk(). On the other hand, that change breaks the patch 3/4. Because it allows the requests to be inserted before adding the queue to all_q_list. As we have disscussed about patch 3/4, there is a problem if the request is inserted on the just onlined CPU before establishing new mapping. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/