Received: by 2002:a6b:500f:0:0:0:0:0 with SMTP id e15csp6399751iob; Tue, 10 May 2022 18:18:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyCBZ1l/3i0lK3WiFImI/mj9m9dGMI5jnkO5sj5Jd48DA5NW2fAJJBvyw90HKhaf1AYA5GX X-Received: by 2002:a63:2a4f:0:b0:3c1:bf77:b81f with SMTP id q76-20020a632a4f000000b003c1bf77b81fmr18896408pgq.364.1652231933377; Tue, 10 May 2022 18:18:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1652231933; cv=none; d=google.com; s=arc-20160816; b=suDlu1U3+p5KNOz1uwUqd6Y+Jeo1Lb5MYzT4xYQ/Z6MdJpjqD8JwBgrlNKFJjhtRnB oVy3kMemX2rQlvPJPlM97lQvOrlAWbYgZvVSt+tdYkUEOPMH5Y4ZDlwAukFBd/peYe/2 W1f1LoeX8b2UkGEEHMzP3kcfLU9Yn8DFTjxcFfFKgehEh9NnKsdrJJAfN/Ig7Odkrk0v jxAICIM3DcZ1CUORxzyBPyk7ZqgnTAZMPBQq3reRFETFEUPzNe6Bu2618eOOwE6zAfzp RdoaMhZXmwvYBbzZvY++G/WoMjDExjH8P4oeG4rExCUQug/zyGqrhdbDKqTJSb6dEG0l 1cww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:user-agent:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date; bh=qZp1cc9qqcq+D4HW7N/qd1kaA70AJ+ViU/NAU26+3zk=; b=QRdMeXJamuL+10pISE6W3L76ah3ox7f9UKsv9kL4d2yzbZRMeJR2kWX4e2nDFLY31z sdYLhYvXvgCy0HqicriTKpjoLrSFp/GIMk3F3utpD/ws2Zog/zQfRbkOqSxWIzSUhS2R G1cmsE4xHmaXwjIwJChIdtL4kmzP+sy0jCYxEm0eyJfkxjCljzjQUILiLFKKyV3+S9j7 WjO6lLcOVjK8TnlMOeGbVEw7XnafObnJtWDC3tXI++YdqWamC0cH8GbhFFJr6F+Hcmqw +pc8RoUuNB2vIFf3N/W9eY9QI/Eh5xyYcF7eLUWdqyP4z3XX4bWuoFMhZDIh429ucmMg CQ8Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e21-20020a656895000000b003c668342553si1117643pgt.351.2022.05.10.18.18.31; Tue, 10 May 2022 18:18:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238413AbiEJXlT (ORCPT + 99 others); Tue, 10 May 2022 19:41:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238541AbiEJXlM (ORCPT ); Tue, 10 May 2022 19:41:12 -0400 Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7174BDEA9 for ; Tue, 10 May 2022 16:41:08 -0700 (PDT) Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 11 May 2022 08:41:07 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: byungchul.park@lge.com Received: from unknown (HELO X58A-UD3R) (10.177.244.38) by 156.147.1.127 with ESMTP; 11 May 2022 08:41:07 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: byungchul.park@lge.com Date: Wed, 11 May 2022 08:39:29 +0900 From: Byungchul Park To: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, chris@chris-wilson.co.uk, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jack@suse.com, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, airlied@linux.ie, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com Subject: Re: [PATCH RFC v6 00/21] DEPT(Dependency Tracker) Message-ID: <20220510233929.GB18445@X58A-UD3R> References: <1651795895-8641-1-git-send-email-byungchul.park@lge.com> <20220509001637.GA6047@X58A-UD3R> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org On Tue, May 10, 2022 at 08:18:12PM +0900, Hyeonggon Yoo wrote: > On Mon, May 09, 2022 at 09:16:37AM +0900, Byungchul Park wrote: > > On Sat, May 07, 2022 at 04:20:50PM +0900, Hyeonggon Yoo wrote: > > > On Fri, May 06, 2022 at 09:11:35AM +0900, Byungchul Park wrote: > > > > Linus wrote: > > > > > > > > > > On Wed, May 4, 2022 at 1:19 AM Byungchul Park wrote: > > > > > > > > > > > > Hi Linus and folks, > > > > > > > > > > > > I've been developing a tool for detecting deadlock possibilities by > > > > > > tracking wait/event rather than lock(?) acquisition order to try to > > > > > > cover all synchonization machanisms. > > > > > > > > > > So what is the actual status of reports these days? > > > > > > > > > > Last time I looked at some reports, it gave a lot of false positives > > > > > due to mis-understanding prepare_to_sleep(). > > > > > > > > Yes, it was. I handled the case in the following way: > > > > > > > > 1. Stage the wait at prepare_to_sleep(), which might be used at commit. > > > > Which has yet to be an actual wait that Dept considers. > > > > 2. If the condition for sleep is true, the wait will be committed at > > > > __schedule(). The wait becomes an actual one that Dept considers. > > > > 3. If the condition is false and the task gets back to TASK_RUNNING, > > > > clean(=reset) the staged wait. > > > > > > > > That way, Dept only works with what actually hits to __schedule() for > > > > the waits through sleep. > > > > > > > > > For this all to make sense, it would need to not have false positives > > > > > (or at least a very small number of them together with a way to sanely > > > > > > > > Yes. I agree with you. I got rid of them that way I described above. > > > > > > > > > > IMHO DEPT should not report what lockdep allows (Not talking about > > > > No. > > > > > wait events). I mean lockdep allows some kind of nested locks but > > > DEPT reports them. > > > > You have already asked exactly same question in another thread of > > LKML. That time I answered to it but let me explain it again. > > > > --- > > > > CASE 1. > > > > lock L with depth n > > lock_nested L' with depth n + 1 > > ... > > unlock L' > > unlock L > > > > This case is allowed by Lockdep. > > This case is allowed by DEPT cuz it's not a deadlock. > > > > CASE 2. > > > > lock L with depth n > > lock A > > lock_nested L' with depth n + 1 > > ... > > unlock L' > > unlock A > > unlock L > > > > This case is allowed by Lockdep. > > This case is *NOT* allowed by DEPT cuz it's a *DEADLOCK*. > > > > Yeah, in previous threads we discussed this [1] > > And the case was: > scan_mutex -> object_lock -> kmemleak_lock -> object_lock > And dept reported: > object_lock -> kmemleak_lock, kmemleak_lock -> object_lock as > deadlock. > > But IIUC - What DEPT reported happens only under scan_mutex and > It is not simple just not to take them because the object can be removed from the > list and freed while scanning via kmemleak_free() without kmemleak_lock and object_lock. That should be one of the following order: 1. kmemleak_lock -> object_lock -> object_lock(nested) 2. object_lock -> object_lock(nested) -> kmemleak_lock > Just I'm still not sure that someone will fix the warning in the future - even if the > locking rule is not good - if it will not cause a real deadlock. There's more important thing than making code just work for now. For example, maintainance, communcation via code between current developers and potential new commers in the future and so on. At least, a comment describing why the wrong order in the code is safe should be added. I wouldn't allow the current order in the code if I were the maintainer. Byungchul > > --- > > > > The following scenario would explain why CASE 2 is problematic. > > > > THREAD X THREAD Y > > > > lock L with depth n > > lock L' with depth n > > lock A > > lock A > > lock_nested L' with depth n + 1 > > lock_nested L'' with depth n + 1 > > ... ... > > unlock L' unlock L'' > > unlock A unlock A > > unlock L unlock L' > > > > Yes. I need to check if the report you shared with me is a true one, but > > it's not because DEPT doesn't work with *_nested() APIs. > > > > Sorry, It was not right just to say DEPT doesn't work with _nested() APIs. > > > Byungchul > > [1] https://lore.kernel.org/lkml/20220304002809.GA6112@X58A-UD3R/ > > -- > Thanks, > Hyeonggon