Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Feedback-ID: iad51458e:Fastmail
Date:   Thu, 12 Jan 2023 23:18:51 -0800
From:   Boqun Feng <boqun.feng@gmail.com>
To:     "Paul E. McKenney" <paulmck@kernel.org>
Cc:     David Woodhouse <dwmw2@infradead.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        seanjc@google.com, Joel Fernandes <joel@joelfernandes.org>,
        Matthew Wilcox <willy@infradead.org>,
        Josh Triplett <josh@joshtriplett.org>, rcu@vger.kernel.org,
        Michal Luczaj <mhal@rbox.co>,
        Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH] Documentation: kvm: fix SRCU locking order docs
Message-ID: <Y8EF24o932lcshKs@boqun-archlinux>
References: <20230111183031.2449668-1-pbonzini@redhat.com>
 <a14a13a690277d4cc95a4b26aa2d9a4d9b392a74.camel@infradead.org>
 <20230112152048.GJ4028633@paulmck-ThinkPad-P17-Gen-1>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20230112152048.GJ4028633@paulmck-ThinkPad-P17-Gen-1>
Precedence: bulk

On Thu, Jan 12, 2023 at 07:20:48AM -0800, Paul E. McKenney wrote:
> On Thu, Jan 12, 2023 at 08:24:16AM +0000, David Woodhouse wrote:
> > On Wed, 2023-01-11 at 13:30 -0500, Paolo Bonzini wrote:
> > > 
> > > +- ``synchronize_srcu(&kvm->srcu)`` is called inside critical sections
> > > +? for kvm->lock, vcpu->mutex and kvm->slots_lock.? These locks _cannot_
> > > +? be taken inside a kvm->srcu read-side critical section; that is, the
> > > +? following is broken::
> > > +
> > > +????? srcu_read_lock(&kvm->srcu);
> > > +????? mutex_lock(&kvm->slots_lock);
> > > +
> > 
> > "Don't tell me. Tell lockdep!"
> > 
> > Did we conclude in
> > https://lore.kernel.org/kvm/122f38e724aae9ae8ab474233da1ba19760c20d2.camel@infradead.org/
> > that lockdep *could* be clever enough to catch a violation of this rule
> > by itself?
> > 
> > The general case of the rule would be that 'if mutex A is taken in a
> > read-section for SCRU B, then any synchronize_srcu(B) while mutex A is
> > held shall be verboten'. And vice versa.
> > 
> > If we can make lockdep catch it automatically, yay!
> 
> Unfortunately, lockdep needs to see a writer to complain, and that patch
> just adds a reader.  And adding that writer would make lockdep complain
> about things that are perfectly fine.  It should be possible to make
> lockdep catch this sort of thing, but from what I can see, doing so
> requires modifications to lockdep itself.
> 

Please see if the follow patchset works:

	https://lore.kernel.org/lkml/20230113065955.815667-1-boqun.feng@gmail.com

"I have been called. I must answer. Always." ;-) 

> > If not, I'm inclined to suggest that we have explicit wrappers of our
> > own for kvm_mutex_lock() which will do the check directly.
> 
> This does allow much more wiggle room.  For example, you guys could decide
> to let lockdep complain about things that other SRCU users want to do.
> For completeness, here is one such scenario:
> 
> CPU 0:  read_lock(&rla); srcu_read_lock(&srcua); ...
> 
> CPU 1:  srcu_read_lock(&srcua); read_lock(&rla); ...
> 
> CPU 2:  synchronize_srcu(&srcua);
> 
> CPU 3: 	write_lock(&rla); ...
> 
> If you guys are OK with lockdep complaining about this, then doing a

Actually lockdep won't complain about this, since srcu_read_lock() is
always a recursive read lock, so it won't break other srcu_read_lock().
FWIW if CPU2 or CPU3 does

	write_lock(&rla); 
	synchronize_srcu(&srcua);

it's a deadlock (with CPU 1)

Regards,
Boqun

> currently mythical rcu_write_acquire()/rcu_write_release() pair around
> your calls to synchronize_srcu() should catch the other issue.
> 
> And probably break something else, but you have to start somewhere!  ;-)
> 
> 							Thanx, Paul