Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 29 Jul 2019 16:33:19 +0100
From:   Catalin Marinas <catalin.marinas@arm.com>
To:     Valentin Schneider <valentin.schneider@arm.com>
Cc:     Nikolay Borisov <nborisov@suse.com>, linux-btrfs@vger.kernel.org,
        paulmck@linux.ibm.com, andrea.parri@amarulasolutions.com,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/2] Refactor snapshot vs nocow writers locking
Message-ID: <20190729153319.GH2368@arrakis.emea.arm.com>
References: <20190719083949.5351-1-nborisov@suse.com>
 <ed015bb1-490e-7102-d172-73c1d069476c@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ed015bb1-490e-7102-d172-73c1d069476c@arm.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Some nitpicking below:

On Mon, Jul 29, 2019 at 03:13:42PM +0100, Valentin Schneider wrote:
> specs.tla:
> 
> ---- MODULE specs ----
> EXTENDS Integers, Sequences, TLC
> 
> CONSTANTS
>     NR_WRITERS,
>     NR_READERS,
>     WRITER_TASK,
>     READER_TASK
> 
> WRITERS == {WRITER_TASK} \X (1..NR_WRITERS)
> READERS == {READER_TASK} \X (1..NR_READERS)
> THREADS == WRITERS \union READERS

Recommendation: use symbolic values for WRITERS and READERS (defined in
.cfg: e.g. r1, r2, r3, w1, w2, w2). It allows you do to symmetry
optimisations. We've also hit a TLC bug in the past with process values
made up of a Cartesian product (though it may have been fixed since).

> macro ReadLock(tid)
> {
>     if (lock_state = "idle" \/ lock_state = "read_locked") {
>         lock_state := "read_locked";
>         threads[tid] := "read_locked";
>     } else {
>         assert lock_state = "write_locked";
>         \* waiting for writers to finish
>         threads[tid] := "write_waiting";
>         await lock_state = "" \/ lock_state = "read_locked";

lock_state = "idle"?

> macro WriteLock(tid)
> {
>     if (lock_state = "idle" \/ lock_state = "write_locked") {
>         lock_state := "write_locked";
>         threads[tid] := "write_locked";
>     } else {
>         assert lock_state = "read_locked";
>         \* waiting for readers to finish
>         threads[tid] := "read_waiting";
>         await lock_state = "idle" \/ lock_state = "write_locked";
>     };
> }

I'd say that's one of the pitfalls of PlusCal. The above is executed
atomically, so you'd have the lock_state read and updated in the same
action. Looking at the C patches, there is an
atomic_read(&lock->readers) followed by a
percpu_counter_inc(&lock->writers). Between these two, you can have
"readers" becoming non-zero via a different CPU.

My suggestion would be to use procedures with labels to express the
non-atomicity of such sequences.

> macro ReadUnlock(tid) {
>     if (threads[tid] = "read_locked") {
>         threads[tid] := "idle";
>         if (\A thread \in THREADS: threads[thread] # "read_locked") {
>             \* we were the last read holder, everyone else should be waiting, unlock the lock
>             lock_state := "idle";
>         };
>     };
> }

I'd make this close to the proposed C code with atomic counters. You'd
not be able to check each thread atomically in practice anyway.

-- 
Catalin