Subject: Re: [PATCH 4/6] x86,kvm: Fix apf_task_wake_one() serialization
To: Davidlohr Bueso <dave@stgolabs.net>, mingo@redhat.com,
        peterz@infradead.org
Cc: npiggin@gmail.com, paulmck@linux.vnet.ibm.com,
        linux-kernel@vger.kernel.org, Davidlohr Bueso <dbueso@suse.de>
References: <20170905190022.1474-1-dave@stgolabs.net>
 <20170905190022.1474-5-dave@stgolabs.net>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <c3f126ad-4248-69a7-4b31-141c39f66988@redhat.com>
Date: Sun, 10 Sep 2017 11:20:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <20170905190022.1474-5-dave@stgolabs.net>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1761
Lines: 57

On 05/09/2017 21:00, Davidlohr Bueso wrote:
> During code inspection, the following potential race was seen:
> 
> CPU0   	    		    	     	CPU1
> kvm_async_pf_task_wait			apf_task_wake_one
>   [S] prepare_to_swait(&n.wq)
> 					  [L] swait_active(&n->wq)
> 					  [S] hlist_del_init(&n->link);
>   [L] if (!hlist_unhahed(&n.link))
> 	schedule()
> 
> Properly serialize swait_active() checks such that a wakeup is
> not missed.
> 
> Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
> ---
>  arch/x86/kernel/kvm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
> index 874827b0d7ca..aa60a08b65b1 100644
> --- a/arch/x86/kernel/kvm.c
> +++ b/arch/x86/kernel/kvm.c
> @@ -180,7 +180,7 @@ static void apf_task_wake_one(struct kvm_task_sleep_node *n)
>  	hlist_del_init(&n->link);
>  	if (n->halted)
>  		smp_send_reschedule(n->cpu);
> -	else if (swait_active(&n->wq))
> +	else if (swq_has_sleeper(&n->wq))
>  		swake_up(&n->wq);
>  }

After Nick's patch, swake_up starts with:

	smp_mb();
        if (!swait_active(q))
                return;

so we can just remove the test here (and in patch 2).

The other patches could also use a better swait API, for example:

1) add a public __swake_up routine that omits the memory barrier, and
which can be used in patch 3.  Perhaps better: omit the out-of-lock
check in __swake_up: then the caller can use it if it knows there is a
waiter.  In those cases the memory barrier is expensive.

2) change swake_up and __swake_up to return true if they woke up a
process (or alternatively 0/-EAGAIN).  Patches 5 and 6 now need not call
anymore either swq_has_sleepers or swait_active, and that saves a memory
barrier too.

What do you think?

Thanks,

Paolo