Subject: Re: [Xen-devel] [PATCH] xen/events: Mask a moving irq
To: David Vrabel <david.vrabel@citrix.com>, konrad.wilk@oracle.com
References: <1458218750-5202-1-git-send-email-boris.ostrovsky@oracle.com>
 <56EAD55A.6020801@citrix.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
        stable@vger.kernel.org
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <56EAE0F7.4060703@oracle.com>
Date: Thu, 17 Mar 2016 12:53:11 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.1.0
MIME-Version: 1.0
In-Reply-To: <56EAD55A.6020801@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1794
Lines: 54

On 03/17/2016 12:03 PM, David Vrabel wrote:
> On 17/03/16 12:45, Boris Ostrovsky wrote:
>> Moving an unmasked irq may result in irq handler being invoked on both
>> source and target CPUs.
>>
>> With 2-level this can happen as follows:
>>
>> On source CPU:
>>          evtchn_2l_handle_events() ->
>>              generic_handle_irq() ->
>>                  handle_edge_irq() ->
>>                     eoi_pirq():
>>                         irq_move_irq(data);
>>
>>                         /***** WE ARE HERE *****/
>>
>>                         if (VALID_EVTCHN(evtchn))
>>                             clear_evtchn(evtchn);
>>
>> If at this moment target processor is handling an unrelated event in
>> evtchn_2l_handle_events()'s loop it may pick up our event since target's
>> cpu_evtchn_mask claims that this event belongs to it *and* the event is
>> unmasked and still pending. At the same time, source CPU will continue
>> executing its own handle_edge_irq().
>>
>> With FIFO interrupt the scenario is similar: irq_move_irq() may result
>> in a EVTCHNOP_unmask hypercall which, in turn, may make the event
>> pending on the target CPU.
>>
>> We can avoid this situation by moving and clearing the event while
>> keeping event masked.
> Can you do:
>
>     if (unlikely(irqd_is_setaffinity_pending(data))) {
>         masked = test_and_set_mask()
>
>         clear_evtchn()
>         irq_move_masked_irq()

I did think about this but then I wasn't sure whether this might open 
some other window for things to sneak in. It shouldn't but these things 
are rather subtle so I'd rather leave the order of how operations are 
done unchanged.

But I should indeed use irq_move_masked_irq() instead of irq_move_irq().

-boris

>
>         unmask(masked);
>     } else
>         clear_evtchn()