Received: by 2002:a05:6a10:9848:0:0:0:0 with SMTP id x8csp3526005pxf; Mon, 15 Mar 2021 11:30:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwflIPfHRdzZDOjVSF4Bh1HUI6kkk8SKBwTJC7CEYgFjlJ+hE/vjHcTtieygxKCRLGD3p86 X-Received: by 2002:aa7:d4cb:: with SMTP id t11mr31158567edr.202.1615833047207; Mon, 15 Mar 2021 11:30:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1615833047; cv=none; d=google.com; s=arc-20160816; b=KR2T/Q7CYocAc6Tdhltx8U5RCsFd7z9YORsdHGTyEJlZW4jHklTZLd+PZc+PUkndF2 nGmJ9LeyqUkGpiw7EbwEGcOX7A8R5Ra7cVSSLQnPdzSIfRtKQc/JyXzZz7fu4JPDheFj iNumzfxqh8MWVJjuZKSGZK0scwGx0z3lLOmJq/iyS0TdfxKC5QA9BPnG//YL7D0mjrNA xtvoLBK7+Dw1WAxOgzYH1utJyHvjn4LIkk7Dop9m7l2AQmkku2WECAOBtG2lv6eMoZCy QnZQYoJa+kyjrJFZxrIIQO/GxTSksKewyd0QmE0Y0Rk7HStP6c7ggrGkUTSlwtIErme3 5AYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=gE2T3PulqSEaoS3bbSn2iqw4eoDsmItrY9YLX/CSIr4=; b=iF6neXObFBQ+KpiBOboHpp+0FLllcEaPcpDmXGnOHKK2YJ3iQAf2E20rxE10Mpjhlz a2dm0FvZ+xQw00fm4n5h4IiztB6QFO4gfOXrGpKtQzFzAJ80oX76V6Tikz8P2VMaqpVb 9h/bGqlpDD5OeNb7hu0/cYrabG+58V6aEy5TTfYKXxNqbCVMCUPUzql8+WzuFxRk2Rl0 vkXjr8/skJxTJAhtWS21CtmGTgc5Ra0jAoFVvXymG3Iw+bozon23US4bm36jF+7W3rd/ Gokp6AUHBLfXki9EmkaTCIniw0ttsv/NiT5lo2IC9K+mx9+YRZibLuKe0932zjx1HqMc gMFA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=zDaNTJNW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id y18si11965768ejp.482.2021.03.15.11.30.23; Mon, 15 Mar 2021 11:30:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=zDaNTJNW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240264AbhCOOh2 (ORCPT + 99 others); Mon, 15 Mar 2021 10:37:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:37836 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233382AbhCOOBh (ORCPT ); Mon, 15 Mar 2021 10:01:37 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6097864FBE; Mon, 15 Mar 2021 14:01:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1615816868; bh=QgHQlinyPsOOtGA8bAPiCfBJ/holMpBW0V8TnTsXwSM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zDaNTJNWnLHeE8CeTTgSytXfIPEKkHB4nmOqAeb5lPPKy+IM40+HTGecD24Wa7glZ Q5fRftFSxWLMSICRVXPEzLlXrWzpn8Ui8YcjPgUYKYv4Rrp7DdZEtqui9OCHxCVKlS iah+HLnEfn6Ux6fosiv39FI1+nwfH+zS7WgFM3fc= From: gregkh@linuxfoundation.org To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Julien Grall , Juergen Gross , Julien Grall , Boris Ostrovsky , Ross Lagerwall Subject: [PATCH 4.19 119/120] xen/events: dont unmask an event channel when an eoi is pending Date: Mon, 15 Mar 2021 14:57:50 +0100 Message-Id: <20210315135723.873674926@linuxfoundation.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315135720.002213995@linuxfoundation.org> References: <20210315135720.002213995@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Greg Kroah-Hartman From: Juergen Gross commit 25da4618af240fbec6112401498301a6f2bc9702 upstream. An event channel should be kept masked when an eoi is pending for it. When being migrated to another cpu it might be unmasked, though. In order to avoid this keep three different flags for each event channel to be able to distinguish "normal" masking/unmasking from eoi related masking/unmasking and temporary masking. The event channel should only be able to generate an interrupt if all flags are cleared. Cc: stable@vger.kernel.org Fixes: 54c9de89895e ("xen/events: add a new "late EOI" evtchn framework") Reported-by: Julien Grall Signed-off-by: Juergen Gross Reviewed-by: Julien Grall Reviewed-by: Boris Ostrovsky Tested-by: Ross Lagerwall Link: https://lore.kernel.org/r/20210306161833.4552-3-jgross@suse.com [boris -- corrected Fixed tag format] Signed-off-by: Boris Ostrovsky Signed-off-by: Greg Kroah-Hartman --- drivers/xen/events/events_2l.c | 7 -- drivers/xen/events/events_base.c | 110 ++++++++++++++++++++++++++--------- drivers/xen/events/events_fifo.c | 7 -- drivers/xen/events/events_internal.h | 13 +--- 4 files changed, 88 insertions(+), 49 deletions(-) --- a/drivers/xen/events/events_2l.c +++ b/drivers/xen/events/events_2l.c @@ -76,12 +76,6 @@ static bool evtchn_2l_is_pending(unsigne return sync_test_bit(port, BM(&s->evtchn_pending[0])); } -static bool evtchn_2l_test_and_set_mask(unsigned port) -{ - struct shared_info *s = HYPERVISOR_shared_info; - return sync_test_and_set_bit(port, BM(&s->evtchn_mask[0])); -} - static void evtchn_2l_mask(unsigned port) { struct shared_info *s = HYPERVISOR_shared_info; @@ -375,7 +369,6 @@ static const struct evtchn_ops evtchn_op .clear_pending = evtchn_2l_clear_pending, .set_pending = evtchn_2l_set_pending, .is_pending = evtchn_2l_is_pending, - .test_and_set_mask = evtchn_2l_test_and_set_mask, .mask = evtchn_2l_mask, .unmask = evtchn_2l_unmask, .handle_events = evtchn_2l_handle_events, --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -98,6 +98,7 @@ static DEFINE_RWLOCK(evtchn_rwlock); * evtchn_rwlock * IRQ-desc lock * percpu eoi_list_lock + * irq_info->lock */ static LIST_HEAD(xen_irq_list_head); @@ -219,6 +220,8 @@ static int xen_irq_info_common_setup(str info->irq = irq; info->evtchn = evtchn; info->cpu = cpu; + info->mask_reason = EVT_MASK_REASON_EXPLICIT; + spin_lock_init(&info->lock); ret = set_evtchn_to_irq(evtchn, irq); if (ret < 0) @@ -366,6 +369,34 @@ unsigned int cpu_from_evtchn(unsigned in return ret; } +static void do_mask(struct irq_info *info, u8 reason) +{ + unsigned long flags; + + spin_lock_irqsave(&info->lock, flags); + + if (!info->mask_reason) + mask_evtchn(info->evtchn); + + info->mask_reason |= reason; + + spin_unlock_irqrestore(&info->lock, flags); +} + +static void do_unmask(struct irq_info *info, u8 reason) +{ + unsigned long flags; + + spin_lock_irqsave(&info->lock, flags); + + info->mask_reason &= ~reason; + + if (!info->mask_reason) + unmask_evtchn(info->evtchn); + + spin_unlock_irqrestore(&info->lock, flags); +} + #ifdef CONFIG_X86 static bool pirq_check_eoi_map(unsigned irq) { @@ -493,7 +524,7 @@ static void xen_irq_lateeoi_locked(struc } info->eoi_time = 0; - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_EOI_PENDING); } static void xen_irq_lateeoi_worker(struct work_struct *work) @@ -680,7 +711,8 @@ static void pirq_query_unmask(int irq) static void eoi_pirq(struct irq_data *data) { - int evtchn = evtchn_from_irq(data->irq); + struct irq_info *info = info_for_irq(data->irq); + int evtchn = info ? info->evtchn : 0; struct physdev_eoi eoi = { .irq = pirq_from_irq(data->irq) }; int rc = 0; @@ -689,14 +721,13 @@ static void eoi_pirq(struct irq_data *da if (unlikely(irqd_is_setaffinity_pending(data)) && likely(!irqd_irq_disabled(data))) { - int masked = test_and_set_mask(evtchn); + do_mask(info, EVT_MASK_REASON_TEMPORARY); clear_evtchn(evtchn); irq_move_masked_irq(data); - if (!masked) - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_TEMPORARY); } else clear_evtchn(evtchn); @@ -749,7 +780,8 @@ static unsigned int __startup_pirq(unsig goto err; out: - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_EXPLICIT); + eoi_pirq(irq_get_irq_data(irq)); return 0; @@ -776,7 +808,7 @@ static void shutdown_pirq(struct irq_dat if (!VALID_EVTCHN(evtchn)) return; - mask_evtchn(evtchn); + do_mask(info, EVT_MASK_REASON_EXPLICIT); xen_evtchn_close(evtchn); xen_irq_info_cleanup(info); } @@ -1635,10 +1667,10 @@ void rebind_evtchn_irq(int evtchn, int i } /* Rebind an evtchn so that it gets delivered to a specific cpu */ -static int xen_rebind_evtchn_to_cpu(int evtchn, unsigned int tcpu) +static int xen_rebind_evtchn_to_cpu(struct irq_info *info, unsigned int tcpu) { struct evtchn_bind_vcpu bind_vcpu; - int masked; + evtchn_port_t evtchn = info ? info->evtchn : 0; if (!VALID_EVTCHN(evtchn)) return -1; @@ -1654,7 +1686,7 @@ static int xen_rebind_evtchn_to_cpu(int * Mask the event while changing the VCPU binding to prevent * it being delivered on an unexpected VCPU. */ - masked = test_and_set_mask(evtchn); + do_mask(info, EVT_MASK_REASON_TEMPORARY); /* * If this fails, it usually just indicates that we're dealing with a @@ -1664,8 +1696,7 @@ static int xen_rebind_evtchn_to_cpu(int if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_vcpu, &bind_vcpu) >= 0) bind_evtchn_to_cpu(evtchn, tcpu); - if (!masked) - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_TEMPORARY); return 0; } @@ -1674,7 +1705,7 @@ static int set_affinity_irq(struct irq_d bool force) { unsigned tcpu = cpumask_first_and(dest, cpu_online_mask); - int ret = xen_rebind_evtchn_to_cpu(evtchn_from_irq(data->irq), tcpu); + int ret = xen_rebind_evtchn_to_cpu(info_for_irq(data->irq), tcpu); if (!ret) irq_data_update_effective_affinity(data, cpumask_of(tcpu)); @@ -1693,37 +1724,39 @@ EXPORT_SYMBOL_GPL(xen_set_affinity_evtch static void enable_dynirq(struct irq_data *data) { - int evtchn = evtchn_from_irq(data->irq); + struct irq_info *info = info_for_irq(data->irq); + evtchn_port_t evtchn = info ? info->evtchn : 0; if (VALID_EVTCHN(evtchn)) - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_EXPLICIT); } static void disable_dynirq(struct irq_data *data) { - int evtchn = evtchn_from_irq(data->irq); + struct irq_info *info = info_for_irq(data->irq); + evtchn_port_t evtchn = info ? info->evtchn : 0; if (VALID_EVTCHN(evtchn)) - mask_evtchn(evtchn); + do_mask(info, EVT_MASK_REASON_EXPLICIT); } static void ack_dynirq(struct irq_data *data) { - int evtchn = evtchn_from_irq(data->irq); + struct irq_info *info = info_for_irq(data->irq); + evtchn_port_t evtchn = info ? info->evtchn : 0; if (!VALID_EVTCHN(evtchn)) return; if (unlikely(irqd_is_setaffinity_pending(data)) && likely(!irqd_irq_disabled(data))) { - int masked = test_and_set_mask(evtchn); + do_mask(info, EVT_MASK_REASON_TEMPORARY); clear_evtchn(evtchn); irq_move_masked_irq(data); - if (!masked) - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_TEMPORARY); } else clear_evtchn(evtchn); } @@ -1734,18 +1767,39 @@ static void mask_ack_dynirq(struct irq_d ack_dynirq(data); } +static void lateeoi_ack_dynirq(struct irq_data *data) +{ + struct irq_info *info = info_for_irq(data->irq); + evtchn_port_t evtchn = info ? info->evtchn : 0; + + if (VALID_EVTCHN(evtchn)) { + do_mask(info, EVT_MASK_REASON_EOI_PENDING); + clear_evtchn(evtchn); + } +} + +static void lateeoi_mask_ack_dynirq(struct irq_data *data) +{ + struct irq_info *info = info_for_irq(data->irq); + evtchn_port_t evtchn = info ? info->evtchn : 0; + + if (VALID_EVTCHN(evtchn)) { + do_mask(info, EVT_MASK_REASON_EXPLICIT); + clear_evtchn(evtchn); + } +} + static int retrigger_dynirq(struct irq_data *data) { - unsigned int evtchn = evtchn_from_irq(data->irq); - int masked; + struct irq_info *info = info_for_irq(data->irq); + evtchn_port_t evtchn = info ? info->evtchn : 0; if (!VALID_EVTCHN(evtchn)) return 0; - masked = test_and_set_mask(evtchn); + do_mask(info, EVT_MASK_REASON_TEMPORARY); set_evtchn(evtchn); - if (!masked) - unmask_evtchn(evtchn); + do_unmask(info, EVT_MASK_REASON_TEMPORARY); return 1; } @@ -1951,8 +2005,8 @@ static struct irq_chip xen_lateeoi_chip .irq_mask = disable_dynirq, .irq_unmask = enable_dynirq, - .irq_ack = mask_ack_dynirq, - .irq_mask_ack = mask_ack_dynirq, + .irq_ack = lateeoi_ack_dynirq, + .irq_mask_ack = lateeoi_mask_ack_dynirq, .irq_set_affinity = set_affinity_irq, .irq_retrigger = retrigger_dynirq, --- a/drivers/xen/events/events_fifo.c +++ b/drivers/xen/events/events_fifo.c @@ -209,12 +209,6 @@ static bool evtchn_fifo_is_pending(unsig return sync_test_bit(EVTCHN_FIFO_BIT(PENDING, word), BM(word)); } -static bool evtchn_fifo_test_and_set_mask(unsigned port) -{ - event_word_t *word = event_word_from_port(port); - return sync_test_and_set_bit(EVTCHN_FIFO_BIT(MASKED, word), BM(word)); -} - static void evtchn_fifo_mask(unsigned port) { event_word_t *word = event_word_from_port(port); @@ -420,7 +414,6 @@ static const struct evtchn_ops evtchn_op .clear_pending = evtchn_fifo_clear_pending, .set_pending = evtchn_fifo_set_pending, .is_pending = evtchn_fifo_is_pending, - .test_and_set_mask = evtchn_fifo_test_and_set_mask, .mask = evtchn_fifo_mask, .unmask = evtchn_fifo_unmask, .handle_events = evtchn_fifo_handle_events, --- a/drivers/xen/events/events_internal.h +++ b/drivers/xen/events/events_internal.h @@ -35,13 +35,18 @@ struct irq_info { struct list_head eoi_list; short refcnt; short spurious_cnt; - enum xen_irq_type type; /* type */ + short type; /* type */ + u8 mask_reason; /* Why is event channel masked */ +#define EVT_MASK_REASON_EXPLICIT 0x01 +#define EVT_MASK_REASON_TEMPORARY 0x02 +#define EVT_MASK_REASON_EOI_PENDING 0x04 unsigned irq; unsigned int evtchn; /* event channel */ unsigned short cpu; /* cpu bound */ unsigned short eoi_cpu; /* EOI must happen on this cpu */ unsigned int irq_epoch; /* If eoi_cpu valid: irq_epoch of event */ u64 eoi_time; /* Time in jiffies when to EOI. */ + spinlock_t lock; union { unsigned short virq; @@ -73,7 +78,6 @@ struct evtchn_ops { void (*clear_pending)(unsigned port); void (*set_pending)(unsigned port); bool (*is_pending)(unsigned port); - bool (*test_and_set_mask)(unsigned port); void (*mask)(unsigned port); void (*unmask)(unsigned port); @@ -138,11 +142,6 @@ static inline bool test_evtchn(unsigned return evtchn_ops->is_pending(port); } -static inline bool test_and_set_mask(unsigned port) -{ - return evtchn_ops->test_and_set_mask(port); -} - static inline void mask_evtchn(unsigned port) { return evtchn_ops->mask(port);