Received: by 2002:ac0:a594:0:0:0:0:0 with SMTP id m20-v6csp1994475imm; Wed, 16 May 2018 06:22:09 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrNe3SE9eBaYGT9qQXCi4FhEPNdzGsp2teiimXzIeNivugYTeygzUqtBAlF6SsEoPTfRnuF X-Received: by 2002:a63:6584:: with SMTP id z126-v6mr765462pgb.168.1526476929497; Wed, 16 May 2018 06:22:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526476929; cv=none; d=google.com; s=arc-20160816; b=whujesU4et8GFYsLJk/QsEoAfuzS6CVKdkzNG6RoQ6ZyBaPEZYjxU6syzkaxxGG2H+ Z3TIqP7FvqLt+0UHprtJZOBRCyeuUqJyCrolD60WxHe6QX/aV05IZTjb8f6MMLfKh7qi J9PeGa3y2qAHSiab8bDiqmFuqCYRAzKX6UqIPr1r/znaogxA41Fd8B3EuSeGmYp8GAmH mqgpa4yVTH6R5DHl1yYV5mjdO1MfTho3qfWKiUKafpCtoBaIRt0AToW07t5qHNpIezOj Tagx2zavdOqWP92ufgliZ81ehd5lipVbIqjHhMKFBupC2h7mcoJ2cxJN3pN8QppnnPkA 4MPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature:arc-authentication-results; bh=lCnMO3e0dZ3j1zjheS8yGP1RBYn6zCGiV9J6/PAOhyo=; b=VjB+3DnCT12RBIF7gofOkv0ILMT5WT2iJblPJE7/GcrHmulu4v5vthvYrzu4zvsxR0 UUws6niJXcwX5uyLFU4O6VUUNoVUGNuRWqHvA7+6gcuMTrpr3uQllhvBUF8HZ2BfXtgh sxEN1x6veuNhYYSpkIG94dDLpiXQM6Dmw/5z2hOiag302ZwfH8xPeXYwFhzSmcDtH7hM hPFktgK+DWPTJoYBo2Ia24vB8IlarAUyUxde2QDoT5kHr6j3OGX3ddttGOi6AZ6GFuXh 60JO/xu9zOUhG7ohVOFhgcqTn2N7ZLOVrd3s4CnZPlGkvNnDJYY30mjUaZ7M+8bdmlPn NPaA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@resnulli-us.20150623.gappssmtp.com header.s=20150623 header.b=wTHRm91M; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m64-v6si2612862pfm.0.2018.05.16.06.21.55; Wed, 16 May 2018 06:22:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@resnulli-us.20150623.gappssmtp.com header.s=20150623 header.b=wTHRm91M; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752410AbeEPNVj (ORCPT + 99 others); Wed, 16 May 2018 09:21:39 -0400 Received: from mail-wr0-f193.google.com ([209.85.128.193]:39571 "EHLO mail-wr0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751319AbeEPNVh (ORCPT ); Wed, 16 May 2018 09:21:37 -0400 Received: by mail-wr0-f193.google.com with SMTP id q3-v6so1118630wrj.6 for ; Wed, 16 May 2018 06:21:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=lCnMO3e0dZ3j1zjheS8yGP1RBYn6zCGiV9J6/PAOhyo=; b=wTHRm91Mk5jLyXjIcMunpzvgQaAlyXKYcFO8D/tMtSkR2o+KYhUgV5wmBjNBEICg3W l39m9N06zM/oWABi0iTSa0d8SPu99AeSWL29q0QM5qxkRm2R92ONihE4aKJAbCfqng3B z3EKo9othTk8qRLeuQoZPKDE/1l4thNBpdqdgk7incdo2xk/OagvCs77Vtr5pE0VneQv 1wF47hSenVYwWQ/bnJsxLNrMOKpcbBbNYDNh4BuVAjrxNKqZg65R37Sr0P95uIyiRhtb TCqAcGs1RodQ0QP1HKJSymya6g9iVMci6zSyQnnhDiwX6EUvP/RX49gZKv7JzycuINQ5 4GPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=lCnMO3e0dZ3j1zjheS8yGP1RBYn6zCGiV9J6/PAOhyo=; b=K+fVjokXqlbIeipkKqfba18YayGYonQXzN5sWslUz82QeXyC7gaTxsJ6dz54iplWqP wg1nXJ/X5t0naHzHEPzupLgitO6JdCYt0EmrQvgehjUyWu0IRh1r2DQMDPoTZWvYIrdJ N32RTarUqxERwfAPgJM2tnrQ0D5Z7M1PFtB5BAPoc2ZfXrLiy2GT5rivac1u87/+5idb DdGgiVFAqFHpasA7b4nlYp9xrXydKb1xuDESldTGlkw1tsU477Hx2wIDvf2eOll8a5I3 LF/z7yAh+wnsgzlzn7aWdPkCXItoBB9rDkJ0UnFNLg6jcsC3xQOwReLMkA6SfJADBbua eUJg== X-Gm-Message-State: ALKqPweRQH3Jh18wT8eSYoDVX8LuSgemAaufAAkWDFNXphwH3UhF9zei lFRLBOOARuIxQIped5rriWhUkw== X-Received: by 2002:adf:bc0f:: with SMTP id s15-v6mr886886wrg.58.1526476896217; Wed, 16 May 2018 06:21:36 -0700 (PDT) Received: from localhost (ip-94-113-127-8.net.upcbroadband.cz. [94.113.127.8]) by smtp.gmail.com with ESMTPSA id g7-v6sm2968482wrb.60.2018.05.16.06.21.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 16 May 2018 06:21:35 -0700 (PDT) Date: Wed, 16 May 2018 15:21:35 +0200 From: Jiri Pirko To: Vlad Buslov Cc: netdev@vger.kernel.org, davem@davemloft.net, jhs@mojatatu.com, xiyou.wangcong@gmail.com, pablo@netfilter.org, kadlec@blackhole.kfki.hu, fw@strlen.de, ast@kernel.org, daniel@iogearbox.net, edumazet@google.com, keescook@chromium.org, linux-kernel@vger.kernel.org, netfilter-devel@vger.kernel.org, coreteam@netfilter.org, kliteyn@mellanox.com Subject: Re: [PATCH 12/14] net: sched: retry action check-insert on concurrent modification Message-ID: <20180516132135.GN1972@nanopsycho> References: <1526308035-12484-1-git-send-email-vladbu@mellanox.com> <1526308035-12484-13-git-send-email-vladbu@mellanox.com> <20180516095953.GI1972@nanopsycho> <20180516122600.GM1972@nanopsycho> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.2 (2017-12-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Wed, May 16, 2018 at 02:43:58PM CEST, vladbu@mellanox.com wrote: > >On Wed 16 May 2018 at 12:26, Jiri Pirko wrote: >> Wed, May 16, 2018 at 01:55:06PM CEST, vladbu@mellanox.com wrote: >>> >>>On Wed 16 May 2018 at 09:59, Jiri Pirko wrote: >>>> Mon, May 14, 2018 at 04:27:13PM CEST, vladbu@mellanox.com wrote: >>>>>Retry check-insert sequence in action init functions if action with same >>>>>index was inserted concurrently. >>>>> >>>>>Signed-off-by: Vlad Buslov >>>>>--- >>>>> net/sched/act_bpf.c | 8 +++++++- >>>>> net/sched/act_connmark.c | 8 +++++++- >>>>> net/sched/act_csum.c | 8 +++++++- >>>>> net/sched/act_gact.c | 8 +++++++- >>>>> net/sched/act_ife.c | 8 +++++++- >>>>> net/sched/act_ipt.c | 8 +++++++- >>>>> net/sched/act_mirred.c | 8 +++++++- >>>>> net/sched/act_nat.c | 8 +++++++- >>>>> net/sched/act_pedit.c | 8 +++++++- >>>>> net/sched/act_police.c | 9 ++++++++- >>>>> net/sched/act_sample.c | 8 +++++++- >>>>> net/sched/act_simple.c | 9 ++++++++- >>>>> net/sched/act_skbedit.c | 8 +++++++- >>>>> net/sched/act_skbmod.c | 8 +++++++- >>>>> net/sched/act_tunnel_key.c | 9 ++++++++- >>>>> net/sched/act_vlan.c | 9 ++++++++- >>>>> 16 files changed, 116 insertions(+), 16 deletions(-) >>>>> >>>>>diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c >>>>>index 5554bf7..7e20fdc 100644 >>>>>--- a/net/sched/act_bpf.c >>>>>+++ b/net/sched/act_bpf.c >>>>>@@ -299,10 +299,16 @@ static int tcf_bpf_init(struct net *net, struct nlattr *nla, >>>>> >>>>> parm = nla_data(tb[TCA_ACT_BPF_PARMS]); >>>>> >>>>>+replay: >>>>> if (!tcf_idr_check(tn, parm->index, act, bind)) { >>>>> ret = tcf_idr_create(tn, parm->index, est, act, >>>>> &act_bpf_ops, bind, true); >>>>>- if (ret < 0) >>>>>+ /* Action with specified index was created concurrently. >>>>>+ * Check again. >>>>>+ */ >>>>>+ if (parm->index && ret == -ENOSPC) >>>>>+ goto replay; >>>>>+ else if (ret) >>>> >>>> Hmm, looks like you are doing the same/very similar thing in every act >>>> code. I think it would make sense to introduce a helper function for >>>> this purpose. >>> >>>This code uses goto so it can't be easily refactored into standalone >>>function. Could you specify which part of this code you suggest to >>>extract? >> >> Hmm, looking at the code, I think that what would help is to have a >> helper that would atomically check if index exists and if not, it would >> allocate one. Something like: >> >> >> int tcf_idr_check_alloc(struct tc_action_net *tn, u32 *index, >> struct tc_action **a, int bind) >> { >> struct tcf_idrinfo *idrinfo = tn->idrinfo; >> struct tc_action *p; >> int err; >> >> spin_lock(&idrinfo->lock); >> if (*index) { >> p = idr_find(&idrinfo->action_idr, *index); >> if (p) { >> if (bind) >> p->tcfa_bindcnt++; >> p->tcfa_refcnt++; >> *a = p; >> err = 0; >> } else { >> *a = NULL; >> err = idr_alloc_u32(idr, NULL, index, >> *index, GFP_ATOMIC); >> } >> } else { >> *index = 1; >> *a = NULL; >> err = idr_alloc_u32(idr, NULL, index, UINT_MAX, GFP_ATOMIC); >> } >> spin_unlock(&idrinfo->lock); >> return err; >> } >> >> The act code would just check if "a" is NULL and if so, it would call >> tcf_idr_create() with allocated index as arg. > >What about multiple actions that have arbitrary code between initial >check and idr allocation that is currently inside tcf_idr_create()? Why it would be a problem to have them after the allocation? There is one issue though with my draft. tcf_idr_insert() function which actually assigns a "p" pointer to the idr index is called later on. Until that happens, the idr_find() would return NULL even if the index is actually allocated. We cannot assign "p" in tcf_idr_check_alloc() because it is allocated only later on in tcf_idr_create(). But that is resolvable by the following trick: int tcf_idr_check_alloc(struct tc_action_net *tn, u32 *index, struct tc_action **a, int bind) { struct tcf_idrinfo *idrinfo = tn->idrinfo; struct tc_action *p; int err; again: spin_lock(&idrinfo->lock); if (*index) { p = idr_find(&idrinfo->action_idr, *index); if (IS_ERR(p)) { /* This means that another process allocated * index but did not assign the pointer yet. */ spin_unlock(&idrinfo->lock); goto again; } if (p) { if (bind) p->tcfa_bindcnt++; p->tcfa_refcnt++; *a = p; err = 0; } else { *a = NULL; err = idr_alloc_u32(idr, NULL, index, *index, GFP_ATOMIC); idr_replace(&idrinfo->action_idr, ERR_PTR(-EBUSY), *index); } } else { *index = 1; *a = NULL; err = idr_alloc_u32(idr, NULL, index, UINT_MAX, GFP_ATOMIC); idr_replace(&idrinfo->action_idr, ERR_PTR(-EBUSY), *index); } spin_unlock(&idrinfo->lock); return err; } > >> >> >>> >>>> >>>> [...] >>> >