Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 2 Nov 2022 19:52:14 +0100
From:   Pablo Neira Ayuso <pablo@netfilter.org>
To:     Michael Lilja <michael.lilja@gmail.com>
Cc:     "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Jozsef Kadlecsik <kadlec@netfilter.org>,
        Florian Westphal <fw@strlen.de>, netdev@vger.kernel.org,
        linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
        netfilter-devel@vger.kernel.org, coreteam@netfilter.org
Subject: Re: [PATCH] Periodically flow expire from flow offload tables
Message-ID: <Y2K8XnFZvZeD4MEg@salvia>
References: <20221023171658.69761-1-michael.lilja@gmail.com>
 <Y1fC5K0EalIYuB7Y@salvia>
 <381FF5B6-4FEF-45E9-92D6-6FE927A5CC2D@gmail.com>
 <Y1fd+DEPZ8xM2x5B@salvia>
 <F754AC3A-D89A-4CF7-97AE-CA59B18A758E@gmail.com>
 <Y1kQ9FhrwxCKIdoe@salvia>
 <25246B91-B5BE-43CA-9D98-67950F17F0A1@gmail.com>
 <03E5D5FA-5A0D-4E5A-BA32-3FE51764C02E@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <03E5D5FA-5A0D-4E5A-BA32-3FE51764C02E@gmail.com>
Precedence: bulk

On Wed, Oct 26, 2022 at 09:40:11PM +0200, Michael Lilja wrote:
> Hi,
> 
> I just quickly tried following the flow_offload_teardown() path instead of clearing IPS_OFFLOAD in flow_offload_del() and it does have some side effects. The flow is added again before the HW has actually reported it to be NF_FLOW_HW_DEAD. 
> 
> The sequence with my patch is:
>   : Retire -> Remove from hw tables -> Remove from sw tables -> kfree(flow) -> flow_offload_add()
> 
> But if flow_offload_teardown() is called on expire I see:
>   : Retire -> Remove from hw tables -> flow_offload_add() -> Remove from sw tables -> kfree(flow)
>  
> I need to investigate why this happens, maybe the IPS_OFFLOAD flag is cleared too early and should not be cleared until the flow is actually removed, like I do? Maybe the issue is not seen before because on timeout or flow_is_dying() no packet arrive to create the flow again prematurely?

Hm, IPS_OFFLOAD should be cleared from flow_offload_del() then, it is
cleared too early.

I'll post a fix for nf.git first then I propose to follow up on this
flowtable feature. I'll keep you on Cc.