Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Uladzislau Rezki <urezki@gmail.com>
Date:   Tue, 11 Apr 2023 16:25:41 +0200
To:     "Zhang, Qiang1" <qiang1.zhang@intel.com>
Cc:     "paulmck@kernel.org" <paulmck@kernel.org>,
        "urezki@gmail.com" <urezki@gmail.com>,
        "frederic@kernel.org" <frederic@kernel.org>,
        "joel@joelfernandes.org" <joel@joelfernandes.org>,
        "qiang.zhang1211@gmail.com" <qiang.zhang1211@gmail.com>,
        "rcu@vger.kernel.org" <rcu@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3] rcu/kvfree: Prevents cache growing when the
 backoff_page_cache_fill is set
Message-ID: <ZDVt5eUAlp4VmbFy@pc636>
References: <20230408142517.800549-1-qiang1.zhang@intel.com>
 <b96953b0-0ac5-477a-a2f0-c211dc17d5dc@paulmck-laptop>
 <PH0PR11MB588072A12543FD617C833AF9DA9A9@PH0PR11MB5880.namprd11.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <PH0PR11MB588072A12543FD617C833AF9DA9A9@PH0PR11MB5880.namprd11.prod.outlook.com>
Precedence: bulk

On Tue, Apr 11, 2023 at 04:04:45AM +0000, Zhang, Qiang1 wrote:
> > Currently, in kfree_rcu_shrink_scan(), the drain_page_cache() is
> > executed before kfree_rcu_monitor() to drain page cache, if the bnode
> > structure's->gp_snap has done, the kvfree_rcu_bulk() will fill the
> > page cache again in kfree_rcu_monitor(), this commit add a check
> > for krcp structure's->backoff_page_cache_fill in put_cached_bnode(),
> > if the krcp structure's->backoff_page_cache_fill is set, prevent page
> > cache growing and disable allocated page in fill_page_cache_func().
> > 
> > Signed-off-by: Zqiang <qiang1.zhang@intel.com>
> >
> >Much improved!  But still some questions below...
> >
> >							Thanx, Paul
> >
> > ---
> >  kernel/rcu/tree.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > index cc34d13be181..9d9d3772cc45 100644
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -2908,6 +2908,8 @@ static inline bool
> >  put_cached_bnode(struct kfree_rcu_cpu *krcp,
> >  	struct kvfree_rcu_bulk_data *bnode)
> >  {
> > +	if (atomic_read(&krcp->backoff_page_cache_fill))
> > +		return false;
> >
> >This will mean that under low-memory conditions, we will keep zero
> >pages in ->bkvcache.  All attempts to put something there will fail.
> >
> >This is probably not an issue for structures containing an rcu_head
> >that are passed to kfree_rcu(p, field), but doesn't this mean that
> >kfree_rcu_mightsleep() unconditionally invokes synchronize_rcu()?
> >This could seriously slow up freeing under low-memory conditions,
> >which might exacerbate the low-memory conditions.
> 
> Thanks for mentioning this, I didn't think of this before????.
> 
> >
> >Is this really what we want?  Zero cached rather than just fewer cached?
> >
> >
> >
> >  	// Check the limit.
> >  	if (krcp->nr_bkv_objs >= rcu_min_cached_objs)
> >  		return false;
> > @@ -3221,7 +3223,7 @@ static void fill_page_cache_func(struct work_struct *work)
> >  	int i;
> >  
> >  	nr_pages = atomic_read(&krcp->backoff_page_cache_fill) ?
> > -		1 : rcu_min_cached_objs;
> > +		0 : rcu_min_cached_objs;
> >  
> >  	for (i = 0; i < nr_pages; i++) {
> >
> >I am still confused as to why we start "i" at zero rather than at
> >->nr_bkv_objs.  What am I missing here?
> 
> 
> No, you are right, I missed this place. 
> 
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -2908,6 +2908,8 @@ static inline bool
>  put_cached_bnode(struct kfree_rcu_cpu *krcp,
>         struct kvfree_rcu_bulk_data *bnode)
>  {
> +       if (atomic_read(&krcp->backoff_page_cache_fill))
> +               return false;
>
This is broken, unfortunately. If a low memory condition we fill
fill a cache with at least one page anyway because of we do not want
to hit a slow path.

>         // Check the limit.
>         if (krcp->nr_bkv_objs >= rcu_min_cached_objs)
>                 return false;
> @@ -3223,7 +3225,7 @@ static void fill_page_cache_func(struct work_struct *work)
>         nr_pages = atomic_read(&krcp->backoff_page_cache_fill) ?
>                 1 : rcu_min_cached_objs;
> 
> -       for (i = 0; i < nr_pages; i++) {
> +       for (i = krcp->nr_bkv_objs; i < nr_pages; i++) {
>                 bnode = (struct kvfree_rcu_bulk_data *)
>                         __get_free_page(GFP_KERNEL | __GFP_NORETRY | __GFP_NOMEMALLOC | __GFP_NOWARN);
> 
> 
IMHO, it should be send as a separate patch explaining why it
it is needed.

--
Uladzislau Rezki