Received-SPF: pass (google.com: domain of linux-kernel+bounces-135660-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240407130850.19625-1-linyunsheng@huawei.com>
 <20240407130850.19625-3-linyunsheng@huawei.com> <43d99616cd4a2a6fce6a6b97f73d08ebc5361a61.camel@gmail.com>
 <cccbddc2-de64-46f9-7d71-60185d364f73@huawei.com>
In-Reply-To: <cccbddc2-de64-46f9-7d71-60185d364f73@huawei.com>
From: Alexander Duyck <alexander.duyck@gmail.com>
Date: Mon, 8 Apr 2024 09:11:27 -0700
Message-ID: <CAKgT0Uf9hWmVjHfrX6yM_LgX6D5m+ktFiJatMhOhuHpADghh4Q@mail.gmail.com>
Subject: Re: [PATCH net-next v1 02/12] mm: page_frag: use initial zero offset
 for page_frag_alloc_align()
To: Yunsheng Lin <linyunsheng@huawei.com>
Cc: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, 
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 
	Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 8, 2024 at 6:39=E2=80=AFAM Yunsheng Lin <linyunsheng@huawei.com=
> wrote:
>
> On 2024/4/8 1:52, Alexander H Duyck wrote:
> > On Sun, 2024-04-07 at 21:08 +0800, Yunsheng Lin wrote:
> >> We are above to use page_frag_alloc_*() API to not just
> >> allocate memory for skb->data, but also use them to do
> >> the memory allocation for skb frag too. Currently the
> >> implementation of page_frag in mm subsystem is running
> >> the offset as a countdown rather than count-up value,
> >> there may have several advantages to that as mentioned
> >> in [1], but it may have some disadvantages, for example,
> >> it may disable skb frag coaleasing and more correct cache
> >> prefetching
> >>
> >> We have a trade-off to make in order to have a unified
> >> implementation and API for page_frag, so use a initial zero
> >> offset in this patch, and the following patch will try to
> >> make some optimization to aovid the disadvantages as much
> >> as possible.
> >>
> >> 1. https://lore.kernel.org/all/f4abe71b3439b39d17a6fb2d410180f367cadf5=
c.camel@gmail.com/
> >>
> >> CC: Alexander Duyck <alexander.duyck@gmail.com>
> >> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> >> ---
> >>  mm/page_frag_cache.c | 31 ++++++++++++++-----------------
> >>  1 file changed, 14 insertions(+), 17 deletions(-)
> >>
> >> diff --git a/mm/page_frag_cache.c b/mm/page_frag_cache.c
> >> index a0f90ba25200..3e3e88d9af90 100644
> >> --- a/mm/page_frag_cache.c
> >> +++ b/mm/page_frag_cache.c
> >> @@ -67,9 +67,8 @@ void *__page_frag_alloc_align(struct page_frag_cache=
 *nc,
> >>                            unsigned int fragsz, gfp_t gfp_mask,
> >>                            unsigned int align_mask)
> >>  {
> >> -    unsigned int size =3D PAGE_SIZE;
> >> +    unsigned int size, offset;
> >>      struct page *page;
> >> -    int offset;
> >>
> >>      if (unlikely(!nc->va)) {
> >>  refill:
> >> @@ -77,10 +76,6 @@ void *__page_frag_alloc_align(struct page_frag_cach=
e *nc,
> >>              if (!page)
> >>                      return NULL;
> >>
> >> -#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE)
> >> -            /* if size can vary use size else just use PAGE_SIZE */
> >> -            size =3D nc->size;
> >> -#endif
> >>              /* Even if we own the page, we do not use atomic_set().
> >>               * This would break get_page_unless_zero() users.
> >>               */
> >> @@ -89,11 +84,18 @@ void *__page_frag_alloc_align(struct page_frag_cac=
he *nc,
> >>              /* reset page count bias and offset to start of new frag =
*/
> >>              nc->pfmemalloc =3D page_is_pfmemalloc(page);
> >>              nc->pagecnt_bias =3D PAGE_FRAG_CACHE_MAX_SIZE + 1;
> >> -            nc->offset =3D size;
> >> +            nc->offset =3D 0;
> >>      }
> >>
> >> -    offset =3D nc->offset - fragsz;
> >> -    if (unlikely(offset < 0)) {
> >> +#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE)
> >> +    /* if size can vary use size else just use PAGE_SIZE */
> >> +    size =3D nc->size;
> >> +#else
> >> +    size =3D PAGE_SIZE;
> >> +#endif
> >> +
> >> +    offset =3D ALIGN(nc->offset, -align_mask);
> >> +    if (unlikely(offset + fragsz > size)) {
> >
> > Rather than using "ALIGN" with a negative value it would probably make
> > more sense to use __ALIGN_KERNEL_MASK with ~align_mask. I am not sure
> > how well the compiler sorts out the use of negatives to flip values
> > that are then converted to masks with the "(a) - 1".
>
> The next patch will remove the '-' in '-align_mask' as the 'ALIGN' operat=
ion
> is done in the inline helper. I am not sure that matter much to use
> __ALIGN_KERNEL_MASK with ~align_mask?

It is a matter of making the negations more obvious. Basically you
could achieve the same alignment by doing:
  (offset + (~align_mask)) & ~(~align_mask)
rather than:
  (offset + ((-align_mask) - 1)) & ~((-align_mask) - 1)

I'm not sure the compiler will pick up on the fact that the two are
identical and can save a number of operations. Also my suggested
approach is closer to how it used to work. Technically the one you are
using only works if align_mask is a negative power of 2.