Received-SPF: pass (google.com: domain of linux-kernel+bounces-58255-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240205072312.2342188-1-howardyen@google.com>
 <ZcDPn5eLvMwpb7N1@smile.fi.intel.com> <ZcJTkQFcVhcGC52Z@smile.fi.intel.com>
 <CAJDAHvYYUcn2HL1QTSG-KA0QcuAOotk5hY4cpuasOu_H=X-V0A@mail.gmail.com> <ZcSzW1kFZIZ_f759@smile.fi.intel.com>
In-Reply-To: <ZcSzW1kFZIZ_f759@smile.fi.intel.com>
From: Howard Yen <howardyen@google.com>
Date: Thu, 8 Feb 2024 22:57:34 +0800
Message-ID: <CAJDAHvaYqq+PFTQRejcbUbyX6zHw5wygF+dRh9HNbJ83fmZacA@mail.gmail.com>
Subject: Re: [PATCH v2] dma-coherent: add support for multi coherent rmems per dev
To: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, 
	gregkh@linuxfoundation.org, rafael@kernel.org, broonie@kernel.org, 
	james@equiv.tech, james.clark@arm.com, masahiroy@kernel.org, 
	linux-kernel@vger.kernel.org, iommu@lists.linux.dev
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 6:56=E2=80=AFPM Andy Shevchenko
<andriy.shevchenko@linux.intel.com> wrote:
>
> On Thu, Feb 08, 2024 at 03:53:37PM +0800, Howard Yen wrote:
> > On Tue, Feb 6, 2024 at 11:43=E2=80=AFPM Andy Shevchenko
> > <andriy.shevchenko@linux.intel.com> wrote:
> > > On Mon, Feb 05, 2024 at 02:08:00PM +0200, Andy Shevchenko wrote:
> > > > On Mon, Feb 05, 2024 at 07:23:00AM +0000, Howard Yen wrote:
>
> ...
>
> > > > > @@ -18,15 +18,9 @@ struct dma_coherent_mem {
> > > > >     unsigned long   *bitmap;
> > > > >     spinlock_t      spinlock;
> > > > >     bool            use_dev_dma_pfn_offset;
> > > > > +   struct list_head        node;
> > > >
> > > > Have you run `pahole`? Here I see wasted bytes for nothing.
> > >
> > > On top of that one may make container_of() to be no-op, by placing th=
is member
> > > to be the first one. But, double check this with bloat-o-meter (that =
it indeed
> > > does better code generation) and on the other hand check if the curre=
nt first
> > > member is not performance critical and having additional pointer arit=
hmetics is
> > > okay.
> > >
> > > > >  };
> >
> > I'm trying to re-org the members as below
> >
> > from =3D=3D=3D>
> >
> > struct dma_coherent_mem {
> > void *                     virt_base;            /*     0     8 */
> > dma_addr_t                 device_base;          /*     8     8 */
> > unsigned long              pfn_base;             /*    16     8 */
> > int                        size;                 /*    24     4 */
> >
> > /* XXX 4 bytes hole, try to pack */
> >
> > unsigned long *            bitmap;               /*    32     8 */
> > spinlock_t                 spinlock;             /*    40     4 */
> > bool                       use_dev_dma_pfn_offset; /*    44     1 */
> >
> > /* XXX 3 bytes hole, try to pack */
> >
> > struct list_head           node;                 /*    48    16 */
> >
> > /* size: 64, cachelines: 1, members: 8 */
> > /* sum members: 57, holes: 2, sum holes: 7 */
> > };
> >
> >
> > to =3D=3D=3D>
> >
> > struct dma_coherent_mem {
> > struct list_head           node;                 /*     0    16 */
> > void *                     virt_base;            /*    16     8 */
> > dma_addr_t                 device_base;          /*    24     8 */
> > unsigned long              pfn_base;             /*    32     8 */
> > int                        size;                 /*    40     4 */
> > spinlock_t                 spinlock;             /*    44     4 */
> > unsigned long *            bitmap;               /*    48     8 */
> > bool                       use_dev_dma_pfn_offset; /*    56     1 */
> >
> > /* size: 64, cachelines: 1, members: 8 */
> > /* padding: 7 */
>
> Which seems better that above, right?
>
> > };
> >
> > Looks like there is about 7 bytes padding at the end of the structure.
> > Should I add __attribute__((__packed__)) to not add the padding?
>
> No, __packed is about alignment, may give you much worse code generation.
> With list_head member first you might get better code from the original,
> check it with bloat-o-meter.
>
> --
> With Best Regards,
> Andy Shevchenko
>
>

From the check result with bloat-o-meter, there is about 3.38%
reduction totally from the
original version. Thanks for the suggestion!

add/remove: 0/0 grow/shrink: 0/7 up/down: 0/-60 (-60)
Function                                     old     new   delta
rmem_dma_device_release                      104     100      -4
dma_release_from_dev_coherent                184     180      -4
dma_release_coherent_memory                  144     140      -4
dma_mmap_from_dev_coherent                   228     224      -4
dma_init_coherent_memory                     292     284      -8
rmem_dma_device_init                         168     152     -16
dma_declare_coherent_memory                  184     164     -20
Total: Before=3D1776, After=3D1716, chg -3.38%
add/remove: 0/0 grow/shrink: 0/0 up/down: 0/0 (0)
Data                                         old     new   delta
Total: Before=3D0, After=3D0, chg +0.00%
add/remove: 0/0 grow/shrink: 0/0 up/down: 0/0 (0)
RO Data                                      old     new   delta
Total: Before=3D216, After=3D216, chg +0.00%

For the dev check, in the previous comment, they're in the static
function and are assigned
to ops function pointers, I think the check is required because they
might be invoked from
others.

I'll submit v3 with the members reorg, the return variable naming
changes and if (!dev) return; .


--=20
Regards,

Howard