Received-SPF: pass (google.com: domain of linux-kernel+bounces-151644-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Precedence: bulk
MIME-Version: 1.0
References: <20240415163527.626541-1-jeffxu@chromium.org> <znrbeb744774vre5dkeg7kjnnt7uuifs6xw63udcyupwj3veqh@rpcqs7dmoxi6>
 <CABi2SkU8B27O28jjTDajFpENgUHhntuRAMKFUMXr6A6AxZeyiQ@mail.gmail.com>
 <CAJuCfpFLwJg4n7wPpT+u9vC4XHoLE_BPPZ0tDKf7W45hGky4_Q@mail.gmail.com>
 <CABi2SkXCC8tvuHTiZ=tYcZw0sQ2SswQqDuFuQi5bKArW9+Nbaw@mail.gmail.com> <CAJuCfpEM0b-_gCbMqcUVvKedqXsSE4di7oqzeQGofVNsHRAqCg@mail.gmail.com>
In-Reply-To: <CAJuCfpEM0b-_gCbMqcUVvKedqXsSE4di7oqzeQGofVNsHRAqCg@mail.gmail.com>
From: Jeff Xu <jeffxu@chromium.org>
Date: Fri, 19 Apr 2024 08:14:58 -0700
Message-ID: <CABi2SkVk-9gQXtWKWs-1pf=Jnm=hNC04-z7qS95sJeCVrEra1Q@mail.gmail.com>
Subject: Re: [PATCH v10 0/5] Introduce mseal
To: Suren Baghdasaryan <surenb@google.com>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>, akpm@linux-foundation.org, 
	keescook@chromium.org, jannh@google.com, sroettger@google.com, 
	willy@infradead.org, gregkh@linuxfoundation.org, 
	torvalds@linux-foundation.org, usama.anjum@collabora.com, corbet@lwn.net, 
	merimus@google.com, rdunlap@infradead.org, jeffxu@google.com, 
	jorgelo@chromium.org, groeck@chromium.org, linux-kernel@vger.kernel.org, 
	linux-kselftest@vger.kernel.org, linux-mm@kvack.org, pedro.falcato@gmail.com, 
	dave.hansen@intel.com, linux-hardening@vger.kernel.org, deraadt@openbsd.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Apr 19, 2024 at 7:57=E2=80=AFAM Suren Baghdasaryan <surenb@google.c=
om> wrote:
>
> On Thu, Apr 18, 2024 at 6:22=E2=80=AFPM Jeff Xu <jeffxu@chromium.org> wro=
te:
> >
> > On Thu, Apr 18, 2024 at 1:19=E2=80=AFPM Suren Baghdasaryan <surenb@goog=
le.com> wrote:
> > >
> > > On Tue, Apr 16, 2024 at 12:40=E2=80=AFPM Jeff Xu <jeffxu@chromium.org=
> wrote:
> > > >
> > > > On Tue, Apr 16, 2024 at 8:13=E2=80=AFAM Liam R. Howlett <Liam.Howle=
tt@oracle.com> wrote:
> > > > >
> > > > > * jeffxu@chromium.org <jeffxu@chromium.org> [240415 12:35]:
> > > > > > From: Jeff Xu <jeffxu@chromium.org>
> > > > > >
> > > > > > This is V10 version, it rebases v9 patch to 6.9.rc3.
> > > > > > We also applied and tested mseal() in chrome and chromebook.
> > > > > >
> > > > > > ---------------------------------------------------------------=
---
> > > > > ...
> > > > >
> > > > > > MM perf benchmarks
> > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > > > > This patch adds a loop in the mprotect/munmap/madvise(DONTNEED)=
 to
> > > > > > check the VMAs=E2=80=99 sealing flag, so that no partial update=
 can be made,
> > > > > > when any segment within the given memory range is sealed.
> > > > > >
> > > > > > To measure the performance impact of this loop, two tests are d=
eveloped.
> > > > > > [8]
> > > > > >
> > > > > > The first is measuring the time taken for a particular system c=
all,
> > > > > > by using clock_gettime(CLOCK_MONOTONIC). The second is using
> > > > > > PERF_COUNT_HW_REF_CPU_CYCLES (exclude user space). Both tests h=
ave
> > > > > > similar results.
> > > > > >
> > > > > > The tests have roughly below sequence:
> > > > > > for (i =3D 0; i < 1000, i++)
> > > > > >     create 1000 mappings (1 page per VMA)
> > > > > >     start the sampling
> > > > > >     for (j =3D 0; j < 1000, j++)
> > > > > >         mprotect one mapping
> > > > > >     stop and save the sample
> > > > > >     delete 1000 mappings
> > > > > > calculates all samples.
> > > > >
> > > > >
> > > > > Thank you for doing this performance testing.
> > > > >
> > > > > >
> > > > > > Below tests are performed on Intel(R) Pentium(R) Gold 7505 @ 2.=
00GHz,
> > > > > > 4G memory, Chromebook.
> > > > > >
> > > > > > Based on the latest upstream code:
> > > > > > The first test (measuring time)
> > > > > > syscall__     vmas    t       t_mseal delta_ns        per_vma %
> > > > > > munmap__      1       909     944     35      35      104%
> > > > > > munmap__      2       1398    1502    104     52      107%
> > > > > > munmap__      4       2444    2594    149     37      106%
> > > > > > munmap__      8       4029    4323    293     37      107%
> > > > > > munmap__      16      6647    6935    288     18      104%
> > > > > > munmap__      32      11811   12398   587     18      105%
> > > > > > mprotect      1       439     465     26      26      106%
> > > > > > mprotect      2       1659    1745    86      43      105%
> > > > > > mprotect      4       3747    3889    142     36      104%
> > > > > > mprotect      8       6755    6969    215     27      103%
> > > > > > mprotect      16      13748   14144   396     25      103%
> > > > > > mprotect      32      27827   28969   1142    36      104%
> > > > > > madvise_      1       240     262     22      22      109%
> > > > > > madvise_      2       366     442     76      38      121%
> > > > > > madvise_      4       623     751     128     32      121%
> > > > > > madvise_      8       1110    1324    215     27      119%
> > > > > > madvise_      16      2127    2451    324     20      115%
> > > > > > madvise_      32      4109    4642    534     17      113%
> > > > > >
> > > > > > The second test (measuring cpu cycle)
> > > > > > syscall__     vmas    cpu     cmseal  delta_cpu       per_vma %
> > > > > > munmap__      1       1790    1890    100     100     106%
> > > > > > munmap__      2       2819    3033    214     107     108%
> > > > > > munmap__      4       4959    5271    312     78      106%
> > > > > > munmap__      8       8262    8745    483     60      106%
> > > > > > munmap__      16      13099   14116   1017    64      108%
> > > > > > munmap__      32      23221   24785   1565    49      107%
> > > > > > mprotect      1       906     967     62      62      107%
> > > > > > mprotect      2       3019    3203    184     92      106%
> > > > > > mprotect      4       6149    6569    420     105     107%
> > > > > > mprotect      8       9978    10524   545     68      105%
> > > > > > mprotect      16      20448   21427   979     61      105%
> > > > > > mprotect      32      40972   42935   1963    61      105%
> > > > > > madvise_      1       434     497     63      63      115%
> > > > > > madvise_      2       752     899     147     74      120%
> > > > > > madvise_      4       1313    1513    200     50      115%
> > > > > > madvise_      8       2271    2627    356     44      116%
> > > > > > madvise_      16      4312    4883    571     36      113%
> > > > > > madvise_      32      8376    9319    943     29      111%
> > > > > >
> > > > >
> > > > > If I am reading this right, madvise() is affected more than the o=
ther
> > > > > calls?  Is that expected or do we need to have a closer look?
> > > > >
> > > > The madvise() has a bigger percentage (per_vma %), but it also has =
a
> > > > smaller base value (cpu).
> > >
> > > Sorry, it's unclear to me what the "vmas" column denotes. Is that how
> > > many VMAs were created before timing the syscall? If so, then 32 is
> > > the max that you show here while you seem to have tested with 1000
> > > VMAs. What is the overhead with 1000 VMAs?
> >
> > The vmas column is the number of VMA used in one call.
> >
> > For example: for 32 and mprotect(ptr,size), the memory range used in
> > mprotect has 32 VMAs.
>
> Ok, so the 32 here denotes how many VMAs one mprotect() call spans?
>
Yes.

> >
> > It also matters how many memory ranges are in-use at the time of the
> > test, This is where 1000 comes in. The test creates 1000 memory
> > ranges, each memory range has 32 vmas, then calls mprotect on the 1000
> > memory range. (the pseudocode was included in the original email)
>
> So, if each range has 32 vmas and you have 1000 ranges then you are
> creating 32000 vmas? Sorry, your pseudocode does not clarify that. My
> current understanding is this:
>
> for (i =3D 0; i < 1000, i++)
>     mmap N*1000 areas (N=3D[1-32])
>     start the sampling
>     for (j =3D 0; j < 1000, j++)
>         mprotect N areas with one syscall
>     stop and save the sample
>     munmap N*1000 areas
> calculates all samples.
>
> Is that correct?
>
Yes, There will be 32000 VMA in the system.

The pseudocode is correct in concept.
The test implementation is slightly different, it uses mprotect to
split the memory and make sure the VMAs  doesn't merge. For detail,
the reference [8]  of the original email link to the test code.

-Jeff