Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 31 May 2023 12:17:23 +0300 (EEST)
From:   =?ISO-8859-15?Q?Ilpo_J=E4rvinen?= <ilpo.jarvinen@linux.intel.com>
To:     "Shaopeng Tan (Fujitsu)" <tan.shaopeng@fujitsu.com>
cc:     "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        Reinette Chatre <reinette.chatre@intel.com>,
        Fenghua Yu <fenghua.yu@intel.com>,
        Shuah Khan <shuah@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v2 21/24] selftests/resctrl: Read in less obvious order
 to defeat prefetch optimizations
In-Reply-To: <OSZPR01MB6328F2713E40CC7D383035A48B489@OSZPR01MB6328.jpnprd01.prod.outlook.com>
Message-ID: <c21fb16d-d3ad-bbcb-daed-28f153b64525@linux.intel.com>
References: <20230418114506.46788-1-ilpo.jarvinen@linux.intel.com> <20230418114506.46788-22-ilpo.jarvinen@linux.intel.com> <OSZPR01MB6328F2713E40CC7D383035A48B489@OSZPR01MB6328.jpnprd01.prod.outlook.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-566315524-1685524665=:5379"
Precedence: bulk

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-566315524-1685524665=:5379
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8BIT

On Wed, 31 May 2023, Shaopeng Tan (Fujitsu) wrote:

> Hi Ilpo,
> 
> > When reading memory in order, HW prefetching optimizations will interfere
> > with measuring how caches and memory are being accessed. This adds noise
> > into the results.
> > 
> > Change the fill_buf reading loop to not use an obvious in-order access using
> > multiply by a prime and modulo.
> > 
> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > ---
> >  tools/testing/selftests/resctrl/fill_buf.c | 17 ++++++++++-------
> >  1 file changed, 10 insertions(+), 7 deletions(-)
> > 
> > diff --git a/tools/testing/selftests/resctrl/fill_buf.c
> > b/tools/testing/selftests/resctrl/fill_buf.c
> > index 7e0d3a1ea555..049a520498a9 100644
> > --- a/tools/testing/selftests/resctrl/fill_buf.c
> > +++ b/tools/testing/selftests/resctrl/fill_buf.c
> > @@ -88,14 +88,17 @@ static void *malloc_and_init_memory(size_t s)
> > 
> >  static int fill_one_span_read(unsigned char *start_ptr, unsigned char
> > *end_ptr)  {
> > -	unsigned char sum, *p;
> > -
> > +	unsigned int size = (end_ptr - start_ptr) / (CL_SIZE / 2);
> > +	unsigned int count = size;
> > +	unsigned char sum;
> > +
> > +	/*
> > +	 * Read the buffer in an order that is unexpected by HW prefetching
> > +	 * optimizations to prevent them interfering with the caching pattern.
> > +	 */
> >  	sum = 0;
> > -	p = start_ptr;
> > -	while (p < end_ptr) {
> > -		sum += *p;
> > -		p += (CL_SIZE / 2);
> > -	}
> > +	while (count--)
> > +		sum += start_ptr[((count * 59) % size) * CL_SIZE / 2];
>
> Could you please elaborate why 59 is used?

The main reason is that it's a prime number ensuring the whole buffer 
gets read. I picked something that doesn't make it to wrap on almost 
every iteration.

-- 
 i.

--8323329-566315524-1685524665=:5379--