Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Alexander Van Brunt <avanbrunt@nvidia.com>
To:     Will Deacon <will.deacon@arm.com>
CC:     Ashish Mhetre <amhetre@nvidia.com>,
        "mark.rutland@arm.com" <mark.rutland@arm.com>,
        "linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>,
        Sachin Nikam <Snikam@nvidia.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH V3] arm64: Don't flush tlb while clearing the accessed bit
Thread-Topic: [PATCH V3] arm64: Don't flush tlb while clearing the accessed
 bit
Thread-Index: AQHUb2mK2jioXCSBXUOglXMUWPpsLKU2DOwAgABGE2iAN2VlVIAEmdOAgAAC4fGAABP4t4ABY6SAgABWZTk=
Date:   Sat, 8 Dec 2018 00:05:44 +0000
Message-ID: <BYAPR12MB271295B398729E07F31082A7CFAA0@BYAPR12MB2712.namprd12.prod.outlook.com>
References: <1540805158-618-1-git-send-email-amhetre@nvidia.com>
 <20181029105515.GD14127@arm.com>
 <BN7PR12MB2708B742FD3448589AD0F259CFF30@BN7PR12MB2708.namprd12.prod.outlook.com>
 <BYAPR12MB27129A4EE7BBF57B1CAB5094CFAE0@BYAPR12MB2712.namprd12.prod.outlook.com>
 <20181206191850.GC20796@arm.com>
 <BYAPR12MB271239AAF4A0B79D756850C6CFA90@BYAPR12MB2712.namprd12.prod.outlook.com>
 <BYAPR12MB27120C304F7B085A0CA36124CFA90@BYAPR12MB2712.namprd12.prod.outlook.com>,<20181207175330.GC11430@edgewater-inn.cambridge.arm.com>
In-Reply-To: <20181207175330.GC11430@edgewater-inn.cambridge.arm.com>
Accept-Language: en-US
received-spf: None (protection.outlook.com: nvidia.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: e844cd0d-fe5d-42aa-6004-08d65ca0e660
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Dec 2018 00:05:44.7741
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2695
X-OriginatorOrg: Nvidia.com
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1544227546; bh=ZF1hFouDXDs4UBLIwuo2/kXizwYZ9ZY2jlI66QU6Hm8=;
        h=X-PGP-Universal:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:
         Message-ID:References:In-Reply-To:Accept-Language:X-MS-Has-Attach:
         X-MS-TNEF-Correlator:authentication-results:x-originating-ip:
         x-ms-publictraffictype:x-microsoft-exchange-diagnostics:
         x-ms-exchange-antispam-srfa-diagnostics:
         x-forefront-antispam-report:
         x-ms-office365-filtering-correlation-id:x-microsoft-antispam:
         x-ms-traffictypediagnostic:x-microsoft-antispam-prvs:
         x-ms-exchange-senderadcheck:x-exchange-antispam-report-cfa-test:
         x-forefront-prvs:received-spf:x-microsoft-antispam-message-info:
         spamdiagnosticoutput:spamdiagnosticmetadata:MIME-Version:
         X-MS-Exchange-CrossTenant-Network-Message-Id:
         X-MS-Exchange-CrossTenant-originalarrivaltime:
         X-MS-Exchange-CrossTenant-fromentityheader:
         X-MS-Exchange-CrossTenant-id:
         X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg:
         Content-Language:Content-Type:Content-Transfer-Encoding;
        b=iCsr4LmCeB+xhMcD6Ux9zVxwXyCpgUJqlOnUZK+nPpxpxf+3Joz1vQEOq4PvTF/YV
         l1QWEZyjW1lrYGPG0uvjPG0Iwo8dWw/m5gNG7urbTqR79E5c4jNqeFZXPtqUcqQluC
         m9kH3NyGih0QBHhc3PaKTZHrutNGb4wHijgfCsUsOEGfuXjRCPOLkv9pNujmXXawOU
         SL0i8XdMrkbFYWDgsAyO/Ls983Jco8ntI0p5hG/T40xJrXQEai7b6P3R0WIrzWyDYT
         Hhvdo01K5uDC4wrp8xtr3uUtK3vSsNduuCd5B+XJf6N8OdZBhXC0+VCR9+oRs60Olr
         S/rdk4sgLMA/g==
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> > > My problem with that is it's not really much different to just skippi=
ng the
> > > page table update entirely. Skipping the DSB is closer to what is don=
e on
> > > x86, where we bound the stale entry time to the next context-switch.
> > =20
> > Which of the three implementations is the "that" and "it" in the first =
sentence?
>
> that =3D it =3D skipping the whole invalidation + the DSB

The TLB is tiny compared to the size of the inactive list. Somehow a TLB ha=
s to
not be evicted during the page's life in the inactive list. That is not an =
easy
feat except for the hottest of pages.

If there is a context-switch, most of the original thread's TLBs will be
evicted because TLBs have a hard time to hold two thread's working sets. So=
, in
practice, that is almost the same as the x86 guarantee.

The worst case cannot have a large impact because the maximum number of pag=
es
that will not have the TLB evicted is the number of pages in the TLB. For
example, a 1024 entry TLB can at worst result in 4 MB of pages erroneously
reclaimed. That is not bad on a system with 4+ GB of memory.

We did benchmark the extreme case where half the pages accessed where not
evicted from the TLB. In the read case, skipping the DSB was ~10% faster th=
an
skipping the invalidate or doing the invalidate and the DSB.

Compared to the improvement in the average performance and variability in t=
he
other cases we tested, the 10% loss in a carefully crafted test is not as
important.