Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   David Laight <David.Laight@ACULAB.COM>
To:     'Hugh Dickins' <hughd@google.com>, Borislav Petkov <bp@suse.de>
CC:     Peter Zijlstra <peterz@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>
Subject: RE: x86: should clear_user() have alternatives?
Thread-Topic: x86: should clear_user() have alternatives?
Thread-Index: AQHYHK8l5MEK9qlbxkOnyE8hF1HtfqyJhs7w
Date:   Tue, 8 Feb 2022 11:45:38 +0000
Message-ID: <9fc41af45fcb40e3ae607eb4f52d7ef9@AcuMS.aculab.com>
References: <2f5ca5e4-e250-a41c-11fb-a7f4ebc7e1c9@google.com>
In-Reply-To: <2f5ca5e4-e250-a41c-11fb-a7f4ebc7e1c9@google.com>
Accept-Language: en-GB, en-US
MIME-Version: 1.0
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Precedence: bulk

From: Hugh Dickins
> Sent: 08 February 2022 05:46
> 
> I've noticed that clear_user() is slower than it need be:
> 
> dd if=/dev/zero of=/dev/null bs=1M count=1M
> 1099511627776 bytes (1.1 TB) copied, 45.9641 s, 23.9 GB/s
> whereas with the hacked patch below
> 1099511627776 bytes (1.1 TB) copied, 33.4 s, 32.9 GB/s
> 
> That was on some Intel machine: IIRC an AMD went faster.
> 
> It's because clear_user() lacks alternatives, and uses a
> nowadays suboptimal implementation; whereas clear_page()
> and copy_user() do support alternatives.
> 
...
> +SYM_FUNC_START(__clear_user)
> +	ASM_STAC
> +	movl %esi,%ecx
> +	xorq %rax,%rax
> +1:	rep stosb
> +2:	movl %ecx,%eax
> +	ASM_CLAC
> +	ret

You only want to even consider than version for long copies
(and possibly only for aligned ones).

The existing code (I've not quoted) does look sub-optimal though.
It should be easy to obtain a write every clock.
But I suspect the loop is too long.
The code gcc generates might even be better!

Note that for copies longer than 8 bytes 'odd' lengths can
be handled by a single misaligned write to the end of the buffer.
No need for a byte copy loop.

I've not experimented with misaligned writes - they might take two clocks.
So it might be worth aligning them - but they may not happen often
enough for it to be an overall gain.
Misaligned reads usually don't make any difference.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)