Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Elliott, Robert (Servers)" <elliott@hpe.com>
To:     Herbert Xu <herbert@gondor.apana.org.au>,
        "Jason A. Donenfeld" <Jason@zx2c4.com>
CC:     "davem@davemloft.net" <davem@davemloft.net>,
        "tim.c.chen@linux.intel.com" <tim.c.chen@linux.intel.com>,
        "ap420073@gmail.com" <ap420073@gmail.com>,
        "ardb@kernel.org" <ardb@kernel.org>,
        "David.Laight@aculab.com" <David.Laight@aculab.com>,
        "ebiggers@kernel.org" <ebiggers@kernel.org>,
        "linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v4 10/24] crypto: x86/poly - limit FPU preemption
Thread-Topic: [PATCH v4 10/24] crypto: x86/poly - limit FPU preemption
Thread-Index: AQHY+XHqfLLvXNGxcESumjNg7U4Mn65BZh+AgA36OwCACrj5IA==
Date:   Fri, 2 Dec 2022 06:21:02 +0000
Message-ID: <MW5PR84MB1842C2D1EA00D5EF65784E25AB179@MW5PR84MB1842.NAMPRD84.PROD.OUTLOOK.COM>
References: <20221103042740.6556-1-elliott@hpe.com>
 <20221116041342.3841-1-elliott@hpe.com>
 <20221116041342.3841-11-elliott@hpe.com> <Y3TF7/+DejcnN0eV@zx2c4.com>
 <Y4B/kjS0lgzdUJHG@gondor.apana.org.au>
In-Reply-To: <Y4B/kjS0lgzdUJHG@gondor.apana.org.au>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?dhmpuTeCGL8/u0Nufuq0ODRracEnwc5D417cuNiPAssBry9BDkEwYcb5QqLU?=
 =?us-ascii?Q?qrkz7tYhilxEP8HiMWrkqINfwxjLX0VV+UbaABJuCeX6w4DDVIy+zXxgD1xt?=
 =?us-ascii?Q?tICVmCSBcF1K0WrUQlQY7M+0+KEDPhUL8ZP6kBnfaoNdzG20ouutafOr1ZTj?=
 =?us-ascii?Q?waN0IiImeE/T02q6A7uGBo31/1zgTMwHt3qfS/yi9BCPSumXlLptQeY0RSVl?=
 =?us-ascii?Q?MYYif3Yk+W5x+AKNAKXRj54vY0UALvB/phwk5xB4Wjxs1+8/5bWWJQDdgsOl?=
 =?us-ascii?Q?dEVWyBlkR1bUnqT4MvGDPwwQXzvmvb807S/OsUcQ7pfxzvuvNkKIuOipMR3+?=
 =?us-ascii?Q?sFYjg2lgiQUyhFUS/aWSy/XXzzBjJpzOxfi8zPx3HOVPvqUuwlcVTpsTrXs7?=
 =?us-ascii?Q?pXXOuPSFqz9gOtbgkJ7Qx1uOAQiPnX2shtZzseadAfh8crKwy4HlsfZFPdIs?=
 =?us-ascii?Q?dekUi/OVJIjTmrrwSWhZDiCas+92o3H28G2i7AqIoIVuyABxtE6mdz46dQnP?=
 =?us-ascii?Q?ZBnl96bYh3f16rWZVgS06CjzsHsO9JO/nny8gu24XMqfZ/GnIeyw0b/inBuy?=
 =?us-ascii?Q?tGOhayIrX4DuNtzCWCL3+2USwXdRslXvCkSAa2bQA23UeAX++psAX6jJBbOY?=
 =?us-ascii?Q?KfaSFpb90bCtL8U27b6H6hhlRvBptDhqokA4DD41yErOm+RZfAGJgEW8uMy6?=
 =?us-ascii?Q?jDkS3SK3zkEjS9ds+qG8ob0PL4TxnC5YLf1zUpWMGnYdD4+DeyBdZld4mOy/?=
 =?us-ascii?Q?cZcbilpdOdCYzuZ1bgE68ahU9IAZkNHbWmFSDgZ/2BRC0BYyL91BMTKQcb+Z?=
 =?us-ascii?Q?fNaX5hSqG17mJE+q5ncHKWriEugHadk52e0tBNZyH119USvQaW06Lk36deP9?=
 =?us-ascii?Q?XvFoDxlSWWcBt88/uTi9ATPnJ3Evd43iFWfCcMENveafe0vNQWom62zSRTHv?=
 =?us-ascii?Q?E1jnF0uFthQxuOTU49CI49RsZsvIMqCSdP3TgNlK08rSvz7WbM5QqPuf7Ksj?=
 =?us-ascii?Q?uVaDVmZVYDbpDMBYru9tAiteQ6/CgYnRplOVYrvNDP0jiKXRCxovaNy8lvmj?=
 =?us-ascii?Q?smqGYSYABRprelMhM5oR+gUzAWkKDxib05ic2nVSNTgB+LVnEaSpH4oY5Ukn?=
 =?us-ascii?Q?9Qo6HOdYE6X8nswyf5tPf5RY8on8H3HvpfVA0fNKB/y53KMTDnqplLV5NtAK?=
 =?us-ascii?Q?yCdqxNY5sRXuooAO9a6kCF50vhH8eu1HGk3/0GLhku/LNvq8V/SmUr3RSPhZ?=
 =?us-ascii?Q?pREtR3M8EhM9KVDTCLGF9/xE3WIyLmTKRSCI7sRw7fvbG5TtRGaGaARd9zPY?=
 =?us-ascii?Q?Ua5E7kBelnAuEBl46J0bK8jqcdOBOag42hEcjY8NdH0Wq253FDQjlRIYW7c3?=
 =?us-ascii?Q?/WsTQVOUbg+3wJ3I9s03u/dneCQf/8a3RoWJAhJxpWn/VBWyyoBTAqwmiPYj?=
 =?us-ascii?Q?jVxQDgdMb5YYDwxUB+N6dy4oCyob+n5pVu9MT6rJ36if+vFNU9thl1UIMMRh?=
 =?us-ascii?Q?PQxfZh2uvei12P+rM4PMFnNjI6k5ZlVOai8fvx5ywFNcaACAUhzXpZCsDCpD?=
 =?us-ascii?Q?7OEsZsQxzbTi67h7E6U=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW5PR84MB1842.NAMPRD84.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-Network-Message-Id: d09dcba0-5acd-4808-9a65-08dad42d62ba
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2022 06:21:02.1204
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: beG0VpmCEo/c9ZenKKeiPhk1eCPoDaym5oDTS9VcJmDu8/s6GOzzN0fFgB1UDO8g
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR84MB1673
X-OriginatorOrg: hpe.com
X-Proofpoint-GUID: Chf3Efrf3jM1VKXNbMhKpVyH-RkxP7UA
X-Proofpoint-ORIG-GUID: Chf3Efrf3jM1VKXNbMhKpVyH-RkxP7UA
X-HPE-SCL: -1
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-02_02,2022-12-01_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0
 priorityscore=1501 clxscore=1015 lowpriorityscore=0 impostorscore=0
 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 bulkscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2210170000 definitions=main-2212020047
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org


> -----Original Message-----
> From: Herbert Xu <herbert@gondor.apana.org.au>
> Sent: Friday, November 25, 2022 2:41 AM
> To: Jason A. Donenfeld <Jason@zx2c4.com>
> Cc: Elliott, Robert (Servers) <elliott@hpe.com>; davem@davemloft.net;
> tim.c.chen@linux.intel.com; ap420073@gmail.com; ardb@kernel.org;
> David.Laight@aculab.com; ebiggers@kernel.org; linux-
> crypto@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v4 10/24] crypto: x86/poly - limit FPU preemption
>=20
> On Wed, Nov 16, 2022 at 12:13:51PM +0100, Jason A. Donenfeld wrote:
> > On Tue, Nov 15, 2022 at 10:13:28PM -0600, Robert Elliott wrote:
> > > +/* avoid kernel_fpu_begin/end scheduler/rcu stalls */
> > > +static const unsigned int bytes_per_fpu =3D 337 * 1024;
> > > +
> >
> > Use an enum for constants like this:
> >
> >     enum { BYTES_PER_FPU =3D ... };
> >
> > You can even make it function-local, so it's near the code that uses it=
,
> > which will better justify its existence.
> >
> > Also, where did you get this number? Seems kind of weird.
>=20
> These numbers are highly dependent on hardware and I think having
> them hard-coded is wrong.
>=20
> Perhaps we should try a different approach.  How about just limiting
> the size to 4K, and then depending on need_resched we break out of
> the loop? Something like:
>=20
> 	if (!len)
> 		return 0;
>=20
> 	kernel_fpu_begin();
> 	for (;;) {
> 		unsigned int chunk =3D min(len, 4096);
>=20
> 		sha1_base_do_update(desc, data, chunk, sha1_xform);
>=20
> 		len -=3D chunk;
> 		data +=3D chunk;
>=20
> 		if (!len)
> 			break;
>=20
> 		if (need_resched()) {
> 			kernel_fpu_end();
> 			cond_resched();
> 			kernel_fpu_begin();
> 		}
> 	}
> 	kernel_fpu_end();
>=20

I implemented that conditional approach in the sha algorithms.=20

The results of a boot (using sha512 for module signatures, with
crypto extra tests enabled, comparing to sha512 with a 20 KiB
fixed limit) are:

  sha1  cond:  14479 calls;   784256 cycles doing begin/end; longest FPU co=
ntext     35828 cycles
sha256  cond:  26763 calls;  1273570 cycles doing begin/end; longest FPU co=
ntext    118612 cycles
sha512  cond:  26957 calls;  1680046 cycles doing begin/end; longest FPU co=
ntext 169140982 cycles
sha512 20KiB: 161011 calls; 16232280 cycles doing begin/end; longest FPU co=
ntext   4049644 cycles

NOTE: I didn't have a patch in place to isolate the counts for each variati=
on
(ssse3 vs. avx vs. avx2) and
- for sha512: sha512 vs. sha384
- for sha256: sha256 vs. sha224
so the numbers include sha256 and sha512 running twice as many tests
as sha1.

This approach looks very good:
- 16% of the number of begin/end calls
- 10% of the CPU cycles spent making the calls
- the FPU context is held for a long time (77 ms) but only while
  it's not needed.

That's much more efficient than releasing it every 30 us just in case.

I'll keep testing this to make sure RCU stalls stay away, and apply
the approach to the other algorithms.

In x86, need_resched() has to deal with a PER_CPU variable, so I'm
not sure it's worth the hassle to figure out how to do that from
assembly code.