Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 28 Oct 2022 10:35:46 -0400
From:   Daniel Jordan <daniel.m.jordan@oracle.com>
To:     Nicolai Stange <nstange@suse.de>
Cc:     Steffen Klassert <steffen.klassert@secunet.com>,
        Herbert Xu <herbert@gondor.apana.org.au>,
        Martin Doucha <mdoucha@suse.cz>, linux-crypto@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/5] padata: make padata_free_shell() to respect pd's
 ->refcnt
Message-ID: <20221028143546.3xc6rnfkfcml373c@parnassus.localdomain>
References: <20221019083708.27138-1-nstange@suse.de>
 <20221019083708.27138-3-nstange@suse.de>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221019083708.27138-3-nstange@suse.de>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3m6DnMDq8cuV6sBcrxDKB6wDGh2dgqcEl33H8nqeFmTthb3tVCr87bmuTs6z?=
 =?us-ascii?Q?/+hefw4XBEUOs8t9keR2xNbhJxH40F70eaCJVlNPpuzg2dmlRe5w6LlG5EFc?=
 =?us-ascii?Q?wqsqrlBbyIPlBjL3qtVPn0HE+TzQW48TmvU+nVjwazOA9cdoutfgErbe7WhB?=
 =?us-ascii?Q?bRIJvMdPYxHV+pRvIMG+g8LrOAYQyneqAdFKZshPCyakKb0WXMoF5wh5xIUF?=
 =?us-ascii?Q?OFiLMZKf6VKv3J4X4Cbwzp2SVOST6mAsP2FAOJWCfIiUhEs+hW8gU+LaqTuN?=
 =?us-ascii?Q?njl5cyPxFoV1qLnp3dj70QceniRsR71faGde06KD7agwDqIv+yq6LpdVOCLU?=
 =?us-ascii?Q?Bq1d6/U9RbEz6hXO3mcFE9Dz/QQZJ0cywX83ivrXX2I6ClJbnIcT5ALcsUWy?=
 =?us-ascii?Q?7F9ZEKVz5azsUcFyuDNf/Oym2lSSs4R2Olm23NVXGePXyJtR6513WbKJQuRv?=
 =?us-ascii?Q?QxfpodSeIyP/g1lKUn76EsWBvcP+yAwWf/71kbiegQLbt3O9EBxM6tyxNM9z?=
 =?us-ascii?Q?bkYbtWZGflg5mytYUwG1OIKTbxyfe/UlQzo04D480N2c3yi+Fr+NXEVddKrh?=
 =?us-ascii?Q?z1L7sy3y2cy5VFyqHV78hwrAb6b+wSSQTlOG5esk3SStTC+9eeY5Heux1K20?=
 =?us-ascii?Q?ey7/vK9YDRqPBz9nCx/rZUL7Q8QR9/whgJOvzsGYFZDtu1MuHFv19cqYKsMo?=
 =?us-ascii?Q?bvEf9i9BZUDHolv+WMidhRp7k5fyuBSku9OG2IVyfTvqQWY488aoKMDp0cK6?=
 =?us-ascii?Q?sp4tVqYrgSpW+tL784eTFaeZntnwedmKOlChSdJuWb0gjsasBN9uaTDzTO9o?=
 =?us-ascii?Q?589kkLoN6Ya3VqMbAnIiLtlq7oFQ3KEYwPntWuQbwGPQd9jnLamCMrZR9O6k?=
 =?us-ascii?Q?VFTP//TfwcgRh9Ngooz1bZC8AysQSEkSXomqjvxY2YLIRVzJQ8LIldgF8bFg?=
 =?us-ascii?Q?7ltq25u1rGsBdHaBEw9v2DfqqyfP5CZZCZS3kccjh/3N+Zq5xqcjSKZyXPUo?=
 =?us-ascii?Q?6JwvhDiamzaO+2l0kREvtjj/rTNy797c/R5zRBdqy/xWdVfVfiIIvc/f3ncQ?=
 =?us-ascii?Q?MnF1HgjjnGiPh5PIsicQF3yj006MbgqhJDCy8aXIxLh/JycGoGwVZXTV7drW?=
 =?us-ascii?Q?IXQ6NK0xJxnWcnY9kbzGGs+mq41Jq0nPbrF7zPDhGXKvEKBl436J/bF8TPC2?=
 =?us-ascii?Q?/qKx/0bgCB6gh+PkzgioOlZmZsiZxvMGRYgiI7iAu9CVcXafNuAJhtIe9V2+?=
 =?us-ascii?Q?Ro++rdx2QvFPLqlvgf4yyMbbEtC8tMJEc/9VwxEaKUjjDyn742WgbWprFxu5?=
 =?us-ascii?Q?Mi45hn7oM9sY9GqiMMTl8IGsPNftdlqEG/bLFW8hD/A+AJn4qRVEIbCR4WMw?=
 =?us-ascii?Q?7fm2HsBaHte8yg/fJ7XqoAW1f8V7r5fLxVXhrZT/sGjTVfO2J3+egVpeq3cb?=
 =?us-ascii?Q?hXUzi3zKkFcbbdD9sBKGeX4MrF6UGo0JLL8q+sFrDDOK1dBij57Y11pN/5dD?=
 =?us-ascii?Q?vHduoMke648E5msI/Hi7Nav7gl8Weoiw5/Wu6IaYKccp9KHVnxcAB4TFDRcp?=
 =?us-ascii?Q?ViR3p5AEolibtHFH+ExO49bBASn7e7UxA4sLmoAw8Vavb7sHat8XrneHMxVm?=
 =?us-ascii?Q?fA=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e7de3f9-4456-49f9-e622-08dab8f1b50a
X-MS-Exchange-CrossTenant-AuthSource: SA1PR10MB5711.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2022 14:35:49.1928
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 07RMXlcfV4FRO4a3mdpXjfUGlM99qw+H1pifYOjzu66gPeMhRYASOkbI5Ms2k9jGdLsssMUFHTImGAHueVmdZGVy8m/kzUkGEUE134MSZlU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR10MB7000
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-10-28_07,2022-10-27_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 malwarescore=0
 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000
 definitions=main-2210280090
X-Proofpoint-ORIG-GUID: Mhy_HQIf1HD9LcOUOVFVwFXPcIUfgpUc
X-Proofpoint-GUID: Mhy_HQIf1HD9LcOUOVFVwFXPcIUfgpUc
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On Wed, Oct 19, 2022 at 10:37:05AM +0200, Nicolai Stange wrote:
> On a PREEMPT kernel, the following has been observed while running
> pcrypt_aead01 from LTP:
> 
>   [ ] general protection fault: 0000 [#1] PREEMPT_RT SMP PTI
>   <...>
>   [ ] Workqueue: pdecrypt_parallel padata_parallel_worker
>   [ ] RIP: 0010:padata_reorder+0x19/0x120
>   <...>
>   [ ] Call Trace:
>   [ ]  padata_parallel_worker+0xa3/0xf0
>   [ ]  process_one_work+0x1db/0x4a0
>   [ ]  worker_thread+0x2d/0x3c0
>   [ ]  ? process_one_work+0x4a0/0x4a0
>   [ ]  kthread+0x159/0x180
>   [ ]  ? kthread_park+0xb0/0xb0
>   [ ]  ret_from_fork+0x35/0x40
> 
> The pcrypt_aead01 testcase basically runs a NEWALG/DELALG sequence for some
> fixed pcrypt instance in a loop, back to back.
> 
> The problem is that once the last ->serial() in padata_serial_worker() is
> getting invoked, the pcrypt requests from the selftests would signal
> completion, and pcrypt_aead01 can move on and subsequently issue a DELALG.
> Upon pcrypt instance deregistration, the associated padata_shell would get
> destroyed, which in turn would unconditionally free the associated
> parallel_data instance.
> 
> If padata_serial_worker() now resumes operation after e.g. having
> previously been preempted upon the return from the last of those ->serial()
> callbacks, its subsequent accesses to pd for managing the ->refcnt will
> all be UAFs. In particular, if the memory backing pd has meanwhile been
> reused for some new parallel_data allocation, e.g in the course of
> processing another subsequent NEWALG request, the padata_serial_worker()
> might find the initial ->refcnt of one and free pd from under that NEWALG
> or the associated selftests respectively, leading to "secondary" UAFs such
> as in the Oops above.
> 
> Note that as it currently stands, a padata_shell owns a reference on its
> associated parallel_data already. So fix the UAF in padata_serial_worker()
> by making padata_free_shell() to not unconditionally free the shell's
> associated parallel_data, but to properly drop that reference via
> padata_put_pd() instead.
> 
> Fixes: 07928d9bfc81 ("padata: Remove broken queue flushing")

It looks like this issue goes back to the first padata commit.  For
instance, pd->refcnt goes to zero after the last _priv is serialized,
padata_free is called in another task, and a particularly sluggish
padata_reorder call touches pd after.

So wouldn't it be

Fixes: 16295bec6398 ("padata: Generic parallelization/serialization interface")

?

Otherwise,

Acked-by: Daniel Jordan <daniel.m.jordan@oracle.com>

> Signed-off-by: Nicolai Stange <nstange@suse.de>
> ---
>  kernel/padata.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/padata.c b/kernel/padata.c
> index 3bd1e23f089b..0bf8c80dad5a 100644
> --- a/kernel/padata.c
> +++ b/kernel/padata.c
> @@ -1112,7 +1112,7 @@ void padata_free_shell(struct padata_shell *ps)
>  
>  	mutex_lock(&ps->pinst->lock);
>  	list_del(&ps->list);
> -	padata_free_pd(rcu_dereference_protected(ps->pd, 1));
> +	padata_put_pd(rcu_dereference_protected(ps->pd, 1));
>  	mutex_unlock(&ps->pinst->lock);
>  
>  	kfree(ps);
> -- 
> 2.37.3
>