Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp3337711pxj; Mon, 14 Jun 2021 21:34:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxkfZOXLsJFblvyy8i25yjF+dyDGSaS6n0UvW2hft6j+fwsUZe570Wjx1JC7MnroX5hydpJ X-Received: by 2002:aa7:cb19:: with SMTP id s25mr21252324edt.194.1623731696246; Mon, 14 Jun 2021 21:34:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623731696; cv=none; d=google.com; s=arc-20160816; b=o2V91UBY7VhPC+h2BJkB8UzxiHRSOY0Ca8RkhWkNW2TPORlDkxyXVk3sTccmb3Il8A A29R5+HaIA8SSHeOuyUpc91putYvOe7KtCLx54BQBjhOqXYSpqzarcQFroIn0+VpzK/A u6emU0Op4wT8FpHqB+jGPoG+9o+UvAez7rYocc8svJAzaAbv67LPdZQI3d8iEvSoYGzg PWHAO+lLdXeXyEbcKQNQvzm/rHOapo57Plg4xfSU22y2eXGPFx3K5Pfprz5ikdJMXnxZ NV5l167c7GYKxpYW2QlGDII8fxtOqiac1anAGA0WOGDn1sFEtL3qs5HtoCF2Emo5CDoE TKHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=052yXA1tUnIeo4m1EI7JxwOSuakJ/bbbyEQYfVZm80E=; b=QQfNF+v7NDITt4DneiBo71PjaMOlNDIiUso1bsfeSOb/mdJSBtlQS141kOqmyXvD4s Eb9rAhtMo4PDXYQpQmP/g8sT3LOnlkf9iPO9TYYOROMTiJc/0d1DqK1E9UAJLgCLBY+n +N1mqcacMahM7E4yYZ/wC7hVGHWgpyF2UWOtKVo9JKJwPNuxj33bf7TXhblb2CmmA3RM uxmJ0A+6OshFm6WLbrnVqhe6dMantqW/Of8ZV8hcqL2zdJC1+CmerYnQZG831HzVFY2G Eh20Uq6xwWzElxHgg32KlGrnYXlnulrqXQufqBl3Pih3IWWY4K1+l20Nuv1pFbEpYhqL ln8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ODH70AH3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l21si12946770eja.295.2021.06.14.21.34.33; Mon, 14 Jun 2021 21:34:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ODH70AH3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229919AbhFOEe5 (ORCPT + 99 others); Tue, 15 Jun 2021 00:34:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229497AbhFOEe5 (ORCPT ); Tue, 15 Jun 2021 00:34:57 -0400 Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F631C0617AF for ; Mon, 14 Jun 2021 21:32:53 -0700 (PDT) Received: by mail-lf1-x12f.google.com with SMTP id p7so24743625lfg.4 for ; Mon, 14 Jun 2021 21:32:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=052yXA1tUnIeo4m1EI7JxwOSuakJ/bbbyEQYfVZm80E=; b=ODH70AH30JafeoPmKjXfsh7WgwsLyEKOZVURLpFogUPmhS03Uw+VEsg9eXnmDOUHkX bkNw7omkKi8Mi8oV6GGKsHVG4I9cFO++QZG+T2Sh1241b3FgplE7BiwdSzMpez2TYSN5 N7mK7bmrbg0Mj0ym27xZ5GVMg6ZpYGRHg9OWTBVWS7Bvt5HQU+5CKBwgdhMQ1rXrVYeI lwvjcayCQbIVrGZbWU2e2VjXDdPPP/L/TjsqJ5ucAVPbihgaHZ3JNkZA6eCZQviX+JbI 70tTbXuOuVJose48vVgWN47OTh8zqY+mXGjO8fnr95O3C7h8x2DH7P6iUgLpFlVs33dy fosw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=052yXA1tUnIeo4m1EI7JxwOSuakJ/bbbyEQYfVZm80E=; b=bM3120Y0UmjlxPIft84AkOXU9avG0pyWz222qIwuPXe+hnKodw4Fhn31S1ReeXF7GB rxHpO/JYV44pVcGrw2KtQw7o4uq0kenDKxcuksImL2Z73QkBgCgVketvKuJh3myv1Skv 8ngnTesg4nwrE4Tw0OnkTw8xk9PxfRNvKqUdFBggpr/vaU6Qjo/kY5rKRPVnIl2hYe8S nz08HO91dBhIaXFh49W8nXN+tZGEF0FGmYiXK41cUoKZB46uGI+RwmXldndppstuzE77 py0V1iBvXO/BVJZVahyQhwjPNAbslAohtVXLs8OprQzeHs6SnUOOTxiSauHg9h/mHVkR Fjhg== X-Gm-Message-State: AOAM533ArYQxlw1vA9aeQU+lDa0HkUnHBwunmf9Aw5HH0ytlTVsNaTNu lv8zJD1bEDXPAeEzD0dpzaREabPuVamDL1FJcbmZsw== X-Received: by 2002:a05:6512:3327:: with SMTP id l7mr14421934lfe.113.1623731571270; Mon, 14 Jun 2021 21:32:51 -0700 (PDT) MIME-Version: 1.0 References: <20210614223317.999867-1-tyhicks@linux.microsoft.com> In-Reply-To: <20210614223317.999867-1-tyhicks@linux.microsoft.com> From: Sumit Garg Date: Tue, 15 Jun 2021 10:02:39 +0530 Message-ID: Subject: Re: [PATCH v5 0/8] tee: Improve support for kexec and kdump To: Tyler Hicks Cc: Jens Wiklander , Allen Pais , Peter Huewe , Jarkko Sakkinen , Jason Gunthorpe , Vikas Gupta , Thirupathaiah Annapureddy , Pavel Tatashin , =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= , op-tee@lists.trustedfirmware.org, linux-integrity , bcm-kernel-feedback-list@broadcom.com, linux-mips@vger.kernel.org, Linux Kernel Mailing List Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Tyler, On Tue, 15 Jun 2021 at 04:03, Tyler Hicks wrote: > > v5: > - Picked up Reviewed-by's from Jens. > - Added 'Cc: stable@vger.kernel.org' to all commits as this is intended > to be a bug fix series. I'm happy to sort out backports with the > stable team. > - Got rid of the bool is_mapped parameter of optee_disable_shm_cache() > by abstracting out the function with two wrappers. One > (optee_disable_shm_cache()) for normal case where the shm cache is > fully mapped and another (optee_disable_unmapped_shm_cache()) for the > unusual case of the shm cache having potentially invalid entries. > - Replaced my previous 'tee: Support kernel shm registration without > dma-buf' patch with a cleaner implementation ('tee: Correct > inappropriate usage of TEE_SHM_DMA_BUF flag') from Sumit Garg. > v4: https://lore.kernel.org/lkml/20210610210913.536081-1-tyhicks@linux.microsoft.com/ > v3: https://lore.kernel.org/lkml/20210609002326.210024-1-tyhicks@linux.microsoft.com/ > v2: https://lore.kernel.org/lkml/20210225090610.242623-1-allen.lkml@gmail.com/ > v1: https://lore.kernel.org/lkml/20210217092714.121297-1-allen.lkml@gmail.com/ > This series looks good to me. Feel free to add: Reviewed-by: Sumit Garg -Sumit > This series fixes several bugs uncovered while exercising the OP-TEE > (Open Portable Trusted Execution Environment), ftpm (firmware TPM), and > tee_bnxt_fw (Broadcom BNXT firmware manager) drivers with kexec and > kdump (emergency kexec) based workflows. > > The majority of the problems are caused by missing .shutdown hooks in > the drivers. The .shutdown hooks are used by the normal kexec code path > to let the drivers clean up prior to executing the target kernel. The > .remove hooks, which are already implemented in these drivers, are not > called as part of the kexec code path. This resulted in shared memory > regions, that were cached and/or registered with OP-TEE, not being > cleared/unregistered prior to kexec. The new kernel would then run into > problems when handling the previously cached virtual addresses or trying > to register newly allocated shared memory objects that overlapped with > the previously registered virtual addresses. The TEE didn't receive > notification that the old virtual addresses were no longer meaningful > and that a new kernel, with a new address space, would soon be running. > > However, implementing .shutdown hooks was not enough for supporting > kexec. There was an additional problem caused by the TEE driver's > reliance on the dma-buf subsystem for multi-page shared memory objects > that were registered with the TEE. Shared memory objects backed by a > dma-buf use a different mechanism for reference counting. When the final > reference is released, work is scheduled to be executed to unregister > the shared memory with the TEE but that work is only completed prior to > the current task returning the userspace. In the case of a kexec > operation, the current task that's calling the driver .shutdown hooks > never returns to userspace prior to the kexec operation so the shared > memory was never unregistered. This eventually caused problems from > overlapping shared memory regions that were registered with the TEE > after several kexec operations. The large 4M contiguous region > allocated by the tee_bnxt_fw driver reliably ran into this issue on the > fourth kexec on a system with 8G of RAM. > > The use of dma-buf makes sense for shared memory that's in use by > userspace but dma-buf's aren't needed for shared memory that will only > used by the driver. This series separates dma-buf backed shared memory > allocated by the kernel from multi-page shared memory that the kernel > simply needs registered with the TEE for private use. > > One other noteworthy change in this series is to completely refuse to > load the OP-TEE driver in the kdump kernel. This is needed because the > secure world may have had all of its threads in suspended state when the > regular kernel crashed. The kdump kernel would then hang during boot > because the OP-TEE driver's .probe function would attempt to use a > secure world thread when they're all in suspended state. Another problem > is that shared memory allocations could fail under the kdump kernel > because the previously registered were not unregistered (the .shutdown > hook is not called when kexec'ing into the kdump kernel). > > The first patch in the series fixes potential memory leaks that are not > directly related to kexec or kdump but were noticed during the > development of this series. > > Tyler > > Allen Pais (2): > optee: fix tee out of memory failure seen during kexec reboot > firmware: tee_bnxt: Release TEE shm, session, and context during kexec > > Jens Wiklander (1): > tee: add tee_shm_alloc_kernel_buf() > > Sumit Garg (1): > tee: Correct inappropriate usage of TEE_SHM_DMA_BUF flag > > Tyler Hicks (4): > optee: Fix memory leak when failing to register shm pages > optee: Refuse to load the driver under the kdump kernel > optee: Clear stale cache entries during initialization > tpm_ftpm_tee: Free and unregister TEE shared memory during kexec > > drivers/char/tpm/tpm_ftpm_tee.c | 8 ++--- > drivers/firmware/broadcom/tee_bnxt_fw.c | 14 ++++++-- > drivers/tee/optee/call.c | 38 +++++++++++++++++++--- > drivers/tee/optee/core.c | 43 ++++++++++++++++++++++++- > drivers/tee/optee/optee_private.h | 1 + > drivers/tee/optee/rpc.c | 5 +-- > drivers/tee/optee/shm_pool.c | 20 +++++++++--- > drivers/tee/tee_shm.c | 20 +++++++++++- > include/linux/tee_drv.h | 2 ++ > 9 files changed, 132 insertions(+), 19 deletions(-) > > -- > 2.25.1 >