Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 15 Dec 2022 16:41:04 -0800
From:   Ira Weiny <ira.weiny@intel.com>
To:     Jens Wiklander <jens.wiklander@linaro.org>
CC:     Linus Torvalds <torvalds@linux-foundation.org>,
        Sumit Garg <sumit.garg@linaro.org>,
        Phil Chang =?utf-8?B?KOW8teS4luWLsyk=?= 
        <Phil.Chang@mediatek.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Al Viro <viro@zeniv.linux.org.uk>,
        "Fabio M. De Francesco" <fmdefrancesco@gmail.com>,
        Christoph Hellwig <hch@lst.de>,
        "op-tee@lists.trustedfirmware.org" <op-tee@lists.trustedfirmware.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH 2/4] tee: Remove vmalloc page support
Message-ID: <Y5u+oOLkJs6jehik@iweiny-desk3>
References: <20221002002326.946620-1-ira.weiny@intel.com>
 <20221002002326.946620-3-ira.weiny@intel.com>
 <CAFA6WYOGT1sJLA4c_B88NaXgxv4fm-idi5QMYvXdXB0acCF3sw@mail.gmail.com>
 <TYZPR03MB65279558CC22F5130B710EA8FB5D9@TYZPR03MB6527.apcprd03.prod.outlook.com>
 <CAFA6WYMT9S1Di6DN_UXc823f0ZTkqerE1PB=oG6wmfx28vEbDg@mail.gmail.com>
 <CAHk-=whVyH-wSWLd=Zn4rwo+91T+qzRvfMPC2yFX98GxykOqOw@mail.gmail.com>
 <CAHUa44GkTLCzuSij5FbjBXFBM1CCQROtrCtHHtj70ZRi-3K7uA@mail.gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAHUa44GkTLCzuSij5FbjBXFBM1CCQROtrCtHHtj70ZRi-3K7uA@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6NlZjiFlP+jOc/F5D6h+n4w1REZzMJSPv40LLs5KQXy/vxe4i6bs24WTZQcg?=
 =?us-ascii?Q?sdKijmhZn/tEKz9Jhx2nluhJnuOT1FXZtCrSBHp7nu1O0S9hM+AxkE4jR4QI?=
 =?us-ascii?Q?e0DqcPEgr3nQJKr+OWOP2bn0QrhSotmkTdvJAtkKPXwQfiHdnRQU75JjLlK5?=
 =?us-ascii?Q?IoDl0CJdhUSte1R1hpmJfqppQqf6AdML8x4DTU2ckViBx80+19FPf5M3O48W?=
 =?us-ascii?Q?oD4SNXwSttavVkm20nC1XPDanj7tFhetlUyFl9C4Q+fXeoNZHNFvI/kEpm7D?=
 =?us-ascii?Q?SCceQEzRll8cgONaxq/iXIW63gYYJkRdkzSDd/GgygrZ7WKuVJnCVgWhYcc6?=
 =?us-ascii?Q?CREPfFlunH9dg/bMNjwroyvXIq5nOMGiPUpUj6Mbavt774lF3sW3B4EkeCS+?=
 =?us-ascii?Q?UXa83sbCFZxDhl8qNGxFQEW9EBLML0xESGVJfU3SrbzwDmr81ze8/Gc6/sMG?=
 =?us-ascii?Q?xnz3fbAdmrKJ7LuaFG0KPSRRO3immtTmh6kMdpiaX3pkHKvkq28xhZ19yeHw?=
 =?us-ascii?Q?h6E+NFibvRl8TBlvA9mbIakriMAZ7G9jV2Kq1LOhme9oq2algEpdgOdBZtT0?=
 =?us-ascii?Q?RwtxMbGcFgwn5A23wdlYjXkgodCgQl457gqNnS+0SvZI4N6fFE6WEAsRxIRS?=
 =?us-ascii?Q?o3QaUekt7oDU8Ylz4bonItyKo4mYzG6n8EfqFF84aTx/EtvUeL2cSkvRhsUM?=
 =?us-ascii?Q?j7Cr1JidtjwhaswiYWNMBeacUzhac4sNfJULekxAAAqG1+Af1k5ZB7t/xko8?=
 =?us-ascii?Q?9Engr0QSM65NN/tiHe2Qs06Y0acAYg22U64G6OFPHcXE1o4lUY9RSwbumvPK?=
 =?us-ascii?Q?eaMEOBYTdaFumUGMGEqzVJW0OCGTkKbkBvQOYNyREO9UGcyHvLaYii48yjPV?=
 =?us-ascii?Q?KFRrcw0bjuvr1d/G3i1u6roTBREki7ZdtlrMoDhIecmChoWCddePEonpwL7I?=
 =?us-ascii?Q?6uISdKZ/8yBifCZ5CwbuLy0LLRrjISrY5lX8gs3mSptxnoStTXBKEH8LwfO+?=
 =?us-ascii?Q?ZUyR+cVppW6dxCKKCvm5zLPzroTyBca6gaFr2sFegs5nuwYK9mbW0yeI+uRA?=
 =?us-ascii?Q?ArXps2xH+PSStUKuv/oRRIPh38jqGTs857paF3NT3XJFOW04kwhvOWGGNr5Q?=
 =?us-ascii?Q?fnkWRubtdJHYxbM0b1JTKeXzg1m3PMt0UTTel4zTl7nl6CidS6VwSjrW3BIB?=
 =?us-ascii?Q?eVSCF7C0wujWuXou5vFeFX1kWQeSsn0fTxLZTkYs/zqZ6dQYtoNI2S8d7hST?=
 =?us-ascii?Q?fJpNiJ2VllrBoQevp+j1XhQhoZZJAAk5Li1SLSmeY7GybYRHtlrte3R1s8gb?=
 =?us-ascii?Q?IBxWLqxNB2k27wJR5JaS7HsP/sbw4eLzbSq2qNU4SCQETvB3MUjbvgHvDTWw?=
 =?us-ascii?Q?745ktT/qt0L12BqJD9NWXAFimhDXVs9bzUoRQuIADCfO7BFLCYi9DqdONXVE?=
 =?us-ascii?Q?r/0oME4R0uj4RJL4NaYZ4LD6e9dI9IpP0TNXaPUtoqZQ6++t8Mbn18d96w+h?=
 =?us-ascii?Q?jrbvC1WUr0/HWvZPYKgWnfYxT/k1ZWWo4E9YnMIBimebt7AjNlu22qv+NQwE?=
 =?us-ascii?Q?jFvqLbrYsKvjbjoBCaTy0Y159SiAC3YwpCX/iSZL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0984cd93-b207-4d7d-6fdd-08dadefe38bd
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Dec 2022 00:41:08.2750
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TNJCxxGYx7uPXVRrzEUWA8iPAG96ZPvvyMryJxvUgI0JKx+nP/0F0qdPhjh4f+rPb4nR8LTtJ+ACkxLyYrY3Eg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB6289
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Oct 07, 2022 at 10:12:57AM +0200, Jens Wiklander wrote:
> On Thu, Oct 6, 2022 at 8:20 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Wed, Oct 5, 2022 at 11:24 PM Sumit Garg <sumit.garg@linaro.org> wrote:
> > >
> > > Sorry but you need to get your driver mainline in order to support
> > > vmalloc interface.
> >
> > Actually, I think even then we shouldn't support vmalloc - and
> > register_shm_helper() just needs to be changed to pass in an array of
> > actual page pointers instead.
> 
> register_shm_helper() is an internal function, I suppose it's what's
> passed to tee_shm_register_user_buf() and especially
> tee_shm_register_kernel_buf() in this case.
> 
> So the gain is that in the kernel it becomes the caller's
> responsibility to provide the array of page pointers and the TEE
> subsystem doesn't need to care about what kind of kernel memory it is
> any longer. Yes, that should avoid eventual complexities with
> vmalloc() etc.

I finally spent some time digging into this again.

Overall I'm not opposed to trying to clean up the code more but I feel like the
removal of TEE_SHM_USER_MAPPED is too complex for the main goal; to remove a
caller of kmap_to_page().

Not only is that flag used in release_registered_pages() but it is also used in
tee_shm_fop_mmap().  I'm not following exactly why.  I think this is to allow
mmap of the tee_shm's allocated by kernel users?  Which I _think_ is
orthogonal to the callers of tee_shm_register_kernel_buf()?

> 
> >
> > At that point TEE_SHM_USER_MAPPED should also go away, because then
> > it's the caller that should just do either the user space page
> > pinning, or pass in the kernel page pointer.
> >
> > JensW, is there some reason that wouldn't work?
> 
> We still need to know if it's kernel or user pages in
> release_registered_pages().

Yes.

As I dug into this it seemed ok to define a tee_shm_kernel_free().  Pull out
the allocation of the page array from register_shm_helper() such that it could
be handled by tee_shm_register_kernel_buf() and this new tee_shm_kernel_free().

This seems reasonable because the only callers of tee_shm_register_kernel_buf()
are in trusted_tee.c and they all call tee_shm_free() on the registered memory
prior to returning.

Other callers[*] of tee_shm_free() obtained tee_shm from
tee_shm_alloc_kernel_buf() which AFAICT avoids all this nonsense.

[*] such as .../drivers/firmware/broadcom/tee_bnxt_fw.c.

> 
> The struct tee_shm:s acquired with syscalls from user space are
> reference counted. As are the kernel tee_shm:s, but I believe we could
> separate them to avoid reference counting tee_shm:s used by kernel
> clients if needed. I'll need to look closer at this if we're going to
> use that approach.
> 
> Without reference counting the caller of tee_shm_free() can be certain
> that the secure world is done with the memory so we could delegate the
> kernel pages part of release_registered_pages() to the caller instead.
> 

I'm not sure I follow you here.  Would this be along the lines of creating a
tee_shm_free_kernel() to be used in trusted_tee.c for those specific kernel
data?

Overall I feel like submitting this series again with Christoph and Al's
comments addressed is the best way forward to get rid of kmap_to_page().  I
would really like to get moving on that to avoid any further issues with the
kmap conversions.

But if folks feel strongly enough about removing that flag I can certainly try
to do so.

Ira

> Cheers,
> Jens
> 
> >
> >                  Linus