Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221002002326.946620-1-ira.weiny@intel.com> <20221002002326.946620-3-ira.weiny@intel.com>
 <CAFA6WYOGT1sJLA4c_B88NaXgxv4fm-idi5QMYvXdXB0acCF3sw@mail.gmail.com>
 <TYZPR03MB65279558CC22F5130B710EA8FB5D9@TYZPR03MB6527.apcprd03.prod.outlook.com>
 <CAFA6WYMT9S1Di6DN_UXc823f0ZTkqerE1PB=oG6wmfx28vEbDg@mail.gmail.com>
 <CAHk-=whVyH-wSWLd=Zn4rwo+91T+qzRvfMPC2yFX98GxykOqOw@mail.gmail.com>
 <CAHUa44GkTLCzuSij5FbjBXFBM1CCQROtrCtHHtj70ZRi-3K7uA@mail.gmail.com>
 <Y5u+oOLkJs6jehik@iweiny-desk3> <CAFA6WYNMRymyqkqfASYAPVoL0iR2kw0h9YKZ1gBTdeSuHMOAtQ@mail.gmail.com>
In-Reply-To: <CAFA6WYNMRymyqkqfASYAPVoL0iR2kw0h9YKZ1gBTdeSuHMOAtQ@mail.gmail.com>
From:   Sumit Garg <sumit.garg@linaro.org>
Date:   Fri, 16 Dec 2022 14:15:46 +0530
Message-ID: <CAFA6WYMqEVDVW-ifoh-V9ni1zntYdes8adQKf2XXAUpqdaW53w@mail.gmail.com>
Subject: Re: [PATCH 2/4] tee: Remove vmalloc page support
To:     Ira Weiny <ira.weiny@intel.com>
Cc:     Jens Wiklander <jens.wiklander@linaro.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        =?UTF-8?B?UGhpbCBDaGFuZyAo5by15LiW5YuzKQ==?= 
        <Phil.Chang@mediatek.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Al Viro <viro@zeniv.linux.org.uk>,
        "Fabio M. De Francesco" <fmdefrancesco@gmail.com>,
        Christoph Hellwig <hch@lst.de>,
        "op-tee@lists.trustedfirmware.org" <op-tee@lists.trustedfirmware.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Fri, 16 Dec 2022 at 10:39, Sumit Garg <sumit.garg@linaro.org> wrote:
>
> On Fri, 16 Dec 2022 at 06:11, Ira Weiny <ira.weiny@intel.com> wrote:
> >
> > On Fri, Oct 07, 2022 at 10:12:57AM +0200, Jens Wiklander wrote:
> > > On Thu, Oct 6, 2022 at 8:20 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > >
> > > > On Wed, Oct 5, 2022 at 11:24 PM Sumit Garg <sumit.garg@linaro.org> wrote:
> > > > >
> > > > > Sorry but you need to get your driver mainline in order to support
> > > > > vmalloc interface.
> > > >
> > > > Actually, I think even then we shouldn't support vmalloc - and
> > > > register_shm_helper() just needs to be changed to pass in an array of
> > > > actual page pointers instead.
> > >
> > > register_shm_helper() is an internal function, I suppose it's what's
> > > passed to tee_shm_register_user_buf() and especially
> > > tee_shm_register_kernel_buf() in this case.
> > >
> > > So the gain is that in the kernel it becomes the caller's
> > > responsibility to provide the array of page pointers and the TEE
> > > subsystem doesn't need to care about what kind of kernel memory it is
> > > any longer. Yes, that should avoid eventual complexities with
> > > vmalloc() etc.
> >
> > I finally spent some time digging into this again.
> >
> > Overall I'm not opposed to trying to clean up the code more but I feel like the
> > removal of TEE_SHM_USER_MAPPED is too complex for the main goal; to remove a
> > caller of kmap_to_page().
> >
> > Not only is that flag used in release_registered_pages() but it is also used in
> > tee_shm_fop_mmap().  I'm not following exactly why.  I think this is to allow
> > mmap of the tee_shm's allocated by kernel users?
>
> No, its rather to allow mmap of tee_shm allocated via
> tee_shm_alloc_user_buf(). Have a look at its user-space usage here
> [1]. So overall I agree here that we can't get rid of
> TEE_SHM_USER_MAPPED completely as it has a valid mmap use-case.
>
> [1] https://github.com/OP-TEE/optee_client/blob/master/libteec/src/tee_client_api.c#L907
>
> >  Which I _think_ is
> > orthogonal to the callers of tee_shm_register_kernel_buf()?
> >
> > >
> > > >
> > > > At that point TEE_SHM_USER_MAPPED should also go away, because then
> > > > it's the caller that should just do either the user space page
> > > > pinning, or pass in the kernel page pointer.
> > > >
> > > > JensW, is there some reason that wouldn't work?
> > >
> > > We still need to know if it's kernel or user pages in
> > > release_registered_pages().
> >
> > Yes.
> >
> > As I dug into this it seemed ok to define a tee_shm_kernel_free().  Pull out
> > the allocation of the page array from register_shm_helper() such that it could
> > be handled by tee_shm_register_kernel_buf() and this new tee_shm_kernel_free().
> >
>
> +1
>
> > This seems reasonable because the only callers of tee_shm_register_kernel_buf()
> > are in trusted_tee.c and they all call tee_shm_free() on the registered memory
> > prior to returning.
> >
> > Other callers[*] of tee_shm_free() obtained tee_shm from
> > tee_shm_alloc_kernel_buf() which AFAICT avoids all this nonsense.
> >
> > [*] such as .../drivers/firmware/broadcom/tee_bnxt_fw.c.
> >
> > >
> > > The struct tee_shm:s acquired with syscalls from user space are
> > > reference counted. As are the kernel tee_shm:s, but I believe we could
> > > separate them to avoid reference counting tee_shm:s used by kernel
> > > clients if needed. I'll need to look closer at this if we're going to
> > > use that approach.
> > >
> > > Without reference counting the caller of tee_shm_free() can be certain
> > > that the secure world is done with the memory so we could delegate the
> > > kernel pages part of release_registered_pages() to the caller instead.
> > >
> >
> > I'm not sure I follow you here.  Would this be along the lines of creating a
> > tee_shm_free_kernel() to be used in trusted_tee.c for those specific kernel
> > data?
>
> I can't find a reason/use-case for the TEE subsystem to keep a
> refcount of memory registered by other kernel drivers like
> trusted_tee.c. So yes I think it should be safe to directly free that
> shm via tee_shm_free_kernel(). Also with that we can simplify shm
> registration by kernel clients via directly passing page pointers to
> tee_shm_register_kernel_buf() (I would rather rename this API as
> tee_shm_register_kernel_pages()).

Okay, so I will take up this work and get rid of kmap_to_page
invocation from the TEE subsystem.

Ira,

You can then rebase your patchset over my work.

-Sumit

>
> >
> > Overall I feel like submitting this series again with Christoph and Al's
> > comments addressed is the best way forward to get rid of kmap_to_page().  I
> > would really like to get moving on that to avoid any further issues with the
> > kmap conversions.
> >
> > But if folks feel strongly enough about removing that flag I can certainly try
> > to do so.
> >
> > Ira
> >
> > > Cheers,
> > > Jens
> > >
> > > >
> > > >                  Linus