Received-SPF: pass (google.com: domain of linux-kernel+bounces-124383-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Precedence: bulk
MIME-Version: 1.0
References: <20240325100359.17001-1-brgl@bgdev.pl> <56e1c63a-4c09-4d92-9ef2-aad5390879cc@gmail.com>
 <CAMRc=Mf_pvrh2VMfTVE-ZTypyO010p=to-cd8Q745DzSDXLGFw@mail.gmail.com>
In-Reply-To: <CAMRc=Mf_pvrh2VMfTVE-ZTypyO010p=to-cd8Q745DzSDXLGFw@mail.gmail.com>
From: Bartosz Golaszewski <brgl@bgdev.pl>
Date: Fri, 29 Mar 2024 11:22:02 +0100
Message-ID: <CAMRc=MfsVWcoMC-dB-fdxy332h-ucUPTfEUMAnCt5L-q3zJxWg@mail.gmail.com>
Subject: Re: [PATCH v9 00/13] firmware: qcom: qseecom: convert to using the TZ allocator
To: Maximilian Luz <luzmaximilian@gmail.com>
Cc: Andy Gross <agross@kernel.org>, Bjorn Andersson <andersson@kernel.org>, 
	Konrad Dybcio <konrad.dybcio@linaro.org>, Elliot Berman <quic_eberman@quicinc.com>, 
	Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>, 
	Guru Das Srinagesh <quic_gurus@quicinc.com>, Andrew Halaney <ahalaney@redhat.com>, 
	Alex Elder <elder@linaro.org>, Srini Kandagatla <srinivas.kandagatla@linaro.org>, 
	Arnd Bergmann <arnd@arndb.de>, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-arm-kernel@lists.infradead.org, kernel@quicinc.com, 
	Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Mar 28, 2024 at 7:55=E2=80=AFPM Bartosz Golaszewski <brgl@bgdev.pl>=
 wrote:
>
> On Thu, Mar 28, 2024 at 5:50=E2=80=AFPM Maximilian Luz <luzmaximilian@gma=
il.com> wrote:
> >
> > If I understand correctly, it enters an atomic section in
> > qcom_tzmem_alloc() and then tries to schedule somewhere down the line.
> > So this shouldn't be qseecom specific.
> >
> > I should probably also say that I'm currently testing this on a patched
> > v6.8 kernel, so there's a chance that it's my fault. However, as far as
> > I understand, it enters an atomic section in qcom_tzmem_alloc() and the=
n
> > later tries to expand the pool memory with dma_alloc_coherent(). Which
> > AFAIK is allowed to sleep with GFP_KERNEL (and I guess that that's the
> > issue here).
> >
> > I've also tried the shmem allocator option, but that seems to get stuck
> > quite early at boot, before I even have usb-serial access to get any
> > logs. If I can find some more time, I'll try to see if I can get some
> > useful output for that.
> >
>
> Ah, I think it happens here:
>
> +       guard(spinlock_irqsave)(&pool->lock);
> +
> +again:
> +       vaddr =3D gen_pool_alloc(pool->genpool, size);
> +       if (!vaddr) {
> +               if (qcom_tzmem_try_grow_pool(pool, size, gfp))
> +                       goto again;
>
> We were called with GFP_KERNEL so this is what we pass on to
> qcom_tzmem_try_grow_pool() but we're now holding the spinlock. I need
> to revisit it. Thanks for the catch!
>
> Bart

Can you try the following tree?

    https://git.codelinaro.org/bartosz_golaszewski/linux.git
topic/shm-bridge-v10

gen_pool_alloc() and gen_pool_add_virt() can be used without external
serialization. We only really need to protect the list of areas in the
pool when adding a new element. We could possibly even use
list_add_tail_rcu() as it updates the pointers atomically and go
lockless.

Bart