Received-SPF: pass (google.com: domain of linux-kernel+bounces-158977-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Precedence: bulk
MIME-Version: 1.0
References: <20240423202356.3372314-2-rdbabiera@google.com> <ZionHYcUvvc865z4@kuha.fi.intel.com>
In-Reply-To: <ZionHYcUvvc865z4@kuha.fi.intel.com>
From: RD Babiera <rdbabiera@google.com>
Date: Thu, 25 Apr 2024 10:37:21 -0700
Message-ID: <CALzBnUHL4kWTQEncWfzNyze25_Ss_Pf8pyR2FOauyoLwNRrrMA@mail.gmail.com>
Subject: Re: [PATCH v1] usb: typec: tcpm: enforce ready state when queueing
 alt mode vdm
To: Heikki Krogerus <heikki.krogerus@linux.intel.com>
Cc: linux@roeck-us.net, gregkh@linuxfoundation.org, hdegoede@redhat.com, 
	badhri@google.com, linux-usb@vger.kernel.org, linux-kernel@vger.kernel.org, 
	stable@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Apr 25, 2024 at 2:49=E2=80=AFAM Heikki Krogerus
<heikki.krogerus@linux.intel.com> wrote:
> I'm probable missing something, but wouldn't it be safer to check
> port->state after grabbing the lock?

I could have been more explicit in describing the deadlock, my bad.
But there are two
threads here:

Thread A starts in the TCPM. It is the port state machine that
transitions to DR_SWAP
and holds port->lock. When it unregisters DisplayPort Alt Mode, it goes int=
o the
DP Alt Mode driver and hangs until any DP Alt Mode work is finished.

Thread B starts in the DP Alt Mode driver. It attempts to call tcpm_enter_m=
ode
and the call to mutex_lock in tcpm_queue_vdm_unlock hangs because Thread A
holds the lock. Thread A will never drop the lock because it waits for Thre=
ad B
to finish.

So, the check is done before grabbing the lock because the thread needs to =
avoid
grabbing the lock in the first place. If port->state changes between
queueing and
sending the message, the VDM state machine will drop the message anyways
because port->state isn't in the ready state as well.

best,
rd