2022-05-06 15:21:48

by Byungchul Park

[permalink] [raw]
Subject: Re: [PATCH RFC v6 00/21] DEPT(Dependency Tracker)

Linus wrote:
>
> On Wed, May 4, 2022 at 1:19 AM Byungchul Park <[email protected]> wrote:
> >
> > Hi Linus and folks,
> >
> > I've been developing a tool for detecting deadlock possibilities by
> > tracking wait/event rather than lock(?) acquisition order to try to
> > cover all synchonization machanisms.
>
> So what is the actual status of reports these days?
>
> Last time I looked at some reports, it gave a lot of false positives
> due to mis-understanding prepare_to_sleep().

Yes, it was. I handled the case in the following way:

1. Stage the wait at prepare_to_sleep(), which might be used at commit.
Which has yet to be an actual wait that Dept considers.
2. If the condition for sleep is true, the wait will be committed at
__schedule(). The wait becomes an actual one that Dept considers.
3. If the condition is false and the task gets back to TASK_RUNNING,
clean(=reset) the staged wait.

That way, Dept only works with what actually hits to __schedule() for
the waits through sleep.

> For this all to make sense, it would need to not have false positives
> (or at least a very small number of them together with a way to sanely

Yes. I agree with you. I got rid of them that way I described above.

> get rid of them), and also have a track record of finding things that
> lockdep doesn't.

I have some reports that wait_for_completion or waitqueue is involved.
It's worth noting those are not tracked by Lockdep. I'm checking if
those are true positive or not. I will share those reports once I get
more convinced for that.

> Maybe such reports have been sent out with the current situation, and
> I haven't seen them.

Dept reports usually have been sent to me privately, not in LKML. As I
told you, I'm planning to share them.

Byungchul

>
> Linus
>


2022-05-07 12:56:03

by Hyeonggon Yoo

[permalink] [raw]
Subject: Re: [PATCH RFC v6 00/21] DEPT(Dependency Tracker)

On Fri, May 06, 2022 at 09:11:35AM +0900, Byungchul Park wrote:
> Linus wrote:
> >
> > On Wed, May 4, 2022 at 1:19 AM Byungchul Park <[email protected]> wrote:
> > >
> > > Hi Linus and folks,
> > >
> > > I've been developing a tool for detecting deadlock possibilities by
> > > tracking wait/event rather than lock(?) acquisition order to try to
> > > cover all synchonization machanisms.
> >
> > So what is the actual status of reports these days?
> >
> > Last time I looked at some reports, it gave a lot of false positives
> > due to mis-understanding prepare_to_sleep().
>
> Yes, it was. I handled the case in the following way:
>
> 1. Stage the wait at prepare_to_sleep(), which might be used at commit.
> Which has yet to be an actual wait that Dept considers.
> 2. If the condition for sleep is true, the wait will be committed at
> __schedule(). The wait becomes an actual one that Dept considers.
> 3. If the condition is false and the task gets back to TASK_RUNNING,
> clean(=reset) the staged wait.
>
> That way, Dept only works with what actually hits to __schedule() for
> the waits through sleep.
>
> > For this all to make sense, it would need to not have false positives
> > (or at least a very small number of them together with a way to sanely
>
> Yes. I agree with you. I got rid of them that way I described above.
>

IMHO DEPT should not report what lockdep allows (Not talking about
wait events). I mean lockdep allows some kind of nested locks but
DEPT reports them.

When I was collecting reports from DEPT on varous configurations,
Most of them was report of down_write_nested(), which is allowed in
lockdep.

DEPT should not report at least what we know it's not a real deadlock.
Otherwise there will be reports that is never fixed, which is quite
unpleasant and reporters cannot examine all of them if it's real deadlock
or not.

> > get rid of them), and also have a track record of finding things that
> > lockdep doesn't.
>
> I have some reports that wait_for_completion or waitqueue is involved.
> It's worth noting those are not tracked by Lockdep. I'm checking if
> those are true positive or not. I will share those reports once I get
> more convinced for that.
>
> > Maybe such reports have been sent out with the current situation, and
> > I haven't seen them.
>
> Dept reports usually have been sent to me privately, not in LKML. As I
> told you, I'm planning to share them.
>
> Byungchul
>
> >
> > Linus
> >

--
Thanks,
Hyeonggon