2009-06-01 13:45:25

by Jan Scholz

[permalink] [raw]
Subject: Re: [regression, bisected] adb trackpad disappears after suspend to ram

"Rafael J. Wysocki" <[email protected]> writes:

> On Friday 29 May 2009, Jan Scholz wrote:
>> Benjamin Herrenschmidt <[email protected]> writes:
>>
>> >> We are too late in the cycle to revert this commit and it really is needed to
>> >> fix a more serious issue. Nevertheless knowing that it caused the problem to
>> >> appear on your system is also important.
>> >>
>> >> Ben, do you have an idea what may be going on here? Does __disable_irq() mask
>> >> the interrupt on this platform?
>> >
>> > I suppose so :-) I'll have to check what's going on, it's not
>> > immediately clear to me.
>> >
>> > Cheers,
>> > Ben.
>> >
>> >>
>> >> I'd like to see a boot log, preferably containing a suspend-resume in which
>> >> the problem was reproduced.
>>
>> Here is a log from booting, through several suspend cycles, until the
>> trackpad disappeared. The first few suspends were done while X was
>> running but from the console. As you can tell from lines like
>> May 28 23:51:26 [kernel] adb devices: [2]: 2 c4 [3]: 3 1 [7]: 7 1f
>> the trackpad was still present. The last suspend was done from within X,
>> here the trackpad did not make it.
>> May 28 23:58:09 [kernel] adb devices: [2]: 2 c4 [7]: 7 1f
>> However, there have been cases, although not in this log, were the
>> trackpad has been alive even when suspending from within X.
>
> This means the problem is probably timing-related.
>
> Can you please try to comment out suspend_device_irqs() and
> resume_device_irqs() in drivers/base/power/main.c and see if that changes
> anything? It's not entirely safe (well, that's why the calls are there after
> all), but hopefully the box won't hang during this test.
>

Tried that against v2.6.30-rc7 and it seems to fix the issue. I did ~10
suspend-resume cycles from the console (which worked just like before)
and ~20 cycles from within X and the trackpad is still alive.

Best,
Jan Scholz


2009-06-01 16:36:18

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [regression, bisected] adb trackpad disappears after suspend to ram

On Monday 01 June 2009, Jan Scholz wrote:
> "Rafael J. Wysocki" <[email protected]> writes:
>
> > On Friday 29 May 2009, Jan Scholz wrote:
> >> Benjamin Herrenschmidt <[email protected]> writes:
> >>
> >> >> We are too late in the cycle to revert this commit and it really is needed to
> >> >> fix a more serious issue. Nevertheless knowing that it caused the problem to
> >> >> appear on your system is also important.
> >> >>
> >> >> Ben, do you have an idea what may be going on here? Does __disable_irq() mask
> >> >> the interrupt on this platform?
> >> >
> >> > I suppose so :-) I'll have to check what's going on, it's not
> >> > immediately clear to me.
> >> >
> >> > Cheers,
> >> > Ben.
> >> >
> >> >>
> >> >> I'd like to see a boot log, preferably containing a suspend-resume in which
> >> >> the problem was reproduced.
> >>
> >> Here is a log from booting, through several suspend cycles, until the
> >> trackpad disappeared. The first few suspends were done while X was
> >> running but from the console. As you can tell from lines like
> >> May 28 23:51:26 [kernel] adb devices: [2]: 2 c4 [3]: 3 1 [7]: 7 1f
> >> the trackpad was still present. The last suspend was done from within X,
> >> here the trackpad did not make it.
> >> May 28 23:58:09 [kernel] adb devices: [2]: 2 c4 [7]: 7 1f
> >> However, there have been cases, although not in this log, were the
> >> trackpad has been alive even when suspending from within X.
> >
> > This means the problem is probably timing-related.
> >
> > Can you please try to comment out suspend_device_irqs() and
> > resume_device_irqs() in drivers/base/power/main.c and see if that changes
> > anything? It's not entirely safe (well, that's why the calls are there after
> > all), but hopefully the box won't hang during this test.
> >
>
> Tried that against v2.6.30-rc7 and it seems to fix the issue. I did ~10
> suspend-resume cycles from the console (which worked just like before)
> and ~20 cycles from within X and the trackpad is still alive.

So, it seems we lose and interrupt during resume and that confuses the
ADB controller driver or something like this. Do you use the keyboard or
the trackpad as a wake-up device?

Please additionally try to go back to the original code, put
'sleepy_trackpad = 1' at the beginning of do_adb_reset_bus() in
drivers/macintosh/adb.c and see if the problem is reproducible with that.

Best,
Rafael

2009-06-01 22:35:20

by Jan Scholz

[permalink] [raw]
Subject: Re: [regression, bisected] adb trackpad disappears after suspend to ram

"Rafael J. Wysocki" <[email protected]> writes:

> On Monday 01 June 2009, Jan Scholz wrote:
>> "Rafael J. Wysocki" <[email protected]> writes:
>>
>> > On Friday 29 May 2009, Jan Scholz wrote:
>> >> Benjamin Herrenschmidt <[email protected]> writes:
>> >>
>> >> >> We are too late in the cycle to revert this commit and it really is needed to
>> >> >> fix a more serious issue. Nevertheless knowing that it caused the problem to
>> >> >> appear on your system is also important.
>> >> >>
>> >> >> Ben, do you have an idea what may be going on here? Does __disable_irq() mask
>> >> >> the interrupt on this platform?
>> >> >
>> >> > I suppose so :-) I'll have to check what's going on, it's not
>> >> > immediately clear to me.
>> >> >
>> >> > Cheers,
>> >> > Ben.
>> >> >
>> >> >>
>> >> >> I'd like to see a boot log, preferably containing a suspend-resume in which
>> >> >> the problem was reproduced.
>> >>
>> >> Here is a log from booting, through several suspend cycles, until the
>> >> trackpad disappeared. The first few suspends were done while X was
>> >> running but from the console. As you can tell from lines like
>> >> May 28 23:51:26 [kernel] adb devices: [2]: 2 c4 [3]: 3 1 [7]: 7 1f
>> >> the trackpad was still present. The last suspend was done from within X,
>> >> here the trackpad did not make it.
>> >> May 28 23:58:09 [kernel] adb devices: [2]: 2 c4 [7]: 7 1f
>> >> However, there have been cases, although not in this log, were the
>> >> trackpad has been alive even when suspending from within X.
>> >
>> > This means the problem is probably timing-related.
>> >
>> > Can you please try to comment out suspend_device_irqs() and
>> > resume_device_irqs() in drivers/base/power/main.c and see if that changes
>> > anything? It's not entirely safe (well, that's why the calls are there after
>> > all), but hopefully the box won't hang during this test.
>> >
>>
>> Tried that against v2.6.30-rc7 and it seems to fix the issue. I did ~10
>> suspend-resume cycles from the console (which worked just like before)
>> and ~20 cycles from within X and the trackpad is still alive.
>
> So, it seems we lose and interrupt during resume and that confuses the
> ADB controller driver or something like this. Do you use the keyboard or
> the trackpad as a wake-up device?

I do the wakeup by pressing keys on the keyboard. I'd try wakeup via the
trackpad's button, but I have no idea how to activate that.

> Please additionally try to go back to the original code, put
> 'sleepy_trackpad = 1' at the beginning of do_adb_reset_bus() in
> drivers/macintosh/adb.c and see if the problem is reproducible with that.

Tried this, but it does not help.

What I haven't noticed before is, that even if the trackpad already
disappeared, an additional suspend-resume cycle from the console brought
it back to life every time. Sometimes this worked from within X as well,
just not with such a high probability as from the console.
However, this is independent from 'sleepy_trackpad = 1'.

Best,
Jan

2009-06-01 22:42:39

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [regression, bisected] adb trackpad disappears after suspend to ram

On Tuesday 02 June 2009, Jan Scholz wrote:
> "Rafael J. Wysocki" <[email protected]> writes:
>
> > On Monday 01 June 2009, Jan Scholz wrote:
> >> "Rafael J. Wysocki" <[email protected]> writes:
> >>
> >> > On Friday 29 May 2009, Jan Scholz wrote:
> >> >> Benjamin Herrenschmidt <[email protected]> writes:
> >> >>
> >> >> >> We are too late in the cycle to revert this commit and it really is needed to
> >> >> >> fix a more serious issue. Nevertheless knowing that it caused the problem to
> >> >> >> appear on your system is also important.
> >> >> >>
> >> >> >> Ben, do you have an idea what may be going on here? Does __disable_irq() mask
> >> >> >> the interrupt on this platform?
> >> >> >
> >> >> > I suppose so :-) I'll have to check what's going on, it's not
> >> >> > immediately clear to me.
> >> >> >
> >> >> > Cheers,
> >> >> > Ben.
> >> >> >
> >> >> >>
> >> >> >> I'd like to see a boot log, preferably containing a suspend-resume in which
> >> >> >> the problem was reproduced.
> >> >>
> >> >> Here is a log from booting, through several suspend cycles, until the
> >> >> trackpad disappeared. The first few suspends were done while X was
> >> >> running but from the console. As you can tell from lines like
> >> >> May 28 23:51:26 [kernel] adb devices: [2]: 2 c4 [3]: 3 1 [7]: 7 1f
> >> >> the trackpad was still present. The last suspend was done from within X,
> >> >> here the trackpad did not make it.
> >> >> May 28 23:58:09 [kernel] adb devices: [2]: 2 c4 [7]: 7 1f
> >> >> However, there have been cases, although not in this log, were the
> >> >> trackpad has been alive even when suspending from within X.
> >> >
> >> > This means the problem is probably timing-related.
> >> >
> >> > Can you please try to comment out suspend_device_irqs() and
> >> > resume_device_irqs() in drivers/base/power/main.c and see if that changes
> >> > anything? It's not entirely safe (well, that's why the calls are there after
> >> > all), but hopefully the box won't hang during this test.
> >> >
> >>
> >> Tried that against v2.6.30-rc7 and it seems to fix the issue. I did ~10
> >> suspend-resume cycles from the console (which worked just like before)
> >> and ~20 cycles from within X and the trackpad is still alive.
> >
> > So, it seems we lose and interrupt during resume and that confuses the
> > ADB controller driver or something like this. Do you use the keyboard or
> > the trackpad as a wake-up device?
>
> I do the wakeup by pressing keys on the keyboard. I'd try wakeup via the
> trackpad's button, but I have no idea how to activate that.
>
> > Please additionally try to go back to the original code, put
> > 'sleepy_trackpad = 1' at the beginning of do_adb_reset_bus() in
> > drivers/macintosh/adb.c and see if the problem is reproducible with that.
>
> Tried this, but it does not help.
>
> What I haven't noticed before is, that even if the trackpad already
> disappeared, an additional suspend-resume cycle from the console brought
> it back to life every time. Sometimes this worked from within X as well,
> just not with such a high probability as from the console.
> However, this is independent from 'sleepy_trackpad = 1'.

OK, thanks for testing. I think I know what to do, but I need some time to
prepare a patch.

Best,
Rafael

2009-06-01 22:54:54

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [regression, bisected] adb trackpad disappears after suspend to ram


> > Tried this, but it does not help.
> >
> > What I haven't noticed before is, that even if the trackpad already
> > disappeared, an additional suspend-resume cycle from the console brought
> > it back to life every time. Sometimes this worked from within X as well,
> > just not with such a high probability as from the console.
> > However, this is independent from 'sleepy_trackpad = 1'.
>
> OK, thanks for testing. I think I know what to do, but I need some time to
> prepare a patch.

I wonder if the trakpad is timing out at some stage, we don't drain it
properly before suspend or something like that. Those ADB things are
a bit strange.

Ben.

2009-06-02 04:50:59

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [regression, bisected] adb trackpad disappears after suspend to ram


> So, it seems we lose and interrupt during resume and that confuses the
> ADB controller driver or something like this. Do you use the keyboard or
> the trackpad as a wake-up device?
>
> Please additionally try to go back to the original code, put
> 'sleepy_trackpad = 1' at the beginning of do_adb_reset_bus() in
> drivers/macintosh/adb.c and see if the problem is reproducible with that.

Well, the ADB controller is also the PMU (drivers/macintosh/via-pmu.c)
which is also the system controller, handles the actual sleep state,
etc... :-)

That code is a bit itchy around the edges. I'm not sure it would have
lost an interrupt, that sounds more like losing an IRQ would have broken
it completely but it's not -impossible- (especially if it's the external
GPIO or CB1 interrupt that notifies of an incoming ADB message).

I haven't managed to reproduce the problem yet here though. I'll see if
I can with a titanium powerbook I have somewhere in storage that might
be a bit closer to your machine than the wallstreet powerbook I've been
using to test at work :-) My other powerbook test machine uses a USB
trackpad.

Cheers,
Ben.