Hello.
Out customer reports that "idle" field is not monotonic. So far this
is all I know. I do not know how to reproduce, etc.
But when I look at this code, this looks really possible even
ignoring drivers/cpuidle/ which plays with update_ts_time_stats().
So, get_cpu_idle_time_us(last_update_time => NULL) does:
if (ts->idle_active && !nr_iowait_cpu(cpu)) {
ktime_t delta = ktime_sub(now, ts->idle_entrytime);
idle = ktime_add(ts->idle_sleeptime, delta);
} else {
idle = ts->idle_sleeptime;
}
Suppose that ts->idle_active == T. By the time we calculate
idle = ktime_add(ts->idle_sleeptime, delta);
this cpu can be already non-idle and ->idle_sleeptime can be already
updated by tick_nohz_stop_idle(), we return the wrong value.
If user-space reads /proc/stat again after that, "idle" can obviously
go back.
get_cpu_iowait_time_us() has the same problem.
Plus nr_iowait_cpu() can change in between even if cpu stays idle,
io_schedule() can return on another CPU.
Questions:
- Any other reason why it can be non-monotonic?
- Should we fix this or should we document that userspace
should handle this itself?
IOW, is this is bug or not?
Oleg.
On Fri, Aug 16, 2013 at 04:47:04PM +0200, Oleg Nesterov wrote:
> Hello.
>
> Out customer reports that "idle" field is not monotonic. So far this
> is all I know. I do not know how to reproduce, etc.
>
> But when I look at this code, this looks really possible even
> ignoring drivers/cpuidle/ which plays with update_ts_time_stats().
>
> So, get_cpu_idle_time_us(last_update_time => NULL) does:
>
> if (ts->idle_active && !nr_iowait_cpu(cpu)) {
> ktime_t delta = ktime_sub(now, ts->idle_entrytime);
>
> idle = ktime_add(ts->idle_sleeptime, delta);
> } else {
> idle = ts->idle_sleeptime;
> }
>
>
> Suppose that ts->idle_active == T. By the time we calculate
>
> idle = ktime_add(ts->idle_sleeptime, delta);
>
> this cpu can be already non-idle and ->idle_sleeptime can be already
> updated by tick_nohz_stop_idle(), we return the wrong value.
>
> If user-space reads /proc/stat again after that, "idle" can obviously
> go back.
>
> get_cpu_iowait_time_us() has the same problem.
>
> Plus nr_iowait_cpu() can change in between even if cpu stays idle,
> io_schedule() can return on another CPU.
>
> Questions:
>
> - Any other reason why it can be non-monotonic?
>
> - Should we fix this or should we document that userspace
> should handle this itself?
>
> IOW, is this is bug or not?
I don't know if we want to fix it (I personally think we should because it is not the
first time I hear complains about this) but if we do, here is a possible fix:
https://lkml.org/lkml/2013/8/8/638
Thanks.
On 08/16, Frederic Weisbecker wrote:
>
> On Fri, Aug 16, 2013 at 04:47:04PM +0200, Oleg Nesterov wrote:
> >
> > Questions:
> >
> > - Any other reason why it can be non-monotonic?
> >
> > - Should we fix this or should we document that userspace
> > should handle this itself?
> >
> > IOW, is this is bug or not?
>
> I don't know if we want to fix it (I personally think we should because it is not the
> first time I hear complains about this) but if we do, here is a possible fix:
>
> https://lkml.org/lkml/2013/8/8/638
Thanks! it is not easy to read the patches on lkml.org and I do
not understand this code enough. But it seems that this should
address my concerns, including the "even ignoring drivers/cpuidle/".
Except, I am not sure that these changes handle the case when
nr_iowait() change in between.
Are you going to resend?
Oleg.
On Fri, Aug 16, 2013 at 05:13:39PM +0200, Oleg Nesterov wrote:
> On 08/16, Frederic Weisbecker wrote:
> >
> > On Fri, Aug 16, 2013 at 04:47:04PM +0200, Oleg Nesterov wrote:
> > >
> > > Questions:
> > >
> > > - Any other reason why it can be non-monotonic?
> > >
> > > - Should we fix this or should we document that userspace
> > > should handle this itself?
> > >
> > > IOW, is this is bug or not?
> >
> > I don't know if we want to fix it (I personally think we should because it is not the
> > first time I hear complains about this) but if we do, here is a possible fix:
> >
> > https://lkml.org/lkml/2013/8/8/638
>
> Thanks! it is not easy to read the patches on lkml.org and I do
> not understand this code enough. But it seems that this should
> address my concerns, including the "even ignoring drivers/cpuidle/".
>
> Except, I am not sure that these changes handle the case when
> nr_iowait() change in between.
Yeah it handles iowait and idle sleeps.
>
> Are you going to resend?
I'm more than happy that you want to review these patches.
I'm resending.
Thanks.
>
> Oleg.
>