2010-11-02 12:15:36

by Daniel J Blueman

[permalink] [raw]
Subject: [2.6.37-rc1] sys_ioprio_set and RCU locking...

With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
across access to the task credentials.

Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
across the __task_cred call, which is presumably sufficient to prevent
the task credentials becoming stale.

Thus, is there preference to take the RCU lock for read across the
credential access eg at [2], or annotate the call?

Thanks,
Daniel

--- [1]

===================================================

[ INFO: suspicious rcu_dereference_check() usage. ]

---------------------------------------------------

kernel/pid.c:419 invoked rcu_dereference_check() without protection!



other info that might help us debug this:




rcu_scheduler_active = 1, debug_locks = 1

1 lock held by start-stop-daem/2246:

#0: (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
sys_ioprio_set+0x8a/0x400



stack backtrace:

Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2

Call Trace:

[<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0

[<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90

[<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20

[<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400

[<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f

[<ffffffff81003482>] system_call_fastpath+0x16/0x1b


--- [2]

Take the RCU lock for read across acquiring the pointer to the task
credentials and dereferencing it.

Signed-off-by: Daniel J Blueman <[email protected]>

diff --git a/fs/ioprio.c b/fs/ioprio.c
index 748cfb9..00cc0e5 100644
--- a/fs/ioprio.c
+++ b/fs/ioprio.c
@@ -139,8 +139,10 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who,
int, ioprio)
break;

do_each_thread(g, p) {
+ rcu_read_lock();
if (__task_cred(p)->uid != who)
continue;
+ rcu_read_unlock();
ret = set_task_ioprio(p, ioprio);
if (ret)
goto free_uid;
@@ -232,8 +234,10 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who)
break;

do_each_thread(g, p) {
+ rcu_read_lock();
if (__task_cred(p)->uid != user->uid)
continue;
+ rcu_read_unlock();
tmpio = get_task_ioprio(p);
if (tmpio < 0)
continue;
--
Daniel J Blueman


2010-11-07 18:54:37

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
> across access to the task credentials.
>
> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
> across the __task_cred call, which is presumably sufficient to prevent
> the task credentials becoming stale.
>
> Thus, is there preference to take the RCU lock for read across the
> credential access eg at [2], or annotate the call?
>
> Thanks,
> Daniel
>
> --- [1]
>
> ===================================================
>
> [ INFO: suspicious rcu_dereference_check() usage. ]
>
> ---------------------------------------------------
>
> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
>
>
>
> other info that might help us debug this:
>
>
>
>
> rcu_scheduler_active = 1, debug_locks = 1
>
> 1 lock held by start-stop-daem/2246:
>
> #0: (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
> sys_ioprio_set+0x8a/0x400
>
>
>
> stack backtrace:
>
> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
>
> Call Trace:
>
> [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
>
> [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
>
> [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
>
> [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
>
> [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>
> [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
>
>
> --- [2]
>
> Take the RCU lock for read across acquiring the pointer to the task
> credentials and dereferencing it.

Jens, does this look sane?

Thanx, Paul

> Signed-off-by: Daniel J Blueman <[email protected]>
>
> diff --git a/fs/ioprio.c b/fs/ioprio.c
> index 748cfb9..00cc0e5 100644
> --- a/fs/ioprio.c
> +++ b/fs/ioprio.c
> @@ -139,8 +139,10 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who,
> int, ioprio)
> break;
>
> do_each_thread(g, p) {
> + rcu_read_lock();
> if (__task_cred(p)->uid != who)
> continue;
> + rcu_read_unlock();
> ret = set_task_ioprio(p, ioprio);
> if (ret)
> goto free_uid;
> @@ -232,8 +234,10 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who)
> break;
>
> do_each_thread(g, p) {
> + rcu_read_lock();
> if (__task_cred(p)->uid != user->uid)
> continue;
> + rcu_read_unlock();
> tmpio = get_task_ioprio(p);
> if (tmpio < 0)
> continue;
> --
> Daniel J Blueman
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2010-11-08 13:28:30

by Jens Axboe

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On 2010-11-07 19:54, Paul E. McKenney wrote:
> On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
>> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
>> across access to the task credentials.
>>
>> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
>> across the __task_cred call, which is presumably sufficient to prevent
>> the task credentials becoming stale.
>>
>> Thus, is there preference to take the RCU lock for read across the
>> credential access eg at [2], or annotate the call?
>>
>> Thanks,
>> Daniel
>>
>> --- [1]
>>
>> ===================================================
>>
>> [ INFO: suspicious rcu_dereference_check() usage. ]
>>
>> ---------------------------------------------------
>>
>> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
>>
>>
>>
>> other info that might help us debug this:
>>
>>
>>
>>
>> rcu_scheduler_active = 1, debug_locks = 1
>>
>> 1 lock held by start-stop-daem/2246:
>>
>> #0: (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
>> sys_ioprio_set+0x8a/0x400
>>
>>
>>
>> stack backtrace:
>>
>> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
>>
>> Call Trace:
>>
>> [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
>>
>> [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
>>
>> [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
>>
>> [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
>>
>> [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>>
>> [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
>>
>>
>> --- [2]
>>
>> Take the RCU lock for read across acquiring the pointer to the task
>> credentials and dereferencing it.
>
> Jens, does this look sane?

Yes, looks clean enough to me.

--
Jens Axboe

2010-11-08 13:52:19

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On Mon, Nov 08, 2010 at 02:28:29PM +0100, Jens Axboe wrote:
> On 2010-11-07 19:54, Paul E. McKenney wrote:
> > On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
> >> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
> >> across access to the task credentials.
> >>
> >> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
> >> across the __task_cred call, which is presumably sufficient to prevent
> >> the task credentials becoming stale.
> >>
> >> Thus, is there preference to take the RCU lock for read across the
> >> credential access eg at [2], or annotate the call?
> >>
> >> Thanks,
> >> Daniel
> >>
> >> --- [1]
> >>
> >> ===================================================
> >>
> >> [ INFO: suspicious rcu_dereference_check() usage. ]
> >>
> >> ---------------------------------------------------
> >>
> >> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
> >>
> >>
> >>
> >> other info that might help us debug this:
> >>
> >>
> >>
> >>
> >> rcu_scheduler_active = 1, debug_locks = 1
> >>
> >> 1 lock held by start-stop-daem/2246:
> >>
> >> #0: (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
> >> sys_ioprio_set+0x8a/0x400
> >>
> >>
> >>
> >> stack backtrace:
> >>
> >> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
> >>
> >> Call Trace:
> >>
> >> [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
> >>
> >> [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
> >>
> >> [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
> >>
> >> [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
> >>
> >> [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
> >>
> >> [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
> >>
> >>
> >> --- [2]
> >>
> >> Take the RCU lock for read across acquiring the pointer to the task
> >> credentials and dereferencing it.
> >
> > Jens, does this look sane?
>
> Yes, looks clean enough to me.

Very good! Are you willing to take the patch in your tree?

Thanx, Paul

2010-11-08 13:55:45

by Jens Axboe

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On 2010-11-08 14:52, Paul E. McKenney wrote:
> On Mon, Nov 08, 2010 at 02:28:29PM +0100, Jens Axboe wrote:
>> On 2010-11-07 19:54, Paul E. McKenney wrote:
>>> On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
>>>> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
>>>> across access to the task credentials.
>>>>
>>>> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
>>>> across the __task_cred call, which is presumably sufficient to prevent
>>>> the task credentials becoming stale.
>>>>
>>>> Thus, is there preference to take the RCU lock for read across the
>>>> credential access eg at [2], or annotate the call?
>>>>
>>>> Thanks,
>>>> Daniel
>>>>
>>>> --- [1]
>>>>
>>>> ===================================================
>>>>
>>>> [ INFO: suspicious rcu_dereference_check() usage. ]
>>>>
>>>> ---------------------------------------------------
>>>>
>>>> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
>>>>
>>>>
>>>>
>>>> other info that might help us debug this:
>>>>
>>>>
>>>>
>>>>
>>>> rcu_scheduler_active = 1, debug_locks = 1
>>>>
>>>> 1 lock held by start-stop-daem/2246:
>>>>
>>>> #0: (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
>>>> sys_ioprio_set+0x8a/0x400
>>>>
>>>>
>>>>
>>>> stack backtrace:
>>>>
>>>> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
>>>>
>>>> Call Trace:
>>>>
>>>> [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
>>>>
>>>> [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
>>>>
>>>> [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
>>>>
>>>> [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
>>>>
>>>> [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>>>>
>>>> [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
>>>>
>>>>
>>>> --- [2]
>>>>
>>>> Take the RCU lock for read across acquiring the pointer to the task
>>>> credentials and dereferencing it.
>>>
>>> Jens, does this look sane?
>>
>> Yes, looks clean enough to me.
>
> Very good! Are you willing to take the patch in your tree?

Certainly, I'm in the middle of patch monkeying now anyway. Will queue
it up.


--
Jens Axboe

2010-11-09 20:35:22

by Jens Axboe

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On 2010-11-08 14:55, Jens Axboe wrote:
>>>>> Take the RCU lock for read across acquiring the pointer to the task
>>>>> credentials and dereferencing it.
>>>>
>>>> Jens, does this look sane?
>>>
>>> Yes, looks clean enough to me.
>>
>> Very good! Are you willing to take the patch in your tree?
>
> Certainly, I'm in the middle of patch monkeying now anyway. Will queue
> it up.

The patch was buggy, I discovered that it does not do the proper
RCU unlock if we fail the uid match. Merged version here:

http://git.kernel.dk/?p=linux-2.6-block.git;a=commit;h=1a882abdbf9579ad0e5655f928e4ede30db301e6

--
Jens Axboe

2010-11-09 21:49:58

by Daniel J Blueman

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On 9 November 2010 20:35, Jens Axboe <[email protected]> wrote:
> On 2010-11-08 14:55, Jens Axboe wrote:
>>>>>> Take the RCU lock for read across acquiring the pointer to the task
>>>>>> credentials and dereferencing it.
>>>>>
>>>>> Jens, does this look sane?
>>>>
>>>> Yes, looks clean enough to me.
>>>
>>> Very good! ?Are you willing to take the patch in your tree?
>>
>> Certainly, I'm in the middle of patch monkeying now anyway. Will queue
>> it up.
>
> The patch was buggy, I discovered that it does not do the proper
> RCU unlock if we fail the uid match. Merged version here:
>
> http://git.kernel.dk/?p=linux-2.6-block.git;a=commit;h=1a882abdbf9579ad0e5655f928e4ede30db301e6

Brown paper bag moment! Good catch, Jens.

Dan
--
Daniel J Blueman

2010-11-09 21:50:50

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

On Tue, Nov 09, 2010 at 09:35:10PM +0100, Jens Axboe wrote:
> On 2010-11-08 14:55, Jens Axboe wrote:
> >>>>> Take the RCU lock for read across acquiring the pointer to the task
> >>>>> credentials and dereferencing it.
> >>>>
> >>>> Jens, does this look sane?
> >>>
> >>> Yes, looks clean enough to me.
> >>
> >> Very good! Are you willing to take the patch in your tree?
> >
> > Certainly, I'm in the middle of patch monkeying now anyway. Will queue
> > it up.
>
> The patch was buggy, I discovered that it does not do the proper
> RCU unlock if we fail the uid match. Merged version here:
>
> http://git.kernel.dk/?p=linux-2.6-block.git;a=commit;h=1a882abdbf9579ad0e5655f928e4ede30db301e6

Ouch!!! I missed that one entirely, good catch!!!

Thanx, Paul