2006-03-02 20:40:53

by Raul

[permalink] [raw]
Subject: Doubt about scheduler

Hello!!!, I?m a student of computer science and I?m doing my final
degree job in linux. It is about "linux kernel modules" , I have to know
some things of the scheduler. The runqueue struct, and so on. The
problem is that in the last linux kernel version in the "sched.h" isn?t
defined these structs (prio_array, runqueue...), and I cann?t access to
runqueue or prio_array fields. I know that in the 2.6.5 kernel version
these fields were accessible and now don?t, could you tell me what is
the reason please?
I think that I?m going to do it (the module) in the 2.6.5 kernel
version and will try to explain why, and for this I need your help.

Please answer to my email, because I?m not subscribe to mailing list.

Thank you very much for your help, sorry for my poor english, it?s not
my mother tongue.

PS: If you think that I can do the module in the last kernel version,
could you tell me how, please?, thank you very much.


2006-03-04 06:33:53

by Jon Masters

[permalink] [raw]
Subject: Re: Doubt about scheduler

On 3/2/06, Ra?l Baena <[email protected]> wrote:

> Hello!!!, I?m a student of computer science and I?m doing my final
> degree job in linux. It is about "linux kernel modules" , I have to know
> some things of the scheduler. The runqueue struct, and so on. The
> problem is that in the last linux kernel version in the "sched.h" isn?t
> defined these structs (prio_array, runqueue...), and I cann?t access to
> runqueue or prio_array fields. I know that in the 2.6.5 kernel version
> these fields were accessible and now don?t, could you tell me what is
> the reason please?

Deliberately, these aren't available outside of the scheduler so that
they can't be played with. Much as things like the symbol table aren't
exported to modules, some things in the kernel aren't even available
to other parts of the core kernel :-)

> I think that I?m going to do it (the module) in the 2.6.5 kernel
> version and will try to explain why, and for this I need your help.

If you really need to play with this stuff, then why not just make
your own kernel tree with this hacked up in the scheduler code itself?
If it's just for you, then that'll work fine. If you would explain
what it is that you need to do, then someone might be able to offer
you advice on the general direction to take - see also
http://www.kernelnewbies.org/

Jon.

2006-03-05 13:13:46

by Raul

[permalink] [raw]
Subject: Re: Doubt about scheduler

Thank you very much Jon. But I think I haven?t explained very well.

I know that now the prio_array and runqueues structs aren?t accesible
for modules, but in the 2.6.5 version they were. I would like to know
the reason, why before they were accesible and now they don?t? If you
could answer me, it would be great. I could to write the reason in my
university job. (In Spain we have to make a final degree job, and mine
is about modules in linux (I chose this), I would like to show
information of the new scheduler, a scheduler monitor, and these fields
are indispensable for me)
I thought in your solution (own kernel tree), but I would like to make a
module that worked in standard distributions. I prefer to make a module
that worked in 2.6.5 version instead to make one that worked in 2.6.12
but with my own kernel tree. So I suposse that I will do that, but
knowing the reason why could serve me to make a better investigation
document.

Thank you again for your help, I will mention you in my final degree job
acknowledge. Please, keep helping me!!! :)



Jon Masters escribi?:

>On 3/2/06, Ra?l Baena <[email protected]> wrote:
>
>
>
>>Hello!!!, I?m a student of computer science and I?m doing my final
>>degree job in linux. It is about "linux kernel modules" , I have to know
>>some things of the scheduler. The runqueue struct, and so on. The
>>problem is that in the last linux kernel version in the "sched.h" isn?t
>>defined these structs (prio_array, runqueue...), and I cann?t access to
>>runqueue or prio_array fields. I know that in the 2.6.5 kernel version
>>these fields were accessible and now don?t, could you tell me what is
>>the reason please?
>>
>>
>
>Deliberately, these aren't available outside of the scheduler so that
>they can't be played with. Much as things like the symbol table aren't
>exported to modules, some things in the kernel aren't even available
>to other parts of the core kernel :-)
>
>
>
>> I think that I?m going to do it (the module) in the 2.6.5 kernel
>>version and will try to explain why, and for this I need your help.
>>
>>
>
>If you really need to play with this stuff, then why not just make
>your own kernel tree with this hacked up in the scheduler code itself?
>If it's just for you, then that'll work fine. If you would explain
>what it is that you need to do, then someone might be able to offer
>you advice on the general direction to take - see also
>http://www.kernelnewbies.org/
>
>Jon.
>
>
>
>
>

2006-03-05 13:30:19

by Nick Piggin

[permalink] [raw]
Subject: Re: Doubt about scheduler

Ra?l Baena wrote:
> Thank you very much Jon. But I think I haven?t explained very well.
>
> I know that now the prio_array and runqueues structs aren?t accesible
> for modules, but in the 2.6.5 version they were. I would like to know
> the reason, why before they were accesible and now they don?t? If you
> could answer me, it would be great.

I don't remember them being available in 2.6.5... but as to why they
aren't available now: it is much cleaner this way. It even benefits
you because now nobody will break your module when they change the
data structure.

> I could to write the reason in my
> university job. (In Spain we have to make a final degree job, and mine
> is about modules in linux (I chose this), I would like to show
> information of the new scheduler, a scheduler monitor, and these fields
> are indispensable for me)

If your task is about modules in Linux, then I don't see how that
involves the scheduler at all?

On the other hand, if you want a scheduler monitor then I can't see
why it would be appropriate to implement as a module (we have schedstats,
which you can read from a userspace program or daemon).

Nick

--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com

2006-03-05 15:21:27

by Raul

[permalink] [raw]
Subject: Re: Doubt about scheduler


>> I could to write the reason in my university job. (In Spain we have
>> to make a final degree job, and mine is about modules in linux (I
>> chose this), I would like to show information of the new scheduler, a
>> scheduler monitor, and these fields are indispensable for me)
>
>
> If your task is about modules in Linux, then I don't see how that
> involves the scheduler at all?
>
> On the other hand, if you want a scheduler monitor then I can't see
> why it would be appropriate to implement as a module (we have schedstats,
> which you can read from a userspace program or daemon).
>
> Nick
>
Thank you very much Nick.

My task is free, I can do everything that I want that involves modules,
I thought that to make the module about the new O(k) scheduler would be
a good idea. I think that it?s not enough for me schedstats, because I
want to make a visual scheduler, I mean, using GTK+ , a module and
something else to make a visual scheduler monitor, how the tasks move
between "active" and "expired", where the task are in prio_array with
the bitmap fields...this module isn?t usefull, only in a didactic way.
This is the reason because I have to make a module althought ( according
with your recomendations, recomendations of a person who knows
infinitely much than me) probably would be better to make another thing.

At any rate, I will search "schedstats" references, in case it was usefull.

Thank all of you very much again , with your help I hope to be able to
make a good final degree job.

2006-03-05 21:27:23

by Jon Masters

[permalink] [raw]
Subject: Re: Doubt about scheduler

On 3/5/06, Ra?l Baena <[email protected]> wrote:

> Thank you very much Jon. But I think I haven?t explained very well.

No, you did. Perhaps I wasn't terse enough in my reply :-)

> I know that now the prio_array and runqueues structs aren?t accesible
> for modules, but in the 2.6.5 version they were. I would like to know
> the reason, why before they were accesible and now they don?t?

Because it's a /bad idea/ to screw around with these. That's why. I
was trying to say that nicely before - it's cleaner and safer if the
scheduler is the only thing playing there as it's the only thing with
a bona fide need.

I'm not trying to be offensive at all - by all means make your own
kernel, perhaps with an exported version of these structure or
introduce your own API for tinkering with them. But just don't
necessarily expect everyone else to jump at it :-)

Jon.

2006-03-05 21:27:32

by Mike Galbraith

[permalink] [raw]
Subject: Re: Doubt about scheduler

On Mon, 2006-03-06 at 00:30 +1100, Nick Piggin wrote:
> Raúl Baena wrote:
> > Thank you very much Jon. But I think I haven´t explained very well.
> >
> > I know that now the prio_array and runqueues structs aren´t accesible
> > for modules, but in the 2.6.5 version they were. I would like to know
> > the reason, why before they were accesible and now they don´t? If you
> > could answer me, it would be great.
>
> I don't remember them being available in 2.6.5... but as to why they
> aren't available now: it is much cleaner this way. It even benefits
> you because now nobody will break your module when they change the
> data structure.

I can't recall having ever seen these exposed.. by Ingo.

-Mike

2006-03-05 21:30:25

by Jon Masters

[permalink] [raw]
Subject: Re: Doubt about scheduler

On 3/5/06, Ra?l Baena <[email protected]> wrote:

> I thought that to make the module about the new O(k) scheduler would be
> a good idea. I think that it?s not enough for me schedstats, because I
> want to make a visual scheduler, I mean, using GTK+ , a module and
> something else to make a visual scheduler monitor, how the tasks move
> between "active" and "expired", where the task are in prio_array with
> the bitmap fields...this module isn?t usefull, only in a didactic way.

If you're seriously interested in this then cool. Let me know how you get on.

I looked at hacking something into gtop etc. previously to use
/proc/kcore and pull out task information - I'd certainly like to see
a visual process monitor that could pull all of this stuff out and
display it for educational interest (page tables, vmas, other
resources). But then, it's probably been done - I didn't look to see
what else is out there.

Jon.

2006-03-05 22:37:19

by Peter Williams

[permalink] [raw]
Subject: Re: Doubt about scheduler

Nick Piggin wrote:
> Ra?l Baena wrote:
>
>> Thank you very much Jon. But I think I haven?t explained very well.
>>
>> I know that now the prio_array and runqueues structs aren?t accesible
>> for modules, but in the 2.6.5 version they were. I would like to know
>> the reason, why before they were accesible and now they don?t? If you
>> could answer me, it would be great.
>
>
> I don't remember them being available in 2.6.5... but as to why they
> aren't available now: it is much cleaner this way. It even benefits
> you because now nobody will break your module when they change the
> data structure.

SuSE patched their 2.6.5 kernel to make the run queues visible in a
header file. Perhaps Raul was using a SuSE version of 2.6.5.

>
>> I could to write the reason in my university job. (In Spain we have to
>> make a final degree job, and mine is about modules in linux (I chose
>> this), I would like to show information of the new scheduler, a
>> scheduler monitor, and these fields are indispensable for me)
>
>
> If your task is about modules in Linux, then I don't see how that
> involves the scheduler at all?
>
> On the other hand, if you want a scheduler monitor then I can't see
> why it would be appropriate to implement as a module (we have schedstats,
> which you can read from a userspace program or daemon).
>
> Nick
>

Peter
--
Peter Williams [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
-- Ambrose Bierce

2006-03-05 22:58:37

by Raul

[permalink] [raw]
Subject: Re: Doubt about scheduler

Peter Williams escribi?:

> SuSE patched their 2.6.5 kernel to make the run queues visible in a
> header file. Perhaps Raul was using a SuSE version of 2.6.5.

Exact!!!! . To be honest, I have the 2.6.12 version to debian, but my
project tutor has this one. And when I argued about my project and I
told him that those fields weren?t accesible, he sent me his sched.h,
and I saw that runqueue and prio_array were accesible in it. I didn?t
think that it was a SUSE patch . Sorry, it's my fault

I have made a litte cheat to be able to see prio_array fields, I have
created a new struct and later with the "array" field of "task_struct" I
have made a cast to my struct. Then I have been able to show these
values, but I don't like this "solution", because I can?t know anything
of tasks in another state ("active" or "expired"), neither which is my
state!! For this reason I would like to know how to get this
information, my project have to be done in June 30 :(.

Thank you very much again for your help, I?m getting a lot of
information about the kernel that I'll be able to use in my report, of
course if you agree.


2006-03-06 07:29:56

by Steven Rostedt

[permalink] [raw]
Subject: Re: Doubt about scheduler


On Sun, 5 Mar 2006, Jon Masters wrote:

> On 3/5/06, Ra?l Baena <[email protected]> wrote:
>
> > I thought that to make the module about the new O(k) scheduler would be
> > a good idea. I think that it?s not enough for me schedstats, because I
> > want to make a visual scheduler, I mean, using GTK+ , a module and
> > something else to make a visual scheduler monitor, how the tasks move
> > between "active" and "expired", where the task are in prio_array with
> > the bitmap fields...this module isn?t usefull, only in a didactic way.
>
> If you're seriously interested in this then cool. Let me know how you get on.
>
> I looked at hacking something into gtop etc. previously to use
> /proc/kcore and pull out task information - I'd certainly like to see
> a visual process monitor that could pull all of this stuff out and
> display it for educational interest (page tables, vmas, other
> resources). But then, it's probably been done - I didn't look to see
> what else is out there.
>

Raul, Also take a look at relayfs. It's a fast way to record data in the
kernel and pass it back to a userland process. You'll have to patch the
kernel as it is said that the data needed is private to sched.c

Look into Documentation/filesystems/relayfs.txt

relayfs entered the kernel in 2.6.14.

-- Steve

2006-03-06 08:56:03

by Raul

[permalink] [raw]
Subject: Re: Doubt about scheduler

Steven Rostedt wrote:

>On Sun, 5 Mar 2006, Jon Masters wrote:
>
>
>
>>On 3/5/06, Ra?l Baena <[email protected]> wrote:
>>
>>
>>
>>>I thought that to make the module about the new O(k) scheduler would be
>>>a good idea. I think that it?s not enough for me schedstats, because I
>>>want to make a visual scheduler, I mean, using GTK+ , a module and
>>>something else to make a visual scheduler monitor, how the tasks move
>>>between "active" and "expired", where the task are in prio_array with
>>>the bitmap fields...this module isn?t usefull, only in a didactic way.
>>>
>>>
>>If you're seriously interested in this then cool. Let me know how you get on.
>>
>>I looked at hacking something into gtop etc. previously to use
>>/proc/kcore and pull out task information - I'd certainly like to see
>>a visual process monitor that could pull all of this stuff out and
>>display it for educational interest (page tables, vmas, other
>>resources). But then, it's probably been done - I didn't look to see
>>what else is out there.
>>
>>
>>
>
>Raul, Also take a look at relayfs. It's a fast way to record data in the
>kernel and pass it back to a userland process. You'll have to patch the
>kernel as it is said that the data needed is private to sched.c
>
>Look into Documentation/filesystems/relayfs.txt
>
>relayfs entered the kernel in 2.6.14.
>
>-- Steve
>
>
>
Thank you very much, I'll see it. I'll tell you my progress!!!

2006-03-06 15:43:15

by Randy Dunlap

[permalink] [raw]
Subject: Re: Doubt about scheduler

On Mon, 06 Mar 2006 09:52:36 +0100 Raul wrote:

> Steven Rostedt wrote:
>
> >On Sun, 5 Mar 2006, Jon Masters wrote:
> >
> >
> >
> >>On 3/5/06, Ra?l Baena <[email protected]> wrote:
> >>
> >>
> >>
> >>>I thought that to make the module about the new O(k) scheduler would be
> >>>a good idea. I think that it?s not enough for me schedstats, because I
> >>>want to make a visual scheduler, I mean, using GTK+ , a module and
> >>>something else to make a visual scheduler monitor, how the tasks move
> >>>between "active" and "expired", where the task are in prio_array with
> >>>the bitmap fields...this module isn?t usefull, only in a didactic way.
> >>>
> >>>
> >>If you're seriously interested in this then cool. Let me know how you get on.
> >>
> >>I looked at hacking something into gtop etc. previously to use
> >>/proc/kcore and pull out task information - I'd certainly like to see
> >>a visual process monitor that could pull all of this stuff out and
> >>display it for educational interest (page tables, vmas, other
> >>resources). But then, it's probably been done - I didn't look to see
> >>what else is out there.
> >>
> >>
> >>
> >
> >Raul, Also take a look at relayfs. It's a fast way to record data in the
> >kernel and pass it back to a userland process. You'll have to patch the
> >kernel as it is said that the data needed is private to sched.c
> >
> >Look into Documentation/filesystems/relayfs.txt
> >
> >relayfs entered the kernel in 2.6.14.
> >
> >-- Steve
> >
> >
> >
> Thank you very much, I'll see it. I'll tell you my progress!!!

note that in recent kernels it has changed to:

config RELAY
bool "Kernel->user space relay support (formerly relayfs)"
help
This option enables support for relay interface support in
certain file systems (such as sysfs, and trivially debugfs).
It is designed to provide an efficient mechanism for tools and
facilities to relay large amounts of data from kernel space to
user space.



---
~Randy