On Thu, Jul 17, 2008 at 10:37:17PM +0530, Sudhir Kumar wrote:
> On Thu, Jul 17, 2008 at 09:47:36AM -0400, Vivek Goyal wrote:
> > On Thu, Jul 17, 2008 at 04:05:17PM +0900, Kazunaga Ikeno wrote:
> > > Vivek Goyal wrote:
> > > > On Mon, Jul 14, 2008 at 10:44:43AM -0400, David Collier-Brown wrote:
> > > > > Vivek Goyal wrote:
> > > > >> If admin has decided to group applications and has written the rules for
> > > > >> it then applications should not know anything about grouping. So I think
> > > > >> application writing an script for being placed into the right group should
> > > > >> be out of question. Now how does an admin write a wrapper around existing
> > > > >> application without breaking anything else.
> > > > >
> > > > > In the Solaris world, processes are placed into cgroups (projects) by
> > > > > one of two mechanisms:
> > > > >
> > > > > 1) inheritance, with everything I create in my existing project.
> > > > > To get this started, there is a mechanism under login/getty/whatever
> > > > > reading the /etc/projects file and, for example, tossing user davecb
> > > > > into a "user.davecb" project.
> > > > >
> > > >
> > > > Placing the login sessions in right cgroup based on uid/gid rules is
> > > > probably easy as check needs to be placed only on system entry upon login
> > > > (Pam plugin should do). And after that any job started by the user
> > > > will automatically start in the same cgroup.
> > > >
> > > > > 2) explicit placement with newtask, which starts a program or moves
> > > > > a process into a project/cgroup
> > > > >
> > > >
> > > > explicit placement of task based on application type will be tricky.
> > > >
> > > > > I have a "bg" project which I use for limiting resource consumption of
> > > > > background jobs, and a background command which either starts or moves
> > > > > jobs, thusly:
> > > > >
> > > > > case "$1" in
> > > > > [0-9]*) # It's a pid
> > > > > newtask -p bg -c $1
> > > >
> > > > Ok, this is moving of tasks from one cgroup to other based on pid. This
> > > > is really easy to do through cgroup file system. Just a matter of writing
> > > > to task file.
> > > >
> > > > > ;;
> > > > > *) # It's a command-line
> > > > > newtask -p bg "$@" &
> > > > > ;;
> > > >
> > > > So here a user explicitly invokes the wrapper passing it the targeted
> > > > cgroup and the application to be launched in that cgroup. This should work
> > > > if there is a facility if user has created its own cgroups (lets say
> > > > under user controlled cgroup dir in the hierarchy) and user explicitly
> > > > wants to control the resources of applications under its dir. For example,
> > > >
> > > > /mnt/cgroup
> > > > | |
> > > > gid1 gid2
> > > > | | | |
> > > > uid1 uid2 uid3 uid4
> > > > | |
> > > > proj1 proj2
> > > >
> > > > Here probably admin can write the rules for how users are allocated the
> > > > resources and give ability to users to create subdirs under their cgroups
> > > > where users can create more cgroups and can do their own resource
> > > > management based on application tasks and place applications in the right
> > > > cgroup by writing wrappers as mentioned by you "newtask".
> > > >
> > > > But here there is no discrimination of application type by admin. Admin
> > > > controls resource divisions only based on uid/gid. And users can manage
> > > > applications within their user groups. In fact I am having hard time thinking
> > > > in what kind of scenarios, there is a need for an admin to control
> > > > resource based on application type? Do we really need setups like, on
> > > > a system databases should get network bandwidth of 30%. If yes, then
> > > > it becomes tricky where admin need to write a wrapper to place the task
> > > > in right cgroup without application/user knowing it.
> > >
> > > I think a wrapper (move to right group and calls exec) will run by user, not by admin.
> > > In explicit placement, user knows what a type of application he/she launch.
> > >
> > > /mnt/cgroup
> > > | |
> > > gid1 gid2
> > > | | | |
> > > uid1 uid2 uid3 uid4
> > > | |
> > > proj1 proj2
> > >
> >
> > This is the easy to handle situation and I am hoping it will work in many
> > of the cases.
>
> This solution seems ok but this looks only one part of the storey. Here
> the top level hierarchy is again user based(gid/uid). What if admin
> wants to manage the system resources per application basis? Say a big
> server in a university is being shared by everyone in the university for
> only 3 application
> 1. http server
> 2. browsing
> 3. computing
> In case the admin wants the system to be always available for computing,
> how should he utilize cgroups for managing the server resources among
> these applications ?
> Isn't such scenarios on the priority now?
We only have FS permissions to play around with. Therefore any hierarchy
we come up with will be uid/gid based. Such scenarios will be handled by
the administrator by ensuring the correct permissions are set for the
cgroup.
>
> >
> > Currently I am writting a patch for libcg which allows querying the
> > destination cgroup based on uid/gid and libcg will also migrate the
> > application there. I am also writing a pam plugin which will move
> > all the login sessions to respective cgroup (as mentioned by rule file).
> > Will also modify "init" so that all the system services to into cgroup
> > belonging to root.
> >
> > Once user is logged in and running into his resource group, he can manage
> > further subgroups at his own based on his application requirements (as you
> > mentioned proj1 and proj2 here).
> >
> > > [uid1/gid1]% newtask.sh proj1app
> > > ... proj1app run under /mnt/cgroup/gid1/uid1
> > >
> >
> > Yes, so if a user does not specifically launch an application targetted
> > for a particular cgroup, then it will run into default group for that
> > user (as specified by rule file). In this case under /mnt/cgroup/gid1/uid1.
> So in this user based approach if admin wants to run 4 major
> applications each one requiring say 15% cpu he needs to create 4
> different gids? Creation of a user account just for running an
> application does not look very flexible to me.
>
A lot of daemons run as specific users. Also its not a good idea to run
daemons/servers as root users. They should run as users who have limited
privileges. With such a model in place, Vivek's comments make sense and
might be the right way to go ahead.
Thanks,
--
regards,
Dhaval
On Fri, Jul 18, 2008 at 01:42:53PM +0530, Dhaval Giani wrote:
[..]
> > > > I think a wrapper (move to right group and calls exec) will run by user, not by admin.
> > > > In explicit placement, user knows what a type of application he/she launch.
> > > >
> > > > /mnt/cgroup
> > > > | |
> > > > gid1 gid2
> > > > | | | |
> > > > uid1 uid2 uid3 uid4
> > > > | |
> > > > proj1 proj2
> > > >
> > >
> > > This is the easy to handle situation and I am hoping it will work in many
> > > of the cases.
> >
> > This solution seems ok but this looks only one part of the storey. Here
> > the top level hierarchy is again user based(gid/uid). What if admin
> > wants to manage the system resources per application basis? Say a big
> > server in a university is being shared by everyone in the university for
> > only 3 application
> > 1. http server
> > 2. browsing
> > 3. computing
> > In case the admin wants the system to be always available for computing,
> > how should he utilize cgroups for managing the server resources among
> > these applications ?
> > Isn't such scenarios on the priority now?
>
> We only have FS permissions to play around with. Therefore any hierarchy
> we come up with will be uid/gid based. Such scenarios will be handled by
> the administrator by ensuring the correct permissions are set for the
> cgroup.
>
> >
> > >
> > > Currently I am writting a patch for libcg which allows querying the
> > > destination cgroup based on uid/gid and libcg will also migrate the
> > > application there. I am also writing a pam plugin which will move
> > > all the login sessions to respective cgroup (as mentioned by rule file).
> > > Will also modify "init" so that all the system services to into cgroup
> > > belonging to root.
> > >
> > > Once user is logged in and running into his resource group, he can manage
> > > further subgroups at his own based on his application requirements (as you
> > > mentioned proj1 and proj2 here).
> > >
> > > > [uid1/gid1]% newtask.sh proj1app
> > > > ... proj1app run under /mnt/cgroup/gid1/uid1
> > > >
> > >
> > > Yes, so if a user does not specifically launch an application targetted
> > > for a particular cgroup, then it will run into default group for that
> > > user (as specified by rule file). In this case under /mnt/cgroup/gid1/uid1.
> > So in this user based approach if admin wants to run 4 major
> > applications each one requiring say 15% cpu he needs to create 4
> > different gids? Creation of a user account just for running an
> > application does not look very flexible to me.
> >
>
> A lot of daemons run as specific users. Also its not a good idea to run
> daemons/servers as root users. They should run as users who have limited
> privileges. With such a model in place, Vivek's comments make sense and
> might be the right way to go ahead.
>
Even if admin is launching the applications/daemons and he wants to have
some control on the resources allocated on these daemons individually,
then he needs to just create four cgroups under his account and launch four
daemons in those four cgroups and control the resources.
/mnt/cgroup
| |
usergroup admingroup
| | | | |
uid1 uid2 ser1 ser2 ser3
Here all the "root" tasks go under /mnt/cgrop/admingroup. Now admin
has created three cgroups "ser1" "ser2" and "ser3". Now admin needs
to explicitly launch applications in right cgroup. Something like.
# newtask.sh -cg /mnt/cgroup/admingroup/ser1 appl1
# newtask.sh -cg /mnt/cgroup/admingroup/ser2 appl2
# newtask.sh -cg /mnt/cgroup/admingroup/ser3 appl3
So admin need not to create separate accounts if services have been
launched by admin. He just needs to launch the services in right cgroup
by using the command line utility, specifying destination cgroup.
(To be written).
Thanks
Vivek