I have enhanced the kernel to allow several independant virtual servers
running on the same box (sharing the same kernel as well). I introduced
2 new system calls (new_s_context and set_ipv4root) allowing
much independance between the virtual servers. Virtual servers are
independant enough and "real" enough that you can supply root password
to the virtual server administrators. Virtual servers may be described
-May run various network services, binding to the same ports
without special configuration. Services are started normally (sysv script, whatever
the distro you are using).
-Have independant process list, so they can't interfere. You can't see or send
signal to process in other vservers (or the root server)
-I have also modified the capability system a little, so those virtual server
administrators can't take over the machine. I have introduced a per-process
capability ceiling, inherited by sub-process. Even setuid program can't grab
more capabilities..
-Update packages normally, create users, Use any admin procedure/tool
Maybe such a project has already been done. Anyway, I have written a lot
of documentation about it (how it works, pro and con and so on). It works
on top of 2.4.10 or 2.4.11 (probably anything). I would really like to get
some comments.
You can find all the documentation and packages at
http://www.solucorp.qc.ca/miscprj/s_context.hc
All this is GPL...
---------------------------------------------------------
Jacques Gelinas <[email protected]>
vserver: run general purpose virtual servers on one box, full speed!
http://www.solucorp.qc.ca/miscprj/s_context.hc
Hi!
> -I have also modified the capability system a little, so those virtual server
> administrators can't take over the machine. I have introduced a per-process
> capability ceiling, inherited by sub-process. Even setuid program can't grab
> more capabilities..
Really? What hardware do they see in /dev/? Do their servers have for
example mouse? What about ethernet cards?
Does /proc/kmem work in virtual servers?
[Why I'm asking? I'm trying to find ways to take over the machine. Do
you want to give me root on your machine stating that I can't
interfere?]
You might want to announce this on bugtraq. [And give solar designer
root account, he might be more creative ;)].
Pavel
--
STOP THE WAR! Someone killed innocent Americans. That does not give
U.S. right to kill people in Afganistan.
In article <[email protected]> you wrote:
> [Why I'm asking? I'm trying to find ways to take over the machine. Do
> you want to give me root on your machine stating that I can't
> interfere?]
Hve you looked on HP compartments, yet? Or on Brickhouse?
Both are there a bit longer, and at least for brickhouse you can get root
access by telneting into it.
Greetings
Bernd
On Fri, 12 Oct 2001 23:01:04 -0500, Pavel Machek wrote
> Hi!
>
> > -I have also modified the capability system a little, so those virtual server
> > administrators can't take over the machine. I have introduced a per-process
> > capability ceiling, inherited by sub-process. Even setuid program can't grab
> > more capabilities..
>
> Really? What hardware do they see in /dev/? Do their servers have for
> example mouse? What about ethernet cards?
In /dev they see very little: full log null ptmx pts random tty urandom zero
The do not have CAP_SYS_MKNOD, so they can't create more than you give.
In fact, the vserver sees whatever you want to give it. So if you intend to run
X in the vserver, give it the mouse device.
> Does /proc/kmem work in virtual servers?
You probably mean /dev/kcore. No they can't read it.
> [Why I'm asking? I'm trying to find ways to take over the machine. Do
> you want to give me root on your machine stating that I can't
> interfere?]
Indeed, I could give you a root password on a vserver and you would not be
able to interfere. Sure enough you would be able to grab resource and slow
down the machine (and potentially work out a DOS attack). We are working
on the schedular right now to solve those issues.
But there is no need to open a crackme vserver. Install it on your machine,
build a vserver. Install the vserver package, reboot a kernel with the patch
with the new new_s_context and set_ipv4root syscall and do
/usr/sbin/vserver test build
enter it and configure a few service
/usr/sbin/vserver test enter
chkconfig crond on
chkconfig sshd on
Then start the server
/usr/sbin/vserver test start
and ssh to it. For now, the build process inherit the user accounts of the
main server, so the root password is the same.
Note that if sshd is already running on the box, you won't be able start
sshd in the test server because sshd on the main server is bound to
0.0.0.0. Just do
/etc/rc.d/init.d/v_sshd restart
or
chkconfig sshd off
chkconfig v_sshd on
> You might want to announce this on bugtraq. [And give solar designer
> root account, he might be more creative ;)].
You don't understand the issue. Anyone can create his own vserver. The
system call controlling this are very simple. It is not a "try to crack my machine"
contest. Anyone can create a vserver and test it.
The security of the vserver is explain on
http://www.solucorp.qc.ca/miscprj/s_context.hc. It relies on the capability
system. So far, I have found one place in the kernel where the capability
was not in place: /proc/sys was changeable if you were root. I added
a capable(CAP_SYS_ADMIN) line to solve this.
--
One nice thing about vserver is the clean separation with the real server.
For example, after having played with a vserver, if you decide you do not like
this concept, then you ca do
reboot the old kernel
erase the new kernel
rpm -e vserver
rm -fr /vservers /etc/vservers
and you are back where you were.
---------------------------------------------------------
Jacques Gelinas <[email protected]>
vserver: run general purpose virtual servers on one box, full speed!
http://www.solucorp.qc.ca/miscprj/s_context.hc
On Mon, 15 Oct 2001, Jacques Gelinas wrote:
> On Fri, 12 Oct 2001 23:01:04 -0500, Pavel Machek wrote
> > Hi!
> >
> > > -I have also modified the capability system a little, so those virtual server
> > > administrators can't take over the machine. I have introduced a per-process
> > > capability ceiling, inherited by sub-process. Even setuid program can't grab
> > > more capabilities..
> >
> > Really? What hardware do they see in /dev/? Do their servers have for
> > example mouse? What about ethernet cards?
>
> In /dev they see very little: full log null ptmx pts random tty
> urandom zero
>
> The do not have CAP_SYS_MKNOD, so they can't create more than you give.
>
> In fact, the vserver sees whatever you want to give it. So if you intend to run
> X in the vserver, give it the mouse device.
Hmmm - does this work with devfs?
> > [Why I'm asking? I'm trying to find ways to take over the machine. Do
> > you want to give me root on your machine stating that I can't
> > interfere?]
>
> Indeed, I could give you a root password on a vserver and you would not be
> able to interfere. Sure enough you would be able to grab resource and slow
> down the machine (and potentially work out a DOS attack). We are working
> on the schedular right now to solve those issues.
Have you looked at the "fairsched" patch for this? It seems to be
unmaintained since 2.4.0-testXX, but look close to your needs...
> But there is no need to open a crackme vserver. Install it on your machine,
> build a vserver.
The question, I think, was would YOU give out root access on vservers on
YOUR box, and be confident people wouldn't be able to escape? :-)
> > You might want to announce this on bugtraq. [And give solar designer
> > root account, he might be more creative ;)].
>
> You don't understand the issue. Anyone can create his own vserver. The
> system call controlling this are very simple. It is not a "try to
> crack my machine" contest. Anyone can create a vserver and test it.
But can you crack your way OUT of the vserver - how confident are you in
the isolation provided?
James.
--
"Our attitude with TCP/IP is, `Hey, we'll do it, but don't make a big
system, because we can't fix it if it breaks -- nobody can.'"
"TCP/IP is OK if you've got a little informal club, and it doesn't make
any difference if it takes a while to fix it."
-- Ken Olson, in Digital News, 1988
you mention problems with interaction if the main sandbox has a service
listening on 0.0.0.0, what happens if a vserver does this (does it only
see it's own IP addresses or does it interfere with other servers?)
David Lang
On
Mon, 15 Oct 2001, Jacques Gelinas wrote:
> Date: Mon, 15 Oct 2001 09:29:05 -0500
> From: Jacques Gelinas <[email protected]>
> To: Linux kernel list <[email protected]>
> Cc: Pavel Machek <[email protected]>
> Subject: re: Re: Announce: many virtual servers on a single box
>
> On Fri, 12 Oct 2001 23:01:04 -0500, Pavel Machek wrote
> > Hi!
> >
> > > -I have also modified the capability system a little, so those virtual server
> > > administrators can't take over the machine. I have introduced a per-process
> > > capability ceiling, inherited by sub-process. Even setuid program can't grab
> > > more capabilities..
> >
> > Really? What hardware do they see in /dev/? Do their servers have for
> > example mouse? What about ethernet cards?
>
> In /dev they see very little: full log null ptmx pts random tty urandom zero
>
> The do not have CAP_SYS_MKNOD, so they can't create more than you give.
>
> In fact, the vserver sees whatever you want to give it. So if you intend to run
> X in the vserver, give it the mouse device.
>
> > Does /proc/kmem work in virtual servers?
>
> You probably mean /dev/kcore. No they can't read it.
>
> > [Why I'm asking? I'm trying to find ways to take over the machine. Do
> > you want to give me root on your machine stating that I can't
> > interfere?]
>
> Indeed, I could give you a root password on a vserver and you would not be
> able to interfere. Sure enough you would be able to grab resource and slow
> down the machine (and potentially work out a DOS attack). We are working
> on the schedular right now to solve those issues.
>
> But there is no need to open a crackme vserver. Install it on your machine,
> build a vserver. Install the vserver package, reboot a kernel with the patch
> with the new new_s_context and set_ipv4root syscall and do
>
> /usr/sbin/vserver test build
>
> enter it and configure a few service
>
> /usr/sbin/vserver test enter
>
> chkconfig crond on
> chkconfig sshd on
>
> Then start the server
>
> /usr/sbin/vserver test start
>
> and ssh to it. For now, the build process inherit the user accounts of the
> main server, so the root password is the same.
>
> Note that if sshd is already running on the box, you won't be able start
> sshd in the test server because sshd on the main server is bound to
> 0.0.0.0. Just do
>
> /etc/rc.d/init.d/v_sshd restart
>
> or
>
> chkconfig sshd off
> chkconfig v_sshd on
>
> > You might want to announce this on bugtraq. [And give solar designer
> > root account, he might be more creative ;)].
>
> You don't understand the issue. Anyone can create his own vserver. The
> system call controlling this are very simple. It is not a "try to crack my machine"
> contest. Anyone can create a vserver and test it.
>
> The security of the vserver is explain on
> http://www.solucorp.qc.ca/miscprj/s_context.hc. It relies on the capability
> system. So far, I have found one place in the kernel where the capability
> was not in place: /proc/sys was changeable if you were root. I added
> a capable(CAP_SYS_ADMIN) line to solve this.
>
> --
>
> One nice thing about vserver is the clean separation with the real server.
> For example, after having played with a vserver, if you decide you do not like
> this concept, then you ca do
>
> reboot the old kernel
> erase the new kernel
> rpm -e vserver
> rm -fr /vservers /etc/vservers
>
> and you are back where you were.
>
> ---------------------------------------------------------
> Jacques Gelinas <[email protected]>
> vserver: run general purpose virtual servers on one box, full speed!
> http://www.solucorp.qc.ca/miscprj/s_context.hc
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Mon, 15 Oct 2001 08:55:51 -0500, David Lang wrote
> you mention problems with interaction if the main sandbox has a service
> listening on 0.0.0.0, what happens if a vserver does this (does it only
> see it's own IP addresses or does it interfere with other servers?)
The vserver can't bind 0.0.0.0. The new set_ipv4root system call (named
after the chroot one) is setting one IP number in the current task_struct.
the system call accept only a transition from 0 to some IP. So like chroot()
once you do it in a process you are trapped.
The system calls affects both the bind() syscall and the connect() one (when
it lookup the route). Once an ipv4 root is set to x.y.z.w, the process (and children) can
do
bind (0.0.0.0) and it gets bind(x.y.z.w)
connect() without binding and they get coonect + bind(x.y.z.w)
bind (x.y.z.w) and it gets bind(x.y.z.w)
bind(another_ip) and it gets an error
connect + bind(another_IP) and it gets an error.
The ability to bind multiple time on a port has always been there. So using
this call, we limit the vserver to a single IP. The nice part is that this is
completly transparent to the vserver services. Services like apache, sshd, xinetd
sybase are running without any special configuration.
Btw, the v_xxx service (v_httpd, v_sshd) are simple wrapper to run service
in the main server without having to change their configuration so they
do not interfere. What they do is simply
#!/bin/sh
/usr/sbin/chbind --ip eth0 /etc/rc.d/init.d/httpd $*
Also you can do the following
/usr/sbin/chbind --ip 1.2.3.4 some_command
and this effectivly prevent the "some_command" to do any IP network activity.
The set_ipv4root() syscall is not privileged at all. Its a one shot thing though :-)
--
For now, I have not tried to hide completly the fact that a vserver is a vserver.
The only feature it has is that a vserver can have its own host and domain name.
This was done because many service use this to setup some defaults and I wanted
vservers to be as natural as possible configuration wise.
I could have added a couple featurism such as
private uptime
hide the network device and fake a single one
what else.
We will see as the need arise.
I also added the /sbin/vreboot command which talks with a socket /dev/reboot
established by the rebootmgr service in the root server (one socket per
vserver). This way a vserver can test its ability to properly shutdown and start.
The reboot manager simply does
/usr/sbin/vserver the-vserver restart
this simply chroot to the vserver, selec the same security context, sets
the ipv4root and then exec
/etc/rc.d/rc 6
Then it kills the remaining processes. To start it, it executes
/etc/rc.d/rc 3
In general, you reboot a vserver in 3 seconds :-)
---------------------------------------------------------
Jacques Gelinas <[email protected]>
vserver: run general purpose virtual servers on one box, full speed!
http://www.solucorp.qc.ca/miscprj/s_context.hc
On Mon, 15 Oct 2001 17:22:43 -0500, James Sutherland wrote
> > But there is no need to open a crackme vserver. Install it on your machine,
> > build a vserver.
>
> The question, I think, was would YOU give out root access on vservers on
> YOUR box, and be confident people wouldn't be able to escape? :-)
Yes this is the goal. At the office we have many developpers doing various
projects. They often have to setup special demos and those demos must be
located in our dmz. So until now, we could not give them root password. This
was kind of annoying (It is always annoying to refuse the root password to
a friend). Now we can do it.
But I suspect many ASP will use this. For now to make the box really robust
we need to enhance the schedular (some fairness) and make the user
resource global to a vserver. For now, you can limit a process (ulimit) and the
number of process a user can start, but you have little control over total
limits used by a user or vserver (right ?, maybe 2.4 has something in this area
I have missed)
> > > You might want to announce this on bugtraq. [And give solar designer
> > > root account, he might be more creative ;)].
> >
> > You don't understand the issue. Anyone can create his own vserver. The
> > system call controlling this are very simple. It is not a "try to
> > crack my machine" contest. Anyone can create a vserver and test it.
>
> But can you crack your way OUT of the vserver - how confident are you in
> the isolation provided?
Highly. Bug pending.
The concept is both very simple and sound
security ID to isolate processes (an integer)
chroot to isolate the files
capabilities to lower the ability of root (I had to enhance that
a little with the capability ceiling)
set_ipv4root to tie all processes in a vserver to one IP.
The "bug pending" means that I may have forgotten one place in the kernel.
For example, when I implemented the new_s_context, I patchs the kill system
call in few places, then later I realized the ptrace syscall was an issue. I added
a one line test there. I may have forgotten one check.
The same applies to capabilities. When we moved from suser() to capable() we may
have forgotten something. Since capabilities are not that much used, some place
may be missing. For example /proc/sys was writtable by root and not protected
by any capabilities.
But the changes in the kernel a trivial. This is why I am posting here to
get some review.
---------------------------------------------------------
Jacques Gelinas <[email protected]>
vserver: run general purpose virtual servers on one box, full speed!
http://www.solucorp.qc.ca/miscprj/s_context.hc
On Mon, 15 Oct 2001, David Lang wrote:
> you mention problems with interaction if the main sandbox has a service
> listening on 0.0.0.0, what happens if a vserver does this (does it only
> see it's own IP addresses or does it interfere with other servers?)
It only sees its own IP address(es). If one vserver could interfere with
another like that, this would be rather a big hole in the vserver
isolation :)
James.
Hi Jacques.
>>>> You might want to announce this on bugtraq. [And give solar
>>>> designer root account, he might be more creative ;)].
>>> You don't understand the issue. Anyone can create his own vserver.
>>> The system call controlling this are very simple. It is not a "try
>>> to crack my machine" contest. Anyone can create a vserver and test
>>> it.
>> But can you crack your way OUT of the vserver - how confident are you
>> in the isolation provided?
> Highly. Bug pending.
> The concept is both very simple and sound
...
> set_ipv4root to tie all processes in a vserver to one IP.
How well does this work on an ipv6 only box?
Best wishes from Riley.
Hi!
> > You might want to announce this on bugtraq. [And give solar designer
> > root account, he might be more creative ;)].
>
> You don't understand the issue. Anyone can create his own vserver. The
> system call controlling this are very simple. It is not a "try to crack my machine"
> contest. Anyone can create a vserver and test it.
Yep. I know that. But I'm not going to install my own vserver. (Too
much work ;-). I might crack yours, OTOH ;-).
Pavel
--
STOP THE WAR! Someone killed innocent Americans. That does not give
U.S. right to kill people in Afganistan.