2002-04-02 22:17:00

by Calin A. Culianu

[permalink] [raw]
Subject: Question about 'Hidden' Directories in ext2


Ok, so some hackers broke into one of our boxes and set up an ftp site.
They monopolized over 70gb of hard drive space with warez and porn. We
aren't really that upset about it, since we thought it was kind of funny.
(Of course we don't like the idea that they are using out bandwidth and
disk space, but we can easily remedy that).

Anyway, the weird thing is they created 2 directories, both of which were
strangely hidden. You can cd into them but you can't ls them. I

/usr/lib/ypx and /usr/man/ypx were the two directories that contained both
the ftp software and the ftp root. When you are in /usr/man and you do an
ls, you don't see the ypx directory (same when you are in /usr/lib). The
ls binary we got is right off the redhat cd so it shouldn't still be
compromised by whatever rootkit was installed.

My question is this: can the data structures in ext2fs be somehow hacked
so a directory can't appear in a listing but can be otherwise located for
a stat or a chdir? I should think no.. maybe we still haven't gotten rid
of the rootkit...

-Calin



2002-04-02 22:44:45

by Andreas Dilger

[permalink] [raw]
Subject: Re: Question about 'Hidden' Directories in ext2

On Apr 02, 2002 17:16 -0500, Calin A. Culianu wrote:
> Ok, so some hackers broke into one of our boxes and set up an ftp site.
> They monopolized over 70gb of hard drive space with warez and porn. We
> aren't really that upset about it, since we thought it was kind of funny.
> (Of course we don't like the idea that they are using out bandwidth and
> disk space, but we can easily remedy that).
>
> Anyway, the weird thing is they created 2 directories, both of which were
> strangely hidden. You can cd into them but you can't ls them. I
>
> /usr/lib/ypx and /usr/man/ypx were the two directories that contained both
> the ftp software and the ftp root. When you are in /usr/man and you do an
> ls, you don't see the ypx directory (same when you are in /usr/lib). The
> ls binary we got is right off the redhat cd so it shouldn't still be
> compromised by whatever rootkit was installed.
>
> My question is this: can the data structures in ext2fs be somehow hacked
> so a directory can't appear in a listing but can be otherwise located for
> a stat or a chdir? I should think no.. maybe we still haven't gotten rid
> of the rootkit...

It could be that glibc is hacked also, unless you have a
statically-linked ls command. Try booting directly from the CD.

Yes, it is possible to hack ext2 to not show directories named "ypx" (or
whatever you want). Non-trivial, but doable. I had read somewhere that
"modern" linux rootkits load kernel modules, so that they intercept
kernel syscalls, like "sys_getdents()" or "sys_getdents64()" to not show
that you have been hacked.

Really, once you are compromized like this, you are better off to back
up your data and reinstall your OS (taking care that no suid binaries
exist in user home directories and such, there shouldn't normally be
any).

You should also take care that you download the latest RPM updates, and
have them available to machine before it is connected to the net again.
I have heard of machines being compromized within minutes of being
installed, before they update to secure RPMs, just because the number
of crack attempts is so high.

Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert

2002-04-02 22:51:56

by Erik Ljungström

[permalink] [raw]
Subject: Re: Question about 'Hidden' Directories in ext2

On Tue, 2 Apr 2002 17:16:42 -0500 (EST)
"Calin A. Culianu" <[email protected]> wrote:

>
> Ok, so some hackers broke into one of our boxes and set up an ftp site.
> They monopolized over 70gb of hard drive space with warez and porn. We
> aren't really that upset about it, since we thought it was kind of funny.
> (Of course we don't like the idea that they are using out bandwidth and
> disk space, but we can easily remedy that).
>
> Anyway, the weird thing is they created 2 directories, both of which were
> strangely hidden. You can cd into them but you can't ls them. I
>
> /usr/lib/ypx and /usr/man/ypx were the two directories that contained both
> the ftp software and the ftp root. When you are in /usr/man and you do an
> ls, you don't see the ypx directory (same when you are in /usr/lib). The
> ls binary we got is right off the redhat cd so it shouldn't still be
> compromised by whatever rootkit was installed.
>
> My question is this: can the data structures in ext2fs be somehow hacked
> so a directory can't appear in a listing but can be otherwise located for
> a stat or a chdir? I should think no.. maybe we still haven't gotten rid
> of the rootkit...
>
> -Calin
>
>
> -

This isn't really my area of expertice, but have you also recovered the original crond ? Perhaps that was compromized as well, and a replacement of the binaries are crontabbed? Search your system for copies of ls, netstat, ps, whatever. This is just a thought that hit me when I read this.

I wish you all of luck in recovering your system(s)


--
Best regards, Erik

2002-04-03 02:29:43

by jw schultz

[permalink] [raw]
Subject: Re: Question about 'Hidden' Directories in ext2

On Tue, Apr 02, 2002 at 05:16:42PM -0500, Calin A. Culianu wrote:
>
> Ok, so some hackers broke into one of our boxes and set up an ftp site.
> They monopolized over 70gb of hard drive space with warez and porn. We
> aren't really that upset about it, since we thought it was kind of funny.
> (Of course we don't like the idea that they are using out bandwidth and
> disk space, but we can easily remedy that).
>
> Anyway, the weird thing is they created 2 directories, both of which were
> strangely hidden. You can cd into them but you can't ls them. I
>
> /usr/lib/ypx and /usr/man/ypx were the two directories that contained both
> the ftp software and the ftp root. When you are in /usr/man and you do an
> ls, you don't see the ypx directory (same when you are in /usr/lib). The
> ls binary we got is right off the redhat cd so it shouldn't still be
> compromised by whatever rootkit was installed.
>
> My question is this: can the data structures in ext2fs be somehow hacked
> so a directory can't appear in a listing but can be otherwise located for
> a stat or a chdir? I should think no.. maybe we still haven't gotten rid
> of the rootkit...
>
> -Calin

It might be much simpler. They may be playing with /etc/profile.
Check your shell aliases and the environment variable LS_OPTIONS.
A simple LS_OPTIONS="$LS_OPTIONS -I ypx" or
alias ls='ls --ignore=ypx' would have the effect you are
talking about.

--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: [email protected]

Remember Cernan and Schmitt

2002-04-03 05:30:25

by Frank Schäfer

[permalink] [raw]
Subject: Re: Question about 'Hidden' Directories in ext2

On Wed, 2002-04-03 at 00:16, Calin A. Culianu wrote:
>
> Ok, so some hackers broke into one of our boxes and set up an ftp site.
> They monopolized over 70gb of hard drive space with warez and porn. We
> aren't really that upset about it, since we thought it was kind of funny.
> (Of course we don't like the idea that they are using out bandwidth and
> disk space, but we can easily remedy that).
>
> Anyway, the weird thing is they created 2 directories, both of which were
> strangely hidden. You can cd into them but you can't ls them. I
>
> /usr/lib/ypx and /usr/man/ypx were the two directories that contained both
> the ftp software and the ftp root. When you are in /usr/man and you do an
> ls, you don't see the ypx directory (same when you are in /usr/lib). The
> ls binary we got is right off the redhat cd so it shouldn't still be
> compromised by whatever rootkit was installed.
>
> My question is this: can the data structures in ext2fs be somehow hacked
> so a directory can't appear in a listing but can be otherwise located for
> a stat or a chdir? I should think no.. maybe we still haven't gotten rid
> of the rootkit...

Hi,

Andreas is right. If you're compromized, You should reinstall your
system.
We catched an atacker at work not long ago. They hadn't the time to
remove their cracking tools, so we were able to analyze them.
This led me to the conclusion not to use a standard distro ( amongst
other things ). The script they used analyzed which distro is in use,
and infected the apropriate places for all popular distributions.
Furtheron I wrote a module to bastillize the box, using exactly the
methods crackers are using ( an example is on phrack ). Oh ... yes, and
last but not least we modified some userspace progs and the kernel (
this task took me to this list ), and we have everything read-only ( tmp
and var on a fs mounted noexec,nosuid,nodev ).

Regards
Frank

BTW: Can anyone point me to a site on which I can find some info about
the usage ot the crypto patches?


2002-04-03 10:44:05

by Craig Knox

[permalink] [raw]
Subject: Re: Question about 'Hidden' Directories in ext2

On Tue, 2002-04-02 at 23:16, Calin A. Culianu wrote:
>
> Ok, so some hackers broke into one of our boxes and set up an ftp site.
> They monopolized over 70gb of hard drive space with warez and porn. We
> aren't really that upset about it, since we thought it was kind of funny.
> (Of course we don't like the idea that they are using out bandwidth and
> disk space, but we can easily remedy that).
>
> Anyway, the weird thing is they created 2 directories, both of which were
> strangely hidden. You can cd into them but you can't ls them. I
>
> /usr/lib/ypx and /usr/man/ypx were the two directories that contained both
> the ftp software and the ftp root. When you are in /usr/man and you do an
> ls, you don't see the ypx directory (same when you are in /usr/lib). The
> ls binary we got is right off the redhat cd so it shouldn't still be
> compromised by whatever rootkit was installed.
>
> My question is this: can the data structures in ext2fs be somehow hacked
> so a directory can't appear in a listing but can be otherwise located for
> a stat or a chdir? I should think no.. maybe we still haven't gotten rid
> of the rootkit...

If you are using the binary "ls" of the redhat CD they are probably
using a kernel module to hide this directory.
Have you tried running -> http://www.chkrootkit.org on the box?


2002-04-13 17:01:09

by Pablo Alcaraz

[permalink] [raw]
Subject: Re: Question about 'Hidden' Directories in ext2

You may use tripwire after you clean your system.
Tripwire will check your system for changes in critical files.

Pablo

Craig Knox wrote:

>On Tue, 2002-04-02 at 23:16, Calin A. Culianu wrote:
>
>>Ok, so some hackers broke into one of our boxes and set up an ftp site.
>>They monopolized over 70gb of hard drive space with warez and porn. We
>>aren't really that upset about it, since we thought it was kind of funny.
>>(Of course we don't like the idea that they are using out bandwidth and
>>disk space, but we can easily remedy that).
>>
>>Anyway, the weird thing is they created 2 directories, both of which were
>>strangely hidden. You can cd into them but you can't ls them. I
>>
>>/usr/lib/ypx and /usr/man/ypx were the two directories that contained both
>>the ftp software and the ftp root. When you are in /usr/man and you do an
>>ls, you don't see the ypx directory (same when you are in /usr/lib). The
>>ls binary we got is right off the redhat cd so it shouldn't still be
>>compromised by whatever rootkit was installed.
>>
>>My question is this: can the data structures in ext2fs be somehow hacked
>>so a directory can't appear in a listing but can be otherwise located for
>>a stat or a chdir? I should think no.. maybe we still haven't gotten rid
>>of the rootkit...
>>
>
>If you are using the binary "ls" of the redhat CD they are probably
>using a kernel module to hide this directory.
>Have you tried running -> http://www.chkrootkit.org on the box?
>
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to [email protected]
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
>
>