Hi Linus, all,
During one recent LKML discussion
(http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going
static you called for LSM users to speak up.
We here at Sophos (the fourth largest endpoint security vendor in the world)
have such a module called Talpa which is a part of our main endpoint security
product for Linux that protects from viruses and malware hosted on Linux,
including those targetting Windows or other connected devices,
(http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
which is GPL code and has been in the field for almost three years now. It's
source code has been shipping with the product from the start. We also have
a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
In essence, what our module does is it intercepts file accesses and allows
userspace daemons to vet them. One of the means we implemented that is
through LSM and although it is not a perfect match for such use we prefer to
use an official interface. Unfortunately, with time it became impossible to
use LSM on some distributions (SELinux) so we had to implement other
intercept methods which are significantly less nice, and which may also
become unworkable over time.
The main point here is that making LSM not available for modules makes our
life even more difficult and we would like to ask you to revert that change
until another acceptable, secure solution is found.
At this point I would not like this to turn into a discussion on technical or
code merits of our current approach. It is just that usage scenarios of
(dis-)allowing access to files based on their content, regardless of the
actual details, do sound like a reasonable idea. What we would like to do is
to start discussion about possible solutions in which other interested
parties will hopefully join and in future create some progress on this front.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
Tel: 01235 559933
Web: http://www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
On Wed, Nov 28, 2007 at 12:42:52PM +0000, Tvrtko A. Ursulin wrote:
>
> Hi Linus, all,
>
> During one recent LKML discussion
> (http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going
> static you called for LSM users to speak up.
>
> We here at Sophos (the fourth largest endpoint security vendor in the world)
> have such a module called Talpa which is a part of our main endpoint security
> product for Linux that protects from viruses and malware hosted on Linux,
> including those targetting Windows or other connected devices,
> (http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
> which is GPL code and has been in the field for almost three years now. It's
> source code has been shipping with the product from the start. We also have
> a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
Thanks for showing a wonderful source code example of why lsm shouldn't
be exported :)
--- Christoph Hellwig <[email protected]> wrote:
> On Wed, Nov 28, 2007 at 12:42:52PM +0000, Tvrtko A. Ursulin wrote:
> >
> > Hi Linus, all,
> >
> > During one recent LKML discussion
> > (http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going
> > static you called for LSM users to speak up.
> >
> > We here at Sophos (the fourth largest endpoint security vendor in the
> world)
> > have such a module called Talpa which is a part of our main endpoint
> security
> > product for Linux that protects from viruses and malware hosted on Linux,
> > including those targetting Windows or other connected devices,
> >
>
(http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
>
> > which is GPL code and has been in the field for almost three years now.
> It's
> > source code has been shipping with the product from the start. We also
> have
> > a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
>
> Thanks for showing a wonderful source code example of why lsm shouldn't
> be exported :)
Would you like to expound on that, or do you feel your claws
are sharp enough already?
Casey Schaufler
[email protected]
On Wed, Nov 28, 2007 at 08:38:43AM -0800, Casey Schaufler wrote:
> Would you like to expound on that, or do you feel your claws
> are sharp enough already?
Just take a look at code.
On Wed, 28 Nov 2007 16:46:13 +0000
Christoph Hellwig <[email protected]> wrote:
> On Wed, Nov 28, 2007 at 08:38:43AM -0800, Casey Schaufler wrote:
> > Would you like to expound on that, or do you feel your claws
> > are sharp enough already?
>
> Just take a look at code.
>
The module in question hooks to the syscall table which is not
acceptable. Including dumpster diving through symbol table to
find the syscall table since it is no longer exported.
--
Stephen Hemminger <[email protected]>
On Wed, 28 Nov 2007 16:46:13 GMT, Christoph Hellwig said:
> On Wed, Nov 28, 2007 at 08:38:43AM -0800, Casey Schaufler wrote:
> > Would you like to expound on that, or do you feel your claws
> > are sharp enough already?
>
> Just take a look at code.
Just to clarify - you're OK with the *concept* (a security model that determines
whether you can do an I/O based on the content), it's just their code that's
ugly?
(Note that the concept has interesting implications in the other direction as
well - rather than stopping you from reading a file that has malware, you could
in theory write an anti-export package that would let you write onto external
memory or outbound e-mail, but prevent the write if it was corporate-sensitive
data, or whatever. Yes, I *know* a smart attacker can bypass it by simply
crypting/compressing it first - but the vast majority of attackers aren't
smart, and will just use 'cp' or the GUI equivalent to move the secret design
documents onto the USB key... )
[email protected] wrote on 28/11/2007 17:39:56:
> On Wed, 28 Nov 2007 16:46:13 +0000
> Christoph Hellwig <[email protected]> wrote:
>
> > On Wed, Nov 28, 2007 at 08:38:43AM -0800, Casey Schaufler wrote:
> > > Would you like to expound on that, or do you feel your claws
> > > are sharp enough already?
> >
> > Just take a look at code.
> >
>
> The module in question hooks to the syscall table which is not
> acceptable. Including dumpster diving through symbol table to
> find the syscall table since it is no longer exported.
That is not completely true.
As I said in the second part of my original email (that part Christoph
deleted so please read it whole) I would prefer not to discuss all the
ugly things some parts of our code do because they are irrelevant to the
LSM discussion. But I still feel some extra explanation might be needed.
Talpa is modular itself being composed of a set of kernel modules of which
not all are loaded simultaneously. Where possible LSM can be used and _no_
messing with syscall table will take place. Unfortunately where another
LSM user is present that won't work and another solution had to be found.
Also that drags history from 2.4 where that was the only solution.
So as there is no question the current code does some ugly things it is
even more true that we would be even more happy to use an official API.
LSM was that and we were happily using it which we won't be able to do if
it abruptly goes away. Yes it is not a perfect match but until it is
modified to be better, or until something appropriate is designed and
implemented, it would be very nice if it could stay.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
Tel: 01235 559933
Web: http://www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
On Wed, Nov 28, 2007 at 01:15:05PM -0500, [email protected] wrote:
> (Note that the concept has interesting implications in the other direction as
> well - rather than stopping you from reading a file that has malware, you could
> in theory write an anti-export package that would let you write onto external
> memory or outbound e-mail, but prevent the write if it was corporate-sensitive
> data, or whatever.
You _can_ _not_ do that. If shared mapping gets dirtied, you have no way to
intercept that. At all. Especially since the page stays mapped while it is
written out, so the next modification can come when hardware had already
started outbound DMA and there's no way to abort it, no matter what your
external scanner would do.
Folks, really, that doesn't work. At all. You can intercept all system
calls you want and it will not be enough to prevent the "bad" contents
from hitting the disk.
And if we are talking about the situation when files are written to in
controlled way (i.e. we are not concerned with malware running on the box
in question and just want to stop it from passing through mailsewer, etc.),
then there's no damn need to play with LSM - just have e.g. coda with its
commit-on-close and run the scanner on commit. End of story. Mind you,
in such setups one would be much better off just having the mail server run
the tests explicitly in the userland, along with the rest of anti-spam, etc.
filters.
"Tvrtko A. Ursulin" <[email protected]> writes:
> We here at Sophos (the fourth largest endpoint security vendor in the world)
> have such a module called Talpa which is a part of our main endpoint security
> product
What is a "endpoint security product" exactly? A gateway that scans
files passing through it?
> In essence, what our module does is it intercepts file accesses and allows
> userspace daemons to vet them.
I suspect the only good way forward would be to collect the
requirements of your user space and then define a proper official
interface to do what they want to do. Ideally would be a shared proposal
from multiple groups who are doing this, e.g. if you could talk to
some others doing similar things that would be best.
Personally I admit I never quite saw the point of intercepting all
file accesses for everything. That will just always be slow as often
demonstrated on other operating systems and racey and unreliable too.
And at least the internal daemons should be already reasonably well
protected by standard (or beyond standard, like AA/SELinux etc.)
security measures, so e.g. it does not really make sense to intercept
all of your /etc file accesses and similar.
It might be better to identify the services (gateway, samba, file
server whatever) that are actually dealing with possible infected
"external" files and then define some generic interface that would
allow you to check those as the data appears.
I would expect such an approach to perform better in the end and be more
reliable too.
Note such a interface might well be user space only.
-Andi
> So as there is no question the current code does some ugly things it is
> even more true that we would be even more happy to use an official API.
> LSM was that and we were happily using it which we won't be able to do if
> it abruptly goes away. Yes it is not a perfect match but until it is
> modified to be better, or until something appropriate is designed and
> implemented, it would be very nice if it could stay.
So for an SELinux based system what you are saying is you want to be able
to stack your module with the SELinux module and after SELinux has
considered policy rules still be able to veto them on the grounds that
you are say about to serve a virus to a windows box ?
Alan
> It might be better to identify the services (gateway, samba, file
> server whatever) that are actually dealing with possible infected
> "external" files and then define some generic interface that would
> allow you to check those as the data appears.
I am wondering if the right interface is actually more related to the
existing audit interfaces ?
On Wed, 28 Nov 2007 19:52:46 GMT, Alan Cox said:
> > It might be better to identify the services (gateway, samba, file
> > server whatever) that are actually dealing with possible infected
> > "external" files and then define some generic interface that would
> > allow you to check those as the data appears.
>
> I am wondering if the right interface is actually more related to the
> existing audit interfaces ?
The problem there is that the audit interface just *records* - it doesn't
have the ability to say "No, I don't *think* so.." that the LSM interface has.
On Wed, 28 Nov 2007, [email protected] wrote:
> So as there is no question the current code does some ugly things it is
> even more true that we would be even more happy to use an official API.
How about becoming involved in creating that official API ?
"A person will stand on the top of a hill for a very long time with their
mouth open before a roast duck will fly in"
> LSM was that and we were happily using it which we won't be able to do if
> it abruptly goes away. Yes it is not a perfect match but until it is
> modified to be better, or until something appropriate is designed and
> implemented, it would be very nice if it could stay.
So, what you're asking for is for us to maintain infrastructure in the
kernel, for your out of tree code, which we could not have even known
about until it was dumped on sourceforge a couple of days ago ?
And you're expecting us to modify it to be "better", for you, but without
us possibly knowing what you want ?
How is this sustainable? Every time we make a change in the kernel code,
do we have to wait for months while all of the (unknown) out of tree users
discover their code is broken and then complain to Linus ?
Do you really thing we should be supporting developers who make zero
effort to engage with the kernel community, and instead abuse kernel APIs
and ship out of tree code to customers?
You are essentially demanding that we provide you with a stable kernel
API. I suggest you review Greg KH's excellent document on the issue:
<http://os.cqu.edu.au/docs/kernel-doc/Documentation/stable_api_nonsense.txt>
Also, your users are likely unaware of the risks associated with these
techniques, such as, that they are not actually running "linux" any more,
and that their kernel is made unsupportable by both the community and
their OS vendors when code like this is being used:
/*
* hidden vfsmnt_lock handling
*/
#ifdef TALPA_USE_VFSMOUNT_LOCK
static inline void talpa_vfsmount_lock(void)
{
spinlock_t* talpa_vfsmount_lock_addr = (spinlock_t *)TALPA_VFSMOUNT_LOCK_ADDR;
spin_lock(talpa_vfsmount_lock_addr);
}
Code quality and correctness issues are definitely relevant, and in fact
are directly linked to the fact that out of tree code receives no
community peer review, and is not able to be considered in the general
kernel design & development process.
Hacks like this effectively void support from the community, which is
another way of saying that the user's kernel becomes proprietary once the
module containing it is loaded.
Actually, it's worse than that, as not only is the kernel no longer "open
source", but also lacks even the level of support provided by proprietary
OS vendors (who provide stable interfaces for such purposes).
If you want to resolve this properly, you'll need to do more than complain
to Linus when the upstream APIs (which you are abusing) change.
You'll need to become productively engaged in designing, developing and
maintaining appropriate APIs, which suit not only your specific needs, but
likely those of others, and submit your code for review and upstream
inclusion.
What I'd suggest, is that you create a public mailing list, get the other
AV projects involved, and then work on a set of requirements so that the
general problem can be understood. Then, propose a solution to the
problem and have it reviewed by core kernel folk (cc it to lkml), so that
it can be evaluated in terms of e.g. VFS correctness, maintainability etc.
That would be at least a useful first step in taking this issue seriously.
Thanks,
- James
--
James Morris
<[email protected]>
On Wed, Nov 28, 2007 at 06:30:40PM +0000, Al Viro wrote:
> On Wed, Nov 28, 2007 at 01:15:05PM -0500, [email protected] wrote:
> > (Note that the concept has interesting implications in the other direction as
> > well - rather than stopping you from reading a file that has malware, you could
> > in theory write an anti-export package that would let you write onto external
> > memory or outbound e-mail, but prevent the write if it was corporate-sensitive
> > data, or whatever.
>
> You _can_ _not_ do that. If shared mapping gets dirtied, you have no way to
> intercept that. At all. Especially since the page stays mapped while it is
> written out, so the next modification can come when hardware had already
> started outbound DMA and there's no way to abort it, no matter what your
> external scanner would do.
>
> Folks, really, that doesn't work. At all. You can intercept all system
> calls you want and it will not be enough to prevent the "bad" contents
> from hitting the disk.
>
> And if we are talking about the situation when files are written to in
> controlled way (i.e. we are not concerned with malware running on the box
> in question and just want to stop it from passing through mailsewer, etc.),
> then there's no damn need to play with LSM - just have e.g. coda with its
> commit-on-close and run the scanner on commit. End of story. Mind you,
> in such setups one would be much better off just having the mail server run
> the tests explicitly in the userland, along with the rest of anti-spam, etc.
> filters.
I've repeated the above statements so many times to a number of the
anti-virus companies, and other people that really should know better,
that I'm really sick of it. For some reason, they keep trying to do
things like this in the kernel, despite it being trivial to do in
userspace properly.
In the end, I even got one company to agree that it should be done in
userspace (McAfee), but they ignored this and went off to update their
kernel code again :(
Just because other operating systems require you to do things like this
within the kernel, doesn't mean that you have to do the same thing on
Linux...
so sad,
greg k-h
On Nov 28 2007 18:22, [email protected] wrote:
>
>Talpa is modular itself being composed of a set of kernel modules of which
>not all are loaded simultaneously. Where possible LSM can be used and _no_
>messing with syscall table will take place. Unfortunately where another
>LSM user is present that won't work
SELinux supports chaining, so if talpa is loaded as a secondary to selinux,
where is the problem? For those LSMs which do not support chaining (*cough*
apparmor *cough* be one, mtadm another), fix them.
On Nov 28 2007 16:38, Greg KH wrote:
>>
>> And if we are talking about the situation when files are written to
>> in controlled way (i.e. we are not concerned with malware running on
>> the box in question and just want to stop it from passing through
>> mailsewer, etc.), then there's no damn need to play with LSM - just
>> have e.g. coda with its commit-on-close and run the scanner on
>> commit. End of story. Mind you, in such setups one would be much
>> better off just having the mail server run the tests explicitly in
>> the userland, along with the rest of anti-spam, etc. filters.
>
>I've repeated the above statements so many times to a number of the
>anti-virus companies, and other people that really should know better,
>that I'm really sick of it. For some reason, they keep trying to do
>things like this in the kernel, despite it being trivial to do in
>userspace properly.
>
Do you mean something along the lines of FUSE?
On Wed, Nov 28, 2007 at 12:42:52PM +0000, Tvrtko A. Ursulin wrote:
>
> Hi Linus, all,
>
> During one recent LKML discussion
> (http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going
> static you called for LSM users to speak up.
>
> We here at Sophos (the fourth largest endpoint security vendor in the world)
> have such a module called Talpa which is a part of our main endpoint security
> product for Linux that protects from viruses and malware hosted on Linux,
> including those targetting Windows or other connected devices,
> (http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
> which is GPL code and has been in the field for almost three years now. It's
> source code has been shipping with the product from the start. We also have
> a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
>
> In essence, what our module does is it intercepts file accesses and allows
> userspace daemons to vet them. One of the means we implemented that is
> through LSM and although it is not a perfect match for such use we prefer to
> use an official interface. Unfortunately, with time it became impossible to
> use LSM on some distributions (SELinux) so we had to implement other
> intercept methods which are significantly less nice, and which may also
> become unworkable over time.
Do you have a patch that shows the type of interface you would like to
see? Like James stated, if you do not participate in the development
process, we have no way of knowing what you even want from the kernel.
What has kept you from submitting your code for inclusion in the main
kernel source tree?
Right now, your customers void their support warranties if they run your
software, as it can not be supported by the distros as an out-of-tree
kernel module. I'm sure your customers would like to not have this
problem.
thanks,
greg k-h
On Thu, Nov 29, 2007 at 01:53:46AM +0100, Jan Engelhardt wrote:
>
> On Nov 28 2007 16:38, Greg KH wrote:
> >>
> >> And if we are talking about the situation when files are written to
> >> in controlled way (i.e. we are not concerned with malware running on
> >> the box in question and just want to stop it from passing through
> >> mailsewer, etc.), then there's no damn need to play with LSM - just
> >> have e.g. coda with its commit-on-close and run the scanner on
> >> commit. End of story. Mind you, in such setups one would be much
> >> better off just having the mail server run the tests explicitly in
> >> the userland, along with the rest of anti-spam, etc. filters.
> >
> >I've repeated the above statements so many times to a number of the
> >anti-virus companies, and other people that really should know better,
> >that I'm really sick of it. For some reason, they keep trying to do
> >things like this in the kernel, despite it being trivial to do in
> >userspace properly.
> >
> Do you mean something along the lines of FUSE?
That is one way, but not the simplest or nicest (people don't want to
run their whole fs on FUSE just yet).
The easiest way is as Al described above, just have the userspace
program that wrote the file to disk, check it then.
There are some nice SAMBA plugins that do just that already out there...
thanks,
greg k-h
--- Jan Engelhardt <[email protected]> wrote:
>
> On Nov 28 2007 18:22, [email protected] wrote:
> >
> >Talpa is modular itself being composed of a set of kernel modules of which
> >not all are loaded simultaneously. Where possible LSM can be used and _no_
> >messing with syscall table will take place. Unfortunately where another
> >LSM user is present that won't work
>
> SELinux supports chaining, so if talpa is loaded as a secondary to selinux,
> where is the problem? For those LSMs which do not support chaining (*cough*
> apparmor *cough* be one, mtadm another), fix them.
Um, cough cough (I ready do have a nasty cold) SELinux supports
a very limited bit of chaining. I don't think you're going to be
chaining security_secid_to_secctx() or security_secctx_to_secid()
with the current SELinux code, but you could prove me wrong there.
Chaining is a red herring. If you want talpa it seems that you
have a use case that isn't going to require the presence of
another LSM. You may have other issues, but at this point I say
throw caution to the wind, clean it up based on the suggestions
you've seen here, and put the patch up as an RFC on the LSM list.
What's the worst that could happen?
Casey Schaufler
[email protected]
Alan Cox <[email protected]> wrote on 28/11/2007 19:50:42:
> > So as there is no question the current code does some ugly things it
is
> > even more true that we would be even more happy to use an official
API.
> > LSM was that and we were happily using it which we won't be able to do
if
> > it abruptly goes away. Yes it is not a perfect match but until it is
> > modified to be better, or until something appropriate is designed and
> > implemented, it would be very nice if it could stay.
>
> So for an SELinux based system what you are saying is you want to be
able
> to stack your module with the SELinux module and after SELinux has
> considered policy rules still be able to veto them on the grounds that
> you are say about to serve a virus to a windows box ?
Basically yes but the effective scenario is a bit wider. Local actions
like disallowing execution of rootkits, exploits and other similar malware
are also interesting. Another example would be enforcing a corporate
policy on which IM clients shouldn't be used so it is not just fileserver
scenario in which Linux machines can be compromised.
But really I am not the best person to know all current attack vectors.
Overall set of requirements and ideas is something we are working on with
other vendors and hopefully with the community. This is one of the two
main things my original post was about.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the
author.
The contents has not been reviewed or approved by Sophos."
Tel: 01235 559933
Web: http://www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Al Viro <[email protected]> wrote on 28/11/2007 18:30:40:
> On Wed, Nov 28, 2007 at 01:15:05PM -0500, [email protected] wrote:
> > (Note that the concept has interesting implications in the other
> direction as
> > well - rather than stopping you from reading a file that has
> malware, you could
> > in theory write an anti-export package that would let you write
> onto external
> > memory or outbound e-mail, but prevent the write if it was
> corporate-sensitive
> > data, or whatever.
>
> You _can_ _not_ do that. If shared mapping gets dirtied, you have no
way to
> intercept that. At all. Especially since the page stays mapped while
it is
> written out, so the next modification can come when hardware had already
> started outbound DMA and there's no way to abort it, no matter what your
> external scanner would do.
>
> Folks, really, that doesn't work. At all. You can intercept all system
> calls you want and it will not be enough to prevent the "bad" contents
> from hitting the disk.
On this level we are not talking about stopping bad stuff from hitting the
disk. There is a lot simpler scenario how that can happen and that is via
a dropper at which point it can be detected and reported when the file is
closed. But more importantly further access to it can be blocked until
appropriate actions are taken which also applies with your example, no? Is
it possible to open for execute and have dirty mappings (or open for
write) on a file at the same time?
> And if we are talking about the situation when files are written to in
> controlled way (i.e. we are not concerned with malware running on the
box
> in question and just want to stop it from passing through mailsewer,
etc.),
> then there's no damn need to play with LSM - just have e.g. coda with
its
> commit-on-close and run the scanner on commit. End of story. Mind you,
> in such setups one would be much better off just having the mail server
run
> the tests explicitly in the userland, along with the rest of anti-spam,
etc.
> filters.
That is true for that scenario.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the
author.
The contents has not been reviewed or approved by Sophos."
Tel: 01235 559933
Web: http://www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
On Thu, 2007-11-29 at 11:12 +1100, James Morris wrote:
> On Wed, 28 Nov 2007, [email protected] wrote:
>
> > So as there is no question the current code does some ugly things it is
> > even more true that we would be even more happy to use an official API.
>
> How about becoming involved in creating that official API ?
Sophos are interested in doing so, and we have spoken about this several
times recently over the phone. This is why they sent the email in
question yesterday, to kickstart debate. And that's awesome. I am trying
to bring a few of these folks together at the moment, so that we can get
a solution that is acceptable to upstream at some point in the future.
So, rather than criticise their current code, or their intentions, or
blanketly dismiss the virus protection market, perhaps we can focus
instead on the fact that there is a known third party who wishes to
perform a task that is not well supportable at this moment. We can all
agree the syscall table hacking isn't such a good idea - but these guys
are *very* open to listening to useful alternative suggestions.
They (virus protection folks) generally think they want to intercept
various system calls, such as open() and block until they have performed
a scan operation on the file. I explained the mmap issue to several of
these companies recently, in quite some detail, and I know they are
interested in listening this time around :-) At the end of the day, what
I have been lead to believe is that they don't care whether they
intercept syscall entries, or use a better method, they just want to
scan files and take some action if a file is "bad". That's it really.
I have been trying to put together an exact feature set that is needed
from these different vendors, so we can discuss it further here, and
hopefully actually get somewhere, too. There have been a few delays
after I pointed out the mmap issues at some length.
Jon.
On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
> The easiest way is as Al described above, just have the userspace
> program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless
myriad of different applications. And if one of them doesn't support
on-access scanning, then the whole solution isn't worth using.
> There are some nice SAMBA plugins that do just that already out there...
That's really not the problem :-)
Jon.
[email protected] wrote on 28/11/2007 19:20:26:
> "Tvrtko A. Ursulin" <[email protected]> writes:
>
> > We here at Sophos (the fourth largest endpoint security vendor in
> the world)
> > have such a module called Talpa which is a part of our main
> endpoint security
> > product
>
> What is a "endpoint security product" exactly? A gateway that scans
> files passing through it?
No, it would be more like a desktop/workstation thingy which protects
against different kinds of malware and can also enforce policies, all
centraly managed. I am afraid to use to more marketing words not to upset
the LKML filters because originaly my posdt was blocked on.. hah! I almost
wrote it now! :) It was "security vendor" but with another term for
vendor.
> > In essence, what our module does is it intercepts file accesses and
allows
> > userspace daemons to vet them.
>
> I suspect the only good way forward would be to collect the
> requirements of your user space and then define a proper official
> interface to do what they want to do. Ideally would be a shared proposal
> from multiple groups who are doing this, e.g. if you could talk to
> some others doing similar things that would be best.
I completely agree and effort to do that is under way. I am not the one
coordinating that but hopefully those who are will post something soon.
> Personally I admit I never quite saw the point of intercepting all
> file accesses for everything. That will just always be slow as often
> demonstrated on other operating systems and racey and unreliable too.
> And at least the internal daemons should be already reasonably well
> protected by standard (or beyond standard, like AA/SELinux etc.)
> security measures, so e.g. it does not really make sense to intercept
> all of your /etc file accesses and similar.
Personally I don't know. I am often surprised when I hear what techniques
the bad guys use and what is possible.
Solution that we currently have is not so slow (btw you are free to try it
out). And by working on a proper interface race and reliability issues
should be solvable.
> It might be better to identify the services (gateway, samba, file
> server whatever) that are actually dealing with possible infected
> "external" files and then define some generic interface that would
> allow you to check those as the data appears.
>
> I would expect such an approach to perform better in the end and be more
> reliable too.
>
> Note such a interface might well be user space only.
That is fine for some classes of servers. But for some other and desktop
more could be needed.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the
author.
The contents has not been reviewed or approved by Sophos."
Tel: 01235 559933
Web: http://www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
> On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
>
> > The easiest way is as Al described above, just have the userspace
> > program that wrote the file to disk, check it then.
>
> But the problem is that this isn't just Samba, this is a countless
> myriad of different applications. And if one of them doesn't support
> on-access scanning, then the whole solution isn't worth using.
Ok, which specific applications do they care about? Last time I asked
it was still limited to a very small handful, all of which would be
trivial to add such a hook to.
> > There are some nice SAMBA plugins that do just that already out there...
>
> That's really not the problem :-)
Yes it is. That's all you want to catch, when a Windows machine wants
to access a file on a SAMBA server. Do the check then, in userspace.
Believe me, I've been over and over and over and over this before...
thanks,
greg k-h
On Thu, Nov 29, 2007 at 11:27:45AM -0500, Jon Masters wrote:
> On Thu, 2007-11-29 at 11:12 +1100, James Morris wrote:
> > On Wed, 28 Nov 2007, [email protected] wrote:
> >
> > > So as there is no question the current code does some ugly things it is
> > > even more true that we would be even more happy to use an official API.
> >
> > How about becoming involved in creating that official API ?
>
> Sophos are interested in doing so, and we have spoken about this several
> times recently over the phone.
Great, but this email did not show that they were interested in doing
so.
> So, rather than criticise their current code, or their intentions, or
> blanketly dismiss the virus protection market, perhaps we can focus
> instead on the fact that there is a known third party who wishes to
> perform a task that is not well supportable at this moment. We can all
> agree the syscall table hacking isn't such a good idea - but these guys
> are *very* open to listening to useful alternative suggestions.
It's not only not a good idea, it's insecure and is trivial to
circumvent, so it's a non-issue here.
> They (virus protection folks) generally think they want to intercept
> various system calls, such as open() and block until they have performed
> a scan operation on the file. I explained the mmap issue to several of
> these companies recently, in quite some detail, and I know they are
> interested in listening this time around :-) At the end of the day, what
> I have been lead to believe is that they don't care whether they
> intercept syscall entries, or use a better method, they just want to
> scan files and take some action if a file is "bad". That's it really.
Yes, I too have talked with people doing this kind of work, see my other
messages in this thread for how it can be done all in userspace, no
kernel changes needed at all.
thanks,
greg k-h
On Nov 29 2007 11:27, Jon Masters wrote:
>
>They (virus protection folks) generally think they want to intercept
>various system calls, such as open() and block until they have performed
>a scan operation on the file. I explained the mmap issue [...]
If open and close was everything, then that would be wonderful. You
could only wonder how many false positives scanners could bring up if
they checked every write() for signatures - not to mention
performance bogdown.
>they just want to scan files and take some action if a file is
>"bad". That's it really.
>
struct security->dentry_open sounds like the candidate, together
with relayfs with submits filenames to userspace.
On Thu, 29 Nov 2007 11:27:45 -0500
Jon Masters <[email protected]> wrote:
> On Thu, 2007-11-29 at 11:12 +1100, James Morris wrote:
> > On Wed, 28 Nov 2007, [email protected] wrote:
> >
> > > So as there is no question the current code does some ugly things it is
> > > even more true that we would be even more happy to use an official API.
> >
> > How about becoming involved in creating that official API ?
>
> Sophos are interested in doing so, and we have spoken about this several
> times recently over the phone. This is why they sent the email in
> question yesterday, to kickstart debate. And that's awesome. I am trying
> to bring a few of these folks together at the moment, so that we can get
> a solution that is acceptable to upstream at some point in the future.
>
> So, rather than criticise their current code, or their intentions, or
> blanketly dismiss the virus protection market, perhaps we can focus
> instead on the fact that there is a known third party who wishes to
> perform a task that is not well supportable at this moment. We can all
> agree the syscall table hacking isn't such a good idea - but these guys
> are *very* open to listening to useful alternative suggestions.
>
> They (virus protection folks) generally think they want to intercept
> various system calls, such as open() and block until they have performed
> a scan operation on the file. I explained the mmap issue to several of
> these companies recently, in quite some detail, and I know they are
> interested in listening this time around :-) At the end of the day, what
> I have been lead to believe is that they don't care whether they
> intercept syscall entries, or use a better method, they just want to
> scan files and take some action if a file is "bad". That's it really.
>
> I have been trying to put together an exact feature set that is needed
> from these different vendors, so we can discuss it further here, and
> hopefully actually get somewhere, too. There have been a few delays
> after I pointed out the mmap issues at some length.
>
Perhaps this kind of scanning belongs in the application. Couldn't an
apache or samba have a plugin to do it?
--
Stephen Hemminger <[email protected]>
On Nov 29 2007 08:47, Greg KH wrote:
>On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
>> On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
>>
>> > The easiest way is as Al described above, just have the userspace
>> > program that wrote the file to disk, check it then.
>>
>> But the problem is that this isn't just Samba, this is a countless
>> myriad of different applications. And if one of them doesn't support
>> on-access scanning, then the whole solution isn't worth using.
>
>Ok, which specific applications do they care about? Last time I asked
>it was still limited to a very small handful, all of which would be
>trivial to add such a hook to.
>
Well, think bash, syscalls. While you can add a plugin to samba "easily",
it seems overkill to do the same for rm, mv, cp, bash.
On Thu, Nov 29, 2007 at 05:53:33PM +0100, Jan Engelhardt wrote:
> >> But the problem is that this isn't just Samba, this is a countless
> >> myriad of different applications. And if one of them doesn't support
> >> on-access scanning, then the whole solution isn't worth using.
> >
> >Ok, which specific applications do they care about? Last time I asked
> >it was still limited to a very small handful, all of which would be
> >trivial to add such a hook to.
> >
> Well, think bash, syscalls. While you can add a plugin to samba "easily",
> it seems overkill to do the same for rm, mv, cp, bash.
Can we please stop this useless discussion? Trying to check the content
of files to see whether they might be malicious is inherently braindead,
and no amounts of plugins in random places will fix this.
On Thu, Nov 29, 2007 at 05:53:33PM +0100, Jan Engelhardt wrote:
>
> On Nov 29 2007 08:47, Greg KH wrote:
> >On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
> >> On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
> >>
> >> > The easiest way is as Al described above, just have the userspace
> >> > program that wrote the file to disk, check it then.
> >>
> >> But the problem is that this isn't just Samba, this is a countless
> >> myriad of different applications. And if one of them doesn't support
> >> on-access scanning, then the whole solution isn't worth using.
> >
> >Ok, which specific applications do they care about? Last time I asked
> >it was still limited to a very small handful, all of which would be
> >trivial to add such a hook to.
> >
> Well, think bash, syscalls. While you can add a plugin to samba "easily",
> it seems overkill to do the same for rm, mv, cp, bash.
Again, these are not things that these companies care about.
Please, let's stop arguing about this, until someone actually produces
some code they want to see help resolve this "issue".
thanks,
greg k-h
On Thu, 2007-11-29 at 08:47 -0800, Greg KH wrote:
> On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
> > On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
> >
> > > The easiest way is as Al described above, just have the userspace
> > > program that wrote the file to disk, check it then.
> >
> > But the problem is that this isn't just Samba, this is a countless
> > myriad of different applications. And if one of them doesn't support
> > on-access scanning, then the whole solution isn't worth using.
>
> Ok, which specific applications do they care about? Last time I asked
> it was still limited to a very small handful, all of which would be
> trivial to add such a hook to.
Like I said, I'm trying to put together a set of "feature requirements"
that we can publish to LKML and get feedback. I am lead to believe that
they basically want to trap every file operation, of every potential
userspace program, so I don't think it's a handful at this point.
> > > There are some nice SAMBA plugins that do just that already out there...
> >
> > That's really not the problem :-)
>
> Yes it is. That's all you want to catch, when a Windows machine wants
> to access a file on a SAMBA server. Do the check then, in userspace.
Oh, I meant that if it was just Samba we'd all be home and dry by now.
> Believe me, I've been over and over and over and over this before...
Yeah. I know, I'm trying not to flog a dead horse here, but I think
there's real genuine interest here this time around. So, maybe I'm just
trying because everyone else already did, but it's worth a shot :-)
Jon.
On Thu, Nov 29, 2007 at 12:05:36PM -0500, Jon Masters wrote:
>
> On Thu, 2007-11-29 at 08:47 -0800, Greg KH wrote:
> > On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
> > > On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
> > >
> > > > The easiest way is as Al described above, just have the userspace
> > > > program that wrote the file to disk, check it then.
> > >
> > > But the problem is that this isn't just Samba, this is a countless
> > > myriad of different applications. And if one of them doesn't support
> > > on-access scanning, then the whole solution isn't worth using.
> >
> > Ok, which specific applications do they care about? Last time I asked
> > it was still limited to a very small handful, all of which would be
> > trivial to add such a hook to.
>
> Like I said, I'm trying to put together a set of "feature requirements"
> that we can publish to LKML and get feedback.
Heh, ok, I did this a while ago, but I'd be interested in seeing this
done in public, on lkml this time around so that we have a thread to
point people at in the future when it comes up again :)
good luck,
greg k-h
On Nov 29, 2007 9:03 AM, Greg KH <[email protected]> wrote:
> On Thu, Nov 29, 2007 at 05:53:33PM +0100, Jan Engelhardt wrote:
> >
> > On Nov 29 2007 08:47, Greg KH wrote:
> > >On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
> > >> On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
> > >>
> > >> > The easiest way is as Al described above, just have the userspace
> > >> > program that wrote the file to disk, check it then.
> > >>
> > >> But the problem is that this isn't just Samba, this is a countless
> > >> myriad of different applications. And if one of them doesn't support
> > >> on-access scanning, then the whole solution isn't worth using.
> > >
> > >Ok, which specific applications do they care about? Last time I asked
> > >it was still limited to a very small handful, all of which would be
> > >trivial to add such a hook to.
> > >
> > Well, think bash, syscalls. While you can add a plugin to samba "easily",
> > it seems overkill to do the same for rm, mv, cp, bash.
>
> Again, these are not things that these companies care about.
Perhaps if you looked at this outside of a file-server scenario, the
problem would be clearer? Anti-malware companies want to check
anything written to disk on a system, either at write time or blocking
the open/mmap. That means proactively protecting email programs with
known vulnerabilities that have yet to be patched, web browsers
writing and reading their caches, an Apache instance running WebDAV,
the list goes on. And these are on desktop systems, with no attached
file/network server.
Yes, each and every one of these programs could have a malware
scanning engine slapped inside of them. But that proves what? That's
like saying each an every program on a system should have the SELinux
policies built into them, and yet we have that in-kernel instead.
> Can we please stop this useless discussion? Trying to check the content
> of files to see whether they might be malicious is inherently braindead,
> and no amounts of plugins in random places will fix this.
Actually it is quite effective especially for files whose content is
expected not to be executable material - there are some very distinct
mathematical signatures.
For the purposes of figuring out what is needed you can consider a random
simple user case such as a system which protects you against the works of
Eric S Raymond. Replace the mathematical analysis and heuristics with a
user space tool which spots the various ESR papers and design it for that
if it makes you happier.
SELinux seems to be able to do most of the lifting around the problem as
it can relabel a file into eric_t and constrain further access to it.
The big problem to me is that this essentially boils down to revoke() on
ordinary files, which as we know is _hard_.
The simple case is
open
write cathedral and bazaar in some order
close
<trap close -> process -> label eric_t>
open (eric_t) - SELinux "no"
Anyone smart will then write it out of order and keep the file open, or
make sure someone else has it open. At that point close ceases to be a
point you can check the labelling. Open doesn't seem too helpful either
as the target may have the file open before it is modified into an ESR
work.
Even if you can spot the surrepticious assembly of an ESR work on your
computer you then have to implement revocation in order to take the file
away from those with it open.
Alan
> closed. But more importantly further access to it can be blocked until
> appropriate actions are taken which also applies with your example, no? Is
That bit is hard- very hard.
> it possible to open for execute and have dirty mappings (or open for
> write) on a file at the same time?
If I want I can have 16 threads executing code in a shared object being
written to by ten other programs at once and shared over a network while
we are at it. Its probably not a good idea but I can do it if I have
reason to.
Unix and Linux likewise take the basic view that locking is a problem for
the user, while file permissions prevent it being made a problem for
other users.
On Thu, Nov 29, 2007 at 09:35:56AM -0800, Ray Lee wrote:
> On Nov 29, 2007 9:03 AM, Greg KH <[email protected]> wrote:
> > On Thu, Nov 29, 2007 at 05:53:33PM +0100, Jan Engelhardt wrote:
> > >
> > > On Nov 29 2007 08:47, Greg KH wrote:
> > > >On Thu, Nov 29, 2007 at 11:36:12AM -0500, Jon Masters wrote:
> > > >> On Wed, 2007-11-28 at 17:07 -0800, Greg KH wrote:
> > > >>
> > > >> > The easiest way is as Al described above, just have the userspace
> > > >> > program that wrote the file to disk, check it then.
> > > >>
> > > >> But the problem is that this isn't just Samba, this is a countless
> > > >> myriad of different applications. And if one of them doesn't support
> > > >> on-access scanning, then the whole solution isn't worth using.
> > > >
> > > >Ok, which specific applications do they care about? Last time I asked
> > > >it was still limited to a very small handful, all of which would be
> > > >trivial to add such a hook to.
> > > >
> > > Well, think bash, syscalls. While you can add a plugin to samba "easily",
> > > it seems overkill to do the same for rm, mv, cp, bash.
> >
> > Again, these are not things that these companies care about.
>
> Perhaps if you looked at this outside of a file-server scenario, the
> problem would be clearer? Anti-malware companies want to check
> anything written to disk on a system, either at write time or blocking
> the open/mmap. That means proactively protecting email programs with
> known vulnerabilities that have yet to be patched, web browsers
> writing and reading their caches, an Apache instance running WebDAV,
> the list goes on. And these are on desktop systems, with no attached
> file/network server.
Ok, if they want to check on every open/mmap then just hook in glibc to
do this. Especially as they want to run userspace code at this point in
time.
Again, let's wait for some code to show up before discussing this
further.
thanks,
greg k-h
On Thu, Nov 29, 2007 at 09:35:56AM -0800, Ray Lee wrote:
> Perhaps if you looked at this outside of a file-server scenario, the
> problem would be clearer? Anti-malware companies want to check
> anything written to disk on a system, either at write time or blocking
> the open/mmap. That means proactively protecting email programs with
> known vulnerabilities that have yet to be patched, web browsers
> writing and reading their caches, an Apache instance running WebDAV,
> the list goes on. And these are on desktop systems, with no attached
> file/network server.
... and anything promising that is a grade-A snake oil.
On Nov 29, 2007 9:45 AM, Greg KH <[email protected]> wrote:
> > Perhaps if you looked at this outside of a file-server scenario, the
> > problem would be clearer? Anti-malware companies want to check
> > anything written to disk on a system, either at write time or blocking
> > the open/mmap. That means proactively protecting email programs with
> > known vulnerabilities that have yet to be patched, web browsers
> > writing and reading their caches, an Apache instance running WebDAV,
> > the list goes on. And these are on desktop systems, with no attached
> > file/network server.
>
> Ok, if they want to check on every open/mmap then just hook in glibc to
> do this. Especially as they want to run userspace code at this point in
> time.
Doesn't help statically linked binaries, or anything else that bypases glibc.
But yes, I'll let them argue their point from here.
On Thu, 2007-11-29 at 11:19 -0700, Justin Banks wrote:
> Ray Lee wrote
> > On Nov 29, 2007 9:45 AM, Greg KH <[email protected]> wrote:
> > > > Perhaps if you looked at this outside of a file-server scenario, the
> > > > problem would be clearer? Anti-malware companies want to check
> > > > anything written to disk on a system, either at write time or blocking
> > > > the open/mmap. That means proactively protecting email programs with
> > > > known vulnerabilities that have yet to be patched, web browsers
> > > > writing and reading their caches, an Apache instance running WebDAV,
> > > > the list goes on. And these are on desktop systems, with no attached
> > > > file/network server.
> > >
> > > Ok, if they want to check on every open/mmap then just hook in glibc to
> > > do this. Especially as they want to run userspace code at this point in
> > > time.
> >
> > Doesn't help statically linked binaries, or anything else that bypases glibc.
>
> Or NFS servers for that matter, either.
Right. I threw all of these examples out there and was given the
impression that a one-size fits all "solution" is needed. And to be
honest, I can understand that desire.
I don't want to fall into any "snake oil" traps, because I know the
reality is that there are holes in any solution. The only way
to /really/ be sure of file integrity is to mark every user-visible
pagecache page read only, trap on every single write, and re-scan the
entirety of the file at that point. And even more silliness.
A compromise is therefore called for. I will go shut up and go talk to
these guys and see if I can hook up a public discussion around a set of
requirements. Preferably some code too, but at least a requirements doc.
Thanks,
Jon.
On Nov 29, 2007 9:36 AM, Alan Cox <[email protected]> wrote:
> > closed. But more importantly further access to it can be blocked until
> > appropriate actions are taken which also applies with your example, no? Is
>
> That bit is hard- very hard.
In some sense it seems like the same problem faced by dynamic
translators such as Qemu. They really want to vet a dirtied or faulted
page before allowing the app to run unhindered. It's be nice to have
some way to do that without virtualizing the whole of userspace.
Ray Lee wrote
> On Nov 29, 2007 9:45 AM, Greg KH <[email protected]> wrote:
> > > Perhaps if you looked at this outside of a file-server scenario, the
> > > problem would be clearer? Anti-malware companies want to check
> > > anything written to disk on a system, either at write time or blocking
> > > the open/mmap. That means proactively protecting email programs with
> > > known vulnerabilities that have yet to be patched, web browsers
> > > writing and reading their caches, an Apache instance running WebDAV,
> > > the list goes on. And these are on desktop systems, with no attached
> > > file/network server.
> >
> > Ok, if they want to check on every open/mmap then just hook in glibc to
> > do this. Especially as they want to run userspace code at this point in
> > time.
>
> Doesn't help statically linked binaries, or anything else that bypases glibc.
Or NFS servers for that matter, either.
-justinb
--
Justin Banks
BakBone Software
[email protected]
On Thu, 2007-11-29 at 10:40 -0800, Ray Lee wrote:
> On Nov 29, 2007 9:36 AM, Alan Cox <[email protected]> wrote:
> > > closed. But more importantly further access to it can be blocked until
> > > appropriate actions are taken which also applies with your example, no? Is
> >
> > That bit is hard- very hard.
>
> In some sense it seems like the same problem faced by dynamic
> translators such as Qemu. They really want to vet a dirtied or faulted
> page before allowing the app to run unhindered. It's be nice to have
> some way to do that without virtualizing the whole of userspace.
Like I hinted at, you can't just "vet a page". Because a page alone is
meaningless garbage, unless it happens to be an extremely small program,
with headers, all nicely aligned. Most likely you don't know if a random
page of data is code from a COFF file, ELF file, or some random crap I
typed in at a terminal after having too much coffee.
So. You'd need to scan *all the pages* of *the entire file*, every time
that you performed any type of operation. Hence, that idea is completely
out right away. The on-access scanning isn't perfect, but it's probably
about as good as you can get and still have a reasonably usable system.
Jon.
On Nov 29, 2007 10:56 AM, Jon Masters <[email protected]> wrote:
> On Thu, 2007-11-29 at 10:40 -0800, Ray Lee wrote:
> > On Nov 29, 2007 9:36 AM, Alan Cox <[email protected]> wrote:
> > > > closed. But more importantly further access to it can be blocked until
> > > > appropriate actions are taken which also applies with your example, no? Is
> > >
> > > That bit is hard- very hard.
> >
> > In some sense it seems like the same problem faced by dynamic
> > translators such as Qemu. They really want to vet a dirtied or faulted
> > page before allowing the app to run unhindered. It's be nice to have
> > some way to do that without virtualizing the whole of userspace.
>
> Like I hinted at, you can't just "vet a page". Because a page alone is
> meaningless garbage, unless it happens to be an extremely small program,
> with headers, all nicely aligned. Most likely you don't know if a random
> page of data is code from a COFF file, ELF file, or some random crap I
> typed in at a terminal after having too much coffee.
>
> So. You'd need to scan *all the pages* of *the entire file*, every time
> that you performed any type of operation.
*blink* Really? To lift Alan's example, a naive first implementation
would be to create a suffix tree of all of ESR's works, then scan each
page on fault to see if there are any partial matches in the tree. If
there's a partial match that ends at the page boundary, mark the page
as questionable, but let execution/reading continue. On the next
linear page, if the match finishes, mark that one as bad, and disallow
access. That wouldn't be very expensive (in terms of scanning effort).
But I'm by no means an anti-malware guy, so perhaps I'm spouting crap,
especially given that I've thought about this a total of five minutes.
Regardless, thanks for doing the coordination work with them, and for
interfacing them with the kernel community. I'm really going to shut
up now.
On Thu, 2007-11-29 at 11:11 -0800, Ray Lee wrote:
> On Nov 29, 2007 10:56 AM, Jon Masters <[email protected]> wrote:
> > On Thu, 2007-11-29 at 10:40 -0800, Ray Lee wrote:
> > > On Nov 29, 2007 9:36 AM, Alan Cox <[email protected]> wrote:
> > > > > closed. But more importantly further access to it can be blocked until
> > > > > appropriate actions are taken which also applies with your example, no? Is
> > > >
> > > > That bit is hard- very hard.
> > >
> > > In some sense it seems like the same problem faced by dynamic
> > > translators such as Qemu. They really want to vet a dirtied or faulted
> > > page before allowing the app to run unhindered. It's be nice to have
> > > some way to do that without virtualizing the whole of userspace.
> >
> > Like I hinted at, you can't just "vet a page". Because a page alone is
> > meaningless garbage, unless it happens to be an extremely small program,
> > with headers, all nicely aligned. Most likely you don't know if a random
> > page of data is code from a COFF file, ELF file, or some random crap I
> > typed in at a terminal after having too much coffee.
> >
> > So. You'd need to scan *all the pages* of *the entire file*, every time
> > that you performed any type of operation.
>
> *blink* Really?
Yeah, really.
> To lift Alan's example, a naive first implementation
> would be to create a suffix tree of all of ESR's works, then scan each
> page on fault to see if there are any partial matches in the tree.
Ah, but I could write a sequence of pages that on their own looked
garbage, but in reality, when executed would print out a copy of the
Jargon File in all its glory. And if you still think you could look for
patterns, how about executable code that self-modifies in random ways
but when executed as a whole actually has the functionality of fetchmail
embedded within it? How would you guard against that?
Jon.
On Thu, 29 Nov 2007 14:45:51 EST, Jon Masters said:
> Ah, but I could write a sequence of pages that on their own looked
> garbage, but in reality, when executed would print out a copy of the
> Jargon File in all its glory. And if you still think you could look for
> patterns, how about executable code that self-modifies in random ways
> but when executed as a whole actually has the functionality of fetchmail
> embedded within it? How would you guard against that?
So, just because Fred Cohen showed in his PhD thesis that *perfect* virus/malware
scanning is equivalent to the Turing Halting Problem, we should abandon
efforts to make a 99.9998% workable system?
Yes, most of these schemes *can* be bypassed because some malicious code does a
mmap() or similar trick. But what is being overlooked here is that in most
cases, what is *desired* is a way to filter things being handled by *non*
malicious code. Yeah, sure, a shar archive can contain a binary that does evil
things - but if we stop /bin/cp from copying the file that has the evil in it,
it's a non-issue.
Let's get real here guys - trying to do *absolutely perfect* security is
pointless. You want to do security that reduces your *total* cost - and in
most cases this means "pretty good security" that stops "almost all issues".
As Linus reminds us once in a while - the perfect is the enemy of the good.
In this case, we don't *need* to be perfect - we only need to be noticably
better than another well-known operating system that isn't even very good at
it.
Alan Cox <[email protected]> writes:
> If I want I can have 16 threads executing code in a shared object being
> written to by ten other programs at once and shared over a network while
> we are at it. Its probably not a good idea but I can do it if I have
> reason to.
Actually the kernel prevents writing to files which are executed currently
and you can also not execute what is currently open for writing.
That is what VM_DENYWRITE is good for.
Does not work for shared libraries though -- you cannot set VM_DENYWRITE
in user mmap aka ld.so Man page says
MAP_DENYWRITE
This flag is ignored. (Long ago, it signalled that attempts to write to the underlying file should fail
with ETXTBUSY. But this was a source of denial-of-service attacks.)
-Andi
> Jargon File in all its glory. And if you still think you could look for
> patterns, how about executable code that self-modifies in random ways
> but when executed as a whole actually has the functionality of fetchmail
> embedded within it? How would you guard against that?
Thats a problem for whoever writes the ESR detection tool and to what
level it works. The question for the kernel is how do we provide a
mechanism to allow (to some extent at least) this kind of tool to run.
Alan
On Thu, Nov 29, 2007 at 03:56:28PM -0500, [email protected] wrote:
> Yes, most of these schemes *can* be bypassed because some malicious code does a
> mmap() or similar trick. But what is being overlooked here is that in most
> cases, what is *desired* is a way to filter things being handled by *non*
> malicious code. Yeah, sure, a shar archive can contain a binary that does evil
> things - but if we stop /bin/cp from copying the file that has the evil in it,
> it's a non-issue.
Then all you need is scan-on-commit with commit-on-close. Which requires
no changes of anything that would run in kernel mode and no bothering with
LSM whatsoever.
Incidentally, I would really love to see the threat profile we are talking
about. I have some impressions in that area, but I would rather keep them
to myself for now - I really want to see the answers.
Alan Cox wrote
> > Jargon File in all its glory. And if you still think you could look for
> > patterns, how about executable code that self-modifies in random ways
> > but when executed as a whole actually has the functionality of fetchmail
> > embedded within it? How would you guard against that?
>
> Thats a problem for whoever writes the ESR detection tool and to what
> level it works. The question for the kernel is how do we provide a
> mechanism to allow (to some extent at least) this kind of tool to run.
In a general way, what about providing a kind of {file,inode}_operations
xchange functionality given a super_block such that filesystems could
essentially be stacked, e.g. a
int op_xchange(struct super_block *sb, struct inode_operations *new_iop,
struct inode_operations *old_iop, struct file_operations *new_fop,
struct file_operations *old_fop);
The caller could/would/should be vetted, but replacing the static ops
with new ones of the caller's specification would allow them to
intercept the i/o, do whatever checking they wanted to do, and then call
the proper underlying operation (even modifying the parameters, I guess,
but that may be too weird). Perhaps separate get_* and set_* so that the
caller can retrieve the current operations vector and only "override"
the functions they really care about?
It's not perfect, but as was recently pointed out, if you can only get
98% of the way there rather than 100% is that a reason for not trying to
make it possible? It'd obviously not work for mmap, but as near as I can
tell the whole point is to get the "normal" malware cases without having
to do icky things like mangle the syscall table.
-justinb
--
Justin Banks
BakBone Software
[email protected]
Alan Cox <[email protected]> writes:
>
> The simple case is
> open
> write cathedral and bazaar in some order
> close
> <trap close -> process -> label eric_t>
>
> open (eric_t) - SELinux "no"
>
>
> Anyone smart will then write it out of order and keep the file open, or
That would assume Eric already has a program running on your system
optimized to inject his works in a obfuscated way. And if he has a
program running he can do nearly everything already. You already
lost the game.
The normal case Tvrtko et.al. are trying to handle would be more the
work getting downloaded from somewhere or read from a usb stick using
normal programs like web browsers or file managers who don't do any
out of order writing tricks and other obfuscation.
Important exception might be things like BitTorrent who write
out of order or parallel downloaders to cheat TCP congestion control.
Or simply tar+gzip with automatic depacking in desktops.
There are probably more and it's probably tricky but it is not a
"need to handle arbitary nastiness by a determined attacker" situation.
Anyways I'm not saying that pattern matching is a useful security
measure (just the interaction with compression and encryption makes it
very dubious), but if you're talking hypothetically you should at
least look closely at the hypothetical use cases @)
-Andi
On Thu, 2007-11-29 at 15:56 -0500, [email protected] wrote:
> On Thu, 29 Nov 2007 14:45:51 EST, Jon Masters said:
> > Ah, but I could write a sequence of pages that on their own looked
> > garbage, but in reality, when executed would print out a copy of the
> > Jargon File in all its glory. And if you still think you could look for
> > patterns, how about executable code that self-modifies in random ways
> > but when executed as a whole actually has the functionality of fetchmail
> > embedded within it? How would you guard against that?
>
> So, just because Fred Cohen showed in his PhD thesis that *perfect* virus/malware
> scanning is equivalent to the Turing Halting Problem, we should abandon
> efforts to make a 99.9998% workable system?
I think you misread what I said. I implied the exact opposite :-)
I'm trying to show that I understand the problem by saying the above,
that doing this perfectly is impossible, but I also happen to believe
that there are those who have solutions that provide a level of
protection to their users, who ask for such things. Hence my point is
that it's not really our place to debate whether virus scanning is
good/bad but more how to provide a sane API. I'll get a spec.
Jon.
On Thu, 2007-11-29 at 21:45 +0000, Alan Cox wrote:
> > Jargon File in all its glory. And if you still think you could look for
> > patterns, how about executable code that self-modifies in random ways
> > but when executed as a whole actually has the functionality of fetchmail
> > embedded within it? How would you guard against that?
>
> Thats a problem for whoever writes the ESR detection tool and to what
> level it works. The question for the kernel is how do we provide a
> mechanism to allow (to some extent at least) this kind of tool to run.
Right. I'm just saying reading a single page out of context (no pun
intended) is not going to be very useful. They need to scan the entire
file, which means that there are limited ways this is practical - it's
not practical to do that on every write into a shared mapping, hence a
solution that scans on open, etc. is probably the best there is.
(I know you know this)
Jon.
On Thu, 29 Nov 2007, Al Viro wrote:
> Incidentally, I would really love to see the threat profile we are talking
> about.
Exactly.
Please come up with a set of requirements that can be reviewed by the core
kernel folk, and perhaps then focus on how to meet those requirements once
they have been accepted.
- James
--
James Morris
<[email protected]>
On Thu, Nov 29, 2007 at 03:12:38PM -0700, Justin Banks wrote:
> It's not perfect, but as was recently pointed out, if you can only get
> 98% of the way there rather than 100% is that a reason for not trying to
> make it possible?
BTW, that's a fine example of a common fallacy: "$FOO is 98% of the way to
$TARGET" does not allow to interpolate the properties of $TARGET to those
of $FOO.
Telling that a condom is a 98% approximation to platonic ideal of such is
not particulary useful, especially if it turns out that what this number
really means is that there's a hole on its tip covering 2% of surface...
On Thu, 29 Nov 2007 18:34:33 EST, Jon Masters said:
>
> On Thu, 2007-11-29 at 21:45 +0000, Alan Cox wrote:
> > > Jargon File in all its glory. And if you still think you could look for
> > > patterns, how about executable code that self-modifies in random ways
> > > but when executed as a whole actually has the functionality of fetchmail
> > > embedded within it? How would you guard against that?
> >
> > Thats a problem for whoever writes the ESR detection tool and to what
> > level it works. The question for the kernel is how do we provide a
> > mechanism to allow (to some extent at least) this kind of tool to run.
>
> Right. I'm just saying reading a single page out of context (no pun
> intended) is not going to be very useful.
Fortunately for all concerned, although Alan's self-modifying code is indeed a
possibility, it's much less of an issue than the sort of malware that can be
found with a simple "find this 27-byte sequence, which will be found in either
block 36 or 37 of the file".
And I'll make the prediction that we won't see anything doing the sorts of
things that Alan's program does, until that's the *easiest* way to get into
a system. Until that time, they're either going to be sending simpler stuff
that a scanner can easily template and find, or using other means of attacks
that are outside the scope of a scanner.
Remember guys - we want to think about *realistic* threat models. The e-mail
virus scanners we use catch hundreds to thousands of known viruses *every day*.
But I can count on the fingers of both hands the number of times I've had to
deal with a *real* "0-day" in a quarter century. The scanner doesn't have to
be perfect - it just has to make it hard enough to bypass to render it
economically infeasible. If you're targeted by a military/govt/political/
religious group that doesn't *care* if it's economically viable, you have
other, bigger problems to deal with...
> Fortunately for all concerned, although Alan's self-modifying code is indeed a
> possibility, it's much less of an issue than the sort of malware that can be
> found with a simple "find this 27-byte sequence, which will be found in either
> block 36 or 37 of the file"
Thats a very old model of detection ;)
There is another usage case for this stuff as well - which is on the fly
indexing of material. An indexer has the same basic needs as a virus
checker (without the blocking aspect) and doesn't have to be robust
against sneaky material.
Alan
Al Viro wrote
> On Thu, Nov 29, 2007 at 03:12:38PM -0700, Justin Banks wrote:
>
> > It's not perfect, but as was recently pointed out, if you can only get
> > 98% of the way there rather than 100% is that a reason for not trying to
> > make it possible?
>
> BTW, that's a fine example of a common fallacy: "$FOO is 98% of the way to
> $TARGET" does not allow to interpolate the properties of $TARGET to those
> of $FOO.
>
> Telling that a condom is a 98% approximation to platonic ideal of such is
> not particulary useful, especially if it turns out that what this number
> really means is that there's a hole on its tip covering 2% of surface...
Understood, but for this use case (and indexing, probably replication,
IDS, and others) hooking at the "normal" VFS layer would provide enough
flexibility to do what's needed. I'm not saying it should be done, only
that it'd work, provide more use cases for linux, and not be totally
unsupportable.
-justinb
--
Justin Banks
BakBone Software
[email protected]
Tvrtko A. Ursulin wrote:
> During one recent LKML discussion
> (http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about
> LSM going
> static you called for LSM users to speak up.
Great big clue: If "LSM" is in the subject line, then cc: the LSM list
[email protected]
For LSM readers seeing this for the first time, the thread starts here
and goes for a while http://lkml.org/lkml/2007/11/28/106
I'm sympathetic to the desire to be able to provide a 3rd party LSM that
end users can install on their systems. That is why I advocated for
keeping the dynamic LSM interface. Getting the dynamic interface
restored faces a lot of challenges, but I hope that some kind of
solution can be found, because the alternative is to effectively force
vendors like Sophos to do it the "dirty" way by fishing in memory for
the syscall table. I would much rather that Linux offers you a way to do
what you need to do than force you to do nasty things.
Crispin
--
Crispin Cowan, Ph.D. http://crispincowan.com/~crispin
CEO, Mercenary Linux http://mercenarylinux.com/
Itanium. Vista. GPLv3. Complexity at work
On Fri, 30 Nov 2007, Crispin Cowan wrote:
> restored faces a lot of challenges, but I hope that some kind of
> solution can be found, because the alternative is to effectively force
> vendors like Sophos to do it the "dirty" way by fishing in memory for
> the syscall table.
I don't think this is quite correct.
The alternative is to engage with the kernel community to become part of
the development process, to ensure that appropriate APIs are implemented,
and to get code upstream before shipping it.
In any case, a patch to revert the dynamic aspect of LSM has been posted
by Arjan (and acked by myself) for the case of valid out of tree users.
The only case of this so far has been Multiadm, although there seems to be
no reason for it to stay out of tree.
- James
--
James Morris
<[email protected]>
James Morris wrote:
> On Fri, 30 Nov 2007, Crispin Cowan wrote:
>> restored faces a lot of challenges, but I hope that some kind of
>> solution can be found, because the alternative is to effectively force
>> vendors like Sophos to do it the "dirty" way by fishing in memory for
>> the syscall table.
>>
> I don't think this is quite correct.
>
> The alternative is to engage with the kernel community to become part of
> the development process, to ensure that appropriate APIs are implemented,
> and to get code upstream before shipping it.
>
That would be part of the "some kind of solution can be found" so I
think we are in agreement here.
> In any case, a patch to revert the dynamic aspect of LSM has been posted
> by Arjan (and acked by myself) for the case of valid out of tree users.
> The only case of this so far has been Multiadm, although there seems to be
> no reason for it to stay out of tree.
>
Dazuko. It has the same yucky code issues as Talpa, but AFAIK is pure
GPL2 and thus is clean on the license issues.
That these modules are valid modules that users want to use, are GPL
clean, and are *not* something LKML wants to upstream because of code
issues, is precisely why the LSM interface makes sense.
Crispin
--
Crispin Cowan, Ph.D. http://crispincowan.com/~crispin
CEO, Mercenary Linux http://mercenarylinux.com/
Itanium. Vista. GPLv3. Complexity at work
On Fri, 30 Nov 2007, Crispin Cowan wrote:
> > The only case of this so far has been Multiadm, although there seems to be
> > no reason for it to stay out of tree.
> >
> Dazuko. It has the same yucky code issues as Talpa, but AFAIK is pure
> GPL2 and thus is clean on the license issues.
>
> That these modules are valid modules that users want to use, are GPL
> clean, and are *not* something LKML wants to upstream because of code
> issues, is precisely why the LSM interface makes sense.
I think the idea is to try and fix code quality issues through
participation in the upstream process, rather than have upstream maintain
stable kernel APIs which are naturally mismatched to the unknown
requirements of out of tree users.
- James
--
James Morris
<[email protected]>
Hi!
> > Personally I admit I never quite saw the point of intercepting all
> > file accesses for everything. That will just always be slow as often
> > demonstrated on other operating systems and racey and unreliable too.
> > And at least the internal daemons should be already reasonably well
> > protected by standard (or beyond standard, like AA/SELinux etc.)
> > security measures, so e.g. it does not really make sense to intercept
> > all of your /etc file accesses and similar.
>
> Personally I don't know. I am often surprised when I hear what techniques
> the bad guys use and what is possible.
>
> Solution that we currently have is not so slow (btw you are free to try it
> out). And by working on a proper interface race and reliability issues
> should be solvable.
Really?
So what you are trying to do is 'application may never read bad
sequence of bits from disk', right?
Now, how do you propose to solve mmap(MAP_SHARED)? The app on the other cpu may
see the bad bits before kernel has chance to see them.
> > It might be better to identify the services (gateway, samba, file
> > server whatever) that are actually dealing with possible infected
> > "external" files and then define some generic interface that would
> > allow you to check those as the data appears.
> >
> > I would expect such an approach to perform better in the end and be more
> > reliable too.
> >
> > Note such a interface might well be user space only.
>
> That is fine for some classes of servers. But for some other and desktop
> more could be needed.
Andi's solution seems the only one that has chance to work _reliably_.
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Sat, 01 Dec 2007 08:43:32 GMT, Pavel Machek said:
> So what you are trying to do is 'application may never read bad
> sequence of bits from disk', right?
No, in many of the use cases, we're trying to do "if application reads certain
specified sequences of bits from disk we know about it", which is subtly
different. Often, *absolute* prevention isn't required, as long as we can
generate audit trails and/or alerts...
> Now, how do you propose to solve mmap(MAP_SHARED)? The app on the other cpu may
> see the bad bits before kernel has chance to see them.
For many usage cases (such as virus scanners), mmap() isn't really an issue,
because if another process is *already* trying to mmap() the file before it's
even finished downloading from the network interface, you have other problems.
In that case, what you *really* want to be hooking is read() from a network
socket, not a disk file - and I don't think you can mmap() a socket anyhow,
right?
Similarly, if you're using these exit points to do *indexing*, you really don't
care if it got mmap()'ed - at worst, it isn't indexed until the other CPU gets
around to it.
On Sun, 02 Dec 2007 14:44:48 -0500
[email protected] wrote:
> On Sat, 01 Dec 2007 08:43:32 GMT, Pavel Machek said:
>
> > So what you are trying to do is 'application may never read bad
> > sequence of bits from disk', right?
>
> No, in many of the use cases, we're trying to do "if application
> reads certain specified sequences of bits from disk we know about
> it", which is subtly different. Often, *absolute* prevention isn't
> required, as long as we can generate audit trails and/or alerts...
.. which breaks down if/when glibc uses mmap() to implement the
fopen/fread etc interface
(note: it already does, just not quite yet by default)
--
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings,
visit http://www.lesswatts.org
> and I don't think you can mmap() a socket anyhow,
> right?
You can mmap packet sockets.
-Andi
Hi!
> > So what you are trying to do is 'application may never read bad
> > sequence of bits from disk', right?
>
> No, in many of the use cases, we're trying to do "if application reads certain
> specified sequences of bits from disk we know about it", which is subtly
> different. Often, *absolute* prevention isn't required, as long as we can
> generate audit trails and/or alerts...
>
> > Now, how do you propose to solve mmap(MAP_SHARED)? The app on the other cpu may
> > see the bad bits before kernel has chance to see them.
>
> For many usage cases (such as virus scanners), mmap() isn't really an issue,
> because if another process is *already* trying to mmap() the file before it's
> even finished downloading from the network interface, you have other
> problems.
Well, if you only want to detect viruses _sometimes_, you can just
LD_PRELOAD your scanner.
I guess the A/V people should describe what they are trying to do, as
in
"forbidden sequences of bits should never hit disk" or "forbidden
sequences of bits should be never read from disk" or something...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Sun, 02 Dec 2007 21:22:40 +0100, Pavel Machek said:
> Well, if you only want to detect viruses _sometimes_, you can just
> LD_PRELOAD your scanner.
And for some use cases, that probably *is* the best answer..
> I guess the A/V people should describe what they are trying to do, as
> in
>
> "forbidden sequences of bits should never hit disk" or "forbidden
> sequences of bits should be never read from disk" or something...
We probably want to hear related usages as well - what *besides* A/V would be
interested? Indexing services? Software that tries to limit the distribution
of sensitive info off the machine - for instance, imagine a rule that said
"Data that comes from a file that contains SSNs or the string 'Corporate
Secret' data isn't allowed to leave the computer" and a Perl-like 'taint'
concept. I'm not saying its *doable*, but it's certainly a goal that somebody
would like to achieve...
On Sun 2007-12-02 16:09:55, [email protected] wrote:
> On Sun, 02 Dec 2007 21:22:40 +0100, Pavel Machek said:
> > Well, if you only want to detect viruses _sometimes_, you can just
> > LD_PRELOAD your scanner.
>
> And for some use cases, that probably *is* the best answer..
I'd say so.
> > I guess the A/V people should describe what they are trying to do, as
> > in
> >
> > "forbidden sequences of bits should never hit disk" or "forbidden
> > sequences of bits should be never read from disk" or something...
>
> We probably want to hear related usages as well - what *besides* A/V would be
> interested? Indexing services?
Well... I'd really like to know what A/V people are trying to do.
Indexing services are really different, and doable with recursive
m-time Jan is preparing...
> Software that tries to limit the
> distribution
> of sensitive info off the machine - for instance, imagine a rule that said
> "Data that comes from a file that contains SSNs or the string 'Corporate
> Secret' data isn't allowed to leave the computer" and a Perl-like 'taint'
> concept. I'm not saying its *doable*, but it's certainly a goal that somebody
> would like to achieve...
That sounds like a task for SELinux, no?
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Dec 2 2007 22:56, Pavel Machek wrote:
>>
>> We probably want to hear related usages as well - what *besides*
>> A/V would be interested? Indexing services?
>
Indexing services would probably benefit much more from a
recursive-aware inotify, though that has its own sort of problems to
solve first.
>Well... I'd really like to know what A/V people are trying to do.
>
>Indexing services are really different, and doable with recursive
>m-time Jan is preparing...
>
m-time <=> modification time?
What am I preparing?
I am actually on a freeze, because I really do not know what to make
of the situation with the static LSM interface.
There is a grave problem with chaining, because you cannot specify
the activation order of one or more LSMs with compiled-in code!
Some kernel Makefiles even contain hints "this depends on link order"
(e.g. net/ipv6/netfilter/Makefile) - and I bet for sure that this
will also be the case for LSM. No thanks.
While we are at it, consider the hypothethical case of a production
server, and the boss tells you to switch to $ThatLSM, with no downtime.
After all, it worked when $Company switched to $ThisLSM with Linux
2.6.x ∀ x<24.
Hi!
> >Well... I'd really like to know what A/V people are trying to do.
> >
> >Indexing services are really different, and doable with recursive
> >m-time Jan is preparing...
> >
> m-time <=> modification time?
Yep.
> What am I preparing?
Not you, Jan Kara. Sorry.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Jon Masters <[email protected]> wrote:
> On Thu, 2007-11-29 at 11:11 -0800, Ray Lee wrote:
>> On Nov 29, 2007 10:56 AM, Jon Masters <[email protected]> wrote:
>> > On Thu, 2007-11-29 at 10:40 -0800, Ray Lee wrote:
>> > > On Nov 29, 2007 9:36 AM, Alan Cox <[email protected]> wrote:
>> > > > > closed. But more importantly further access to it can be blocked
>> > > > > until appropriate actions are taken which also applies with your
>> > > > > example, no? Is
>> > > >
>> > > > That bit is hard- very hard.
>> To lift Alan's example, a naive first implementation
>> would be to create a suffix tree of all of ESR's works, then scan each
>> page on fault to see if there are any partial matches in the tree.
>
> Ah, but I could write a sequence of pages that on their own looked
> garbage, but in reality, when executed would print out a copy of the
> Jargon File in all its glory. And if you still think you could look for
> patterns, how about executable code that self-modifies in random ways
> but when executed as a whole actually has the functionality of fetchmail
> embedded within it? How would you guard against that?
You can't scan all possible code for malware:
Take a random piece of code, possibly halting. Replace all halting conditions
using a piece of malware. Scan it. If it were possible to detect the malware
without false positives, you'd have solved the halting problem.
In practice, this does not hinder virus scanners from preventing most damage.
Therefore I think it's OK to have one.
If I had to design a virus scanner interface, I'd e.g. create a library*
providing an {open|mmap}_and_scan() function that would give me a clean
copy/really-private mapping of a scanned file, and a scan_{blob,file}()
function that would scan a block of memory/a file. Then, it's up to the
application to ensure that it uses that library. As a result, you could
e.g. run "less eicar.sh", but you could not run "bash eicar.sh"**, and an
application receiving a strangely encoded piece of malware into it's
memory has a chance of avoiding an infection without writing it to a file.
Maybe gpg < eicar.gpg.sh|sh will unintendedly work, but I don't think
scanning pipes would be easy anyway. OTOH, maybe the library would make
it feasible at all, provided the malicious code is not located way before
the signature.
Off cause I'd need to do something about binaries. At first glance, this
does not seem too bad, since there is a way to run ld*.so. I'd just use it
to enforce a preloader for static binaries, too. (I'm glad I can leave the
implementation details to somebody else.-)
* Without having a virus scanner installed, this library will just NOOP
by default.
** Bonus: I can unzip open_office_file; rm macros; zip open_office_file.
OTOH, the scanner should provide a cleaner for those simple cases.
On Mon, 2007-12-03 at 23:45 +0100, Bodo Eggert wrote:
> Jon Masters <[email protected]> wrote:
> > On Thu, 2007-11-29 at 11:11 -0800, Ray Lee wrote:
> >> On Nov 29, 2007 10:56 AM, Jon Masters <[email protected]> wrote:
> >> > On Thu, 2007-11-29 at 10:40 -0800, Ray Lee wrote:
> >> > > On Nov 29, 2007 9:36 AM, Alan Cox <[email protected]> wrote:
>
> >> > > > > closed. But more importantly further access to it can be blocked
> >> > > > > until appropriate actions are taken which also applies with your
> >> > > > > example, no? Is
> >> > > >
> >> > > > That bit is hard- very hard.
>
> >> To lift Alan's example, a naive first implementation
> >> would be to create a suffix tree of all of ESR's works, then scan each
> >> page on fault to see if there are any partial matches in the tree.
> >
> > Ah, but I could write a sequence of pages that on their own looked
> > garbage, but in reality, when executed would print out a copy of the
> > Jargon File in all its glory. And if you still think you could look for
> > patterns, how about executable code that self-modifies in random ways
> > but when executed as a whole actually has the functionality of fetchmail
> > embedded within it? How would you guard against that?
>
> You can't scan all possible code for malware:
> Take a random piece of code, possibly halting. Replace all halting conditions
> using a piece of malware. Scan it. If it were possible to detect the malware
> without false positives, you'd have solved the halting problem.
Good. I think you got the point of my sarcasm. My *point* was that we
have two different camps of people here:
* Those who think some solution is better than none.
* Those who want an unobtainable, perfect solution.
I'm not criticising, each has their position. However, I was attempting
to explain that I do fully "get it" by running through an example of how
to work around more elementary on-access scanning schemes. I know that
(no matter what marketing exists to the contrary), it is never possible
to have perfect anti-malware software. But I do think there is a time
and a place for Linux to help make some folks feel safer - on access
file scanning isn't evil, and you don't have to use it! Freedom! :-)
Having spoken to a few people, I've created the following mailing list,
so we can rant away and come up with a list of requirements to present
for further discussion. Note that this is a case where I actually expect
people to be *happy* with yet another email list :-)
http://lists.printk.net/cgi-bin/mailman/listinfo/malware-list
Please sign up, and encourage interested third parties to do so too.
Let's work this all out. Then I'll come back sometime over the holidays
with a summary and some followup.
> If I had to design a virus scanner interface, I'd e.g. create a library*
> providing an {open|mmap}_and_scan() function that would give me a clean
> copy/really-private mapping of a scanned file, and a scan_{blob,file}()
> function that would scan a block of memory/a file.
Although I'm open to the idea, I'm almost 100% convinced that nobody is
going to buy modifying userspace applications one at a time. I think
there is a legitimate feeling of this needing to be massaged by the
kernel on some level. But I might be wrong - don't flame me.
Jon.
Jon Masters wrote:
> On Mon, 2007-12-03 at 23:45 +0100, Bodo Eggert wrote:
>> Jon Masters <[email protected]> wrote:
>>> On Thu, 2007-11-29 at 11:11 -0800, Ray Lee wrote:
>>>> On Nov 29, 2007 10:56 AM, Jon Masters <[email protected]> wrote:
>>>> To lift Alan's example, a naive first implementation
>>>> would be to create a suffix tree of all of ESR's works, then scan each
>>>> page on fault to see if there are any partial matches in the tree.
>>> Ah, but I could write a sequence of pages that on their own looked
>>> garbage, but in reality, when executed would print out a copy of the
>>> Jargon File in all its glory. And if you still think you could look for
>>> patterns, how about executable code that self-modifies in random ways
>>> but when executed as a whole actually has the functionality of fetchmail
>>> embedded within it? How would you guard against that?
>> You can't scan all possible code for malware:
>> Take a random piece of code, possibly halting. Replace all halting conditions
>> using a piece of malware. Scan it. If it were possible to detect the malware
>> without false positives, you'd have solved the halting problem.
>
> Good. I think you got the point of my sarcasm. My *point* was that we
> have two different camps of people here:
>
> * Those who think some solution is better than none.
But we are talking about malicious programs, and so
there is a common motto:
"Poor Security Can Be Worse Than No Security", so
in this field often "none" is better that "some"
Really i don't understand why you push such module.
Malicious software in few generation (few years)
will use alternate methods. So the linux kernel
will be worse (and maybe will expose more bugs because
of complexity, and no problem are solved) but no
problem are solved.
See windoze: it is a patch after an other, so
the system is complex, unmaintainable and surely
not more secure. or do you want to change our behavior
as windows users: they compress files before to send
it, because of antiviruses policies.
If antiviruses will add security, we will not
have such big bot-nets and worms from the concurrent
OS. Antiviruses offers only a short term cure.
ciao
cate
> * Those who want an unobtainable, perfect solution.
>
> I'm not criticising, each has their position. However, I was attempting
> to explain that I do fully "get it" by running through an example of how
> to work around more elementary on-access scanning schemes. I know that
> (no matter what marketing exists to the contrary), it is never possible
> to have perfect anti-malware software. But I do think there is a time
> and a place for Linux to help make some folks feel safer - on access
> file scanning isn't evil, and you don't have to use it! Freedom! :-)
>
> Having spoken to a few people, I've created the following mailing list,
> so we can rant away and come up with a list of requirements to present
> for further discussion. Note that this is a case where I actually expect
> people to be *happy* with yet another email list :-)
>
> http://lists.printk.net/cgi-bin/mailman/listinfo/malware-list
>
> Please sign up, and encourage interested third parties to do so too.
> Let's work this all out. Then I'll come back sometime over the holidays
> with a summary and some followup.
>
>> If I had to design a virus scanner interface, I'd e.g. create a library*
>> providing an {open|mmap}_and_scan() function that would give me a clean
>> copy/really-private mapping of a scanned file, and a scan_{blob,file}()
>> function that would scan a block of memory/a file.
>
> Although I'm open to the idea, I'm almost 100% convinced that nobody is
> going to buy modifying userspace applications one at a time. I think
> there is a legitimate feeling of this needing to be massaged by the
> kernel on some level. But I might be wrong - don't flame me.
>
> Jon.
On Thu 2007-11-29 23:58:44, Andi Kleen wrote:
> Alan Cox <[email protected]> writes:
> >
> > The simple case is
> > open
> > write cathedral and bazaar in some order
> > close
> > <trap close -> process -> label eric_t>
> >
> > open (eric_t) - SELinux "no"
> >
> >
> > Anyone smart will then write it out of order and keep the file open, or
>
> That would assume Eric already has a program running on your system
> optimized to inject his works in a obfuscated way. And if he has a
> program running he can do nearly everything already. You already
> lost the game.
>
> The normal case Tvrtko et.al. are trying to handle would be more the
> work getting downloaded from somewhere or read from a usb stick using
> normal programs like web browsers or file managers who don't do any
> out of order writing tricks and other obfuscation.
Fortunately normal programs tend to be dynamically linked, so
LD_PRELOAD is fine to handle them. And we know we can't handle nasty
programs. Seems like LD_PRELOAD is the way to go.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html