*sigh*, anyone object to patches marking such mailing lists in
MAINTAINERS as 'subscription only' ?
Dave
On Wed, Jan 21, 2004 at 08:39:54PM +0100, linuxtv-listserver wrote:
>
> Sorry, this is a closed list.
>
> To subscribe please mail to [email protected] with
> subject "subscribe linux-dvb".
>
> If you get this message even if you _are_ on the list your actual
> identity differs from the email address you subscribed with.
> So you have to change one of them, either by adjusting your actual
> email address in your email client or by unsubscribing an re-subscribing
> with your valid address.
>
> If you are still having problems, please contact [email protected].
>
>
> ---
> Ecartis v1.0.0 - job execution complete.
---end quoted text---
On Wed, 21 Jan 2004 19:43:15 GMT, Dave Jones <[email protected]> said:
> *sigh*, anyone object to patches marking such mailing lists in
> MAINTAINERS as 'subscription only' ?
Only if it's recognized that although many people will send a bug report to a
mailing list, they won't bother if they have to subscribe to post it, and then
figure out how to get unsubscribed...
On Wed, Jan 21, 2004 at 11:56:02AM -0800, Linus Torvalds wrote:
>
>
> On Wed, 21 Jan 2004, Dave Jones wrote:
> >
> > *sigh*, anyone object to patches marking such mailing lists in
> > MAINTAINERS as 'subscription only' ?
>
> Sounds like they shouldn't be in MAINTAINERS at all if they can't be
> posted to. I mean, what's the point?
I see the point of having a 'this is our mailing list' entry,
but having that as the sole contact point when its a closed list
is damned rude IMO.
Dave
On Wed, 21 Jan 2004, Dave Jones wrote:
>
> *sigh*, anyone object to patches marking such mailing lists in
> MAINTAINERS as 'subscription only' ?
Sounds like they shouldn't be in MAINTAINERS at all if they can't be
posted to. I mean, what's the point?
Linus
On Wed, Jan 21, 2004 at 07:43:15PM +0000, Dave Jones wrote:
> *sigh*, anyone object to patches marking such mailing lists in
> MAINTAINERS as 'subscription only' ?
Better remove them completly. A subscription-only ML is worthless
for bug reports.
Quote from Dave Jones <[email protected]>:
> On Wed, Jan 21, 2004 at 11:56:02AM -0800, Linus Torvalds wrote:
> >
> >
> > On Wed, 21 Jan 2004, Dave Jones wrote:
> > >
> > > *sigh*, anyone object to patches marking such mailing lists in
> > > MAINTAINERS as 'subscription only' ?
> >
> > Sounds like they shouldn't be in MAINTAINERS at all if they can't be
> > posted to. I mean, what's the point?
>
> I see the point of having a 'this is our mailing list' entry,
> but having that as the sole contact point when its a closed list
> is damned rude IMO.
Quite possibly some lists weren't closed when they were first added to
the MAINTAINERS file, but have since become closed.
John.
On Wed, 2004-01-21 at 12:56, Linus Torvalds wrote:
> On Wed, 21 Jan 2004, Dave Jones wrote:
> >
> > *sigh*, anyone object to patches marking such mailing lists in
> > MAINTAINERS as 'subscription only' ?
>
> Sounds like they shouldn't be in MAINTAINERS at all if they can't be
> posted to. I mean, what's the point?
>
> Linus
Compared to the time required to write up a good bug report or patch,
what's the problem with the couple minutes needed to subscribe,
authorize and _then_ post to the list?
--
Zan Lynx <[email protected]>
Zan Lynx <[email protected]> wrote:
> On Wed, 2004-01-21 at 12:56, Linus Torvalds wrote:
> > On Wed, 21 Jan 2004, Dave Jones wrote:
> > >
> > > *sigh*, anyone object to patches marking such mailing lists in
> > > MAINTAINERS as 'subscription only' ?
> >
> > Sounds like they shouldn't be in MAINTAINERS at all if they can't be
> > posted to. I mean, what's the point?
>
> Compared to the time required to write up a good bug report or patch,
> what's the problem with the couple minutes needed to subscribe,
> authorize and _then_ post to the list?
The time it takes is beside the point. If I find a bug, I'll write it up and
send it in -- but I'm not going to jump through arbitrary hoops to do it. I
may as well just fork the code and fix the damned thing myself :).
If they want to close their list, fine -- but then they should set up a
separate bug-submission address which is open.
Charles
--
-----------------------------------------------------------------------
Charles Cazabon <[email protected]>
GPL'ed software available at: http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------
On Wed, Jan 21, 2004 at 01:38:19PM -0700, Zan Lynx wrote:
> Compared to the time required to write up a good bug report or patch,
> what's the problem with the couple minutes needed to subscribe,
> authorize and _then_ post to the list?
Closed lists are just *rude*. The only intention is to stop spam,
but personally, I find hoop-jumping like this *more* of an inconvenience
than spam, (and I get a *lot* of spam, several megs a day on a quiet day)
Some maintainers get *really pissed off* when you short-circuit them
and send fixes to Linus directly. But if I'm forced to jump through hoops
each time I fix up random parts of the kernel, I'm going to be equally
as rude as I consider they are by doing the best they can to inconvenience
people with good intentions, and mail them straight to Linus/Andrew/Marcelo.
Dave
On Wed, Jan 21, 2004 at 09:15:50PM +0000, Dave Jones wrote:
> On Wed, Jan 21, 2004 at 01:38:19PM -0700, Zan Lynx wrote:
>
> > Compared to the time required to write up a good bug report or patch,
> > what's the problem with the couple minutes needed to subscribe,
> > authorize and _then_ post to the list?
>
> Closed lists are just *rude*. The only intention is to stop spam,
> but personally, I find hoop-jumping like this *more* of an inconvenience
> than spam, (and I get a *lot* of spam, several megs a day on a quiet day)
>
> Some maintainers get *really pissed off* when you short-circuit them
> and send fixes to Linus directly. But if I'm forced to jump through hoops
> each time I fix up random parts of the kernel, I'm going to be equally
> as rude as I consider they are by doing the best they can to inconvenience
> people with good intentions, and mail them straight to Linus/Andrew/Marcelo.
What do you think about individual email (non-list) using a confirmation
based spam blocking system.
I currently use spamassassin to filter my messages, but I saw recently a
project that asks you to reply to a confirmation message if you're not
already on the white-list.
I'm not sure how acceptable it would be, and this is a little OT, but I'm
wondering if I should spend the time testing that for my corp.
Mike
On Wed, 21 Jan 2004 21:15:50 +0000 Dave Jones <[email protected]> wrote:
| On Wed, Jan 21, 2004 at 01:38:19PM -0700, Zan Lynx wrote:
|
| > Compared to the time required to write up a good bug report or patch,
| > what's the problem with the couple minutes needed to subscribe,
| > authorize and _then_ post to the list?
|
| Closed lists are just *rude*. The only intention is to stop spam,
| but personally, I find hoop-jumping like this *more* of an inconvenience
| than spam, (and I get a *lot* of spam, several megs a day on a quiet day)
Same here.
| Some maintainers get *really pissed off* when you short-circuit them
| and send fixes to Linus directly. But if I'm forced to jump through hoops
| each time I fix up random parts of the kernel, I'm going to be equally
| as rude as I consider they are by doing the best they can to inconvenience
| people with good intentions, and mail them straight to Linus/Andrew/Marcelo.
Agreed. I usually copy such maintainers in private email, but
it's too much hassle to subscribe to a mailing list just to post
one patch, especially if there are other ways to have it merged.
--
~Randy
>>>>> "Dave" == Dave Jones <[email protected]> writes:
Dave> On Wed, Jan 21, 2004 at 11:56:02AM -0800, Linus Torvalds wrote:
>> Sounds like they shouldn't be in MAINTAINERS at all if they can't
>> be posted to. I mean, what's the point?
Dave> I see the point of having a 'this is our mailing list' entry,
Dave> but having that as the sole contact point when its a closed list
Dave> is damned rude IMO.
Honestly, can a list be considered a contact point at all if you can't
post to it? If they are so afraid of outside posters, they can setup a
specific list for bug reporting only thats open or something. IMHO
allowing closed lists to be listed in the MAINTAINERS file is
to support the stupidity.
Cheers,
Jes
El Wed, 21 Jan 2004 14:57:18 -0600 Charles Cazabon <[email protected]> escribi?:
> The time it takes is beside the point. If I find a bug, I'll write it up and
> send it in -- but I'm not going to jump through arbitrary hoops to do it. I
> may as well just fork the code and fix the damned thing myself :).
>
> If they want to close their list, fine -- but then they should set up a
> separate bug-submission address which is open.
Or something which allows to the kernel bugzilla send them bug reports (like
it's donde with several subsystems)
On Wed, 21 Jan 2004, Zan Lynx wrote:
> >
> > Sounds like they shouldn't be in MAINTAINERS at all if they can't be
> > posted to. I mean, what's the point?
>
> Compared to the time required to write up a good bug report or patch,
> what's the problem with the couple minutes needed to subscribe,
> authorize and _then_ post to the list?
Sorry, bu that just isn't how it works.
Bug reports should be more important to the recipient than to the sender.
Anything you do to make it more of a bother is WRONG.
Linus
On Wed, Jan 21, 2004 at 01:30:27PM -0800, Mike Fedyk wrote:
>
> What do you think about individual email (non-list) using a confirmation
> based spam blocking system.
>
> I currently use spamassassin to filter my messages, but I saw recently a
> project that asks you to reply to a confirmation message if you're not
> already on the white-list.
>
> I'm not sure how acceptable it would be, and this is a little OT, but I'm
> wondering if I should spend the time testing that for my corp.
I'm not a big fan of this solution, but it's definitely a better
solution than a closed list. The acceptance depends on whether the group
of people you want emails from accepts it (and if you subscribe to a
mailing list be sure to exclude it from this mechanism or the mailing
list administrator will immediately kick you from the list).
And be aware that it doesn't help against all spam - spammers know how
to fake headers...
> Mike
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
> What do you think about individual email (non-list) using a confirmation
> based spam blocking system.
Consider a spammer using your address and spams people. Say 25000 of those
use this method (Called challenge response authentication protocol). You'll
get bombarded with 25000 challenge message.
You put the burden on the sender, not the spammer which is pretty much
useless. There was a discussion about this on exim-users and someone posted
a web page. http://kmself.home.netcom.com/Rants/challenge-response.html
> I currently use spamassassin to filter my messages, but I saw recently a
> project that asks you to reply to a confirmation message if you're not
> already on the white-list.
>
> I'm not sure how acceptable it would be, and this is a little OT, but I'm
> wondering if I should spend the time testing that for my corp.
I for one refuse to answer those challenges unless I know it was due to a
spammer. Defeats the purpose.
--
Lab tests show that use of micro$oft causes cancer in lab animals
Linus Torvalds writes:
>
>
> On Wed, 21 Jan 2004, Zan Lynx wrote:
> > >
> > > Sounds like they shouldn't be in MAINTAINERS at all if they can't be
> > > posted to. I mean, what's the point?
> >
> > Compared to the time required to write up a good bug report or patch,
> > what's the problem with the couple minutes needed to subscribe,
> > authorize and _then_ post to the list?
>
> Sorry, bu that just isn't how it works.
>
> Bug reports should be more important to the recipient than to the sender.
> Anything you do to make it more of a bother is WRONG.
>
That's true. And being on the linux-dvb list I can't see why they
would want to give it as a maintainer address.
AFAIR, recently there was only one guy from convergence sending in
patches for the kernel anyway. But maybe they have some problems
within the company so they don't want to name one single person as a
maintainer.
Still they could give some joint mailing address. I don't think it is
necessary to discuss all patches on the list, especially since those
coming from people that are not on the list will probably be quite
trivial or more kernel related than DVB hardware related.
Marcus
--
/--------------------------------------------------------------------\
| Dr. Marcus O.C. Metzler | |
| [email protected] | http://www.metzlerbros.de/ |
\--------------------------------------------------------------------/
|>>> I like GNU, but I couldn't eat a whole one <<<|
For the bridge list, I just setup the list up so all non-subscriber messages
have to be approved by the list maintainer. This adds a slight delay until
I get a round to approving the message; but seems like a fair compromise
to keep everyone else from reading the spam.
After the first message, I also put the person on the "approved senders"
list so the discussion can continue unhindered.
Also, listing just a mailing list (with no human) contact seems pretty
vague.
--
Stephen Hemminger mailto:[email protected]
Open Source Development Lab http://developer.osdl.org/shemminger
On Wed, Jan 21, 2004 at 09:15:50PM +0000, Dave Jones wrote:
> Closed lists are just *rude*. The only intention is to stop spam,
> but personally, I find hoop-jumping like this *more* of an inconvenience
> than spam, (and I get a *lot* of spam, several megs a day on a quiet day)
Umm, I don't think that's entirely correct. There's a problem in the
UK and EU with 100% open mailing lists - it's called data protection,
where the owner of their personal data has certain rights. I believe
that it is questionable whether such open lists can be run in the EU,
or even by EU individuals.
There appears to be two ways around this problem:
1. Only allow list members to be able to look at the list archives.
This precludes the ability of search engines (eg google) to be able
to index such archives. It also means that archive sites outside
the EU must not be permitted to subscribe to such a list. (List
owner control over this is virtually impossible since archive sites
don't generally ask permission before hand.)
2. Allow member-only posting and make membership of the list conditional
on an agreement which allows all information contained within the
message to be made publically available, and that such information
will be internationally distributed. This means that people who
have not agreed to the terms must be prevented from posting to the
list.
(1) is fine for things like LUG lists where the membership is fairly
small and the subject matter is not of international or universal
interest. However, it is unacceptable for highly technical lists
and/or those of international interest.
Note that I'm not saying that closed lists are a good thing either -
I don't like closed lists either, both from the user and more especially
the admin point of view, but I've have come to the conclusion that it
is a necessary evil which we have to bear in our increasingly litigious
world.
[Please note: I am aware of an instance where someone has indeed used
the UK's Data Protection Act in connection with a mailing list to get
the lists archives modified or the list shut down.]
--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 PCMCIA - http://pcmcia.arm.linux.org.uk/
2.6 Serial core
On Wed, 21 Jan 2004 21:44:37 +0000, Mike Fedyk wrote:
> What do you think about individual email (non-list) using a confirmation
> based spam blocking system.
for personal email it is plain asocial. it tells me that
a person does not want to receive mail from me.
for ML i would propably accept a confirmation email
(similar to the gmame news server confirmations I have
to send if I want to post a message).
if you have a well working ML, I'm sure you can find some
person that will receive outside postings and click on
"delete" or "post" to confirm them. moderating outside
postings is ok IMO. but make sure that person does not
drop dead or is on a long vacation.
Andreas
On Wed, 2004-01-21 at 16:40, Andreas Jellinghaus wrote:
> On Wed, 21 Jan 2004 21:44:37 +0000, Mike Fedyk wrote:
> > What do you think about individual email (non-list) using a confirmation
> > based spam blocking system.
>
> for personal email it is plain asocial. it tells me that
> a person does not want to receive mail from me.
For me, that isn't what it says at all. It tells me that he or she is
tired of receiving and sorting all of the spam every day. Since I feel
exactly the same way about spam, I cooperate and reply with a
confirmation.
--
Zan Lynx <[email protected]>
On Wed, 21 Jan 2004, Zan Lynx wrote:
> For me, that isn't what it says at all. It tells me that he or she is
> tired of receiving and sorting all of the spam every day. Since I feel
> exactly the same way about spam, I cooperate and reply with a
> confirmation.
If somebody asks me a question, but the reply gets stuck
in a C-R system ... too bad, you won't get your answer.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
On 21 Jan 2004, Jes Sorensen wrote:
> Honestly, can a list be considered a contact point at all if you can't
> post to it? If they are so afraid of outside posters, they can setup a
> specific list for bug reporting only thats open or something. IMHO
> allowing closed lists to be listed in the MAINTAINERS file is
> to support the stupidity.
Agreed, we should remove all addresses from MAINTAINERS
where bug reports by email aren't welcome.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
On Wed, 21 Jan 2004, Mike Fedyk wrote:
> What do you think about individual email (non-list) using a confirmation
> based spam blocking system.
>
> I currently use spamassassin to filter my messages, but I saw recently a
> project that asks you to reply to a confirmation message if you're not
> already on the white-list.
That used to be fairly ok, until the introduction of
virusses-with-forged-sender a year or two ago ...
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
On Wed, 2004-01-21 18:01:54 -0500, Wakko Warner <[email protected]>
wrote in message <[email protected]>:
> Consider a spammer using your address and spams people. Say 25000 of those
> use this method (Called challenge response authentication protocol). You'll
^ ^ ^ ^
You name it: crap...
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg
fuer einen Freien Staat voll Freier B?rger" | im Internet! | im Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
I get about one mail per month from a person I never ever had
contact before, didn't know and so a challange response system
couldn't know. If I used one, I propably would never get that
mail.
If I report problems, bugs, or answer calls for help, and
I'm annoyed with a challange response system, I never
answer, simply try to swallow my hatred and forget it.
no matter how much spam you get, it is not a valid reason
for making other peoples live worse. challange response
systems do that, so they should be out lawed.
Note: mailing lists subscriptions are a different case:
if *I* want on that list, I'm happy to answer the challange,
because I want on the list, and it is good if the ML manager
cannot be tricked.
I don't like closed mailing lists. They are ok, if some person
moderates them, and always approves posts from the outside fast.
still using a closed mailing list as address for bug reports
is not a good idea.
btw: please, all challange-response people add a footer
(or maybe even better: a first text line) announcing the fact they do
so. I will not waste my time to send you an email you will never read.
Andreas
>>>>> "Zan" == Zan Lynx <[email protected]> writes:
Zan> On Wed, 2004-01-21 at 16:40, Andreas Jellinghaus wrote:
>> On Wed, 21 Jan 2004 21:44:37 +0000, Mike Fedyk wrote: > What do you
>> think about individual email (non-list) using a confirmation >
>> based spam blocking system.
>>
>> for personal email it is plain asocial. it tells me that a person
>> does not want to receive mail from me.
Zan> For me, that isn't what it says at all. It tells me that he or
Zan> she is tired of receiving and sorting all of the spam every day.
Zan> Since I feel exactly the same way about spam, I cooperate and
Zan> reply with a confirmation.
I've had people pull the authentication game on me before. I just
stopped replying to them, waste of my time.
Fixing the spam problem is a lot easier without losing contact with
all your friends in the proces:, train your Bayesian filters and be
done with it. Mine were a mess, deleted all the data and fed 10 days
of spam and some proper mail through sa-learn. Since then I have seen
1 spam make it through during the last week, it used to be 20-40/day
(and some 200-300/day caught by the filters).
Jes
On Thu, 22 Jan 2004, David Ford wrote:
> Considering that Bayesian filters are useless against the new spam that
> is proliferating these days, that's laughable. Spam now comes with a
> good 5-10K of random dictionary words.
so we need to extend the Bayesian filters to deal with multi-word combos,
how many legit mail has those dictionary words in them? properly traind
their presence should help identify the spam.
not that you will ever see this (other then through the list) as I won't
respond to your confirmation message.
David Lang
On Thu, 2004-01-22 at 12:41, David Ford wrote:
> No, actually you can a) reply to my -plain text- email and yes...this is
> plain text that has been trimmed, b) wait because I'm involved in this
> discussion, I go to my queue and pre-auth people involved.
>
> So it's fine for you to have filters of your choosing, but someone
> else's filter's of their choosing are obnoxious...very social.
>
> [email protected] wrote:
>
Well it's all very fun, but I expect the spammers to figure out
challange and response soon enough.
> >Well, isn't it just fscking great... So in order to send you an email
> >I have to
> > a) cut the URL from your reply
> > b) suspend mutt(1)
> > c) type lynx '' and paste the damn thing in there
> > d) pray that your setup doesn't use Javashit or something equally
> >obnoxious
> >That, BTW, assumes that your reply will make it through the filters on
> >my side. The most obvious ones take care of HTML mail. As in "Dave
> >Null might care, I don't"...
> >
> >
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Brian Beattie | Experienced kernel hacker/embedded systems
[email protected] | programmer, direct or contract, short or
http://www.beattie-home.net | long term, available immediately.
"Honor isn't about making the right choices.
It's about dealing with the consequences." -- Midori Koto
On Wed, Jan 21, 2004 at 11:08:51PM +0000, Russell King wrote:
> Umm, I don't think that's entirely correct. There's a problem in the
> UK and EU with 100% open mailing lists - it's called data protection,
> where the owner of their personal data has certain rights. I believe
> that it is questionable whether such open lists can be run in the EU,
> or even by EU individuals.
>
> There appears to be two ways around this problem:
There is a third, actually:
3. Abandon running mailing lists in the E.U., and let them be run in
the other countries that don't have these sorts of insane laws.
"The Net interprets censorship as damage, and routes around it."
- Ted
P.S. Not that I think that concept of data protection is a bad thing,
but if it prevents open mailing lists --- if posting to a list isn't
considered an implicit permission to redistribute the mail and have it
be archived --- something is badly, badly, badly wrong.
P? to , 22/01/2004 klokka 11:56, skreiv David Ford:
> Considering that Bayesian filters are useless against the new spam that
> is proliferating these days, that's laughable. Spam now comes with a
> good 5-10K of random dictionary words.
The solution is obvious: learn a foreign language...
Cheers,
Trond
On Thu, 22 Jan 2004, Linus Torvalds wrote:
>
> Do a google search for "Mark V Shaney", and you should get some idea
> about this.
Oh, damn. I shouldn't have reminded myself. It's been so long since I did
this, that I had forgotten all about it, and I'm just happy that I'm
working from home, because if I'd been in an office, they'd have come to
take me away already, I was laughing so hysterically.
Some twisted soul put Mark V Shaney to work on a combination of Bible and
UNIX newsgroups. My favourite so far:
..
This is supported by Jesus's use of low cost eight bit micros and
small amounts of RAM. When you find salvation.
..
If God truly loves humankind then why does He create sinners? If
human is His creation, then who is the ultimate in all shells?
I know at one point Jesus said "no one may come to grips with the
cpio header blown away".
It speaks of the original ftpd.
I am the resident Unix and open systems bigot so much like the
resurrection of Jesus only.
..
...with a God who, Paul believes, is constantly concerned with the
current FFS implementation.
Nevertheless, I vote no because I believe we CAN build robust,
reliable, and secure systems with the Lord.
Mark V. Shaney
Hey, spammers, please start sending me those emails.
Linus
Unfortunately, 99% of emails that I have received in a foreign language
are spam. Spam has no language boundaries.
Trond Myklebust wrote:
>P? to , 22/01/2004 klokka 11:56, skreiv David Ford:
>
>
>>Considering that Bayesian filters are useless against the new spam that
>>is proliferating these days, that's laughable. Spam now comes with a
>>good 5-10K of random dictionary words.
>>
>>
>
>The solution is obvious: learn a foreign language...
>
>Cheers,
> Trond
>
>
On Thu, Jan 22, 2004 at 01:49:02PM -0500, David Ford wrote:
> I've been amusing myself once or twice a week by studying some of these
> emails. Due to the use of common words just like your email below,
> bayesian score is far too low (granting it a negative point value in SA).
>
> The problem is that properly trained is too fluid. It'd be far more
> achievable if I only talked geek.. Or if I only talked automotive. Or
> that I only talked medical. However, my "vocabulary" is far to varied
> to train a bayesian filter that the use of medical terms, computer
> terms, or a given topic, is taboo.
>
> It cuts the gray area far to close to the middle of the road and thus
> makes marking the email as probable spam useless. All I'm doing now is
> wasting CPU because in the end I'm doing the job of dealing with the
> spam myself.
Most of the spam using that technique get flagged on other
rules so they get scores of at least 8 but i've been
considering writing a rule to catch them and up the score.
Beyes is the wrong aproach for those random words from the
dictionary blocks.
Those i've seen seem to be a long string of words all longer
than 4 characters. A rule that gave a score of based on the
number of consecutive words longer than some number or
characters would catch those fairly easily. If i get
annoyed enough i may figure out how to write such a rule.
On the downside, once a rule becomes common to catch these
random word lists the spammers will start salting the lists
with short common words. Then when we get something that
somehow measures semantic content they will shift to random
random sentence construction and/or quotations.
What we need is a bounty on these scum. $1000 fine per
reported recipient with half going to the reporter would be
nice.
> David Lang wrote:
>
> >On Thu, 22 Jan 2004, David Ford wrote:
> >
> >
> >>Considering that Bayesian filters are useless against the new spam that
> >>is proliferating these days, that's laughable. Spam now comes with a
> >>good 5-10K of random dictionary words.
> >>
> >>
> >so we need to extend the Bayesian filters to deal with multi-word combos,
> >how many legit mail has those dictionary words in them? properly traind
> >their presence should help identify the spam.
> >
> >not that you will ever see this (other then through the list) as I won't
> >respond to your confirmation message.
> >
> >David Lang
> >
> >
--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: [email protected]
Remember Cernan and Schmitt
diff -ura xx-linux-2.6.1/MAINTAINERS xx-linux-2.6.1.p/MAINTAINERS
--- xx-linux-2.6.1/MAINTAINERS 2004-01-09 09:22:32.000000000 +0100
+++ xx-linux-2.6.1.p/MAINTAINERS 2004-01-22 15:44:08.000000000 +0100
@@ -680,7 +680,8 @@
DVB SUBSYSTEM AND DRIVERS
P: LinuxTV.org Project
-L: [email protected]
+M: [email protected]
+L: [email protected] (subscription required)
W: http://linuxtv.org/developer/dvb.xml
S: Supported
Johannes Stezenbach writes:
> Marcus Metzler wrote:
> > Linus Torvalds writes:
> > >
> > > Bug reports should be more important to the recipient than to the sender.
> > > Anything you do to make it more of a bother is WRONG.
> >
> > That's true. And being on the linux-dvb list I can't see why they
> > would want to give it as a maintainer address.
> > AFAIR, recently there was only one guy from convergence sending in
> > patches for the kernel anyway.
>
> Someone has to do the job, and IMHO Michael does it very well.
> Why do you think it would be a good idea if every patch would
> be submitted by a differnt person?
Who said that. If only one poerson submits, why not give his address
as maintainer.
>
> > But maybe they have some problems
> > within the company so they don't want to name one single person as a
> > maintainer.
>
> There isn't a single maintainer. The different drivers in linuxtv.org
> CVS are maintained by different people.
>
I think only the kernel tree is relevant for the MAINTAINER list.
> > Still they could give some joint mailing address. I don't think it is
> > necessary to discuss all patches on the list, especially since those
> > coming from people that are not on the list will probably be quite
> > trivial or more kernel related than DVB hardware related.
>
> I beg to differ. IMHO patches sent to the list from anyone are
> invaluable input, especially (but not only) if they solve
> actual bugs.
>
Anyone being able to patch something that really concerns the DVB
relevant hard or software will probably be on the list. Any other
patches will probably concern minor things or changes in the overall
kernel structure.
> Anyway, the *only* reason why the linux-dvb list is closed is
> the fear of spam. We are currently discussing whether to open
> linux-dvb or create a second, open list for patch submission
> only. It depends on the ability of the list admins to configure
> effective spam filters.
But how will the person submitting a patch get feedback about the
patch when he or she is not on the list. I don't think a list is a
good idea as a maintainer address.
The problem is not that the patches may or may not go to the list, you
can always forward them anyway. The problem is that some people not
familiar with DVB may have patches but are not that interested in
DVB that they would like to join the list.
So, you don't have to open the list. You just have to give a different
maintainer address, like [email protected].
Marcus
--
/--------------------------------------------------------------------\
| Dr. Marcus O.C. Metzler | |
| [email protected] | http://www.metzlerbros.de/ |
\--------------------------------------------------------------------/
|>>> I like GNU, but I couldn't eat a whole one <<<|
No, actually you can a) reply to my -plain text- email and yes...this is
plain text that has been trimmed, b) wait because I'm involved in this
discussion, I go to my queue and pre-auth people involved.
So it's fine for you to have filters of your choosing, but someone
else's filter's of their choosing are obnoxious...very social.
[email protected] wrote:
>Well, isn't it just fscking great... So in order to send you an email
>I have to
> a) cut the URL from your reply
> b) suspend mutt(1)
> c) type lynx '' and paste the damn thing in there
> d) pray that your setup doesn't use Javashit or something equally
>obnoxious
>That, BTW, assumes that your reply will make it through the filters on
>my side. The most obvious ones take care of HTML mail. As in "Dave
>Null might care, I don't"...
>
>
On Thu, Jan 22, 2004 at 04:15:00PM +0100, Michael Hunold wrote:
> diff -ura xx-linux-2.6.1/MAINTAINERS xx-linux-2.6.1.p/MAINTAINERS
> --- xx-linux-2.6.1/MAINTAINERS 2004-01-09 09:22:32.000000000 +0100
> +++ xx-linux-2.6.1.p/MAINTAINERS 2004-01-22 15:44:08.000000000 +0100
> @@ -680,7 +680,8 @@
>
> DVB SUBSYSTEM AND DRIVERS
> P: LinuxTV.org Project
> -L: [email protected]
> +M: [email protected]
> +L: [email protected] (subscription required)
> W: http://linuxtv.org/developer/dvb.xml
> S: Supported
Thanks for doing this, much appreciated.
Dave
On Thu, 22 Jan 2004, jw schultz wrote:
>
> Beyes is the wrong aproach for those random words from the
> dictionary blocks.
Bayes is not wrong per se, but doing bayes on pure word statistics is
wrong. It always was. People knew how it could be broken. The current rash
of spams is just the obvious way to do it.
> Those i've seen seem to be a long string of words all longer
> than 4 characters. A rule that gave a score of based on the
> number of consecutive words longer than some number or
> characters would catch those fairly easily. If i get
> annoyed enough i may figure out how to write such a rule.
Don't. That's easily broken too, as you realized yourself.
> What we need is a bounty on these scum. $1000 fine per
> reported recipient with half going to the reporter would be
> nice.
What you should aim for, and which should be much harder to break, is to
realize that random words that make no sense give a really unlikely
score when you build up a markov chain of them.
So to avoid the random words problem, do Bayes on the _chain_ of words
instead.
Now, you can try to overcome this by spamming with something that makes
"sense" from the markov chain standpoint, but by then that spam is going
to be hilarious. Once I start getting spams that are generated by markov
generators and read like "real" email, I might stop filtering them, just
because they are bound to be a lot of fun to read.
Have you played with Markov chains? What happens is that you don't just
build up a list of words and their likelihood of being spam or ham, you
build up a list of word _combinations_ and the likelihood of one
particular word following another one.
That's how a lot of the "random phrase" generators on the web work.
They can be absolutely hilarious, exactly because the sentences they
generate actually _almost_ make sense. Sometimes you get an almost
readable story, but one that reads like somebody having a bad trip and his
reality just shifted 90 degrees. (Usually the best stories come if the
training material is coherent, which email sadly usually isn't).
Do a google search for "Mark V Shaney", and you should get some idea
about this.
Linus
In article <[email protected]> you write:
>
>so we need to extend the Bayesian filters to deal with multi-word combos,
>how many legit mail has those dictionary words in them? properly traind
>their presence should help identify the spam.
Been there, done that. Take a look at spamprobe, it does two-word
bayes and is freakishly effective. I haven't had a single false
positive since the first week of training, and it's blocking well over
99% of all incoming spam. I no longer have a spam problem.
Scott
* [email protected] <[email protected]>:
> Well, isn't it just fscking great... So in order to send you an email
> I have to
> a) cut the URL from your reply
> b) suspend mutt(1)
> c) type lynx '' and paste the damn thing in there
> d) pray that your setup doesn't use Javashit or something equally
> obnoxious
No, you can simply use mutt's urlview (usually CTRL+b), this eliminates
a), b) and c) -- d) still stays true. BTW, elinks renders much better
than lynx.
--
Ralf Hildebrandt (Im Auftrag des Referat V a) [email protected]
Charite - Universit?tsmedizin Berlin Tel. +49 (0)30-450 570-155
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. +49 (0)30-450 570-916
Referat V a - Kommunikationsnetze - AIM. ralfpostfix
Please keep me in CC
> > Consider a spammer using your address and spams people. Say 25000 of those
> > use this method (Called challenge response authentication protocol). You'll
> ^ ^ ^ ^
> You name it: crap...
I was wondering if anyone would catch on to that =)
The acronym was chosen wisely.
--
Lab tests show that use of micro$oft causes cancer in lab animals
On Thu, Jan 22, 2004 at 11:56:33AM -0500, David Ford wrote:
> Considering that Bayesian filters are useless against the new spam that
> is proliferating these days, that's laughable. Spam now comes with a
> good 5-10K of random dictionary words.
>
> I use challenge-response and the only spam that gets to my inbox now
> comes from lists. I pre-listed all my buddies in my whitelist, only new
> senders that I'm not yet aware of have to go thru the challenge process.
>
> If you can't handle clicking on a link to authorize your email, then I'm
> not interested in your email. If that tiny few seconds of effort is a
> waste of your time, then writing your email to me was also a waste of
> your time.
Well, isn't it just fscking great... So in order to send you an email
I have to
a) cut the URL from your reply
b) suspend mutt(1)
c) type lynx '' and paste the damn thing in there
d) pray that your setup doesn't use Javashit or something equally
obnoxious
That, BTW, assumes that your reply will make it through the filters on
my side. The most obvious ones take care of HTML mail. As in "Dave
Null might care, I don't"...
Marcus Metzler wrote:
> Johannes Stezenbach writes:
> > Marcus Metzler wrote:
> > > Linus Torvalds writes:
> > > >
> > > > Bug reports should be more important to the recipient than to the sender.
> > > > Anything you do to make it more of a bother is WRONG.
> > >
> > > That's true. And being on the linux-dvb list I can't see why they
> > > would want to give it as a maintainer address.
> > > AFAIR, recently there was only one guy from convergence sending in
> > > patches for the kernel anyway.
> >
> > Someone has to do the job, and IMHO Michael does it very well.
> > Why do you think it would be a good idea if every patch would
> > be submitted by a differnt person?
>
> Who said that. If only one poerson submits, why not give his address
> as maintainer.
Because that one person might go on vacation, or other events keep
her from processing her mail. You would hate it if your patches would
get lost or unnecessarily delayed, don't you?
> But how will the person submitting a patch get feedback about the
> patch when he or she is not on the list. I don't think a list is a
> good idea as a maintainer address.
Acknowledged.
> The problem is not that the patches may or may not go to the list, you
> can always forward them anyway. The problem is that some people not
> familiar with DVB may have patches but are not that interested in
> DVB that they would like to join the list.
> So, you don't have to open the list. You just have to give a different
> maintainer address, like [email protected].
OK, we did that now, but for reasons stated above it is
a list, not a single person.
Johannes
I've been amusing myself once or twice a week by studying some of these
emails. Due to the use of common words just like your email below,
bayesian score is far too low (granting it a negative point value in SA).
The problem is that properly trained is too fluid. It'd be far more
achievable if I only talked geek.. Or if I only talked automotive. Or
that I only talked medical. However, my "vocabulary" is far to varied
to train a bayesian filter that the use of medical terms, computer
terms, or a given topic, is taboo.
It cuts the gray area far to close to the middle of the road and thus
makes marking the email as probable spam useless. All I'm doing now is
wasting CPU because in the end I'm doing the job of dealing with the
spam myself.
Yes, I did see this. I'm not so spiteful and actively pay attention to
my queue when having this type of correspondence.
David
David Lang wrote:
>On Thu, 22 Jan 2004, David Ford wrote:
>
>
>>Considering that Bayesian filters are useless against the new spam that
>>is proliferating these days, that's laughable. Spam now comes with a
>>good 5-10K of random dictionary words.
>>
>>
>so we need to extend the Bayesian filters to deal with multi-word combos,
>how many legit mail has those dictionary words in them? properly traind
>their presence should help identify the spam.
>
>not that you will ever see this (other then through the list) as I won't
>respond to your confirmation message.
>
>David Lang
>
>
Marcus Metzler wrote:
> Linus Torvalds writes:
> >
> > Bug reports should be more important to the recipient than to the sender.
> > Anything you do to make it more of a bother is WRONG.
>
> That's true. And being on the linux-dvb list I can't see why they
> would want to give it as a maintainer address.
> AFAIR, recently there was only one guy from convergence sending in
> patches for the kernel anyway.
Someone has to do the job, and IMHO Michael does it very well.
Why do you think it would be a good idea if every patch would
be submitted by a differnt person?
> But maybe they have some problems
> within the company so they don't want to name one single person as a
> maintainer.
There isn't a single maintainer. The different drivers in linuxtv.org
CVS are maintained by different people.
> Still they could give some joint mailing address. I don't think it is
> necessary to discuss all patches on the list, especially since those
> coming from people that are not on the list will probably be quite
> trivial or more kernel related than DVB hardware related.
I beg to differ. IMHO patches sent to the list from anyone are
invaluable input, especially (but not only) if they solve
actual bugs.
Anyway, the *only* reason why the linux-dvb list is closed is
the fear of spam. We are currently discussing whether to open
linux-dvb or create a second, open list for patch submission
only. It depends on the ability of the list admins to configure
effective spam filters.
Johannes
P? to , 22/01/2004 klokka 12:10, skreiv David Ford:
> Unfortunately, 99% of emails that I have received in a foreign language
> are spam. Spam has no language boundaries.
I beg to differ. Looking at my Bayesian filter dumps, I see something
quite different. All the words are English (if you will allow me to
count those various dyslectic spellings as being "English")...
I do on occasion get the odd oriental, French or Turkish(???) spam, but
the volumes are so small, it doesn't even register on the filters.
I have yet to receive any spam in my native Norwegian tongue. 8-)
Cheers,
Trond
David Ford is the typical challange responce user:
He writes emails, e.g. to me, but does not whitelist
me. I was fooled, answered his email, and now his
software wants me to confirm myself.
<insert some very unfriendly words here/>
I should experiment with newsreaders to read mail,
that way I could plonk him.
Andreas
Considering that Bayesian filters are useless against the new spam that
is proliferating these days, that's laughable. Spam now comes with a
good 5-10K of random dictionary words.
I use challenge-response and the only spam that gets to my inbox now
comes from lists. I pre-listed all my buddies in my whitelist, only new
senders that I'm not yet aware of have to go thru the challenge process.
If you can't handle clicking on a link to authorize your email, then I'm
not interested in your email. If that tiny few seconds of effort is a
waste of your time, then writing your email to me was also a waste of
your time.
Getting well over 900 spams a day on average, almost double on mondays,
just isn't my cup of tea. There is no one solution to spam. I
pre-filter with spamassassin using all it's tools, anything scoring high
automatically gets /dev/nulled. Those include bayesian, pattern
matches, DNSBL, etc. Next I attempt to filter viruses and the like. The
remainder goes through TMDA.
Spamassassin cuts it down to less than 100 typically, and of that, about
50 are on the border. TMDA takes care of the rest. The majority of
spam making it through SA is the dictionary attack spam. My
retro-impact on spam is minimized.
It's getting really annoying because spammers are taking input emails
like LKML and making word lists out of the emails.
Hmm, 900 spams in my mailbox, or half a dozen due to lists. I'll take
the second.
David
Jes Sorensen wrote:
>>>>>>"Zan" == Zan Lynx <[email protected]> writes:
>>>>>>
>>>>>>
>
>Zan> On Wed, 2004-01-21 at 16:40, Andreas Jellinghaus wrote:
>
>
>>>On Wed, 21 Jan 2004 21:44:37 +0000, Mike Fedyk wrote: > What do you
>>>think about individual email (non-list) using a confirmation >
>>>based spam blocking system.
>>>
>>>for personal email it is plain asocial. it tells me that a person
>>>does not want to receive mail from me.
>>>
>>>
>
>Zan> For me, that isn't what it says at all. It tells me that he or
>Zan> she is tired of receiving and sorting all of the spam every day.
>Zan> Since I feel exactly the same way about spam, I cooperate and
>Zan> reply with a confirmation.
>
>I've had people pull the authentication game on me before. I just
>stopped replying to them, waste of my time.
>
>Fixing the spam problem is a lot easier without losing contact with
>all your friends in the proces:, train your Bayesian filters and be
>done with it. Mine were a mess, deleted all the data and fed 10 days
>of spam and some proper mail through sa-learn. Since then I have seen
>1 spam make it through during the last week, it used to be 20-40/day
>(and some 200-300/day caught by the filters).
>
>
>
David Ford <[email protected]> writes:
> If you can't handle clicking on a link to authorize your email, then
> I'm not interested in your email.
Well, I think I can't handle clicking on any link most of the time.
You know, UUCP doesn't carry HTTP, while it transfers mail quite
effectively.
Yes, I can PPP-connect over the dial-up, click on your link, disconnect.
Costs money and time. Multiply this by everyone who demands such
a click.
--
Krzysztof Halasa, B*FH
On Thu, 22 Jan 2004 14:58:54 -0800 (PST)
Linus Torvalds <[email protected]> wrote:
> Have you played with Markov chains? What happens is that you don't just
> build up a list of words and their likelihood of being spam or ham, you
> build up a list of word _combinations_ and the likelihood of one
> particular word following another one.
>
> That's how a lot of the "random phrase" generators on the web work.
>
> They can be absolutely hilarious, exactly because the sentences they
> generate actually _almost_ make sense. Sometimes you get an almost
> readable story, but one that reads like somebody having a bad trip and his
> reality just shifted 90 degrees. (Usually the best stories come if the
> training material is coherent, which email sadly usually isn't).
This reminds me of the literary work done by William S. Burrough and his
infamous "cut ups". Similar result for the reader.
On Thu, Jan 22, 2004 at 01:20:26PM -0500, Brian Beattie wrote:
> On Thu, 2004-01-22 at 12:41, David Ford wrote:
> > No, actually you can a) reply to my -plain text- email and yes...this is
> > plain text that has been trimmed, b) wait because I'm involved in this
> > discussion, I go to my queue and pre-auth people involved.
> >
> > So it's fine for you to have filters of your choosing, but someone
> > else's filter's of their choosing are obnoxious...very social.
> >
> > [email protected] wrote:
> >
>
> Well it's all very fun, but I expect the spammers to figure out
> challange and response soon enough.
very easy : they will send the mail with a reply-to set to an open ML, then
collect challenges from this ML and send their shit back to you. Imagine a
reply-to set to lkml ? all subscribed people allowed to send their challenges
to the list, and the spammer reading them from marc.theaimsgroup.com :-/
Cheers,
Willy
>>>>> "David" == David Ford <[email protected]> writes:
David> Considering that Bayesian filters are useless against the new
David> spam that is proliferating these days, that's laughable. Spam
David> now comes with a good 5-10K of random dictionary words.
Well if thats the case, then I wonder why the random word spams
stopped getting through my filters after I spent the appropriate time
training my Bayesian filters. I must have done something that made the
spammers laugh so hard they didn't bother spamming me anymore.
David> If you can't handle clicking on a link to authorize your email,
David> then I'm not interested in your email. If that tiny few seconds
David> of effort is a waste of your time, then writing your email to
David> me was also a waste of your time.
Well now we're getting to the laughable part: by expecting people to
'click on a link you obviously assume that everybody uses a GUI mail
client. Guess what, I don't have the patience for that. So guess what,
having to launch a web browser and copy-pasting a random link into it
in order to send you email makes me uninterested in mailing you in the
first place.
David> Getting well over 900 spams a day on average, almost double on
David> mondays, just isn't my cup of tea. There is no one solution to
David> spam. I pre-filter with spamassassin using all it's tools,
David> anything scoring high automatically gets /dev/nulled. Those
David> include bayesian, pattern matches, DNSBL, etc. Next I attempt
David> to filter viruses and the like. The remainder goes through
David> TMDA.
Maybe you should consider using it properly, guess what it does work.
David> Hmm, 900 spams in my mailbox, or half a dozen due to lists.
David> I'll take the second.
600 spams in my spambox over the last 48 hours, _1_ in my inbox, guess
that speaks for itself.
'nuff said, back to spending time on real issues, such as fixing
kernel bugs.
Jes
On Thu, 22 Jan 2004, David Ford wrote:
> Considering that Bayesian filters are useless against the new spam
> that is proliferating these days, that's laughable. Spam now comes
> with a good 5-10K of random dictionary words.
Right, but random words result in strange couplings of words.
Statistical filters should be working on phrases, not just individual
words. So to the statistical filter random words will just be
meaningless noise, neither an indicator of goodness nor of spamness.
(unless a spammer reuses a boilerplate 'random word' section - in
which case it'll be an indicator of spamness).
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
warning: do not ever send email to [email protected]
Fortune:
Men of lofty genius when they are doing the least work are most active.
-- Leonardo da Vinci
On Thu, 22 Jan 2004, Linus Torvalds wrote:
> > Beyes is the wrong aproach for those random words from the
> > dictionary blocks.
>
> Bayes is not wrong per se, but doing bayes on pure word statistics is
> wrong. It always was. People knew how it could be broken. The current rash
> of spams is just the obvious way to do it.
I use spamprobe - spamprobe.sf.net - which works on arbitrary length
phrases. (2 is the reccomended maximum phrase length).
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
warning: do not ever send email to [email protected]
Fortune:
System going down at 1:45 this afternoon for disk crashing.
On Thu, 22 Jan 2004, Brian Beattie wrote:
> Well it's all very fun, but I expect the spammers to figure out
> challange and response soon enough.
It works while the level of use is insignificant. If it ever were to
have significant adoption the spammers would indeed then adapt.
Challenge/Response is an arms race waiting to happen, with a
turing-test every time you send an email as its destiny.
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
warning: do not ever send email to [email protected]
Fortune:
Demographic polls show that you have lost credibility across the board.
Especially with those 14 year-old Valley girls.
On Thu, 22 Jan 2004, David S. Miller wrote:
> On Thu, 22 Jan 2004 14:58:54 -0800 (PST)
> Linus Torvalds <[email protected]> wrote:
>
> > Have you played with Markov chains? What happens is that you don't just
> > build up a list of words and their likelihood of being spam or ham, you
> > build up a list of word _combinations_ and the likelihood of one
> > particular word following another one.
> >
> > That's how a lot of the "random phrase" generators on the web work.
> >
> > They can be absolutely hilarious, exactly because the sentences they
> > generate actually _almost_ make sense. Sometimes you get an almost
> > readable story, but one that reads like somebody having a bad trip and his
> > reality just shifted 90 degrees. (Usually the best stories come if the
> > training material is coherent, which email sadly usually isn't).
>
> This reminds me of the literary work done by William S. Burrough and his
> infamous "cut ups". Similar result for the reader.
jwz's DadaDodo program <http://www.jwz.org/dadadodo/> was inspired by
Burroughs' cut ups, and it does this sort of Markov chain-based
generation....
I'm not so sure that Markov chain analysis for spam filtering will work well
in practice for email, though, particularly when dealing with international
email. I don't write / read German or French well, but I can write in them
enough that a German / French speaker can figure out what I'm trying to say,
maybe. My German in particular is bad enough that it probably wouldn't fit
Markov chain models of how someone fluent in high German would write,
though. Similarly, look at some of the fractured English emails that appear
on this list. I can understand them, but a Markov model relaxed enough to
allow them will also include a lot of random spam....
later,
chris
Hi!
> > Beyes is the wrong aproach for those random words from the
> > dictionary blocks.
>
> Bayes is not wrong per se, but doing bayes on pure word statistics is
> wrong. It always was. People knew how it could be broken. The current rash
> of spams is just the obvious way to do it.
You want to do it on trigrams (groups of three works). Anything longer
than trigrams is not likely to be effective.
> > What we need is a bounty on these scum. $1000 fine per
> > reported recipient with half going to the reporter would be
> > nice.
>
> What you should aim for, and which should be much harder to break, is to
> realize that random words that make no sense give a really unlikely
> score when you build up a markov chain of them.
>
> So to avoid the random words problem, do Bayes on the _chain_ of words
> instead.
>
> Now, you can try to overcome this by spamming with something that makes
> "sense" from the markov chain standpoint, but by then that spam is going
> to be hilarious. Once I start getting spams that are generated by markov
> generators and read like "real" email, I might stop filtering them, just
> because they are bound to be a lot of fun to read.
Even if you get 100 of them per day?
I'm doing language modeling in school, and generating text that "looks
like meaningfull" to everyone but human is too easy.
[Take your favourite voice-recognition software, and speak in another
language to it. It will generate plausible-looking sentences at the
output. This could be easily automated.]
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
On Thu, Jan 22, 2004 at 10:35:54AM -0800, David Lang wrote:
> On Thu, 22 Jan 2004, David Ford wrote:
> > Considering that Bayesian filters are useless against the new spam that
> > is proliferating these days, that's laughable. Spam now comes with a
> > good 5-10K of random dictionary words.
I'm curious what Bayesian filters you're using. The filter I use
(bogofilter.sf.net) regularly catches and properly categorizes these
spams.
A good Bayesian spam filter isn't nearly as susceptible to random words as
some people think. Words that are likely to be spam (along with words that
are frequently "ham") are given _exponentially_ more weight than other
words. The only way a group of random words is likely to sway the score is
if it happens upon enough "ham" words to outweigh the message's "spam"
words, and there is just as much chance of randomly picking a "spam" word
as there is of randomly finding "ham". In any case, you'd need random word
blocks _much_ bigger than 5-10k to make it statistically likely of catching
"ham" tokens.
> so we need to extend the Bayesian filters to deal with multi-word combos,
> how many legit mail has those dictionary words in them? properly traind
> their presence should help identify the spam.
If filters start looking for grammatically correct phrases or sentences,
the spammers will just start pasting in random sections of books or web
pages. Multi-word and "markov chains" tests will only be helpful if the
filter also does proper weighting of the results. And, since my filter
works fine today, I'm in no rush to upgrade to a more complex one.
-Kevin
--
---------------------------------------------------------------------
| Kevin O'Connor "BTW, IMHO we need a FAQ for |
| [email protected] 'IMHO', 'FAQ', 'BTW', etc. !" |
---------------------------------------------------------------------
On Sat, 24 Jan 2004, Kevin O'Connor wrote:
>
> A good Bayesian spam filter isn't nearly as susceptible to random words as
> some people think. Words that are likely to be spam (along with words that
> are frequently "ham") are given _exponentially_ more weight than other
> words.
Especially if the "random words" in the spam end up being weighted by real
frequency, you just _cannot_ use single-word bayes filters on it. Or if
you do, you'll eventually have those words either being neutral, or (worst
of all cases) you'll have real mail be marked as spam after having
aggressively trained the filter for the spams.
It might not be that big of a deal especially if you have a fairly narrow
scope of emails in your ham-list, but people who get mail from varied
sources _will_ get screwed by this, one way or the other.
Of course, the spam filters will catch on to other things. I find that the
DNS lookups take care of most of it, to the point where the other rules
don't even much matter.
Linus
On Sat, Jan 24, 2004 at 01:12:34PM -0800, Linus Torvalds wrote:
>
> On Sat, 24 Jan 2004, Kevin O'Connor wrote:
> >
> > A good Bayesian spam filter isn't nearly as susceptible to random words as
> > some people think. Words that are likely to be spam (along with words that
> > are frequently "ham") are given _exponentially_ more weight than other
> > words.
>
> Especially if the "random words" in the spam end up being weighted by real
> frequency, you just _cannot_ use single-word bayes filters on it. Or if
> you do, you'll eventually have those words either being neutral, or (worst
> of all cases) you'll have real mail be marked as spam after having
> aggressively trained the filter for the spams.
A "random" word will not occur frequently enough in spam messages (when
measured over a large sample of spam) to become a "spam" token or to
adversely effect it becoming a "ham" token. (If it did, then it would be a
good indicator of "spam", and spammers wouldn't be using it in their
"random" word blocks.) Also, the algorithms in the filter make sure that
words don't become spam tokens just because one receives more spam than ham
(ie. you can't train the filter to over aggressively catch spam).
Don't get me wrong - I agree that multi-word bayes filters can be more
precise. But, I don't see them as being significantly different from
single word filters -- they're susceptible to the same "attack" you outline
above (with blocks of random sentences) -- and they will have larger
dictionaries and require more training time.
> It might not be that big of a deal especially if you have a fairly narrow
> scope of emails in your ham-list, but people who get mail from varied
> sources _will_ get screwed by this, one way or the other.
A lot of testing has been done on these filters (see for example
http://spambayes.sourceforge.net/background.html) with a very large email
corpus. If you haven't looked at bayes filters recently, or have only been
looking at the simplistic ones, then I think you might have better luck
trying again.
-Kevin
--
---------------------------------------------------------------------
| Kevin O'Connor "BTW, IMHO we need a FAQ for |
| [email protected] 'IMHO', 'FAQ', 'BTW', etc. !" |
---------------------------------------------------------------------
>
> David> Getting well over 900 spams a day on average, almost double on
> David> mondays, just isn't my cup of tea. There is no one solution to
> David> spam. I pre-filter with spamassassin using all it's tools,
> David> anything scoring high automatically gets /dev/nulled. Those
> David> include bayesian, pattern matches, DNSBL, etc. Next I attempt
> David> to filter viruses and the like. The remainder goes through
> David> TMDA.
>
> Maybe you should consider using it properly, guess what it does work.
>
> David> Hmm, 900 spams in my mailbox, or half a dozen due to lists.
> David> I'll take the second.
>
> 600 spams in my spambox over the last 48 hours, _1_ in my inbox, guess
> that speaks for itself.
Why some people gets so much spam ??, I'm getting like 20-50 at
most(usually less), and I post to slashdot, i'm suscribed to several
mailing lists. I have 3 mail accounts, all of the for years, more than
7.
I get about 1 or 2 spams on my inbox, and sa-learn has worked quite well
on my box.
I have never received a "random words" spam.
It's kind of weird.
Do the pople that gets ~600 spamd daily get a lot of repeated messages
or they are all different ?
Max