Anoying Peter Whalley Spam messages.

[Scot L. Harris]

On Tue, 2005-04-05 at 00:59, Craig White wrote:
> On Mon, 2005-04-04 at 16:17 -0400, Scot L. Harris wrote:
> > On Mon, 2005-04-04 at 15:34, David Hoffman wrote:
> 
> > But I don't understand your last sentence.  Not sure if you are talking
> > about the recipient of a TMDA message or the user that implemented it?
> > 
> > Either way you can achieve similar results that reduce spam by 99%+ by
> > using greylisting and spamassassin with a well trained bayes database
> > which does not require the sender to do anything new or different.  Yes,
> > at some point the spammers can start retrying messages to get around
> > greylisting but they have not done so yet and when they do it will cost
> > them more to maintain the list of messages to retry, consuming more
> > resources on the bots they are using, making it more likely that they
> > will be discovered.  It also slows down the delivery of more spam to
> > someone else since they now need to send the message at least twice
> > possibly more than that.  
> > 
> > Because of this I don't think the spammers will change anytime soon as
> > it will start costing them to much to send spam, which is the basic
> > idea.
> > 
> > And if they do start retrying messages in large numbers greylisting can
> > utilize various RBLs which will most likely catch such spammers during
> > the wait period and the next time they come in you reject their messages
> > due to the RBLs that were populated from spam collectors.
> ----
> FWIW - RBL's sometimes blacklist rather idiotically and cause issues and
> it can be hard to get off the lists once on it.
> 
> That said though, I agree with your thoughts on RBL's
> 

I think the trick here is to use the right RBL.  Some do have a rather
draconian set of rules and are better avoided due to that.

In combination with spamassassin the RBL used would need to be specially
setup to collect data from dedicated spam traps only.  The idea being
that only spam is received at those spam traps and appropriate checks
are made on what is received before the sender gets listed.

> We have been having a rather interesting discussion on CentOS mail list
> - I will take credit for starting the thread called 'Postfix tightening'
> and what I thought seemed to be a rather simple question turned into a
> real eye opening experience for me as I have found postfix to be an
> extremely granular system - much more configurable and comprehendible
> than sendmail and I'm actually starting to wonder if I am needing
> greylisting at all in a setup that includes MailScanner, Clamav,
> SpamAssassin and a well planned set of rules within Postfix.
> 

IMHO the real benefit of greylisting is that your MTA turns away 90%+ of
the spam before your system wastes any resources on trying to analyze
the message.  Reduction in system resource usage is dramatic.  One of
the main reasons I implemented greylisting to start with was the
periodic spam storms that would just about swamp the email server due to
the load placed on spamassassin.  While spamassassin does an excellent
job it can become resource intensive when you are literally receiving
thousands of messages an hour and most of them are spam.

> The thing that opened my eyes is the amazing amount of qualifications
> that you can put into smtp-accept/reject within postfix itself.
> 
> My disillusionment with greylisting came last week when I had to explain
> to a fairly important end user why a particular person couldn't get an
> email through...search the logs I found that this person tried 3 times
> on 3 separate occasions to send her an email. Evidently the smtp server
> doesn't accept the mail for delivery until the end point accepts the
> mail and she got a tempfail (450) and gave up each time. Her system gave
> her a different smtp server each time so each attempt was separately
> greylisted for 1 minute and each time, she gave up before the 1 minute
> passed. 
> 

One known issue with greylisting are some email server farms where they
may retry the email from different servers each time.  Sounds like what
you hit.  This is normally handled by whitelisting those server farms so
those messages are not delayed by greylisting.  Most greylisting
implementations I have looked at have a pre-populated whitelist that
includes known email server farms for well known companies.  If you have
identified another one like this it would be appropriate to add them to
the whitelist so there would be no problems going forward.

In practice I have found this to be a rare occurrence.  And once
identified it is easy to resolve.
  
> Though I have installed greylisting on a number of systems that I handle
> for my clients, I am seriously watching the impact - and fine tuning the
> rule sets in Postfix and will probably turn off greylisting at some
> point to see if it ultimately makes a difference. My thinking is that
> the RBL's and an expansive use of Postfix rules will pick off the same
> low hanging fruit that greylisting handles.
> 

Could be.  I have found that greylisting is largely a setup and forget
solution.  Very little on going maintenance is required.  Would be
interesting to see if the rule sets used by postfix are equally low
maintenance or not.  I also suspect you will see an increase in the
system resources needed using postfix without greylisting.  

> As for bayes training - I'm not convinced that users are going to
> actively participate and I'm struggling to find a way that is simple
> enough for them.

One company that I implemented spamassassin for uses a system wide bayes
database.  There is a designated user that monitors the spam bucket and
periodically runs sa-learn to keep the bayes database up to date on
confirmed spam and occasionally on a sampling of ham messages.  This
setup seems to work quite well.  Since implementing greylisting for them
the amount of spam per week that spamassassin deals with is minimal. 
Prior to that they were handling anywhere from 5000 to 8000 spam
messages a day.  Now they are down to 5 or 10 spam messages a day which
spamassassin processes.  


> ----
> > 
> > The problem is solvable.  Of course the best solution is to hunt down
> > the few people that actually buy stuff from the spam email and take
> > their computers away and have all ISP's ban them for life.  Then there
> > will be no reason for spam anymore.  :)
> ----
> it's simply not possible to idiot proof things. The problem is ALWAYS
> gonna be - that some people don't inherently distrust things in their
> mailbox - why should they? It is probably their nature to trust things.
> 

Yeah, nature seems to be able to evolve better idiots all the time.  :)

> For example - many of the phishing schemes have an official looking
> email from a bank, completely with logos and reasonable looking email
> address. PT Barnum was a man ahead of his time.

Maybe there should be a license process similar to drivers licenses that
prevents people from using a computer that have not been trained to not
respond to random junk that shows up in their mailbox.  And if they are
caught responding to spam they get re-trained using electric shock,
tests are run by sending them spam/phish messages and if they open them
up hit them with 50,000 volts.  :)


-- 
Scot L. Harris
webid at cfl.rr.com

Garbage In, Gospel Out