[K12OSN] OT: just reduced spam by 95% with Free Software

Mon Jan 29 17:54:41 UTC 2007

You bring up interesting and relevant points.  Responses are below. 

BTW, earlier this morning, I posted a paper that happens to address
several of the concerns you point out.  It is in .RTF format.  Due to
its size, it is awaiting moderator approval.

--TP
_______________________________
Do you GNU!?
Microsoft Free since 2003 <http://www.gnu.org/>--the ultimate antivirus
protection!

Michael Blinn wrote:
> Yikes. This is a really important problem - I hate it when my adult
> users come to me telling stories of a particularly graphic or nasty
> email they've received about 'hot teens' or other trash. Giving
> students email accounts.. well that's another level of responsibility.
>
> I've been spamfighting for awhile, and can give very good reasons why
> these solutions may not be the best out there.
>
> First, with the BSD spamd box, well that's simple. That's another box.
> Also, it's not going to save you any bandwidth if you have a small
> pipe.. the spam still comes IN to the spam box before being
> processed.. adding RTBL checks on top of this will much more quickly
> use your available bandwidth as signatures are checked for every
> incoming email. The benefits just don't warrant the additional level
> of complexity.. and good luck if it breaks.
>

That's the point.  You stop the vast majority of spam from ever getting
downloaded to your machine *in the first place.*

> (There are certain log analyzers that will add a ipchains/iptables
> entry to block access to a particular IP or block of IPs when multiple
> spams within a time threshold are found, and this would be the only
> way that you could lower your bandwidth. There's a program that does
> it automagically, something like 'vispan' or 'vespar' or something.. I
> cannot recall. I did not like this solution because it required a
> certain log format, and I also manage my own iptables chains, blocking
> out entire /8 blocks when necessary.)
>

Actually, no, throwing up iptables rules isn't the only way to lower the
bandwidth that spam uses.

> Spamassassin and its ilk (http://www.mailscanner.info is a nice
> spamassassin package that does it all for you) rely heavily on perl
> and real-time blacklists. Perl will bring a fast server to its knees
> when the volume is cranked up to 11... and, judging by the way spam
> has jumped in the last 5 years, it's only going to get worse. I'd like
> to save my CPU and memory for my terminals, thank you very much.
> Real-time blacklists are just wrong in principle. If you've ever been
> mistakingly put on one, or been behind a /24 subnet because of another
> spammer, you'll understand why they're inherently evil.
>
I agree.  That's the point of OpenBSD spamd and why it was written.  It
avoids all that heavy-duty processing overhead, specifically because of
how it works.  It is not SpamAssassin.

>
> I found a great package called 'DSpam'
> (http://dspam.nuclearelephant.com) that I've been using for a few
> years now. It's 100% adaptive. After the first thousand or so emails
> that come in (yes, you can train/seed it for quick results), the thing
> is basically trouble-free. The author, Johnathan Zdziarski, has
> written some great books about syntactical pattern-recognition and the
> algorithms behind them - He's a Smart Guy. The community contributes
> fixes/enhancements. It's rock-solid, with plenty of time between
> releases. It's written in C, with a plenthora of backend storage
> mechanisms from which to choose. I've got it hooked into procmail and
> dovecot (my IMAP server) using a plugin so that all incoming mail
> marked spam is delivered to a Spam folder, and removed if left for > 2
> weeks. Dragging into and out of the IMAP folder causes automagic spam
> retraining. If you prefer, there's also a web interface each user can
> access to retrain, see pretty graphs, etc. I've got clamd hooked into
> it for virus checking, though one could use Sophos or any other. It's
> bulletproof.
>
Great!  The more anti-spam solutions out here, the better.  Given our
clientele (schoolchildren), we should indeed be discussing as many F/OSS
solutions as we can.

> After trying many others others, I've found my spam happy place in
> DSpam. Go to the website, read the description. You'll save so much
> time in the end, by never worrying about spam again, and your server
> won't be struggling under the weight of perl scripts or your bandwidth
> from checking every email against a RTBL. I guarantee you that if you
> mess with spamassassin, you'll see some results, but after some months
> of use, you'll be dealing with spam on a weekly, if not a daily basis.
>
Again, that's a good thing.  As one who despises spam email, I myself
will check it out.

> If anyone would like some additional information on how my system is
> set up, I'd be glad to offer assistance.
>
> Cheers,
>   Michael Blinn
>
>
> Nils Breunese wrote:
>> Terrell Prudé Jr. wrote:
>>
>>> Nils Breunese wrote:
>>>> Terrell Prudé Jr. wrote:
>>>>
>>>>> I will first admit that this is somewhat off-topic from K12LTSP.
>>>>> That said, schools could benefit from this.  This is definitely
>>>>> applicable for those of you who asked about using K12LTSP as an email
>>>>> server for your students.
>>>>>
>>>>> We all know about the spam problem.  Well, over this last week, I
>>>>> have been playing with OpenBSD's spamd as a possible solution.
>>>>> Basically, I put the spamd box in front of my (yes, GNU/Linux) email
>>>>> server.  I have now reduced the spam count in my inbox from close to
>>>>> 200 a day down to...five.  FIVE.  This is without false positives.  I
>>>>> have verified that by studying my spamd logs all week and comparing
>>>>> them to my real email server's logs.
>>>>>
>>>>> For those of you with small pipes to the Internet, this is
>>>>> *definitely* something you might want to consider.  It saves you some
>>>>> bandwidth.
>>>>>
>>>>> If anyone's interested, let me know.
>>>>
>>>> If your K12LTSP server can handle it, why not just run spamd (which is
>>>> just the SpamAssassin daemon, right?) on your K12LTSP server directly?
>>>> I don't think there is a difference between OpenBSD's spamd and Fedora
>>>> Core's spamd, is there?
>>>
>>> Good question.  Actually, there is a big difference, and a lot of
>>> people
>>> confuse OpenBSD's spamd with that of SpamAssassin, since the name of
>>> the
>>> executable happens to be the same.  They are in fact different programs
>>> with different strategies of dealing with spam.  They are not
>>> replacements for each other; rather, they are complements.
>>
>> Ah, I found it [0]. Looks like 'just a collection of blacklists'. I'm
>> not sure I'd setup a separate box with another OS just for that
>> (different if you're familiar with OpenBSD), but yes, you might want
>> to offload the load that filtering spam takes to another box if your
>> K12LTSP needs all its power to serve your thin clients.
>>
>> I don't have a K12LTSP server at the moment, but I use SpamAssassin
>> with dcc [1], pyzor [2] and razor [3] to fight spam on my servers
>> (running Plesk with qmail as MTA) which works nicely. You could add
>> MAPS zones (like spamhaus.org's zen.spamhaus.org, etc.) as blocklists
>> or plug them into SpamAssassin for extra scoring, but make sure you
>> know which ones you're using and why (they all have different
>> policies and some include others). Also, make sure to keep your list
>> of zones up to date, because a MAPS zone that no longer exists can
>> delay your mail delivery pretty bad.
>>
>> If you happen to run servers running Plesk check out
>> atomicrocketturtle.com's free Project Gamera [4] if you'd like to
>> setup a dedicated spam and virus filtering gateway.
>>
>> Nils Breunese.
>>
>> [0] http://www.openbsd.org/spamd/
>> [1] http://www.rhyolite.com/anti-spam/dcc/
>> [2] http://pyzor.sourceforge.net/
>> [3] http://razor.sourceforge.net/
>> [4] http://www.atomicrocketturtle.com/Joomla/content/view/77/29/
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> K12OSN mailing list
>> K12OSN at redhat.com
>> https://www.redhat.com/mailman/listinfo/k12osn
>> For more info see <http://www.k12os.org>
>
> -- 
>  
>
> CONFIDENTIALITY NOTICE:
> This message, and any attachments that may accompany it, contain information that is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential, or otherwise exempt from disclosure under applicable law. If the recipient of this message is not the intended recipient, any disclosure, copying, or other use of this communication or any of the information, which it contains is unauthorized and prohibited.  If you have received this message in error, please notify the original sender by return mail and delete this message, along with any attachments, from your computer. Thank you.  
> ------------------------------------------------------------------------
>
> _______________________________________________
> K12OSN mailing list
> K12OSN at redhat.com
> https://www.redhat.com/mailman/listinfo/k12osn
> For more info see <http://www.k12os.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/k12osn/attachments/20070129/788d03a1/attachment.htm>