[geeklog-devtalk] surbl.org SpamX implentation

[Keith Geffert]

Hello all,

First post to the list, and btw the mailman subscribe webform gives out an 
internal server error.  Doesn't seem to affect the confirmation emails back 
and forth though.

I've scanned and searched the dev / dev-talk lists for information about 
implementing a surbl functionality into GL, more specifically as an add on 
to the SpamX plugin, however I only came up with this:
http://lists.geeklog.net/pipermail/geeklog-users/2004-May/001027.html

I spent this weekend on this and I've got a working prototype and after 
looking at the SpamX plugin it shouldn't be too difficult integrating it 
(the spamx stuff is really well designed liked it alot).

I'd like to see if any of your are interested in looking at it and trying 
it out. I'm still in the process of converting much of our LUG site over to 
GL so I don't have an active implementation to hit it against.

Features:

Implements the major details of the suggested features for www.surbl.org
Scans any input for url looking strings (very liberal)
Verifies hosts against the list of proper TLD's
Verifies 2nd level TLD against list linked at www.surbl.org
Handles Ip addresses
Handles URL redirection recursively with configurable max depth
Support for Header and Meta tag redirection.
Considers the comment bad / spam if the link redirection depth is hit (10)
Won't bounce a comment if a link doesn't resolve or is otherwise dead.

Todo:

Provide configuration interface -- first release, you'll have to edit the 
config options by hand

Store TLD lists in the database
Provide a way to update those lists
Store bounced comments for later perusal and submission to bad link db?
Check user IP against it?  Don't know how well that'd fair.
etc.. etc.. etc...



Anyway.  Let me know if any of you are interested.  I don't know what the 
effectivness will be.  I would assume the comment spammers would also have 
to be sending spam email with the same URL's for them to make it into the 
surbl db.  BUT that doesn't mean you can't run your own :)