Subject: RE: Re[2]: Syntax question: smtp-policy.spam

From: Marek Kowal <[email-masked]>
Date: Fri, 14 Jun 2002 11:37:25 +0200

> With the recent exposion of spam and virus traffic, I became 
> concerned about content filtering.  But I am wandering if extra 
> fork()/exec() for every message is practical...  

Exactly. This opens the computer to DoS attack - just send many letters and
you've got fork-bomb. Even very good linuxes act slow above 50 forks()/sec.

> We currently process well over a million messages per day.  
> Ideally, content filter might be built in smtpserver binary, with 
> filtering rules dynamically reloadable the same way as current policy
database.

I doubt it should be in the binary - it will grow big and maintenance will
become real burden. The best way - I believe - is to create separate
program, which listens on the socket for incoming letters, parses them, and
then answers (in unerstandable way) - what should be done to the letter. it
might also return reprocessed letter (i.e. with viruses removed). once the
interface is well documented, anyone can write his/her own content filter,
the zmailer just contacts it.

In this way the program gets separated from smtpserver, you do not have
forks, and you can develop filter in any language (probably starting off
with perl, but ending up in pure C for sites requiring bigger efficiency). 

Also, there is question, where the filter should be installed - in the
smtp-server (better response to still-connected SMTP client, but bigger
openess to DoS attack (smtpserver works on-line!), or in router (off-line,
but in case of problems no way to contact SMTP client).

Cheers,
Marek.
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to [email-masked]

---

From: Eugene Crosser <[email-masked]>
Date: Fri, 14 Jun 2002 14:15:34 +0400 (MSD)

On Fri, 14 Jun 2002 11:37:25 +0200 Marek Kowal <[email-masked]> wrote:

> > filtering rules dynamically reloadable the same way as current policy database.
> 

This is not a viable option.  If the checker is one process, it may become
a bottleneck. If there are several, you'll have to schedule requests, which
is expensive and fragile.  Performing the check within the space of a
single process looks more practical.

--
Eugene Crosser, head of Internet Applications section, +7 501 787 1000
ROL, Teleross, Golden Telecom, http://user.rol.ru/~crosser/

-

---

From: Matti Aarnio <[email-masked]>
Date: Fri, 14 Jun 2002 13:27:11 +0300

> you've got fork-bomb. Even very good linuxes act slow above 50 forks()/sec.

  If you look at how the contentfilter interface is written at ZMailer,
  non-forking is a big thing in it.   All this presuming that incoming
  messages are not coming in separate connections...

  A multithreaded smtp input engine in which single process instance
  handles lots of incoming sessions might alleviate some of the problems.
  If you look at the smtpserver code, it has quite a lot of things in
  "SS" state variable structure, and very little in scattered global
  state variables.   Adding multiplexed access to permanently running
  router server process(es), and content filter(s) (pools of resources
  if single instances are bottlenecks.)

  However things won't be "low technology approach", once threads step
  into the picture..

  (And once the smtpserver is threaded, scheduler will actually get more
   benefits from the same treatment.)

> the zmailer just contacts it.

  Definitely a binary engine, which understands various things about
  the message -- like MIME structure, decodes BASE64 encoded parts
  in the analysis, etc.

  Having smart ways to define rules for downloading & internal compilation
  would be bonus.  (E.g. RE patterns downloaded, and on-demand optimized
  for re-interpretation, think of PERL RE's "o" option.)

> with perl, but ending up in pure C for sites requiring bigger efficiency). 

  Quite so.  Bottleneck issues need to be handled, though.

> but in case of problems no way to contact SMTP client).

  I begun to use the contentfilter myself to showe a middle finger on
  HTML email senders.  It does incidentially also cut down about 90+%
  of incoming spams and viruses...

  At one system I have a reply like:

  return "-1 550 5.7.1 The message contains HTML, therefore we consider".
         "it SPAM.  Send pure TEXT/PLAIN if you are not a spammer.";

-- 
/Matti Aarnio	<[email-masked]>
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to [email-masked]

---

From: Rik van Riel <[email-masked]>
Date: Sun, 16 Jun 2002 21:30:28 -0300 (BRT)

>   of incoming spams and viruses...

Hmmm, guess I should finish my contentfilter script, which
I want to be able to use domain-based dnsbl systems.

In particular {postmaster,abuse,dsn}.rfc-ignorant.org would
be interesting.

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

-