Subject: rbl problems and questions

From: "Daryle A. Tilroe" <[email-masked]>
Date: Fri, 14 Jun 2002 16:19:42 -0600

I have subscribed to the MAPS rbl+ service and everything is working fine
from the DNS lookup standpoint.  However despite appearing to find an
ip from the MAPS DNS check the smtpserver is allowing the mail.  From
smtpserver log:

-------------------------------------------------------
NVmk02147#      connection from rrss.crynwr.com ipcnt 1 childs 1 ident:
NO-IDENT-SERVICE[2]
NVmk02147#      Looked up DNS A object: 70.178.203.192.rbl-plus.mail-abuse.org
-> 127.1.0.4
NVmk02147#      Looking up DNS A object: 70.178.203.192.bl.spamcop.net
NVmk02147w      220 iris.micralyne.com ZMailer Server 2.99.56-pre1 #1
ESMTP+IDENT ready at Fri, 14 Jun 2002 15:42:40 -0600
NVmk02147#      remote from [192.203.178.70]:14599
NVmk02147r      helo rrss.crynwr.com
NVmk02147w      250 iris.micralyne.com Hello rrss.crynwr.com
NVmk02147r      mail from:<>
NVmk02147w      250 2.1.0 Sender syntax Ok
NVmk02147r      rcpt to:<[email-masked]>
NVmk02147w      250 2.1.5 Recipient address syntax Ok;
rcpt=<[email-masked]>
NVmk02147r      data
NVmk02147w      354 Start mail input; end with <CRLF>.<CRLF>
NVmk02147w      250 2.6.0 S4817070AbSFNVml message accepted
NVmk02147#      S4817070AbSFNVml: 581 bytes
NVml02151r      helo dul.crynwr.com
NVml02151w      250 iris.micralyne.com Hello dul.crynwr.com
NVmk02147r      quit
NVmk02147w      221 2.0.0 iris.micralyne.com Out
-------------------------------------------------------

Note that a lookup resolves to 127.1.0.1, 127.1.0.2, or 127.1.0.4
depending on what list it matches.  Anyhow what ever the address it
returns the mail is accepted.  The salient parts of my smtp-policy.src
look like this:

--------------------------------------------------------
.                       relaycustomer - relaytarget - senderokwithdns + = _rbl1
[0.0.0.0]/0             relaycustomer - relaytarget - senderokwithdns + = _rbl0
--------------------------------------------------------
_rbl0          test-dns-rbl      rbl-plus.mail-abuse.org:bl.spamcop.net
_rbl1          # Nothing at late phase
--------------------------------------------------------

Am I doing/forgetting something stupid?

Also I am not sure I understand the difference between or intentions of:

_rbl0          test-dns-rbl		[lookup services]  vs
_rbl0          rcpt-dns-rbl		[lookup services]  vs
_rbl1          test-rcpt-dns-rbl	[lookup services]

Should I add or change something here?

-- 
Daryle A. Tilroe
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to [email-masked]

---

From: "Daryle A. Tilroe" <[email-masked]>
Date: Sat, 15 Jun 2002 17:06:55 -0600

Hello?  Anyone on the list this weekend :-).  Anyhow, after looking at my
logs, it appears that the spamcop block is working fine:

-------------------------------------------------------
NVQW27874#      connection from [202.101.191.226] ipcnt 1 childs 2 ident: NO-IDE
NT-SERVICE[2]
NVQW27874#      Looking up DNS A object: 226.191.101.202.rbl-plus.mail-abuse.org
NVQW27874#      Looked up DNS A object: 226.191.101.202.bl.spamcop.net -> 127.0.
0.2
NVQW27874w      550-Blocked - see http://spamcop.net/bl.shtml?202.101.191.226
NVQW27874w      550-If you feel we mistreat you, do contact us.
NVQW27874w      550 Ask HELP for our contact information.
NVQW27874#      remote from [202.101.191.226]:57055
NVQW27874#      -- policyresult=-1 initial policy msg: Blocked - see http://spam
cop.net/bl.shtml?202.101.191.226
-------------------------------------------------------

The spamcop lookup resolves to 127.0.0.2 (rather than the MAPS 127.1.0.1,
127.1.0.2, or 127.1.0.4).  Now I would have thought that any positive lookup
should result in a block but it appears that zmailer may be looking for
127.0.0.2 only/exactly.  I have had a quick peek in the source code
(policytest.c, mxverify.c, and smtpserver.c) but cannot immediately see
where this may be hardcoded or set dynamically.  Maaaattiiiiii!  Help
meeeeeeee! :-)

---

From: Matti Aarnio <[email-masked]>
Date: Sun, 16 Jun 2002 04:11:59 +0300

> logs, it appears that the spamcop block is working fine:

  Besides of midnight oil (being up at 4 am..) I am burning a bit
  of CPU cycles at my home system to process few massive datasets.
  While waiting those to come through, I have a moment to look at this..

> NVQW27874#      connection from [202.101.191.226] ipcnt 1 childs 2 ident: NO-IDENT-SERVICE[2]
> NVQW27874#      Looked up DNS A object: 226.191.101.202.bl.spamcop.net -> 127.0.0.2
> NVQW27874#      -- policyresult=-1 initial policy msg: Blocked - see http://spamcop.net/bl.shtml?202.101.191.226

  The address data verification (at the last routine of  mxverify.c)
  does look (in text form!) that the data is of  "127.0.0."  and
  that it isn't  "127.0.0.4"

  What is that   127.1.0.*  thing ?

  I thought that MAPS RBL was the first who specified that the lookup
  produces valid rejection entry ONLY when it is 127.0.0.2
  It was possibly ORBS, that added ...3 and ...4, very least, into this
  repertoire of data variants.

-- 
/Matti Aarnio	<[email-masked]>
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to [email-masked]

---

From: "Daryle A. Tilroe" <[email-masked]>
Date: Sat, 15 Jun 2002 21:46:23 -0600

I don't know why I missed that the first time.  So is it as simple as
changing:

            if (strncmp("127.0.0.",abuf,8) != 0) {
              has_ok = 1;
              continue; /* Isn't  127.0.0.* */
            }

to?:

            if (strncmp("127.",abuf,4) != 0) {
              has_ok = 1;
              continue; /* Isn't  127.* */
            }

Are there any unforeseen repercussions with this change?

Also with orbs gone for a while now is there any use in this code
that ignores a 127.0.0.4 return; in other words is the 127.0.0.4
convention for untestable still being used?:

            if (strcmp("127.0.0.4",abuf) == 0) {
              /* ORBS NETBLOCK */
              if (has_ok) continue;
            }

---

From: "Daryle A. Tilroe" <[email-masked]>
Date: Sun, 16 Jun 2002 08:19:23 -0600

>               continue; /* Isn't  127.* */

Well I made the above, simple, change and so far so good.  The only
thing I notice is that MAPS does not appear to have any TXT records
for thier lookups so I think I may also change the default 550:

550-iris.micralyne.com - You are on our reject-IP-address -list, GO AWAY!

to something that points to MAPS.

---

From: Eugene Crosser <[email-masked]>
Date: Wed, 3 Jul 2002 12:20:45 +0400 (MSD)

>   repertoire of data variants.

I am under impression that if there is *any* result from the DNS lookup
than the address should be considered listed.  NXDOMAIN means that it isn't.

Eugene, responding that late because he's just returned from a leave spent
in Utah and Wyoming :)

-

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 3 Jul 2002 13:46:23 +0200

Eugene Crosser <[email-masked]>
> than the address should be considered listed.  NXDOMAIN means that it isn't.

ok.orbs.org contained 127.127/16. I presume "ok" means the hosts were OK.
No idea whether anyone uses 127.127/16 now that ORBS is dead. ISTR someone
using one particular 127.0.0.x for either "OK" or "untested", too.

A repulsive mess, rather.

--Arnt
-

---

From: Rik van Riel <[email-masked]>
Date: Sat, 6 Jul 2002 00:57:04 -0300 (BRT)

> than the address should be considered listed.  NXDOMAIN means that it isn't.

There are so many different meanings that you'd almost want
the flexibility to specify the interpretation of the result
in the config file ;)

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

-

---

From: "Daryle A. Tilroe" <[email-masked]>
Date: Fri, 30 Aug 2002 14:21:19 -0600

Is there a way (in scheduler.conf?) to force independent delivery
sessions to a particular domain/mx?  I.E. if I have a message going
to 5 recipients @fu.bar I would like to force the scheduler to deliver
it with 5 separate sessions to mx.fu.bar with 1 recipient rather than
1 session to mx.fu.bar with 5 recipients.

-- 
Daryle A. Tilroe

-

---

From: Matti Aarnio <[email-masked]>
Date: Sat, 31 Aug 2002 01:02:15 +0300

> 1 session to mx.fu.bar with 5 recipients.

  At scheduling level we can not do that.  Rewriting parts of routing
  might do that, though.  -- 10 seconds latter I realized that no, that
  can't be done.  (channel/host/user/privilege -quad, the 'host'
  part must be the user's destination domain.  It can't be the full
  recipient address, which would take care of parallellizing
  each recipient address just fine...)

  The "EZMLM" tricks in smtp transport agent are related to what
  you are looking for, although those don't do 5 parallel sessions
  either, merely 5 successive deliveries of the same message with
  separate MAIL FROM addresses.

  Approaches:

    1) New code at SMTP-ta to enable the 'host' part to be full
       user address, and make changes into router scripts to have
       them produce the address quad as channel/user/user/priv
       This gives parallel deliveries to the system.

    2) New code at SMTP-ta to control behaviour somewhat alike
       the present "ezmlm" to split deliveries into N ( << 100 )
       rcpt per transmission.  Also to limit the number of delivered
       messages per connection...

  All those are what I call "anti-performance" features, but out
  there are a few ISP sites which presume they can e.g. limit
  the number of messages delivered in single connection.

  Of course the _spammers_ send single recipient single message
  per session, only weird systems running busy lists send
  multirecipient multi-messages per connection...

  For the  2)  there needs to be some better way to make
  parametrization, than at scheduler's command line..
  I have though of a few ways, including something which
  does greatly resemble  scheduler.conf  file syntax...

---

From: "Daryle A. Tilroe" <[email-masked]>
Date: Fri, 30 Aug 2002 19:54:02 -0600

>   separate MAIL FROM addresses.

Do these tricks involve using EZMLM or hooks for EZMLM?  I guess I am
somewhat clueless wrt this reference.  If there is a simple way to do
the above mentioned '5 successive deliveries' it might do the trick
for my specific application.  Could you elaborate?  Thanks!

-- 
Daryle A. Tilroe

-

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 31 Aug 2002 09:10:20 +0530

Matti Aarnio <[email-masked]>
> Of course the _spammers_ send single recipient single message per 
session
...

Not any more. I run a spam honeypot (a Zmailer configured to forward 
relay probes and freeze everything else) and I see only multi-recipient 
spam, and mostly multiple messages per session.

Everyone using open relays use multi-recipient messages; most spammers 
using hashbusters or personalization use multiple messages per session.

They even use PIPELINING, sometimes correctly.

Btw, if any of you feel like setting up a honeypot catch lots of spam, 
please send me mail off-line. (Certain details are useful only until 
the spammers catch on, so I don't want them in a web-searchable 
archive.)

--Arnt
-

---

From: Matti Aarnio <[email-masked]>
Date: Sat, 31 Aug 2002 13:41:21 +0300

On Fri, Aug 30, 2002 at 07:54:02PM -0600, Daryle A. Tilroe wrote:
...
> for my specific application.  Could you elaborate?  Thanks!

  The "EZMLM" is (to my knowledge) doing email deliveries with
  individual source addresses.  Traditional way to make that to
  happen is to send separate messages for them all thru the MTA.

  We needed something a bit like it at vger.kernel.org, but still
  retaining speed advantages of processing 3000+ recipient lists
  by the routing and scheduling, and then do the customization of
  MAIL FROM/RCPT TO at the smtp transport agent.

  If you look into the source of   transports/smtp/smtp.c  and
  search for string "ezmlm", you encounter spots where the
  delivery is split into individual recipients.  Now if you
  want to experiement by adding some command-line parameter
  to the module, and have them split the delivery, those
  are the places where the action needs to be.
  I haven't thought of where connection restart needs to be,
  if anywhere at all.

---

From: Eugene Crosser <[email-masked]>
Date: 02 Sep 2002 11:52:04 +0400

>   Of course the _spammers_ send single recipient single message

Speaking of which, we are currently planning to deploy a thing that
would collect "RCPT TO" rate by peer's IP address in realtime and
degrade the service accordingly (a la "tarpit").  Anyone already done
that with Zmailer?

Eugene

-

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 2 Sep 2002 13:36:12 +0200

> the spammers catch on, so I don't want them in a web-searchable 

The way I saw spammers working was submitting a mail message to about
30 envelope recipients of which one address, a hotmail address was the
same in each batch.  I suppose that address was used for flow control.

  Ralf
-

---

From: "Mariano Absatz" <[email-masked]>
Date: Wed, 4 Sep 2002 16:08:18 -0300

> degrade the service accordingly (a la "tarpit").  Anyone already done
Nope... but are definitively interested in it... hoy do you plan to keep the rate info?  
in-core? in a database? collecting info in a log and parsing it every xxx seconds?

I thought about it a couple of times, but never went ahead with it... I'm also a slow Perl 
programmer and 2 old 2 C seriously... :-(

--
Mariano

-

---

From: Eugene Crosser <[email-masked]>
Date: 05 Sep 2002 11:48:19 +0400

> in-core? in a database? collecting info in a log and parsing it every xxx seconds?

In-core, collected directly from smtpserver over UDP.  We are too
resource constrined to deply kludges - crunching about 1.5 million
messages in a good day.  I am sorry to say that but *maybe* our solution
will not be open source because we *may* use our in-house middleware for
communication, which is not open source.

-- 
Eugene Crosser, head of Internet Applications section, +7 501 787 1000
ROL, Teleross, Golden Telecom, http://user.rol.ru/~crosser/

-

---

From: Matti Aarnio <[email-masked]>
Date: Thu, 5 Sep 2002 12:21:30 +0300

On Thu, Sep 05, 2002 at 11:48:19AM +0400, Eugene Crosser wrote:
> > Nope... but are definitively interested in it... hoy do you plan to keep 
> > the rate info?  in-core? in a database? collecting info in a log and
> > parsing it every xxx seconds?

  What about busy email lists outside your own area with a plenty
  of subscribers in your system(s) / serviced by your ISP relay ?

  What about there being a network break, and such a list not being
  delievered to you in a day, and after service restoration, all
  of the accumulated backlog hits you ?

  I have seen these things mis-applied so much that I am extremely
  wary to implement and deploy any such a thing...

  The slow-down tarpit is less harmfull to legitimate feeds (if
  monitored properly!), than yielding 500-series responses, of course.

---

From: Eugene Crosser <[email-masked]>
Date: 05 Sep 2002 13:32:23 +0400

>   monitored properly!), than yielding 500-series responses, of course.

We thought about that.  The same sentiment applies to big mail relays:
they may produce high traffic which is still legitimate.  We are
planning to have a dynamically modifiable "whitelist" where you will be
able to specify non-default "rating allowance" for ranges of IP
addresses.  And of course it will be necessary to constantly monitor the
status of rating cache.

Another interesting point (for us) is handling dialup customers.  We are
planning to attach the rating value to the customer rather than IP
address if that address belongs to a dialup pool.

Eugene

-