Re: SMTP file naming issue?

[Matti Aarnio <mea@nic.funet.fi>]

On Mon, Jun 05, 2006 at 06:38:24AM -0400, Carlos Mendioroz wrote:
> Here's what I'm doing, it may suit your needs:
> You can insert a call to an external pocess in the loop of router
> processing. That would be in router.cf, this is what I do:
> 
>         [0-9]*)         TAG=$(/usr/bin/sb_client.py -T $file)
>                         rfc822 "$file" ;;
> 
> I'm just "reading" the file there. Zmailer speed comes in part from the
> fact that imposed message is not altered anytime.
> 
> Notice also that I'm keeping some info inside router. You can also keep
> some more metadata outside.
> I've altered router to place that info in a new X-xxxx header in the
> transport control file, which the transport agents end up copying to
> the output message. If you need further changing, you can change
> the transport sequence, like insert another process in the chain
> by using sm.

Yes, that is a nice method - call interface to external server so that
likely slow server startup will get distributed over many invocations
of that program.   That is what ZMailer does with its routers et.al.
after all.

There should be a decent size pool of those external servers to match
number of parallelly running ZMailer routers -- and if that external 
server does something exessively slow (like DNS queries),  number of
parallel routers as well as external servers should be increased...

Another would be to integrate sendmail milter library to ZMailer's router
so that external milter applications could be used.  It would allow 
additional header manipulations.

Meanwhile..  I have thought also about redoing the smtpserver's policy
framework so that there really is a programming language behind it all.
What that language would be ?  Plain PERL ?  (Or embedded variant of it.)
This has been motivated primarily by a desire to have more flexible
policy functions, not just some small numeric limits but even something
where one could say that "authenticated user NN has following limit values
L1, L2, L3, L4" -- and specifically that "authenticated user" could be
a very flexible thing indeed  -  query from any database supported by
the language.

However my time has gone for other things for past year or so..

> Hope this helps...
> -Carlos

    /Matti Aarnio


> Neal Morgan  @ 5/6/2006 2:23 -0400 dixit:
> > After some time looking through the code, I see my previous questions
> > were dumb.  
> > 
> > So, let me try again:
> > 
> > Is there a way to configure router/transport(/scheduler) so that:
> > 
> > 1) messages received by smtpserver are "routed" to an alternate
> > directory
> > 2) metadata about the message is maintained
> > 3) an external process operates on the message, then
> > 4) the external processes "resubmits" or "renables" the message, router
> > and transport pick back up with the SAME message information (but
> > altered message contents) and continue where they left off with routing
> > and delivery?
> > 
> > It seems like this should be possible with router and sm? I just can't
> > figure out the correct configuration to make ZMailer view this as the
> > same message, rather than a new one.
> > 
> > 
> > 
> > 
> > 
> > -----Original Message-----
> > From: Carlos Mendioroz [mailto:tron@huapi.ba.ar] 
> > Sent: Saturday, June 03, 2006 4:29 PM
> > To: Neal Morgan
> > Cc: zmailer@nic.funet.fi
> > Subject: Re: SMTP file naming issue?
> > 
> > I would use message-id, I guess, but there may be issues with that.
> > Sorry, I'm not able to help you more...
> > 
> > 
> > Neal Morgan  @ 3/6/2006 17:36 -0400 dixit:
> >> Thanks.  I don't see any indication of a problem with the messages
> > being
> >> handled properly.  I agree Zmailer is working fine.
> >>
> >> My issue is visibility.  I don't see an easy way from my logs to match
> >> messages leaving SMTP with those traversing external processes, and
> > back
> >> when re-entering the router.
> >>
> >> Maybe I should ask my question differently.  Is there an easy way to
> >> include the spoolID in the envelope (on the comment line) or append it
> >> to the message header?
> >>
> >> Alternatively, how difficult would it be to use spoolID as the file
> >> name?  ...would that cause problems elsewhere?
> >>
> >> I'm guessing the envelope approach would be easiest.  Suggestions?
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: zmailer-owner@nic.funet.fi [mailto:zmailer-owner@nic.funet.fi]
> > On
> >> Behalf Of Carlos Mendioroz
> >> Sent: Saturday, June 03, 2006 4:40 AM
> >> To: Neal Morgan
> >> Cc: zmailer@nic.funet.fi
> >> Subject: Re: SMTP file naming issue?
> >>
> >> If the naming scheme is the old one, name comes from inode number.
> >> So if the name is reused, you should count on the old one having been
> >> taken care of.
> >> -Carlos
> >>
> >> Neal Morgan  @ 3/6/2006 2:58 -0400 dixit:
> >>> Hello Matti!
> >>>
> >>> I've been watching the files dropped into postoffice/router by
> >> incoming
> >>> SMTP sessions and I note the same file name is used several times.
> >> I'm
> >>> wondering if this is configurable, or a bug...  I have an external
> >>> process that acts on the messages before the router picks them up for
> >>> delivery.  I'd like to be able to track "chain of custody" per
> > message
> >>> using this file name, but am unable to do so if SMTP reuses the same
> >>> name.
> >>>
> >>>
> >>> Regards,
> >>>
> >>> Neal Morgan
> >>>
> >>>
> >>> -
> >>> To unsubscribe from this list: send the line "unsubscribe zmailer" in
> >>> the body of a message to majordomo@nic.funet.fi
> >>>
> >> -
> >> To unsubscribe from this list: send the line "unsubscribe zmailer" in
> >> the body of a message to majordomo@nic.funet.fi
> >>
> >> -
> >> To unsubscribe from this list: send the line "unsubscribe zmailer" in
> >> the body of a message to majordomo@nic.funet.fi
> >>
> > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe zmailer" in
> > the body of a message to majordomo@nic.funet.fi
> > 
> -
> To unsubscribe from this list: send the line "unsubscribe zmailer" in
> the body of a message to majordomo@nic.funet.fi

-- 
/Matti Aarnio	<mea@nic.funet.fi>
FUNET:  Finnish Academic and Research Network
	Network Information/Software Archival Service
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to majordomo@nic.funet.fi