Re: parallel smtp sessions to same target ost

[mea@nic.funet.fi]

> >      sizebins="4 64 1000 0"  # Bin upper bound values in kilobytes
> >                     # "0" is magic meaning "no limit"
> 
> There might be situation when you have an unknown mix of jobs and, say,
> a sattelite link with looong roundtrip.  Then you would prefer to feed
> jobs to several threads *un*sorted, in random order.  Actually, I beleive
> that this would be more useful than `bins'.  Just get next message from
> the queue and pass it to an agent that have just done previous job...

	Method called PIPELINING was created to solve this exact
	problem.  It turns the half-duplex behaviour of the SMTP
	into far more streamlined where it essentially does check-
	point at DATA and at "." verbs.  You can feed all of your
	MAIL FROM and RCPT TO lines without waiting their individual
	reports to arrive.  Then you send DATA, and start listening
	on all of the reports up to and including the DATA.

	Of course this turns the control flow somewhat weird..
	(see ZMailer's smtp transport agent, and if you can follow
	 the pipelined flow without severe headache, congratulations.)

	The receiving end must support PIPELINING at reception,
	but at simplest that means just that it does not throw
	anything away at the smtp input stream even with an error
	status at previous command.  For example Dan Bernstain
	(of qmail fame) was at first very much opposed to the
	whole idea, until I (and perhaps others too) were able
	to get him to verify the server behaviour, and just slap
	in the magic EHLO capability token.
	(Server side is easy, even the optimized case, transport-
	 agent is not..)

	My primary inspiration for implementing it were sluggish
	feed problems on Linux lists from  vger.rutgers.edu to
	here at nic.funet.fi -- classical half-duplex facility
	was vacuuming hard silicates with messages carrying up
	to 2000 recipient addresses, and perhaps 1 kB of headers
	and body..  With pipelining such is handled with 2-3
	second time from MAIL FROM to the ack of ".", while
	classical system took 2000 * 0.3 seconds = 600 seconds...
	(Ok, perhaps it takes 10 seconds for the feed with pipe-
	 line, but not more than that!)


	I think the parallel sub-channel priorization and SMTP
	PIPELINING can handle most narrow/long pipe problems.

	... and with priorized sub-channels, users will soon learn
	that small messages get thru, and they start asking mail
	from (file-)servers in small multipart messages...

> Eugene

/Matti Aarnio