patch2 (2003-01-10):

- sender notification were not sent for messages containing banned
  filenames but no virus; fixed (thanks to Philip Ross);

- added virus name 'Sobig' to the default $viruses_that_fake_sender_re
  (amavisd.conf);

- extended default $banned_filename_re to match also a longer
  (4 character) middle file extension (e.g. Movie_0074.mpeg.pif)
  (amavisd.conf);

- add explicit path to the program vscan in the 'Trend Micro FileScanner'
  entry (amavisd.conf), thanks to Matthew Hobbs;

- improvements in the setup of helper programs (files configure.in,
  Makefile.in, config.h, amavis.c and amavis-milter.c), by Radu Greab;

- add Sophos error code 527 ('Not supported in this SAVI implementation')
  to the list of (mostly)harmless codes in the SAVI-Perl interface routine
  (thanks to Paul B. Henson and Tim Winders for discussion);

- initialize SAVI-Perl module (if available) in each new child, instead of
  only once during daemon startup time. This avoids the need to reload
  amavisd for every change in the Sophos IDE database - at the expense
  of additional SAVI-Perl startup time every $max_requests messages;


patch1 (2002-12-30):
  fix for the problem when using multiple virus
  scanners, and the last scanner doesn't find a virus, but some other
  scanner does, and amavisd-new reports a false negative.
  Thanks to Okke Timm, Henrik Larsson and Eduardas Paulavicius;

---------------------------------------------------------------------------
                                                          December 27, 2002
amavisd-new-20021227 release notes


amavisd-new-20021227 is primarily a maintenance release, consolidating
amavisd-new-20021116 with its mandatory patches, updating documentation,
but also brings a couple of niceties.


Changes since amavisd-new-20021116 patch level 5

- The new configure/makefile in the helper_progs subdirectory is
  contributed by Stephane Lentz, based on MIMEDefang configure.in.
  It is needed to build helper programs for sendmail (milter and nonmilter)
  setups (but not needed for Postfix and Exim v4 setups, or any dual-MTA
  setup). Similar to the prior work done by Ramiro Morales - thanks to both!

- Updated documentation (README, INSTALL, RELEASE_NOTES, README.lookups,
  README.customize, README.postfix).

- new subdirectory test-messages, containing sample mail messages to test
  virus and spam detection - see README there

- For daemonized antivirus scanners (only) the semantics of few parameters
  in the @av_scanners list has been incompatibly changed and enhanced.
  The same internal mechanism now handles both Unix sockets and inet socket.
  More than one socket may now be specified, to be tried one after the other
  until one is successfully connected to (useful for 'FRISK F-Prot Daemon').
  Also the usual regexp pattern parameters in the @av_scanners are now
  observed by Sophie and Trophie interfaces as well, bringing them more
  in line with other scanners.

- The (?m) regexp flag in @av_scanners patterns is now turned-on
  by default, so it is no longer necessary to specify it in @av_scanners,
  and has been removed to shorten the expressions.

- The SAVI-Perl interface routine now turns on most Sophos engine options,
  enabling recognition of some more file formats.

- All active antivirus scanners are now called, the scan no longer stops
  when the first one finds a virus. This is on the average a very small
  price to pay (only payed in virus storms), but makes possible to see
  which virus scanners found the virus and which did not, and to get their
  timing in the logs. The list of virus scanners that noticed the virus
  is now included in the notification message to the virus administrator.
  A new macro %W is available for this purpose.

- More sensible logic when checking more than one file for those
  daemonized av scanners that need to be handed individual files instead
  of directory name.

- The 'Clam Antivirus-clamd' daemon is now supported, both via Unix
  socket, as well as via inet socket (but Unix socket should be used,
  due to lack of self-protection in this respect by clamd).

  clamd is a fast and open-source anti-virus scanner
  ( http://clamav.elektrapro.com/ ), and I recommend it to be used in
  combination with other commercial virus scanners as the last resort
  scanner, to avoid bouncing mail in case primary scanner(s) stumble
  across some unhandled situation, like corrupted or encrypted mail.

- New utility program 'amavisdconf' to check the values of configurable
  variables set by /etc/amavisd.conf or defaulted (modelled after
  Postfix utility 'postconf', but -e option is not yet supported).

  Usage:
    amavisdconf [-d | -n] [-c conf-file] [variable]

  Options:
   (none) show the value as will be used by amavisd;
    -d  show default value as in the absence of the config file;
    -n  only show variable if its value is different from the default;
    -c conf-file ... use the specified file instead of /etc/amavisd.conf

  If a variable is specified show only that variable, otherwise show all
  configurable variables. (Note: the leading $, @ or % must be included
  with the variable. Make sure to protect $ from shell evaluation,
  e.g. by enclosing variable name in single quotes, or prepending
  a backslash:
    $ amavisdconf -d '$forward_method'
    $ amavisdconf -n

  This is the first attempt at such a utility, some details would need
  to be polished (e.g. how to report code reference), but it is hoped
  to be usable already in its present form.

- Two new subroutines (read_hash and read_text) are now available
  for use from amavisd.conf file, with a primary intent to move larger
  texts out of the amavisd.conf file:

  * read_hash allows for lookup hashes (e.g. white/blacklist,
    static %spam_lovers table, ...) to be read from a file during startup;
    It takes quoted form of addresses, optionally enclosed in angle
    brackets, and converts them into unquoted (raw) form, as needed
    for the hash keys. Format: one address per line, everything from
    '#' to end of line is a comment, leading and trailing whitespace
    removed, empty (or comment-only) lines are ignored.

  * read_text allows for reading more complex strings (e.g. multiline,
    or including special characters such as backslash) from a file.
    Useful to override the default template text for notification messages.

  See example usage in amavisd.conf.

- Three synonym (alias) subroutine names are available for use
  in amavisd.conf to shorten the expressions:

    new_RE       same as   Amavis::Lookup::RE->new
    ask_daemon   same as   Amavis::AV::ask_daemon
    sophos_savi  same as   Amavis::AV::sophos_savi

  amavisd.conf file is changed to use the short names, but full
  subroutine names are still valid.

- Provide protection mechanism for SMTP/LMTP server code to limit the
  number of recipients in a mail transaction to some sane value
  ($smtpd_recipient_limit, default 1000). Even if the number is exceeded,
  mail will still be normally delivered in more than one transaction
  (provided your MTA is rfc2821 compliant).

- SMTP client code now correctly handles the situation where remote
  SMTP server returns '452 Too many recipients'. When this happens the
  mail forwarding or submission is split into several SMTP transactions,
  and as long as the remote SMTP server is willing to accept at least
  one recipient per transaction, mail delivery is assured.

- Auto white-listing (AWL) by SpamAssassin may be turned on by
  setting $sa_auto_whitelist to true (contributed by Hamish Marson).
  This only works as one global database, the per-recipient AWL
  is presently not available.

- New configuration variables $hdrfrom_notify_sender, $hdrfrom_notify_admin,
  and $hdrfrom_notify_spamadmin, to remove the overloading (double semantics)
  from variables $mailfrom_notify_*. Now $mailfrom_notify_* control ONLY
  envelope addresses, while $hdrfrom_notify_* only controls From header
  in notifications. The default values for $hdrfrom_notify_* are chosen
  to preserve compatibility.

- The spam headers in quarantined message now reflect the lowest
  tag and kill levels when more than one recipient is given, instead
  of reporting the levels for the first specified recipient.

- Tested with SpamAssassin 2.50 and with its new Bayesian classification
  (used as a global database by amavisd-new). Given two sets of spam
  and non-spam (=ham) messages, one can build a database by using new
  SpamAssassin utilities sa-learn-spam and sa-learn-nonspam as user amavis,
  and the database built this way will be used by amavisd-new/SpamAssassin.

- Initialize SpamAssassin as $daemon_user (no longer as root),
  as a safety measure, and to ensure the files it creates are still
  accessible after the UID change.



Incompatible changes to 20021116:

- Different parameters in @av_scanners list for daemonized antivirus
  scanners (as described above).

- lookup_hash: an undefined hash value was previously treated as a
  spacial case, resulting in value true. This is no longer the case,
  which brings it in line with other lookup methods. It is not likely
  anyone will be affected, as this practice was only documented but
  not encouraged by any example.


---------------------------------------------------------------------------
Here is a brief description of patches that needed to be applied to
amavisd-new-20021116, and were available from the amavisd-new web page:


patch5 (2002-12-09):
- fix a simple test to decide if the anti-virus and anti-spam code is needed;
- use the /m regexp modifier by default when parsing av-scanner output
  for virus names - seems like (?m) in qr is ignored (have we found
  a Perl bug?);
- fix DSN to report original smtp response in case reporting MTA is remote
  (thanks to Chris Hastie for his keen eye in understanding DSN);
- fix reporting quarantine mailbox name(s) in case of per-recip quarantine
  or quarantine forwarding to MTA;
- macro %q (quarantine id) is now a list of quarantine mailbox names or
  addresses (may have more than one entry in case of per-user quarantines);
- macro %v (av-scanner output) is now a list of scanner output lines,
  to facilitate formatting of multi-line av-scanner output;
- fix spam admin notifications, which used the virus template instead of
  the spam template;
- fix problem of undefined entity causing spam notification failure
  in case other content checking (virus/banned) is not enabled;
- make SQL lookups (DBI access) optional - no DBI code is loaded if
  @lookup_sql_dsn of empty;
- support SAVI-Perl module 0.10 or later (direct access to the Sophos library
  engine) ( http://www.csupomona.edu/~henson/www/projects/SAVI-Perl/ );
- add macro %j that evaluates to message 'Subject:' header field;
- fix minor SMTP protocol inconsistency when multiple MAIL FROM
  commands are issued by the client;
- more informative timing report when more than one av-scanner is used;
- make sendmail initial submission happy when null reverse-path is specified
  (instead of the correct empty argument, which works but makes sendmail
  issue a warning, we now use <> in arguments to specify null paths),
  (thanks to Sebastian Hagedorn and Dale Perkel);
- replace log entry "spam from=<%s>, to=%s" with "SPAM, <%s> -> %s ...".
- dropped (redundant) historical %bypass_checks @bypass_checks_acl
  (use bypass_virus_checks and %bypass_spam_checks instead);

patch4 (2002-11-21):
  In amavisd:
- SQL lookup fix;
- finished per-recipient header edits and split-forwarding, where
  one multi-recipient message may be forwarded as several messages
  if different headers need to be inserted for different recipients
  (not with milter);
- the check for banned file contents now looks for both the short
  and the original full type classification as provide by 'file' utility
  (thanks to Wolfgang Lumpp for the idea);
- #!/usr/local/bin/perl => #!/usr/bin/perl
- cosmetics

  In amavisd.conf:
- suggested group name change $daemon_group 'sweep'=>'amavis';
  so that non-Sophos users are not puzzled where 'sweep' group comes from;
- added path and fixed args for KasperskyLab AVPDaemonClient
  (thanks to Mike Hall);
- added path for Trend Micro FileScanner vscan
  (thanks to Eduardas Paulavicius);
- added 'use strict;' and '1;' at the end; prevent read_config from failing
  if the config file does not return true (thanks to Vivek Khera);
- cosmetics, fixed typos in comments;

patch2 & patch3 (2002-11-18, 2002-11-19):
- fixes a problem with 'recip_done' object method (causing TEMPFAIL)
  when $warnvirusrecip is enabled (thanks to Chris Hastie, Dale Perkel,
  and Didi Rieder);
- pass null return path to sendmail in such a way to make it happy
  (thanks to Sebastian Hagedorn and Dale Perkel);
- added a safety precaution to prevent somebody shooting himself in the
  foot by specifying an empty $forward_method while using the SMTP in/out
  setup (e.g. Postfix), which could send mail down the bit bucket;

patch1 (2002-11-17):
- fixes a problem in milter setup where per-recipient response codes
  needed to be handled differently (the problem was causing a TEMPFAIL)
  (thanks to Stephane Lentz and Didi Rieder for the initial milter testing
  and problem reports);


Mark

-- 
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  !!  Mark Martinec (system manager)     tel  +386 1 4773-575 !!
  !!  J. Stefan Institute, Jamova 39     fax  +386 1 2519-385 !!
  !!  SI-1000 Ljubljana, Slovenia        mark.martinec@ijs.si !!
  !!!!!!!!!!!!!!!!!!!!!!!!!! http://www.ijs.si/people/mark/ !!!!


---------------------------------------------------------------------------
Previous release notes
---------------------------------------------------------------------------
(the rest may differ from the original posting in small detail,
mostly to fix typos or clarify a sentence)


---------------------------------------------------------------------------
                                                          November 16, 2002
amavisd-new-20021116 release notes

- provide a mechanism to load only required code sections
  (anti-virus scanning, anti-spam scanning, SMTP/LMTP server module,
  traditional amavis client & milter server module),
  resulting in reduced memory usage and less installation dependencies;

- introduce the per-recipient status handling and make possible some
  per-recipient functionality that was not possible before, such as
  handling the per-recipient spam thresholds. This required a major
  rewrite and cleanup of some sections, hopefully providing cleaner
  mechanism for possible future advances in this area.

  Consider for example a previously incorrectly handled situation
  where there is more than one recipient, and some (but not all) of them
  get a reject from the (outgoing) relay MTA - it is not possible for a
  single SMTP response or status code to describe the situation;

- generate proper (non-)delivery status notifications (DSN), compliant
  with rfc1892 and rfc1894. This form supersedes the simple virus/spam
  sender notifications, but also covers the area of genuine delivery
  failures, such as selective recipient rejects by the outgoing relay MTA,
  which could previously lead to multiple deliveries;

  NOTE: to avoid sender getting two non-delivery messages (one from
  MTA, and another from amavisd), do not set $warnvirussender and
  $final_virus_destiny=-1 (REJECT) at the same time (and equivalently
  for spam settings);

- enhanced mail system status codes (rfc1893, rfc2034) are now included
  with all SMTP responses and DSN notifications;

- added checking for banned MIME types and names. If any mail part
  matches, the whole mail is rejected, much like the way viruses
  are handled. A list @banned_filename_patterns_re can be defined
  to provide a list of Perl regular expressions to be matched against
  each part's:

  * Content-Type value (both declared and effective mime-type),
    including the possible security risk content types
    message/partial and message/external-body, as specified by rfc2046;

  * declared (recommended) file names as specified by MIME subfields
    Content-Disposition.filename and Content-Type.name, both in their
    raw (encoded) form and in rfc2047-decoded form if applicable;

  * file content type as guessed by 'file' utility and classified into
    short type names such as .asc, .txt, .html, .doc, .jpg, .pdf,
    .zip, .exe, ... - see subroutine determine_file_types().
    This step is done only if $bypass_decode_parts is not set.

  NOTE: by default the $final_banned_destiny is set to 1 (pass),
  so detected banned file names only cause a header line to be added,
  quarantining, and added address extension - but the mail is delivered
  nevertheless. This default is set so that we can get initial experience
  without being too obtrusive. Change as you prefer.

- besides SMTP/ESMTP protocol, the server now also accepts LMTP protocol
  (rfc2033). This now explains why I had to implement PIPELINING,
  ENHANCEDSTATUSCODES and 8BITMIME SMTP extensions, as these are required
  by rfc2033.

  One advantage of using LMTP to feed content filter is that LMTP uses
  per-recipient status response, as opposed to 'one-size-fits-all'
  SMTP status response, which require that the SMTP client (e.g. content
  filter) is capable of generating DSN.

  Another advantage is specific to the Postfix setup, as Postfix
  LMTP client supports multiple transactions per session, saving on
  connection teardown/reconnect for every message being checked.

  To enable LMTP feed from Postfix, add 'max_use=10' to main.cf, and
  replace last argument 'smtp' with 'lmtp' in the master.cf line:
  'smtp-amavis unix - - n - 2 lmtp'. No changes are needed in amavisd-new,
  both protocols can coexist, the distinction is based on HELO/EHLO
  vs. LHLO command.

  Btw, older versions of Postfix lmtp client inappropriately lowercased
  the envelope addresses. This is fixed in Postfix Snapshot 1.1.11-20021015,
  and in the regular Postfix release 1.2 (when available). Lowercasing
  can be a problem for addresses where local part is case-sensitive,
  although such setups are rare.

- improved per-recipient SMTP response code handling when sending mail
  via SMTP. Previously one rejected recipient (by MTA) in a multi-recipient
  message caused the whole message to me rejected. This is now correctly
  handled with the new ability to send DSN;

- added command line option '-c config-file' so one can override the
  default location of the configuration file (/etc/amavisd.conf);

- explicitly set PATH and HOME environment variables
  (settings: $path and $helpers_home)

- added another form of lookups: Perl regular expression matching.
  See README.lookups for details. Corresponding new variables in amavisd.conf
  are: $virus_lovers_re, $spam_lovers_re, $bypass_virus_checks_re,
  $bypass_spam_checks_re, $local_domains_re, as well as $banned_filename_re,
  $viruses_that_fake_sender_re, $keep_decoded_original_re,
  $whitelist_sender_re, $blacklist_sender_re.

- besides whitelist_sender* lookup tables, there is now also
  a blacklist_sender* set of tables, which causes mail to be declared spam
  and to skip remaining spam checks. See examples in amavisd.conf.

- provide a configurable lookup table $keep_decoded_original_re
  of file types, for cases where unpacker is not very trustworthy.
  The lookup key is what 'file' utility returned. If the part contents
  match the lookup table, we keep both the original and the unpacked file
  for virus scanner to check;

- provide a configurable list of regular expressions
  $viruses_that_fake_sender_re, which recognizes viruses that usually
  fake envelope sender address. Don't send sender notification if a match
  is found. For syntax of the new regexp lookup tables see README.lookups;

- specifying per-recipient quarantine address or location is now possible
  by setting $virus_quarantine_to and $spam_quarantine_to be a ref to a
  hash lookup table. Thanks to Vivek Khera for the idea;

- no 'configure' and 'make' for the daemon; all configuration is done
  via amavisd.conf configuration file at the daemon startup time;
  Also the ./mta, ./av and ./notify configuration subdirectories
  are now gone;

- provided a sample init shell script amavisd_init.sh (edit to will,
  and move it to /etc/init.d/amavisd if you want). Based on the script
  from Wil Cooley;

- oneshot debugging mechanism: if envelope sender matches @debug_sender_acl
  lookup table, turn debugging fully up just for this one message and
  cause temp file and directories not to be cleaned for this message.
  This facilitates debugging a particular problem even in the presence
  of regular traffic;

- cleaner debug log entries for multi-transaction SMTP/LMTP protocol sessions;

- cleaner log entries - avoid misleading and incomplete text
  when quarantine is disabled; thanks to Michael Leone;

- avoid using (nonstandard) field width in the %e format specifier
  when calling strftime, which lead to bad date syntax on some systems;

- local time zone offset is now automatically computed, no more '-0000'
  in rfc2822 dates;

- put Message-ID field value in angle brackets as required by rfc2822
  when generating notifications;

- add a In-Reply-To field to notifications when original Message-ID is known;

- don't bounce a virus (or spam) back to a mailing lists even if
  $final_virus_destiny (or $final_spam_destiny) is set to REJECT;
  a patch by Brian May from the Debian support crew, thanks;

- because some external module may play games with STDIN and STDOUT
  (like SpamAssassin seems to do when local_tests_only=>0)
  run the input protocol directly on the Net::Server's socket,
  not on STDIN and STDOUT, which are aliases to the socket;

- a patch to amavis-milter.c by Didi Rieder to support REJECTing mail,
  by instructing sendmail to return non-delivery notification to sender;

- a modified amavis.c helper program to make possible to invoke
  local delivery agent from it, for those still using such a setup;

- a new macro %l is available for use in notification messages (via
  'expand'); it evaluates to true (1) if the sender matches @local_domain,
  and returns empty otherwise; by default it is now used in creating
  a 'Subject:' line, inserting word 'LOCAL' before the sender name
  when appropriate;

- new macros %D and %N expand to lists of recipients that got the mail
  delivered (%D), or not-delivered (%N).  Union of both sets gives %R,
  i.e. a list of all recipients as specified in the envelope;

- dropped macro %a

- improved parsing of e-mail addresses according to rfc2821
  (full address literals syntax, etc.);

- proper line folding for generated rfc2822 header fields;

- proper SMTP response wrapping (for very long responses such as
  the ones that include a trouble report) according to rfc2821;

- allow for obsolete rfc822 syntax of permitting whitespace before
  colon in header field name;

- added optional spam-sender nondelivery notifications, based on patch
  from Lazslo E. Miranda (lazslo@dcc.ufmg.br) and Fernando F. Morais
  (frota@cecom.ufmg.br);

- dynamically change process name (Perl variable $0) to reflect
  the process state; suggested by Chip Paswater. Not all operating
  systems make this process state visible by ps(1);

- determine location of external programs (or their absence) at
  startup time; in amavisd.conf one may specify absolute path or just
  rely on PATH. This mechanism is also used to determine absolute path
  of the daemon itself, making reload (after HUP) more predictable;

- explicitly specify lock file for serialization to be used by the
  Net::Server::PreForkSimple module, instead of relying on default
  provided by POSIX::tmpnam. The default approach has a possibility
  for a minor security problem, because the lockfile is created with
  open()..., so it will follow symlinks. Observed, and a patch
  provided, by Jarno Huuskonen;

- new defense against mail bombs: for the cumulative total of all
  decoded mail parts we set max storage size. The formula is:
    quota = max($MIN_EXPANSION_QUOTA,
                $mail_size*$MIN_EXPANSION_FACTOR,
                min($MAX_EXPANSION_QUOTA, $mail_size*$MAX_EXPANSION_FACTOR))
  In plain words (later condition overrules previous ones):
    allow MAX_EXPANSION_FACTOR times initial mail size,
    but not more than MAX_EXPANSION_QUOTA,
    but not less than MIN_EXPANSION_FACTOR times initial mail size,
    but never less than MIN_EXPANSION_QUOTA

- if the permitted quota is exceeded (or the defense triggered by other
  similar safeguards), the virus scanning is skipped to protect the
  virus scanner from tripping over the mail bomb, a header field
  is inserted:
     X-Amavis-Hold: ... reason ...
  and a log entry 'Placing on HOLD: reason' at level 0 is produced.
  Also the temporary directory is preserved.

  The 'X-Amavis-Hold: ...' header field can be used by your MTA to
  put the message 'on hold' (freeze). If MTA is not set up to catch
  messages with this tag (the default), they are passed normally to
  recipients. This likely to be the best action under the circumstances.

- tested with razor-agents-2.20 and SpamAssassin-2.43 (Razor2 is called
  by SpamAssassin). New configuration variable $helpers_home, which
  defaults to $TEMPBASE. Thanks for hints by Chris Hastie and John Stewart;

- to avoid CPU loop in SpamAssassin-2.43 and earlier, my patch needs
  to applied to SA - see amavisd-new web page;

- to avoid taint problem in Razor 2.20 (if SpamAssassin-2.43 is configured
  to call it), my patch needs to be applied to it - see amavisd-new web page;

- chroot available (but not well tested): $daemon_chroot_dir = '/var/amavis'

- provide a failover mechanism for SQL database connect - given a list
  of SQL servers/databases, pick the first that is available.
  Thanks to Ken McKittrick for making available the patch,
  and to Ben Ransford <ben@terakeet.net> for writing it.

- remove existing virus-related and spam-related headers (some of them
  optionally) if we'll be providing our own; suggested by Borut Mrak;

- avoid 'insecure dependency' in lookup_sql when calling DBI::execute
  (thanks to ric* at mpc.com.br)

- fixed macro %H to provide original header lines, not the ones
  stored internally by MIME::Entity, which may have been modified;
  (thanks to Chris Hastie for noticing the problem);

- no longer inserts X-Razor-id header field;


INCOMPATIBLE CHANGES:

- when specifying boolean values to variables in the amavisd.conf file,
  please specify 1 (or old style "yes") for true, and 0 or "" or undef
  for false. The old style "no" yields true for Perl, and is only still
  supported (converted to 0) for some traditional variables for
  compatibility with amavisd(-snapshot);

- removed variables $sendmail_wrapper*, and changed the syntax for
  specifying $forward_methor/$notify_method, with the intention to
  do all mail sending settings at one place. See amavisd.conf for examples;

- variable $mailto is now deprecated (but still works as a fallback default
  for compatibility with previous version). Use $virus_admin and $spam_admin
  lookup tables instead, they also offer a mechanism to specify per-domain
  administrator address;

- variable $warnadmin is no longer used. Use $virus_admin and $spam_admin
  lookup tables instead. Not specifying administrator address
  (e.g. leaving $virus_admin, $spam_admin and $mailto undefined)
  turns off admin warnings.

- dropped variable $LOGDIR, the variable $LOGFILE now specifies the full path;

- renamed: $warnsender -> $warnvirussender, $warnrecip -> $warnvirusrecip

- dropped variables $enable_relay, $sendmail_cf_orig, $QMAILDIR;
  they were never used in amavisd-new;

- dropped $MAX_ARCHIVE_NESTING, which is replaced by new storage limitations:
  $MIN_EXPANSION_QUOTA, $MIN_EXPANSION_FACTOR
  $MAX_EXPANSION_QUOTA, $MAX_EXPANSION_FACTOR

- SQL database is now specified differently (@lookup_sql_dsn)


MTA-SPECIFIC: sendmail/milter

- $notify_methos now specifies deferred delivery mode ('-odd') by defaut,
  when submitting notifications to sendmail. This is to avoid calling milter
  immediately during submission, which in turn calls amavisd-new, possibly
  leading to a deadlock situation when the number of amavisd-new child
  processes is small. Seems like this change is needed since Sendmail 8.12
  or so.

  The following recommendation is from mimedefang-filter man page:

  | You MUST run a client-submission queue processor if you use
  | Sendmail 8.12.  We recommend executing this command as part of the
  | Sendmail startup sequence:
  |     sendmail -Ac -qp1m


DECODERS-UNPACKERS:

- rewritten decode_parts() to allow for retaining source text if the
  unpacker sw is considered unreliable. This more cleanly resolves
  the problem reported on the amavis-user mailing list on 2002-06-06:
    "Amavisd passing through VBS/VBSWG.gen@MM"
  A nice side-product is that a directory search is avoided for
  each nested unpacking step;

- refined do_unzip to control and limit the size of decompressed
  members (among others it defends against the 42.zip-type bombs);

- rewritten external decompressors interface to the gzip/bzip2/compress
  family. Instead of using a 'system' call, they are now called through
  fh_copy, making possible to control and limit the size of the
  decompressed contents on the fly, avoiding denial-of-service attacks.
  Affected: b(un)zip2, g(un)zip, (un)compress;

- fh_copy now uses IO::Handle object to assure the forked process
  gets reclaimed even in case of aborted contents extraction;
  stdin gets redirected to /dev/null or to a specified input file
  for the exec'd process;

- Convert::UUlib is called again (the amavisd-new-20020630 removed its
  usage due to problems with improper decoding). This time the originals
  are kept, so that virus checker sees both the original and the
  attempted-decoded part;

- completely rewritten do_ascii and its usage of Convert::UUlib
  to fix:
  * coding error (bitwise op treated as logical op and improperly negated):
      ... if (!$uu->state || !FILE_OK || -z $newpart);
  * it never reset the state, so if a successfully decoded ASCII file
    contained another ASCII file, each decoding level would decode all
    previously decoded parts again, plus add new ones at this level;
    recursion would not stop until the hard limit, resulting in TEMPFAIL;
  * it never checked nor reported errors that should have been detected
    (I/O errors, out of memory, trouble accessing or creating files);
  * more informative log entry;

- save MIME preambles and epilogues (if nonempty) as extra (pseudo)parts
  to be scanned. This also mitigates the problem of syntactically-incorrect
  MIME mail as produced by some user agents, which (rightfully) gets
  treated as one long preamble by MIME-Tools, and previously went by
  unchecked;

- supports unpacking arc archives using 'nomarch' (by Russell Marks,
  http://rus.members.beeb.net/nomarch.html), (thanks to David D. Kilzer
  for the initial code). Using 'nomatch' fixes a nasty habit of arc
  which gratuitously appends a form-feed at the end of file when using
  the 'p' (pipe) option, which might mislead a virus scanner.
  Besides, 'nomarch' is GPL licensed;

- fixed a 'broken pipe' problem when calling unrar,
  thanks to Ricardo Campos Passanezi and Rainer Link;


VIRUS SCANNERS:

- rewritten interfacing for most command-line virus scanners.
  All settings for them is now done in amavisd.conf. New ones may be
  added without having to modify the daemon source. More complex scanners
  (e.g. daemonized scanners) still need to have a corresponding interface
  routine in the daemon;

- rewritten Sophie and Trophie interfaces to be more resilient
  to Sophie/Trophie daemon restarts during virus database reloads,
  avoiding an unecessary retry (TEMPFAIL) - thanks to Cor Bosman
  for the suggestion and code, and to Dale Perkel for testing the
  Trophie interface;

- make sophos sweep tolerant to encrypted attachments: if all files are
  password protected, then the scanner failure is ignored and the message
  is allowed to pass. Based in patch by Radu Greab;

- updated nai uvscan interface to recognize the result
  'Found trojan or variant Exploit-CodeBase !!!', a patch by Anton Berezin;

- Clam Antivirus supported;
  
- fixed a problem in 'avp' scanner interface ($TEMPBASE not imported),
  thanks to Joshua E Warchol;

- avpdc (KasperskyLab AVPDaemonClient): recognize additional exit codes,
  based on patch from Christian Hammers;

- Panda new regexps, no TERM vt100 setting (thanks to Benjamin Zwittnig);

- Trend vscan exit code seems to be the number of infected parts.
  Updated the test to reflect that, based on observation from Stephane Lentz;

- MksVir scanner interface returns error code 2 if viruses are removed
  (if --clean option is passed) - a patch by Robert Litwiniec;

----------------------------------------------------------------------------
                                                               June 30, 2002
amavisd-new-20020630 release notes

Since it seems like several people are adapting amavisd-new
in details to their requirements, and certain improvement requests
have much in common, I'm making available the 20020630 release,
shortly before leaving on vacation. As this means my support
will be absent for the comming few weeks, and this release is
perhaps by few days premature, please consider it primarily a
development and new features release.

It is available at the usual location at:

  http://www.ijs.si/software/amavisd/
  (or ask Google about 'amavisd-new')

Having said that, it is still a fully functional and tested version,
and it is running in production at a couple of sites now. Also it is
my version of choice in view of dependability, having it running
at our site while I'm away. If you get into trouble, you can still go
back to amavisd-new-20020517 with which it is fully upwards compatible.

The main changes and features since amavisd-new-20020517 are:

- the code is thoroughly rearranged, interfaces cleaned, separated
  into namespaces (packages), several sections generalized (e.g. lookups,
  appending/editing header lines). The AV scanner and unpacking sections
  are still mostly the same and compatible with amavisd, so whatever
  improvements and new AV scanner support becomes available for amavisd,
  applies almost without a change to amavisd-new;

- SMTP on the input side (used with Postfix and Exim) now talks ESMTP
  (rfc2821) and not just rfc821, including some SMTP extensions:
  command pipelining (rfc2920), message size declarations (rfc1870), and
  8bit-MIME transport (rfc1652). The main reason for this was the change
  in recent Postfix varsions which can now do MIME transformations to support
  7bit transports (implied by SMTP). To ensure the transparent 8bit path and
  avoid message transformation by MTA, amavisd-new needs to declare it does
  present an 8bit-clean path. A side benefit is a little speedup in passing
  chunks of mailing list addresses due to pipelining support;

- split certain previous amavisd.conf settings (variables)
  into several variables or lookup tables:

  * sender address for notifications:
    $mailfrom -> $mailfrom_notify_admin, $mailfrom_notify_sender,
                 $mailfrom_notify_recip, $mailfrom_notify_spamadmin
    (these may also be empty to specify null reverse path <>,
    which is most useful for sender notification);

  * administrator address for notifications:
    $mailto -> $virus_admin, $spam_admin (per-sender lookups)

  * %bypass_checks -> %bypass_virus_checks, %bypass_spam_checks

- supports SQL database lookups via Perl module DBI (interface to
  popular database types). Some examples are provided, if you need
  other SQL lookups just modify the code by analogy. SQL lookups
  are most useful for per-user settings of virus_lovers, bypass_virus_checks,
  bypass_spam_checks, spam thresholds etc. when the user base is large
  and subgroups can not be identified through their (sub)domains.
  Another use is for dynamically changing settings without having
  to restart amavisd-new;

- can optionally insert 'Received:' header if acting as a mail relay
  (not with milter); does loop detection as required by rfc2821 section 6.2;

- notification messages now contain 'Date:' and 'Message-id:' headers;

- quarantined viruses contain X-AMaViS-Alert header line with names
  of detected viruses;

- quarantined spam contains X-Spam-Status and X-Spam-Level header lines;

- optionally send spam admin notifications, which include the full
  SpamAssassin spam report and message header;

- when started as root, changes UID and GID to $daemon_user, $daemon_group;

- to facilitate startup scripts and debugging, supports few simple
  command line parameters:
    amavisd        ... standard run: changes uid/gid and daemonizes
    amavisd start  ... same thing
    amavisd debug  ... starts with full debug level, stays attached
    amavisd reload ... finds amavisd master process and sends it a HUP
    amavisd stop   ... finds amavisd master process and sends it a TERM

- some more (minor) configurable options:
    $daemonize, $pid_file,
    $replace_existing_extension, $localpart_is_case_sensitive

- no longer calls Convert::UUlib for uuencoded, xxencoded, and binhex
  attachments. The first two are handled by MIME::Parser, the remaining
  are likely to be handled by anti-virus scanners, especially when some
  virus would use such encoding. The use of Convert::UUlib is dropped
  because it was causing recent problems with garbling virus so that it
  could no longer be recognized by AV scanners, and because the underlying
  library does not seem very dependable. See the thread 'Amavisd passing
  through VBS/VBSWG.gen@MM' from the beginning of June 2002 in amavis-user
  mailing list archives;

- quarantine files now include internal amavis id in the file name,
  instead of the process number; previous naming scheme could stumble
  across a name contention on a busy system;

- only a recommendation: logging via syslog is now preferred to direct
  logging to a file. It serializes the logging, and avoids locking/unlocking
  and reopening a log file by amavisd* for every log entry.
  The syslog daemon does it more efficiently and reliably.

  NOTE for Linux users: make sure you prefix the file name in syslogd.conf
  with a '-' tag to disable fsync after every write; this is most necessary
  for heavy logging such as from MTA and/or amavisd-new.  The downside
  is that you may lose the last few log entries in case of machine crash.
  See man page of your syslogd for details.

- changed examples according to rfc2606 and recent complaints on the
  postfix-users list.


CAVEATS:

- header rewriting is only available in SMTP-in/SMTP-out setup,
  i.e. with Postfix and Exim, but not with sendmail milter setup;

- the SQL lookups are a very recent addition and not so well tested
  as the rest of the program. Also since SQL lookups are supposed
  to introduce per-user settings (e.g. spam thresholds), the code
  still does not accomodate it, and spam thresholds of the first
  recipient in a message affects the whole mesage;

- as Razor2 is still pretty much unstable, I left the spam_scan()
  routine much as it was in the May version. Contributions welcome;

- as I ran out of my time for this release, I didn't prepare a
  separate version without SpamAssassin and Razor 1.20 support,
  so you will need to strip it out if you do not need it.
  Similarly for the required DBI Perl module. I'm very sorry;

- the time stamp used in the 'Date:' header in notifications,
  and in an optional 'Received:' header, does not contain true time zone
  offset, but -0000 instead (standard meaning for unknown). I did not want
  to include a fat Perl package for handling time zones. For now just edit
  the subroutine rfc2822_timestamp() if you want to change that;

- amavisd.conf settings $daemon_user and $daemon_group are not
  automatically set by ./configure options. Please set them manually.

I would like to thank many people on the amavis-user list, on the
postfix-users list, and in private converations, who contributed
valuable ideas and improvements, and offered much appreciated
encouragements.

---------------------------------------------------------------------------
                                                               May 17, 2002
Available at:
  http://www.ijs.si/software/amavisd/

amavisd-new-20020517 is primarily a response to popular demand for
Mail::SpamAssassin support. If amavisd-new-20020424 meets your needs,
there is no urgent need to upgrade. There are no incompatible changes
between these two versions, except some new (optional) amavisd.conf
variables, so you may keep old amavisd.conf file if you wish.

A new file README.exim is now provided, so that Exim 4.x is now a
supported and tested configuration, besides the usual Postfix and
sendmail/milter MTAs. Also included is a brand new qmail amavis client
by Lars Hecking (untested, please try it), plus his updated version of
amavis.c (no longer needed in recommended Postfix and Exim configurations,
and Sendmail milter and qmail configurations use a different client).

Several files are unchanged from the base amavisd CVS release 2002-05-13
and do not reflect the amavisd-new state: FAQ, HINTS, INSTALL, BUGS,
NEWS, TODO, ChangeLog, tests/, doc/. Please start with this file
README.amavisd-new-RELNOTES and follow it. More recent instructions
and last-minute changes are available from the web page.


Changes since amavisd-new-20020424:

- supports Mail::SpamAssassin and Vipul's Razor (1.20 required)
  for spam checking (but not for modifying mail body - only add headers
  and/or address extension, or reject/discard/quarantine spam).

  NOTE: spam checks are off in the default amavisd setup.
        Copy file ./amavis/amavisd.in.all to ./amavis/amavisd.in
        before running ./configure and make, to use the
        spam-check-enabled version!

  Amavisd calls Mail::SpamAssassin directly, avoiding the need to set up
  spamc/spamd or to chain filters. This is more efficient, one daemon
  less to worry about, although maybe less flexible for some taste.
  At the moment the SA per-user database is not used. Feel free to
  experiment with it and let me know what you came up with.

  The usual SA config files are observed, but remote tests are disabled
  by default (Razor is handled directly, RBL lookups can/should be
  performed early by MTA which knows what IP address mail came from,
  and that information is (mostly) lost afterwards). If you feel otherwise,
  change the hard-wired settings in the call:

    $spamassasin_obj = Mail::SpamAssassin->new(
      { dont_copy_prefs => 1, local_tests_only => 1 } )

  (also the Razor score contribution is hard-wired,
  modify: '$razor_spam_found ? 3 : 0' to will).

  Amavisd-new handles Razor checks directly to be able to exercise
  more control over it than would otherwise be available through SA:
  timing, signatures are needed for insertion into header, skips one-liner
  body checks which are common Razor false-positives. To avoid SA calling
  Razor again, either keep the default setting 'local_tests_only=>1',
  or set 'score RAZOR_CHECK 0' in the SA configuration.

  Added are 'whitelist_sender' hash and ACL lookups (see README.lookups),
  which approves spam from specified SMTP originator addresses - SpamAssassin
  can only check and whitelist rfc822 headers, not the envelope addresses,
  and I see no way of passing envelope addresses to it - which is a pitty,
  as important information is lost.

  SpamAssassin checks are computationally quite expensive compared to other
  amavisd activities, and the time needed for SA check goes up significantly
  with the message size. SA check are skipped (but not the Razor check)
  if mail size exceeds 64k. My analysis shows that presently less than 1%
  of spam exceeds 64k characters, and this is probably well below the
  false-negative SA rate, so it is not worth wasting time to check large mail.

  
remaining changes:

- new file README.exim (thanks to Jochen Erwied, Patrice Fournier
  and Igor D'Astolfo);

- updated README.postfix to describe how to avoid running header_checks,
  body_checks and dns_lookups in Postfix twice;

- put back file README.customize, which was omitted from amavisd-new-20020224
  by mistake (but available with previous version and on the web);

- added introductory paragraph to README, the rest still needs to be updated;

- added missing last-step check for '.' in hash lookups to match examples
  and make it more useful;

- added timeouts to certain tasks to make it better suitable for unattended
  operation (less, but still subject to certain DoS attacks, similar to
  the official amavisd). The main reason for adding timeouts is that we
  don't want spam checking to slow down amavisd operation too much: if it
  takes too long, just skip it (assume not spam) and move on;

- change unmangle_sender() to believe sender address for Klez viruses;
  this is sometimes wrong, but seems like people prefer to sacrifice
  few false accusations in favour of some warranted sender notifications.
  Feel free to hack this routine at will (and publish good ideas),
  it is intended to be modified;

- use Perl module Errno instead of errno.ph to avoid using broken
  file errno.ph on some popular platforms;

- make possible to have per-sender-domain administrator e-mail address
  for admin notifications (hash lookup %mailto, sub warn_admin() )

- relax temporary file/dir protection to allow them to be readable
  by the group. This makes possible for anti-virus (daemon) checker
  to run under a different user (but in the same group). Doing so should
  be safer as it makes impossible for virus-checking daemon to clobber
  files. It is only supported in the all-SMTP configuration. To use
  it with traditional amavis clients (e.g. milter), you will have to
  modify their sources to change umask and mods for file/dir they create;

- avoid changing sender address <> to <""> under certain circumstances
  (not strictly wrong, but still a bad idea);
  
- replaced header 'X-Razor-Warning:' with SA-compatible 'X-Spam-Status:' ;
  the 'X-Razor-id:' is still provided to facilitate user spam reporting;

- added '--' between options and argument when calling $sendmail_wrapper
  to be triply sure we avoid problems with some mailers (note that
  $sendmail_wrapper is only still needed in the sendmail setup,
  Postfix and Exim do not need it in the recommended configuration);

- tidy the %local_delivery_aliases mechanism for local delivery / quarantine;

- allow MIME::Parser to decode uuencoded parts, if it feels
  it can (and should) do it;

- write warnings from MIME::Parser to the log (at log level 1 or higher),
  instead of discarding them.


The package is available at the usual location at:

  http://www.ijs.si/software/amavisd/

where also the most up-to-date version of FAQ
and certain other fresh documentation files live.


---------------------------------------------------------------------------
                                                             April 24, 2002
amavisd-new-20020424 is primarily a maintenance release
to summarize one week's worth of experience with amavisd-new-20020418
and to implement some good ideas from the amavis-user mailing list.
It also brings one or two new features.

It is available at:
  http://www.ijs.si/software/amavisd/

or more specifically, at:
  http://www.ijs.si/software/amavisd/amavisd-new-20020424.tar.gz


Changes since amavisd-new-20020418:

- removed Perl 'my' declaration from configurable variables which prevented
  them from being changed in amavisd.conf (thanks to Sebastian Hagedorn
  and Wouter de Jong for reporting)

- introduced child timeouts to prevent bad amavis client from monopolizing
  a child forever (thanks to Sebastian Hagedorn for reporting the problem)

- supported and documented (in README.postfix) configuration where multiple
  remote or local SMTP-in/SMTP-out MTAs (e.g. Postfix) can use the same
  amavisd server, by making it deliver checked mail back to the same
  IP address it came from (see variable $relayhost_is_client in amavisd.conf)
  (thanks to Wouter de Jong for the splendid idea which fitted naturally
  into the overall scheme)

- in certain log messages include the SMTP-in and SMTP-out MTA IP address

- new access list checking for IP addresses - used to limit SMTP access
  to authorized MTAs only: @inet_acl; access control is now enabled by default

- slightly more sophisticated hash-based access list lookups, modeled after
  Postfix map lookups. The sequence now goes through the following steps:
    - hash lookup for user+foo@do.ma.in
    - hash lookup for user@do.ma.in  (only if $recipient_delimiter is nonempty)
    - hash lookup for do.ma.in
    - hash lookup for .do.ma.in
    - hash lookup for .ma.in
    - hash lookup for .in
    - hash lookup user+foo@
    - hash lookup user@  (only if $recipient_delimiter is nonempty)

- updated README.lookups

- cleaner quarantine code; new variable $mailfrom_quarantine allows
  to choose either the original envelope sender, or admin-specified
  fixed sender address; include a special per-user quarantine example
  (look for 'trouble-user-quarantine' in the amavisd)

- fixed problem with localized system error messages - use numeric errno
  instead of strings in Sophie and Trophie clients (thanks to Igor D'Astolfo
  for reporting the problem with italianized version of Linux and for a good
  suggestion)

- successfully tested with Exim
  (thanks to Igor D'Astolfo for testing and for reporting
  the <<doubly-quoted>> sender (but not recipient!) problem)

- wrong variable used in the spam section ($final_virus_destiny
  instead of $final_spam_destiny) (thanks to Wayne Smith for reporting
  and for testing Vipul's Razor 'plugin' patch)

- changed pattern match in the ./av/oav to support new version of
  OpenAntiVirus ScannerDaemon (thanks to Rainer Link)

- changed 'configure' to make it recognize Sophie 1.33rc1
  (thanks to Igor D'Astolfo and Lars Hecking)

- include acinclude.m4/acx_pthread.m4 macro
  (thanks to Rainer Link)

- updated README.postfix: includes instructions on how to avoid body_checks
  and header_checks for reinserted mail (contributed by Wayne Smith,
  works nicely, requires Postfix version 1.1.7-20020331 or later)

- clarified comments in amavisd.conf

- optionally keep existing X-Virus-Scanned: header lines, or remove them
  before adding our own header line - see $remove_existing_x_scanned_headers
  (requested by Darryl Harvey)

- avoid historical (misleading) parameter name $localhost_ip;
  use $relayhost and $relayhost_port instead, but take old variables into
  account for backwards compatibility with existing amavisd.conf files

- Here is an overall picture (sequence of events)
  of how pieces fit together:

    bypass_checks? ==> PASS
    no viruses?    ==> PASS
    log virus     if $log_templ is nonempty
    quarantine    if $virus_quarantine_to is nonempty
    notify admin  if $warnadmin
    notify sender if $warnsender
    notify recips if $warnrecip
    final_destiny==pass?  ==> PASS
    virus_lovers?         ==> PASS
    DISCARD or REJECT (depending on final_*_destiny)


---------------------------------------------------------------------------
                                                             April 18, 2002
This it to announce the second release of amavisd-new-20020418,
available at:

  http://www.ijs.si/software/amavisd-new-20020418.tar.gz

It is a version of amavisd (a daemonized AMaViS, which is an interface
between MTA and virus scanners), based on amavisd CVS from today (20020418)
(same configuration, amavis clients in C, instructions, AV client code),
while also being a successor of the initial release of amavisd-new-20020329,
which is a performance-enhanced pre-forked Net::Server -based amavisd
with SMTP-in/SMTP-out capability, written in Perl.

(I tried to make this release based on amavisd-snapshot-20020300, but failed,
as that version still has $errval semantics bug (the bitwise-operations
problem was discussed on the AMaViS-user list some time ago).
This was fixed in the CVS version and most AV clients were changed then.
As the CVS version contains AV client code for new AV scanners, this makes
it incompatible with config stuff from amavisd-snapshot-20020300.

If you have problems with ./configure or make, these problems would be
common to both versions. Also the unpackers and decompressors code is mostly
the same for all recent amavisd versions, so any problems in this arena
(like DoS mishandling) are most likely to be common for both the official
amavisd and the amavisd-new version.

Compared to the first release of amavisd-new-20020329, the second release
brings further significant performance improvements especially in the
SMTP-in/SMTP-out configuration, e.g. with Postfix, but also brings
some interesting new features and new configuration possibilities.

No important bug fixes were needed, so upgrade at your leisure if you are
running my initial version. The Postfix users would perhaps want to rush
a bit though, to put new performance improvements into use.

The summary of changes since the initial version:

- significant SMTP-in speedups (25% with fast AV scanner), file reuse
- pass reject reason to MTA on the input side
- more informative MTA log entries in the SMTP-in/SMTP-out setup
- amavis internal id (am_id) in log entries and passed to MTA in SMTP response
- ISP features: specify subgroups of users who want to receive viruses
- address extensions: e.g. user@domain -> user+virus@domain if virus detected
- can specify final_virus_destiny: reject, discard, pass
- quarantine new options: save to individual file, save to mailbox, pass to MTA
- new headers in quarantined viruses preserve envelope addresses
  and quarantine id (similar to the suggestion from Furio Ercolessi - see code)
- detailed timing breakdown report for each passed message
- anti-spam hooks and examples, example patch to integrate Vipul's Razor client
- body cache now always enabled - heavy speedup for mailing list bursts
- Sophie 1.33-ready
- rewritten Trophie client, based on new Sophie client code
- rewritten README.postfix, describing new setup possibilities
- new file README.lookups (to be used with virus_lovers and bypass_checks)
- new file README.customize (same as in the initial version)
- new file README.performance (unfinished)
- new amavisd.conf options, documented there
- code heavily commented, cleaned, generalized again
- does not accuse innocent users of sending viruses if we are suspicious
  of sender address (see FAQ below) (same as in the initial version)

CAVEATS:

- no test mode;

- only Postfix and sendmail milter are fully supported and tested;
  to integrate with other mailers one would need some understanding of their
  operation to set up properly; contributions are welcome;

(P.S. note: this is probably a non-issue, related to the file system problem:
  All amavisd versions seem to share one still unresolved problem, probably
  with amavis-milter.c client, its use of libmilter, or perhaps even in the
  libmilter code itself - see recent thread on the AMaViS-user list
  (subject: Leftover email.txt files with amavisd 'standard' also).
  The problem is most pronounced in burst of heavy traffic.
)

FAQ:

- Net::Server 0.82 triggers a Perl 5.005 bug (the problem is obvious:
  you get syntax errors). Either upgrade to Net::Server 0.83, or
  upgrade your Perl - 5.6.x should be ok.

- if you intend to play with customized notifications, it is wise
  to remove the '-t' option from $sendmail_wrapper_args in amavisd.conf.
  That way you are free to screw up notification mail headers any way you want,
  and the message would still be delivered to the correct recipients.
  Removing '-t' is now the recommended setting, but both variants should work.
  If you use SMTP-out method for notifications (e.g. Postfix recommended
  and default setting), this does not affect you.

- if you see virus notifications claiming the virus originator is <?>
  or <?@some.domain> and sender notifications are not sent, this is not a bug,
  but a feature - see comments at the subroutine unmangle_sender().
  The original idea comes from Furio Ercolessi: as some viruses tend to use
  forged or corrupted sender or 'From:' addresses, we try to determine
  the true virus sender, and if we can not do that, we avoid accusing
  innocent users of sending viruses.

- if you kill or HUP amavisd, temporary directories may be left undeleted;
  this is normal and mails are not lost;

- if amavisd does not restart after receiving HUP, a possible reason may be
  that amavisd can not be found in the path as set in the $ENV{PATH} variable
  (near the beginning of amavisd program). Another reason may be a syntax
  error if you changed the amavisd.conf file. Try to start it manually:
    $ su vscan -c amavisd
  If that does not make you wiser, set $DEBUG = "yes" and retry.
  This is also the recommended first-time start method.

- after changing $inet_socket_bind in amavisd.conf, you must stop amavisd
  and start it anew. The HUP method causes amavisd to stumble over its feet.

Below is my announcement notice for the initial version of amavisd-new,
just slightly edited to remove some mistakes.


---------------------------------------------------------------------------
Date: Sat, 30 Mar 2002 04:13:25 +0100
From: Mark Martinec <Mark.Martinec@ijs.si>
Subject: [AMaViS-user] ANNOUNCE: new amavisd - leaner and meaner
To: amavis-user@lists.sourceforge.net
Message-id: <01KFYI1DJ2O200AMKT@CATHY.IJS.SI>

I would like to announce an updated version of amavisd,
based on recent CVS code of amavisd (which is not far
from the February amavisd snapshot). It is a result of my
three weeks work on the code, caused by our needs for:
- better reliability;
- higher throughput (less overhead);
- versatility (e.g. separating amavis and MTA hosts, load sharing),

Available at:
  http://www.ijs.si/software/amavisd-new-20020329.tar.gz

This is now finally a version which I can recommend to friends :)
TODO: better DoS handling in unpackers.

Main features - in brief:

- pre-forked reusable children - saving on process creations;
- persistent connections to certain AV scanners, e.g. Sophie, saving on forks;
- both SMTP and pipe (sendmail wrapper) interfaces independently available
  and configurable on all three sides: input, output, notifications;
- sendmail Milter interface supported and tested; Postfix supported and
  thoroughly tested (Exim untested; classical sendmail untested, no qmail);
- customizable notification messages;
- compatibility with existing configurations;
- cleanups, generalizations, speedups, fixes, better code documentation;
- HUP signal causes restart with new configuration;
- ISP feature: certain recipients may be allowed to receive viruses
  (with alert header line added (not with milter), notifications are
  still generated);
- anti-spam hooks, caching-ready, Vipul's Razor interface in a form
  of a patch included - should make integration with SpamAssassin easier.

Install:

  Unpack the tar over the checked-out CVS version of amavisd branch
  (or February snapshot). It overlays some files, the rest is unchanged.
  For milter interface make sure you use amavis-milter.c from CVS,
  as the snapshot version contains a bug which can cause message loss
  if amavisd dies.

Details:

- child and socket handling is now delegated to Perl module Net::Server,
  which gives us pre-forked children which are able to do more than one
  mail-check during their lifetime, saving on process creations
  and giving better response time;

- Net::Server controls number of children, does signal handling,
  takes care of dead children, handles listening on multiple sockets
  (both Unix and TCP/UDP), delegating tasks and synchronizing 'accept's;

- as a consequence, certain AV-scanner interfaces (most notably Sophie,
  and soon to follow Trophie) can keep persistent connections to the
  AV checking daemon, saving on AV scanner process creations
  (forks on accept), and socket setups/teardowns;

- on the input side: both SMTP (TCP) and traditional amavis client
  protocol (Unix socket) are now accepted - even both at the same time
  (by default) to ease transition to SMTP interface;

- new input-side SMTP interface (SMTP server) is easier to set up
  (no need to worry about file ownerships, UIDs and GIDs),
  and more versatile (e.g. SMTP responses carry more information
  that sysexits.h-based status codes, SMTP can talk to remote host);
  It is based on Perl native I/O and it is quite fast - no OO overhead,
  _not_ SMTP::Server-based (which I do not consider production quality);

- a nice by-product of SMTP input interface is the extra information
  available in MTA logs, e.g. Postfix log:
  postfix/smtp[7656]: DA7B147FA6:
    to=<user@domain>, relay=localhost[127.0.0.1], delay=5,
    status=sent (250 Ok, discarded - VIRUS: EICAR-AV-Test)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- notifications messages can be delivered either via SMTP or
  as traditionally via pipe to sendmail wrapper / MTA pickup.
  This means that in the Postfix setup for example, one needs
  to worry only about one output interface (SMTP or pipe).
  This also saves unnecessary work of checking the just-generated
  notification messages for viruses;

- the output pipe interface is now more careful with status
  checking and forking;

- output SMTP interface error handling more compact;

- sender notifications can be sent to more than one contact address
  (not used at the moment, but the mechanism is available);

- sender address unmangling patch incorporated (but I'm willing
  to take it out if considered inappropriate for the base distribution);

- see file ./amavis/mta/postfix_init for comments describing
  how to select SMTP- or pipe-based output interface
  (for reinjection and notifications);

- customizable notification messages and log entry text
  (see README.customize);

- clean notion of when mail addresses are in their quoted and
  when in unquoted form (RFC2821). All internal handling uses unquoted
  form, addresses get quoted as required by the output interface,
  and quoting gets stripped away as required by each input interface;

- besides traditional choices where a mail can be forwarded/accepted,
  discarded, or temporary failure/retry indicated, there is now
  a fourth choice: REJECT. Depending on the MTA this requires a
  message rejection to be done by the input side MTA itself.
  This comes handy in cases when amavisd accepts a non-infected mail,
  but outgoing MTA does not want to take it back for final delivery,
  e.g. in case of some policy violation. Traditionally amavisd
  would indicate temporary failure on its input side, causing the
  message to be retried and re-scanned over and over again,
  without having a chance of ever being accepted;

- all existing virus scanners are still there (one little change
  in each: replaced a call to do_virus with return 1);
  Sophie client rewritten to take advantage of persistent connections;

- decoding sections are mostly unchanged. This area needs more work
  in the future;

- includes exit status codes from sysexits.ph
  instead or having them hard-wired;

- anti-spam code easier to integrate into amavisd due to some code
  rearrangements; a patch to integrate Vipul's Razor is included
  as an example, but it may be better to tie amavisd with SpamAssasin.
  Anti-spam code will NOT be integrated into amavisd, but the least
  we can do is making it easier for people to add their own code;

- cache-ready (example in the included Razor patch). This works by
  calculating a message digest (hash, signature) of the message body
  and keeping it in storage for a short while, e.g. for the lifetime
  of a child process (10 consecutive requests by default).
  If another message with the same body content arrives in the near future
  we can skip a virus check. This comes handy where mailing list traffic
  is frequent, especially if we have to deal with poorly done mailing
  list managers or heavy spam traffic;

  To support this concept the functions of MIME decoding and
  unpacking of archives are now separated.

- MIME parsing is now supplied with our own Filer subclass.
  This was necessary to avoid MIME parser complaints when it tried
  to reconstruct file name extensions from file names in strange
  character sets. Its work was completely unnecessary and harmful,
  since we want to supply our own file names and do not care
  for file name extensions.

- as a consequence of our own Filer subclass, we now avoid the first
  (or the only) directory traversal (reading) in the first decoding pass;
  Not much, but every little bit counts.

- when using sendmail wrapper it is no longer necessary to supply
  the sendmail -t option. If -t is not specified, sendmail wrapper
  receives addresses via command arguments (exec, no shell),
  which is more reliable than having to parse mail headers - which are
  now more error-prone due to user-customizable notifications.
  The -t option is still supported though, but not recommended.

- do_unzip no longer complains with multi-line backtrace when it dies;
  just a single message is issued, like with other decoders;

- HUP signal causes restart and re-reading of config file;

- many new comments, code unifications, supplied some missing
  error checks, code generalizations;


MTA support:

- thoroughly tested with Postfix in all combinations of input,
  output, and notifications interfaces; in production use;

- big thanks to Sebastian Hagedorn who helped to test the milter
  interface on his Solaris!

- Exim and traditional sendmail interface untested, but should
  work without much work.

- Qmail interface is still missing, like in current CVS amavisd.
  I believe the amavis client for qmail can be written as
  a few-dozen line Perl program. Anybody want to try?


A quick cookbook on how to set up Postfix / amavisd interface
using SMTP on both amavisd input and output side (including notifications).
Amavisd by default now accepts both Unix socket and SMTP on the loopback
interface, so the transition is easier: first install new amavisd,
then at some other time change Postfix configuration (if desired) to:

master.cf:
  # MTA -> amavisd
  smtp-amavisd      unix - - n - 2 smtp

  # amavisd -> MTA
  localhost:10025   inet n - n - - smtpd -o content_filter=

main.cf:
  # choose transport to amavisd
  content_filter = smtp-amavisd:localhost:10024

[see new file README.postfix for details]

The amavisd can now easily be located on a different host than MTA,
also Postfix load-balancing transport methods can be used (e.g. multiple
MX records). Although amavisd now talks SMTP and incorporates some rudimentary
defenses against malicious SMTP clients, do not expose its SMTP server
directly to the world - always front-end it with MTA. By default it
binds to the local interface only, but other access restrictions are
also available.

Happy amavising!  Experiences and comments are most welcome.
