Get your hands dirty with Procmail

Warning

Maildir on netbsd

pkg_add procmail

PATH=/bin:/usr/bin:/usr/local/bin:/usr/pkg/bin
SHELL=/bin/ksh
DROPPRIVS=yes
VERBOSE=no
ORGMAIL=$HOME/Maildir/
MAILDIR=$HOME/Maildir
DEFAULT=$ORGMAIL
SYSYEAR=`date +%Y`
LOGFILE=$HOME/procmail.log.$SYSYEAR

MH on gnu/linux

apt install ksh procmail

PATH=/bin:/usr/bin:/usr/local/bin:/usr/local/bin
SHELL=/bin/ksh
DROPPRIVS=yes
VERBOSE=no
ORGMAIL=$HOME/mailbox/inbox/.
MAILDIR=$HOME/mailbox
DEFAULT=$ORGMAIL
SYSYEAR=`date +%Y`
LOGFILE=$HOME/procmail.log.$SYSYEAR

using an inbox inside the mail folder in view to avoid this error

procmail: Lock failure on "/var/spool/mail/USERNAME.lock"

you could also try ORGMAIL=/var/mail/$USER and make sure the variable works

Useless

This should not happen, as we are the MX and we block spam at session time, but it does not harm

:0
* ^X-Spam-Status: Yes
spam-status/.

Auto-archives

save the date part of the last Received header

:0
* $ ^Received:.*for .*; \/(Mon|Tue|Wed|Thu|Fri|Sat|Sun), .+
{
        MFROM_ = $MATCH
        #LOG="MFROM_ = $MFROM_"

        :0
        * MFROM_ ?? ()\/[1-9][0-9][0-9][0-9]
        { YEAR = $MATCH }
        #LOG="YEAR = $YEAR"
}

deal with sent-mail (Received header is missing)

:0
* ? [[ -z $YEAR ]]
{
        :0
        * $ ^Date: \/(Mon|Tue|Wed|Thu|Fri|Sat|Sun), .+
        {
                MFROM_ = $MATCH
                #LOG="MFROM_ = $MFROM_"

                :0
                * MFROM_ ?? ()\/[1-9][0-9][0-9][0-9]
                { YEAR = $MATCH }
                #LOG="YEAR = $YEAR"
        }
}

define and do some tests against YEAR otherwise you may end up with .archives..inbox

:0
* !? [[ $SYSYEAR = $YEAR ]]
{ prefix = archives/${YEAR}- }

:0
* ? [[ -z $YEAR ]]
{ prefix = archives/DONNO- }

also see * !? [[ $SYSYEAR = $YEAR ]] in the Finish-up section

Sent-mail incl. alt-domain

:0
* !^Received:
* ^From:.*email@domain\.tld
${prefix}sent/.

:0
* !^Received:
* ^From:.email@domain2\.tld
${prefix}domain2.sent/.

:0
* !^Received:
${prefix}sent.donno/.

Alt-domain

:0
* ^(To|Cc):.email@domain2\.tld
${prefix}domain2/.

Cron jobs

:0
* ^From:.*(root@|USERNAME@|Cron Daemon|logwatch@)
${prefix}cron/.

Mailing-lists

avoid anoying mailing-list membership reminders. note there is no space between subject: and Rappel_pour

:0
* ^From: .*mailman-owner@
* ^Subject:.*( mailing list memberships reminder|Rappel_pour_les_abonnements_aux_listes_sur_)
/dev/null

dealing with mailing-lists Sender: "some name" <blah@blah.org> or "some name" <blah+user=domain@blah.org>

:0
* ^Sender: .*<\/([^+@]+)
{
    SENDERID=$MATCH

    :0 
    * SENDERID ?? (owner-|-owner|-bounces)
    ${prefix}list-${MATCH}/.
}

dealing with mailing-lists Sender: "some name" <blah@blah.org>

:0
* ^Sender: .*<\/([^@]+)
{
    SENDERID=$MATCH

    :0 
    * SENDERID ?? (owner-|-owner|-bounces)
    ${prefix}list-${MATCH}/.
}

dealing with mailing-lists Sender: blah@blah.org

:0
* ^Sender: .*\/([^@]+)
{
    SENDERID=$MATCH

    :0
    * SENDERID ?? (owner-|-owner|-bounces)
    ${prefix}list-${MATCH}/.
}

note senders that did not comply with SENDERID ?? are passing through

now dealing with mailing-lists List-I[Dd]: <blah@blah.org>

:0 
* ^List-ID: <\/([^\.]*)
{ 
    LISTID=$MATCH

    :0 
    * LISTID ?? (early-questions|gr-gsm|fabelier)
    ${prefix}list-${MATCH}/.
}

note lists that did not comply with LISTID ?? are still sorted out

dealing with other kinds of mailing-lists

:0
* ^(List-Id|List-ID|X-(Mailing-)?List):
${prefix}listloose/.

Finish-up

everything that did not get sorted out before and that is not To nor Cc

:0
* !^\
(To:.*(USER|sales|abuse|contact|info)@domain\.tld\
|Cc:.*(USER|sales|abuse|contact|info)@domain\.tld)
${prefix}bcc/.

archiving older incomming messages while defining YEAR for DEFAULT

:0
* !? [[ $SYSYEAR = $YEAR ]]
* ^\
(To:.*(USER|sales|abuse|contact|info)@domain\.tld\
|Cc:.*(USER|sales|abuse|contact|info)@domain\.tld)
${prefix}inbox/.

the rest (incomming messages for the current year) goes to DEFAULT

Ready to go

tail -100 -F ~/procmail.log.`date +%Y`

(anything)

vi ~/.forward

"|IFS=' '&&p=/usr/pkg/bin/procmail&&test -f $p&&exec $p -f-||exit 75"

"|IFS=' '&&p=/usr/bin/procmail&&test -f $p&&exec $p -f %F||exit 75"

Postfix

vi /etc/postfix/main.cf

mailbox_command = /usr/bin/procmail

Fetchmail

mda "/usr/bin/procmail -f %F"

Additional notes

this might be incompatible with heavy re-processing (header might be added multiple times)

# @sales
:0 hfw
* ^(To|Cc):.*sales@nethence.com
* ^Subject:\/.+
| formail -i "Subject: [sales]$MATCH"

# @abuse
:0 hfw
* ^(To|Cc):.*abuse@nethence\.com
* ^Subject:\/.+
| formail -i "Subject: [abuse]$MATCH"

# @contact @info
:0 hfw
* ^(To|Cc):.*contact@nethence\.com
* ^(To|Cc):.*info@nethence\.com
* ^Subject:\/.+
| formail -i "Subject: [contact]$MATCH"

Re-processing & format conversions

see procmail.reproc

Troubleshooting & Repairs

Find broken messages

looking for new line and Return-Path instead of From

15:12:51 < geirha> elge: awk -v RS= '!/^From/' mbox
15:13:17 < geirha> or awk -v RS= '/^Return-Path/'
15:13:45 < geirha> and maybe throw in   -v ORS='\n\n'  too

==> the second one works out of the box

Resources

Procmail FAQ http://porkmail.org/era/procmail/mini-faq.html

Procmail Tips & Tricks http://novosial.org/procmail/

Automatic folders for mailing lists using procmail https://blog.sandipb.net/2011/05/06/automatic-folders-for-mailing-lists-using-procmail/

Jari’s Procmail Tips Page https://www.howtoforge.com/procmail_tips_recipes

prevent procmail from adding Return-path https://serverfault.com/questions/408849/prevent-procmail-from-adding-return-path

Maildir

Converting Mbox mailboxes to Maildir format http://batleth.sapienti-sat.org/projects/mb2md/

Maildir Configuration https://wiki.debian.org/MaildirConfiguration

procmail is creating Maildir files instead of mbox https://groups.google.com/forum/#!topic/comp.mail.misc/GbGwg31uDk4

Troubleshooting

3.0 Dry run testing http://pm-doc.sourceforge.net/doc/#dry_run_testing

Debugging Procmail Recipes: Some Tips http://porkmail.org/era/mail/procmail-debug.html

procmail: lock file failures https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=160753

Procmail lock failures and errors while writing https://serverfault.com/questions/212794/procmail-lock-failures-and-errors-while-writing

How to Open MBOX files in Thunderbird. https://www.wintips.org/how-to-open-mbox-files-in-thunderbird/


Nethence | Pub | Lab | Pbraun | SNE Russia | xhtml