Mailbox re-processing & format conversions

Re-deliver locally stored mail

check the destination

    head -20 /var/mail/root | grep To

re-deliver previously stored messages while checking the mail logs

    ls -ldF /var/mail/
    ls -lF /var/mail/
    formail -s sendmail -t < /var/mail/root && rm -f /var/mail/root
    mailq

KISS

reprocess/convert mbox

mv mbox mbox.lala
formail -s procmail < mbox.lala
rm -f mbox.lala

reprocess/convert Maildir

find cur #empty
find tmp #empty
for msg in `find new -type f`; do formail < $msg | procmail; done; unset msg
rm -rf new
rmdir cur tmp

RE-PROCESSING

dealing with duplicates

rm -f $HOME/msgid.cache

This rule uses formail to remove emails that have a message-id that has already passed through the system. It keeps an 8K log. “W” waits until it gets the exit code from formail and filters if appropriate. the ‘h’ means pipe the headers only. it uses a user-defined lockfile for this task.

# GET RID OF DUPLICATES
:0 Wh: msgid.lock
| formail -D 8192 $HOME/.msgid.cache

mbox re-processing

we need to split the messages with formail -s

mv -i mbox mbox.tmp
formail -s procmail < mbox.tmp && rm -f mbox.tmp

maildir re-processing

no need for Formail here

find folder/tmp/ -type f #should be empty
procmail < folder/new/message && rm -f folder/new/message
procmail < folder/cur/message && rm -f folder/cur/message
rm -rf folder/

FORMAT CONVERSIONS

maildir2mbox conversion

we need to add ^From and Formail does it so well compared to procmail -f -

find folder/tmp/ -type f #should be empty
formail < folder/new/message | procmail && rm -f folder/new/message
formail < folder/cur/message | procmail && rm -f folder/cur/message
rm -rf folder/

mbox2split conversion

mbox to Maildir or MH

formail -s procmail < mbox && rm -f mbox

SAVE THE WORLD

splitting files,

grep '^Return-Path: ' brokenmbox | wc -l 

that number minus 2 (think about it),

csplit -n 4 -s brokenmbox /^Return-Path:/ {901} 

find files about 10MB,

find . -type f -size +$((1024 * 1024 * 10))c             

move all matching files to another folder,

find .lists.* -type f -print0 | xargs -0 -I file mv file tosort/
rm -rf .lists.*

ALTERNATIVES

Dovecot’s dsync https://wiki2.dovecot.org/Migration/MailFormat

RESOURCES

re-processing / duplicates

\~/.procmailrc.html https://bethesignal.org/dotfiles/procmailrc.html

postfix/procmail - preventing duplicate email in procmailrc https://serverfault.com/questions/650023/postfix-procmail-preventing-duplicate-email-in-procmailrc

Procmail copies message instead of moving it https://stackoverflow.com/questions/28120828/procmail-copies-message-instead-of-moving-it

conversions

Faking “From ” header with procmail/formail https://www.depesz.com/2010/09/22/faking-from-header-with-procmailformail/

Procmail: Filter and Split Incoming Email https://www.togaware.com/linux/survivor/Procmail_Filter.html

Converting between mailbox formats https://wiki.dovecot.org/Migration/MailFormat

How can I apply my procmail filters to my maildir inbox? https://unix.stackexchange.com/questions/41971/how-can-i-apply-my-procmail-filters-to-my-maildir-inbox

How to convert maildir to mbox format https://mutt-users.mutt.narkive.com/14tojofq/how-to-convert-maildir-to-mbox-format

Convert Maildir to mbox https://stackoverflow.com/questions/2501182/convert-maildir-to-mbox

save the world


HOME | GUIDES | BENCHMARKS | html