I got off GMail back in 2013, right after the Snowden leaks. I temporarily moved my email to my hosting provider at the time, and then tried to follow a guide "NSA-proof your e-mail in 2 hours":

=> https://sealedabstract.com/code/nsa-proof-your-e-mail-in-2-hours/index.html

By the time I got to "Open DKIM", "SPF", "Reverse PTR", etc. I'd given up. It's not that it's hard, but I could see how getting myself off spam blacklists and avoiding other people's spam boxes would end up becoming a full time job for me. So I decided to try something slightly different.

Goals

To figure out how to handle my email, I needed to have some explicit goals in mind:

• I want all my email on my own server, under my control, and searchable/archivable in an easy plain text format, like Maildir or mbox.
• I would prefer email I send to my friends/colleagues to be routed and stay within my country, rather than being routed to and stored in/via another country (unless they are using GMail or similar).
• I don't want to have to work constantly with domain settings and requesting myself removed from spam blacklists, etc.
• If I use a hosting provider for my email, I'd like to be able to quickly switch to another hosting provider with minimal downtime. I don't want to be tied to a specific provider.

The method

The method for achieving this is quite simple:

• Sign up for a cheap email hosting account at a local hosting provider. The local part is important for many reasons: your email doesn't need to be routed unnecessarily far away; you may get better support locally; you may get a better price locally; your provider is beholden to local laws; you support a local business.
• On a regular basis (e.g. once a minute), pull down all email from the email account into your own server, and serve it using IMAP. This part is easier than it sounds, see below.
• Filter spam out while pulling in the email into your IMAP server.
• Optionally install a webmail service on your server so that you have GMail-like usability.

This is what I refer to as semi-self-hosted email, since the actual mail and SMTP server is at a hosting provider of my choosing, but I store and interact with my email from my own server.

Dovecot

Dovecot is the IMAP server I use. The first part of the afore-mentioned article on setting up your own email server outlines how to install and configure Dovecot. I installed SSL certificates to allow access to the IMAP server over TLS and firewalled it to the IP address range of my ISP to reduce the chances of it being hacked. If I need to access it while on a trip, for example, I use the webmail interface instead (see below). I wouldn't bother with the encryption of the mail directory though, I can't think of a scenario where that might be useful - your email needs to stay in unencrypted form on your server anyway, so if it's hacked, your email can be copied.

=> Dovecot

Fetchmail and Procmail

Now to pull new email from my hosting provider into Dovecot. For this, I use Fetchmail. It's as easy as installing it and adding a .fetchmailrc config file to tell it where to fetch email from, and then run it on boot. And example config file:

poll pop3.myhostingprovider.com protocol POP3
   user "user@mydomain.com" is toby here
   password 'thepasswordformyemailaccountatmyhostingprovider'
   fetchall
   fetchlimit 500
   nokeep
   mda "/usr/bin/procmail -m /home/toby/procmail/procmail.conf"

In the above example, I tell Fetchmail to connect to POP3 server "pop3.myhostingprovider.com", then fetch the email from my registered account there "user@mydomain.com" with the specified password, and pull it into the local "toby" email account in Dovecot. The other options mean that it fetches all email on the server and deletes them off the server (nokeep).

=> Fetchmail

Fetchmail can pull email from multiple accounts into one (just repeat the poll block above), which is quite handy! It also supports multiple protocols like POP3 and IMAP. It can push the email straight into Dovecot, however as per my example above, I invoke Procmail on each received e-mail (the line starting with mda), passing in the 'procmail.conf' configuration.

=> Procmail

Procmail allows me to act on the incoming email in almost unlimited ways, for example I can invoke programs on the email, I can filter the email, I can send auto-reponses, I can set up auto-forwards, etc. My main use for Procmail is to filter out spam. I used to simply add regular expressions to filter spam such as:

# ------------------- Blacklist
:0
* ^(Subject:) .*(viagra|cialis|unforgettable|pills|hack you|michel kors.)
/dev/null

:0
* ^(From:|Reply-To:) .*(no-reply@|noreply@|messages-noreply@|promotions)
/dev/null

However, eventually after adding rules for whitelists and blacklists and pre-whitelist-blacklist (no kidding), I decided I needed a new approach to spam filtering, where I'd rather err on the side of losing a few legit emails than have to constantly update my rules for spammers. My new simplified procmail config only has 2 rules, and it looks like this:

# ALLOW: If it mentions my name or contains my signature
:0 B:
* ( Toby | Toby!| Toby\.| Toby,| Toby$| Kurien | Kurien!| Kurien\.| Kurien,| Kurien$| @tobykurien | @tobykurien$)
toby/

# DISALLOW: If it's not in my whitelist (email addresses in my email box)
FROM=`formail -XFrom: | formail -r -xTo: | grep -i -o '[A-Z0-9._%+-]\+@[A-Z0-9.-]\+\.[A-Z]\{2,4\}'`
:0
* ! ? grep -F -i -q "$FROM" /home/toby/whitelist.txt
/home/toby/spam

# -------------------- Catch all
:0
toby/

So basically, only allow an email through if it mentions my name (e.g. starts with "Hi Toby!") or is a reply to my email, where I know my signature contains "@tobykurien"; or allow an email through if I've already received (and kept in my mail box) emails from them before. Any other email goes into my spam box (which I can still access via Dovecot), but I have a cron job to clear that spam box every night. This means I can still get at any two-factor e-mails or the like if I need to. This is a fairly hard-line stance for managing email, but it's worked incredibly well for me so far - I get at most 1 spam email per month that sneaks through, but rarely miss any important email (at least, to my knowledge!)

How does the second rule work, i.e. how does procmail know what email addresses are in my mail box? The rule checks to see if the email occurs in the "whitelist.txt" file. This file is re-generated every night by the following script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#!/bin/sh

# create a list of email addresses from my mailbox
grep "From: " -r /Mailbox/toby/* | cut -d ' ' -f2-  > whitelist2.txt

# strip out the exact email addresses and remove duplicates
grep -i -o '[A-Z0-9._%+-]\+@[A-Z0-9.-]\+\.[A-Z]\{2,4\}' whitelist2.txt | sort| uniq -i > whitelist.txt

# remove some email addresses from the whitelist
egrep -v -i "tobykurien|noreply|no-reply|marketing" whitelist.txt > whitelist2.txt && mv whitelist2.txt whitelist.txt

mv whitelist.txt ~/whitelist.txt

SMTP

So with the above setup, I can use an email client like Thunderbird/Evolution on desktop, or K-9 mail/FairEmail on Android to send and receive e-mail. For sending email, I can simply configure the client with the SMTP server details provided by my hosting provider. I did also opt to install an SMTP server on my own server, which has been working fine for me. This did require me to set up SPF, Reverse PTR, etc.

Web mail

Since I move around devices and laptops often, I wanted the ability to access my email via a web browser. This also allows me to add stricter firewall rules to deny access to my IMAP and SMTP servers from outside my ISP's IP address range, for additional security. I initially tried a few options like Horde and Roundcube, but by far the easiest and nicest (for me) solution has been Rainloop. Installing it is as easy as unzipping it into your web server directory and then editing a config file with the details of your IMAP and SMTP server. It has a beautiful and simple interface that works on both desktop and mobile. It even integrates with my WebDAV server to pull in all my contacts, so for me it has all the features I could need, so much so that I no longer use email clients, I just use Rainloop exclusively.

=> Rainloop

Summary

In summary, I semi-self-host my email by using a local email hosting provider to provide the email service. I then pull in all email from the provider into my own Dovecot server and then delete the mail off the hosting provider, giving me storage and control over my email archive. In the process, I also filter out spam using two simple rules: either you greet me by name, or I've already contacted you before. This almost entirely eliminates spam. I use Rainloop web mail interface to read and respond to emails from my web browser on either desktop or smartphone. This setup gives me control over my email, allowing me to easily backup my entire email archive using rsync, and I can search and process my email as plain text files (a fact I use to generate my whitelist, for example). Finally, I can easily add additional email accounts or domains by simply adding them to my fetchmail config, and I can move email hosting providers easily.