Zum Inhalt

A private mailserver

Somehow you need to archive your personal eMails. I don't like to store all Mails on Mail Hoster like here in Germany GMX, Web or Google mail. I'd like to store the mails on my private server space.

Since year I work with a private Raspberry Pi and a Cyrus imap server handling my mails locally. Each night the mails are transferred using POP3 to my local environment. I can search for specific topics of all my mails and work with my Mails locally. First I used a NAS server and later a Raspberry Pi with a USB-Disk. A second mirror inside the network helps to improve the secure storage of my Backup solution.

Old Cyrus-based IMAP solution

In this old solution the SPAM filtering does not work well and sorting of mails into specific directories was not working. Sure it is possible to do this with Postfix and/or Cyrus IMAP, but it was too complicated.

The corresponding configuration must be done in the system main configuration in Unix and so it is not possible to separate it like it is possible in Microservice environments. There is no way back or cannot check new configurations before activate.

In addition the afford to handle and administrate this environment is getting bigger. It seems that the new versions and migration took a lot of work to be done. That is independent to Postfix or Cyrus IMAP configuration changes. I used this approach for all family members and the administration of users and corresponding security is getting complicated. You need to add the user to the Linux users.

New Docker-based mailserver

The old Cyrus-based IMAP is included in an Microservice based approach where the overall system configuration is transfered to an Microservice configuration. Any small Docker or Podman service can start the service. A small hard disc contains configuration and mail data. A backup can by done using a simple rsync command to synchronize the single directory data to another destination. Either a backup or the move of the service are very quickly done. Maybe a cluster of Microservice instances can be started in parallel. This all is prepackaged by a solution I've found. Better then any native Linux based solution.

I use following mailserver-configuration for this Microservice:

https://github.com/docker-mailserver/docker-mailserver

This solution works inside a Docker container and contains an IMAP server, different SPAM filter you can enabled and a filtering or manage tools to pre-sort mails received.

In the old Cyrus IMAP approach I've used already the mail storage in 'Maildir'. I only need to copy with rsync to another directory and start the service. The configuration is strict forward and easy. Only the usage of sieve was a little bit of learning curve needed.

An example to get started is here:

DMS_GITHUB_URL="https://raw.githubusercontent.com/docker-mailserver/docker-mailserver/master"
wget "${DMS_GITHUB_URL}/compose.yaml"
wget "${DMS_GITHUB_URL}/mailserver.env"

Following Docker-compose configuration I used to start the service:

services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:15.0.2
    container_name: mailserver
    # Provide the FQDN of your mail server here (Your DNS MX record should point to this value)
    hostname: <mail host alias>
    env_file: mailserver.env
    # More information about the mail-server ports:
    # https://docker-mailserver.github.io/docker-mailserver/latest/config/security/understanding-the-ports/
    ports:
      - "25:25"    # SMTP  (explicit TLS => STARTTLS, Authentication is DISABLED => use port 465/587 instead)
      - "143:143"  # IMAP4 (explicit TLS => STARTTLS)
      - "465:465"  # ESMTP (implicit TLS)
      - "587:587"  # ESMTP (explicit TLS => STARTTLS)
      - "993:993"  # IMAP4 (implicit TLS)
    environment:
      - ENABLE_FETCHMAIL=1
      - FETCHMAIL_POLL=60
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
    restart: always
    stop_grace_period: 1m
    # Uncomment if using `ENABLE_FAIL2BAN=1`:
    cap_add:
      - NET_ADMIN
    healthcheck:
      test: "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1"
      timeout: 3s
      retries: 0

To start the mailserver just start the docker-composer startup like

docker-compose up -d

The mailserver service can be stopped like that

docker-compose down

Create users

The 'mailserver' solution creates users inside the Docker instance by creating email infrastructure. After the Docker container is started the first time you need to setup the new environment. You need to start the 'setup' script to add users or configure some specific topics.

It is possible to start the command using docker or podman commands inside the mailserver container. Starting the command without parameters and you will get the possible options of the setup script.

docker exec mailserver  setup

You can add the email address to the configuration using following command:

docker exec mailserver  setup add test@example.com

A corresponding user can use a configuration to retrieve the mail using POP3 or IMAP. You can use either fetchmail or getmail defining the email address as an destination of the Mail.

Mails received periodically

My origin solution was that each night between 2 and 3 o'clock all mails will be fetched from all mail host to my local environment. The mails will be deleted on the remote mail host. In this case all read and not deleted emails will be kept at home. This will be done by starting a fetchmail command in the night. At 5 o'clock the current mail will be synchronized to an internal second NAS server. Especially mails with receipt or important information need to be archived at home.

In the local environment I can access all my mail using normal IMAP access with any Mail program I like. It is not SSL encrypted and sending can be done in parallel Mail accounts and not through the mail server. The fetchmail were done in night for all users. Each user can access his mail with different user and search or read his mail. You need to review all mails once more because they are displayed unread. I got the impression, that reviewing email was good instead of annoying.

The new architecture I can use the same functionality. In this case the IMAP service configuration need not to be adapted for all users, only migration the service to another host the host need to be adapted. On problem in the new version is that the periodically fetch of all mails is done not in an specific point in time but in an time period defined during start. In my depend on the start of the service that the interval is not in the night. In addition the mails on the server are defined as read with fetchmail. This is not good because user do not see the mails they already read or not. A alternative fetching tool is getmail which reads the mails but let the reading flag on remote server unattached.

Fetchmail

In <mail-data>/dms/config/fetchmail it can be added IMAP email addressesn. fetchmail is activated using following environment variables:

ENABLE_FETCHMAIL=1
 FETCHMAIL_POLL=60

Here every 60 seconds will start fetchmail to fetch the mails to the local mailserver environment. In the file <mail-data>/dms/config/fetchmail it looks like the normal fetchmail syntax und you can take over each configuration referencing local email address:

poll pop.gmx.net with proto POP3 uidl
    user 'user@gmx.xxx' there with password 'ekrferoe' is 'user@example.de' here fetchall ssl keep
    sslfingerprint "1A:EF:49:31:84:6C:87:8D:11:5E:2A:E6:C1:48:08:AF"

GetMail

In the <mail-data>/dms/config/getmail directory the configuration of all users are defined. Every user mail host configuration has it's own In the directory are all configuration for all users. Everybody is getting either in IMAP or POP3 service. In comparison of fetchail Getmail can not only POP3, instead you can use IMAP to load the mails. You need to add some additional IMAP referenced path to load from mail hoster these paths as well.

Generic configuration for all mail configuration are done in getmail/getmailrc_general.cf:

[options]
verbose = 0
read_all = false
delete = false
max_messages_per_session = 500
received = false
delivered_to = false

Here is a user specific configuration for GMX

[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmx.net
username = <GMX email>
password = <GMX password>
mailboxes = ("INBOX","SubMail") // zusätzliche Pfad vom IMAP laden

[destination]
type = MDA_external
path = /usr/lib/dovecot/deliver
allow_root_commands = true
arguments =("-d","test@example.com")

Finally the night solution

Because Getmail does not define the read flag, the mail is fetched but not missed. The second service trigger the fetchmail with all corresponding fetchmail reads in the night only. This download and final delete on the remote service is done in the night only. With this I don't need to rsync the services in the night once more.

Sieve filter configuration

Some mails should be moved to an corresponding sub-directory for sorting or filtering the mails. In mailserver the sieve filtering is included. You need to define the filtering rules in an programmable sieve configuration file. In the email/user directory exists a home directory containing some user specific configuration . One of this configuration might be the sieve filter. In the directory dms/mail-data/<user>/home put the file .dovecot.sieve containing filter option. The syntax can be search in the web, a lot of information are available there.

This is an example configuration of different simple sorting of mails into IMAP path sub-folders:

require ["fileinto", "reject"];

if address "From" "noreply@transactions.paramountplus.com" {
  fileinto "Spam";
} elsif address "From" "user@googlemail.com" {
  fileinto "TestMail";
} elsif address "From" "jobs@mail.xing.com" {
  fileinto "Privat/OpenBC";
} else {
  keep;
}

Mailserver configuration

To initial configure the mailserver service you need to start the container. There is a time frame in the first start where you can add new users to the mailserver container. It is independent if the mailserver container is started inside Docker or Podman. Once a user is added the mailserver will working and you can connect from any mail program you like.

The mail program parameters are standard. If you have not configured SSL it may need to connect using a non-SSL connection. The user is the name of the email address (e.g. user@example.com).

You can generate and use a self-signed certificate to encrypt access to the mailserver. At the moment I need to review the certificate usage, becaause a number of service already use SSL in my local environment. Each time I need to add the certificate on all client devices.

You can enable mails using the SMTP service of mailserver as well but I do not use that at the moment.

TO_DO

  • Using the SSL access by creating self-signed SSL Certificates
  • Evaluate further filter possibilities