Data and mail

Mail services

A mail server for your own domains: mailboxes reached over IMAP, POP3, JMAP and SMTP submission from any mail app, and mail from the internet taken on port 25. It runs on a single server or highly available.

This is a different thing from Settings → System → Mail server, which is the outside SMTP server the panel sends its own email through.

Topologies

A mail service is created like a database, with engine mail (the New Mail form). stackd runs it on Stalwart.

Kind Mail servers Survives
mail single 1 nothing: mail waits at the senders' end while it's down
mail ha 3, 5, 7 or 9, one per server losing a minority: the others serve every mailbox

Creating a service also creates the stores it keeps its data in, on the same servers and with the same topology: mail-db (PostgreSQL), mail-blobs (object storage) and mail-cache (Redis). They don't appear in lists, and their pages say which service they belong to. Links, external access, domains, admin credentials and backups are refused on them, and they go only when the service is deleted. Their members can be added or removed on their own pages.

Any mail server serves any mailbox. A server holds one mail service at most, because it has one port 25.

The mail host

The service's external_host (mail.example.com) is the name its mail servers answer as (SMTP greeting, MX target), what mail apps connect to, and where the web is served:

  • HTTPS. Every mail server's proxy serves the host: Stalwart's web admin at /admin, JMAP at /jmap, and its account page.
  • Certificate. A Let's Encrypt certificate is obtained for the host as for an app's domain, once the host's A and AAAA records point at the servers. It's used for HTTPS, SMTP, IMAP, POP3 and ManageSieve, and renewed like any other. Until then, the mail servers serve a self-signed certificate.
  • Firewall. While any mail service exists, the host firewall opens 25, 465, 587, 143, 993, 995 and 4190 on every server. It's harmless where nothing listens. Port 24480 stays closed to the internet; without the host firewall, it's as open as the panel's 7080. Banned addresses and blocking feeds are dropped on the mail ports, and the mail servers' own bans (failed logins, abuse, loitering, scanning) become bans on every server (Firewall).

The host must be unique among databases' hosts and apps' domains. App ports 24379–24480 are refused (Ports).

Reverse DNS

Receivers look up the PTR record of the address mail comes from, and check that the name it gives resolves back to that address (forward-confirmed); Gmail refuses mail without one. Each server's addresses should name the mail host. That ties the PTR to the mail host only, never to the service's domains: they publish an MX to the mail host and sign with their own DKIM keys, which is what DMARC checks. A host of your own that doesn't change (mx.yourcompany.net) keeps it right whatever domains are added, as mail providers do.

GET /api/databases/{id}/mail/reverse-dns looks up every mail server's public addresses in public DNS: the PTR name, then its A or AAAA records. Each address is:

  • matches: names the mail host, which resolves back;
  • other: another name that resolves back (accepted, less trusted);
  • no_forward: the name has no A (or, for IPv6, AAAA) record: add them in the name's zone, or point the PTR at the mail host;
  • unconfirmed: the name resolves, but not to the address;
  • missing, or unknown when public DNS didn't answer.

It also gives cloud (hetzner, digitalocean) when stackd made the server, so it can set the record. The Domains tab shows it per server, with what to ask the provider for.

POST on the same path (admins: it uses the cloud provider's token) points every address whose PTR doesn't resolve back at the mail host, for servers stackd made. On DigitalOcean the PTR comes from the droplet's name, so the droplet is renamed to the mail host there; the server keeps its name in stackd. It answers what was set, server by server, with the provider's error for those that weren't. Public DNS can show the old name until it expires.

Each mail server greets (EHLO) as its own address's PTR name once that resolves back, and as the mail host until then. Changes are picked up within 15 minutes, or at once after a reverse DNS check. Each mail domain's records then include v=spf1 a -all at every name a server greets as, and Set up DNS publishes them with the rest.

Domains and mailboxes

The panel's Mail page manages them (GET /api/databases/{id}/mail and below). Anyone who may change the service may change them; viewers read them.

  • Domains. POST …/mail/domains takes mail for another domain, with its own DKIM keys (Ed25519 and RSA, rotated every 90 days). DELETE …/mail/domains/{id} removes one with no mailboxes left. The first domain can't be removed.
  • DNS records. Each domain lists what to publish: the mail host's A and AAAA records (every mail server's public addresses), then MX, SPF (for the domain and the host), DKIM, DMARC, TLS reporting, and SRV records for IMAP, POP3, submission, JMAP, CalDAV and CardDAV. MTA-STS, autoconfig/autodiscover and TLSA (DANE) records aren't listed: a TLSA record pins the certificate's key, which every renewal replaces, and a stale one makes senders that check DANE refuse the mail.
  • Setting up DNS (admins only). Publishes a domain's records at Cloudflare or DigitalOcean with a token kept under Settings → System, DNS providers (the same as for wildcard certificates, Domains). POST …/mail/domains/{id}/dns with provider_id previews first. For each record it says whether the record is published, will be created, or conflicts. A conflict is something that contradicts the record: another address for the mail host, another MX, another SPF, DMARC, DKIM or TLS-RPT policy, a CNAME at the name, or an address proxied through Cloudflare (mail can't pass its proxy). With apply it creates what's missing. With replace it first deletes what conflicts; otherwise conflicts are left alone. Records stackd doesn't need are never touched, so a site verification TXT or the web's A record stays. Addresses go in unproxied. Once anything is published, the mail host's certificate is tried again at once. In the panel, each domain is collapsed to one line (mailboxes, record count); opening it shows its records and the DNS setup, with each record's state at the provider after a preview.
  • Mailboxes. POST …/mail/accounts takes a local part, a domain, aliases in the same domain, a description and a limit in bytes. The password is made unless given, and returned once. PATCH changes the password (or makes a new one with reset_password), the description, the aliases or the limit. DELETE deletes the mailbox and its mail. An address already taken by a mailbox or alias is refused.
  • Web admin. For what the panel doesn't cover (groups, mailing lists, spam settings, sieve, the queue, logs), sign in at https://<host>/admin as stackd. Admins can reveal the password (POST …/mail/credentials). Changing listeners or stores there undoes what stackd set up.

Test messages

The Domains tab's Send a test message, or POST /api/databases/{id}/mail/test (from, one of the service's mailboxes or aliases; to, any address; server_id, optional), sends a test message from the mailbox through the mail server on that server, the first that answers when left out. The message is the mailbox's own, DKIM-signed for its domain and sent from that server's addresses; no mailbox password is needed or changed. It answers once the message is queued, with the queue id; delivery follows from the queue. Anyone who may change the service may send one.

Mail apps

The full address is the user name. IMAP is <host>:993 (TLS) or 143 (STARTTLS), POP3 <host>:995, and sending <host>:465 (TLS) or 587 (STARTTLS). JMAP is at https://<host>/jmap, and ManageSieve at <host>:4190.

Operations

  • Members. Mail servers can be added (another server, any time, even above the stores' count) and removed (down to two) on the Servers tab. A new one joins as it starts. Drains move a mail server like any member.
  • Updates. When a stackd release changes the mail servers, they're redeployed one at a time, and only while all are healthy, so the others keep taking mail. A single-server service is down for the few seconds it takes to restart.
  • Reboots. A coordinated reboot waits for the stores and every mail server to be healthy. It doesn't take the last server with a mail server.
  • Diagnostics. A member's diagnostics list the mail servers the cluster knows and the messages waiting to go out.
  • Deleting stops the mail servers, then deletes the stores and the service, with their data unless kept.

Limitations

  • Backups. The service can't be backed up as one yet. Its PostgreSQL store can be dumped on its own, but restoring it without its messages, which are in the object store, gives mailboxes whose messages are missing.
  • No webmail. Mail apps, or a webmail app deployed from an image (IMAP and SMTP to the mail host), cover it.
  • Reverse DNS and port 25 are the servers' provider's. Many block outgoing 25 on new accounts, and receivers distrust addresses whose PTR record doesn't name the mail host. stackd checks PTR records, and sets them for servers it made at Hetzner or DigitalOcean (see Reverse DNS); others' are their provider's.
  • One service per server, since there's one port 25. Apps can still publish 25 or 993 on servers without a mail server, and on a server with one they fail to start, with Docker's reason.
  • MTA-STS and client autoconfiguration aren't served.