How to Bootstrap MariaDB Galera on Ubuntu 22.04

This walkthrough is the Ubuntu 22.04 (Jammy) implementation of the seed-node procedure in Bootstrapping Your First Galera Cluster, and it answers one focused question: what does the Ubuntu package layout, AppArmor confinement, and systemd unit design change about forming the initial Primary Component? The generic bootstrap rule — one node forms the group, every other node joins — is unchanged. What differs on Jammy is where the files live (/etc/mysql/mariadb.conf.d/), how the service is started (galera_new_cluster exporting a systemd environment variable rather than a raw mariadbd flag), and what silently blocks a clean start (AppArmor denials, LimitNOFILE, and ufw). Get those distribution specifics right and the bootstrap is deterministic; miss one and you get a Permission denied mid-SST or a node that comes up standalone.

Why the Distribution Details Matter #

On Ubuntu 22.04, MariaDB runs under an AppArmor profile (/etc/apparmor.d/usr.sbin.mysqld) and a systemd unit that injects options through an environment file rather than the command line. Bootstrapping is the one moment where those two layers matter most, because the seed node commits to being the source of truth for the whole group before any peer can correct it. If AppArmor blocks the datadir or the socket during the first State Snapshot Transfer, the founding member never reaches Synced; if a stray MYSQLD_OPTS in a systemd drop-in overrides wsrep_cluster_address, the node forms a rival component and the group silently diverges. That divergence is exactly the split-brain the single-bootstrap rule exists to prevent, and once concurrent writes land on two components the write-set certification process can no longer reconcile them. Everything below is the Jammy-specific way to keep the seed node honest.

Pre-Flight: AppArmor, systemd Limits, and Ports #

Three Ubuntu subsystems must be reconciled with Galera’s synchronous replication before the seed starts. Each is a distinct gate that fails quietly rather than loudly.

AppArmor. The bundled profile confines mysqld to its datadir. Confirm it permits read/write on /var/lib/mysql/ and the InnoDB logs; if the first SST stalls with Permission denied, reload the profile and restart the service so the confinement is re-evaluated:

sudo systemctl reload apparmor
sudo systemctl restart mariadb

systemd resource limits. The default LimitNOFILE is too low for the file-descriptor pressure of an initial snapshot transfer, surfacing as Too many open files. Add a drop-in override rather than editing the packaged unit, so a package upgrade never reverts it:

sudo systemctl edit mariadb.service

Insert the override block, then reload the daemon:

[Service]
LimitNOFILE=1048576
LimitMEMLOCK=infinity
LimitNPROC=65535

sudo systemctl daemon-reload

Firewall and name resolution. Galera needs bidirectional TCP/UDP 4567 (group communication), TCP 4568 (Incremental State Transfer), TCP 4444 (State Snapshot Transfer), and TCP 3306 (client traffic). Open them in ufw with no NAT in the path, and make hostname resolution deterministic — map each node’s wsrep_node_address to a static IP in /etc/hosts or internal DNS. Ambiguous DNS caching during bootstrap is a frequent cause of failed to open gcomm backend connection: 110 (Connection timed out). Locking these ports to known peers is covered in Network Security & Firewall Rules for Galera.

The Galera Drop-In on Ubuntu #

On Jammy the configuration belongs in /etc/mysql/mariadb.conf.d/, which the packaged my.cnf already pulls in via !includedir. Create /etc/mysql/mariadb.conf.d/99-galera.cnf so it sorts last and wins over the distribution defaults:

[mysqld]
wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://"
wsrep_node_address="192.168.10.10"
wsrep_node_name="galera-node-01"
wsrep_sst_method=mariabackup
wsrep_sst_auth="sstuser:StrongSSTPassword!"
binlog_format=ROW
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2

The empty wsrep_cluster_address="gcomm://" is the whole trick: it tells the daemon to form a new group rather than join one. This value is transient — you should never leave the empty form baked into a config that persists across restarts, or every reboot re-bootstraps a fresh group. On the other nodes, set the full member list, gcomm://192.168.10.10,192.168.10.11,192.168.10.12. The wsrep_sst_method (see Initial Data Synchronization Methods) is mariabackup, the native non-blocking backup shipped with MariaDB 10.6+, which supersedes the deprecated xtrabackup-v2. Loading precedence, and why a second wsrep_provider_options line silently replaces the first, are detailed in the wsrep.cnf Configuration Deep Dive.

Bootstrap Execution with galera_new_cluster #

The bootstrap runs on exactly one node — the seed that forms the initial Primary Component. On Ubuntu 22.04, systemd owns mariadb.service, and invoking mariadbd --wsrep-new-cluster by hand bypasses the unit’s environment injection and socket setup, which is what triggers AppArmor denials or a missing MYSQLD_OPTS. Use the packaged wrapper instead:

sudo systemctl stop mariadb
sudo galera_new_cluster

The wrapper starts the service with --wsrep-new-cluster by exporting _WSREP_NEW_CLUSTER='--wsrep-new-cluster' into the systemd unit, so the daemon forms a new Primary Component instead of joining. The safe_to_bootstrap flag in /var/lib/mysql/grastate.dat is managed by the server itself on clean shutdown — the wrapper does not rewrite it. Watch the journal for the component to form:

sudo journalctl -u mariadb -f --no-pager | grep -E "WSREP|gcomm|Primary"

A healthy first bootstrap emits a sequence like:

[Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer '192.168.10.10:'
[Note] WSREP: declaring 192.168.10.10 at tcp://192.168.10.10 stable
[Note] WSREP: New cluster view: global state: 1234abcd-...:0, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3

Once the seed is Primary, the remaining nodes are added with a plain sudo systemctl start mariadb — never a second galera_new_cluster — so each requests a state transfer and joins the existing component. The controlled sequencing that keeps each rejoin on the fast IST path is covered in Graceful Node Join and Leave Procedures.

Parameter & Path Reference #

The values and Jammy-specific paths that determine whether the bootstrap succeeds:

Verification #

Immediately after galera_new_cluster, confirm the seed formed a writable component before adding any joiner:

SHOW GLOBAL STATUS WHERE Variable_name IN (
  'wsrep_cluster_size',          -- expect 1
  'wsrep_ready',                 -- expect ON
  'wsrep_local_state_comment',   -- expect Synced
  'wsrep_cluster_status'         -- expect Primary
);

Cross-check the on-disk state; after a clean bootstrap the flag must read 1:

sudo cat /var/lib/mysql/grastate.dat
# ... safe_to_bootstrap: 1

For a repeatable provisioning gate, wrap the check in a Python probe. This one uses PyMySQL and treats the two transient wsrep conflict codes — 1213 (deadlock / certification conflict) and 1205 (lock wait timeout) — as retryable rather than as a bootstrap failure, because a live multi-master group can legitimately surface them under load:

import sys
import pymysql
from pymysql.err import OperationalError

RETRYABLE = {1205, 1213}  # lock wait timeout, deadlock / certification conflict


def seed_is_primary(host: str, user: str, password: str) -> bool:
    """Return True only when the seed node holds a Synced Primary Component."""
    try:
        conn = pymysql.connect(host=host, user=user, password=password,
                               connect_timeout=5, read_timeout=5)
    except OperationalError as exc:
        print(f"[FATAL] {host} unreachable: {exc}", file=sys.stderr)
        return False

    want = {
        "wsrep_cluster_status": "Primary",
        "wsrep_ready": "ON",
        "wsrep_local_state_comment": "Synced",
    }
    try:
        with conn.cursor() as cur:
            cur.execute(
                "SHOW GLOBAL STATUS WHERE Variable_name IN "
                "('wsrep_cluster_status', 'wsrep_ready', 'wsrep_local_state_comment')"
            )
            status = {name: val for name, val in cur.fetchall()}
    except OperationalError as exc:
        if exc.args and exc.args[0] in RETRYABLE:
            print(f"[WARN] {host} contended ({exc.args[0]}); retry", file=sys.stderr)
            return False
        raise
    finally:
        conn.close()

    for key, value in want.items():
        if status.get(key) != value:
            print(f"[FAIL] {host} {key}={status.get(key)} (want {value})")
            return False
    print(f"[OK] {host} is Synced and Primary — safe to add joiners.")
    return True


if __name__ == "__main__":
    ok = seed_is_primary("192.168.10.10", "monitor", "secure_pass")
    sys.exit(0 if ok else 1)

Reusable versions of this probe, wired into alerting, are covered in Monitoring Galera Cluster State with Python.

Edge Cases & Gotchas #

safe_to_bootstrap: 0 after a crash or double bootstrap. If two nodes attempt to bootstrap, Galera’s quorum logic rejects the second and can leave the flag at 0. Recover deliberately: find the node with the highest committed sequence number (grep seqno /var/lib/mysql/grastate.dat), flip the flag on that one node only (sudo sed -i 's/safe_to_bootstrap: 0/safe_to_bootstrap: 1/' /var/lib/mysql/grastate.dat), and re-run sudo galera_new_cluster. Forcing it on more than one node guarantees data divergence. Decoding the exact startup lines behind this is covered in Handling Galera Startup Errors & Logs.

SST auth fails with mariabackup finished with error: 1. Because any node can become a donor, the sstuser account must exist as 'sstuser'@'localhost' on every node with the grants mariabackup needs. Create it once on the seed so it replicates to joiners:

CREATE USER IF NOT EXISTS 'sstuser'@'localhost' IDENTIFIED BY 'StrongSSTPassword!';
GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON *.* TO 'sstuser'@'localhost';
FLUSH PRIVILEGES;

Cloud images and Docker. On Ubuntu cloud images the packaged unit may enable ProtectHome/ProtectSystem sandboxing that clashes with a non-default datadir — mount your datadir path explicitly in the systemd drop-in. In containers there is no systemd at all, so galera_new_cluster does not exist; start mariadbd --wsrep-new-cluster directly in the seed container’s entrypoint, and make sure LimitNOFILE is raised at the container runtime level (--ulimit nofile=1048576) rather than in a unit file. AppArmor on the host still applies to containerized mysqld unless the container runs with a profile that permits the datadir.

Never bootstrap a node with live clients. Do not run galera_new_cluster on a node with active connections or pending transactions; isolate it, stop the service, and if data integrity is uncertain restore from a verified backup before re-initializing. Kernel keepalive defaults on Jammy (net.ipv4.tcp_keepalive_time=7200) are also far too high for Galera’s failure detection — lower them in /etc/sysctl.d/99-galera-net.conf (net.ipv4.tcp_keepalive_time=30) so partitions are detected in seconds, not hours.

Related #

• Bootstrapping Your First Galera Cluster — the platform-agnostic seed-and-join procedure this page implements on Jammy
• wsrep.cnf Configuration Deep Dive — loading precedence and the full provider-option matrix
• Graceful Node Join and Leave Procedures — clean shutdown that keeps rejoins on the fast IST path
• Handling Galera Startup Errors & Logs — decoding wsrep and gcomm log lines at boot