This guide builds a central log server with rsyslog on Red Hat Enterprise Linux, AlmaLinux, Rocky Linux, Oracle Linux and CentOS Stream. Clients forward their logs over TCP, the server writes them into one directory per host, the forwarding queue survives server outages, and the last part wraps everything in TLS with mutual certificate checks. The steps are tested against the rsyslog packages in AppStream/BaseOS on EL9 and EL10 and also work on EL8.
What you will build
loghost.example.com(192.168.1.50): the central rsyslog server, receiving on TCP 514 (plain) and TCP 6514 (TLS).web1.example.com,db1.example.com: clients forwarding everything with a disk-assisted queue.- Remote logs stored under
/var/log/remote/<hostname>/<program>.log, kept separate from the server’s own/var/log/messages. - Daily rotation with logrotate, SELinux left in enforcing mode, firewalld open only for the log ports.
Prerequisites
- EL8, EL9 or EL10 (RHEL, AlmaLinux, Rocky Linux, Oracle Linux or CentOS Stream) with root or sudo.
- Working forward and reverse DNS (or
/etc/hostsentries) for the server and every client. TLS name checks depend on it. - Time in sync on all hosts with chronyd (
chronyc tracking), otherwise timestamps and certificate validity get confusing. - firewalld running and SELinux enforcing (the defaults).
How rsyslog works on Enterprise Linux
On EL systems, systemd-journald collects messages first and rsyslog reads them through the imjournal module, then filters them by facility and priority and writes them to files under /var/log or forwards them. The main file is /etc/rsyslog.conf, and it includes every *.conf file in /etc/rsyslog.d/ in lexical order. Forwarding uses the built-in omfwd module, and receiving uses imtcp (or imudp). Prefer TCP: UDP drops messages silently and cannot use an action queue.
Step 1: Install rsyslog on the server
rsyslog is installed by default on most EL images, but minimal and container-based installs sometimes leave it out.
sudo dnf install -y rsyslog policycoreutils-python-utils
sudo systemctl enable --now rsyslog
rsyslogd -v | head -1Step 2: Configure the server to receive logs over TCP
Create a drop-in that loads the TCP listener, binds it to its own ruleset and writes each host’s messages into its own directory. Binding the input to a ruleset is what keeps remote messages out of the server’s local log files.
sudo tee /etc/rsyslog.d/10-remote-server.conf >/dev/null <<'CONF'
# Per-host file template: /var/log/remote/<host>/<program>.log
template(name="RemoteHostFile" type="list") {
constant(value="/var/log/remote/")
property(name="hostname" SecurePath="replace")
constant(value="/")
property(name="programname" SecurePath="replace")
constant(value=".log")
}
ruleset(name="remote") {
action(type="omfile" dynaFile="RemoteHostFile"
dirCreateMode="0750" fileCreateMode="0640"
dynaFileCacheSize="200")
stop
}
module(load="imtcp" MaxSessions="500")
input(type="imtcp" port="514" ruleset="remote")
CONF
sudo mkdir -p /var/log/remote
sudo chmod 0750 /var/log/remote
sudo restorecon -Rv /var/log/remoteSecurePath="replace" stops a client from sending a hostname like ../../etc to write outside /var/log/remote. Raise dynaFileCacheSize if you have many hosts and programs, so rsyslog does not keep reopening files.
Step 3: Open the firewall and check SELinux
Port 514 is already labeled for syslog in the SELinux targeted policy, so only the firewall needs changing. Restrict the source network if you can.
# Simple: open TCP 514 in the default zone
sudo firewall-cmd --permanent --add-port=514/tcp
sudo firewall-cmd --reload
# Tighter: allow only the server subnet
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.0/24" port port="514" protocol="tcp" accept'
sudo firewall-cmd --reload
# SELinux: confirm the syslog port labels
sudo semanage port -l | grep syslogIf you move the listener to a non-standard port, label it first, for example sudo semanage port -a -t syslogd_port_t -p tcp 30514, or rsyslog will fail to bind with an AVC denial.
Step 4: Validate and restart the server
sudo rsyslogd -N1
sudo systemctl restart rsyslog
sudo ss -tlnp | grep 514rsyslogd -N1 parses every file without starting the daemon and prints the line number of any error. You should see rsyslogd listening on 0.0.0.0:514 and [::]:514.
Step 5: Configure clients to forward logs
On each client, add a forwarding action with a disk-assisted queue. If the log server is down, messages queue in memory, spill to disk under /var/lib/rsyslog if needed, and are sent when the server comes back.
sudo tee /etc/rsyslog.d/90-forward.conf >/dev/null <<'CONF'
*.* action(type="omfwd"
target="loghost.example.com" port="514" protocol="tcp"
queue.type="LinkedList"
queue.filename="fwd_loghost"
queue.maxDiskSpace="1g"
queue.saveOnShutdown="on"
action.resumeRetryCount="-1"
action.resumeInterval="10")
CONF
sudo rsyslogd -N1
sudo systemctl restart rsyslogLocal logging keeps working as before; this action only adds a copy. To send less, replace *.* with a selector such as *.info;mail.none;cron.none or authpriv.*.
Step 6: Verify end to end
# On a client
logger -p user.notice "central logging test from $(hostname -s)"
# On the server
sudo ls /var/log/remote/
sudo tail /var/log/remote/web1/*.log
sudo ss -tn state established '( sport = :514 )'Each client should appear as a directory, and the logger line shows up in a file named after the sending user or program (for example root.log).
Step 7: Rotate the remote logs
The stock /etc/logrotate.d/rsyslog file only covers the local logs. Add a rule for the remote tree:
sudo tee /etc/logrotate.d/rsyslog-remote >/dev/null <<'CONF'
/var/log/remote/*/*.log {
daily
rotate 30
compress
delaycompress
missingok
notifempty
dateext
sharedscripts
postrotate
/usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
endscript
}
CONF
sudo logrotate -d /etc/logrotate.d/rsyslog-remoteStep 8: Encrypt forwarding with TLS
Plain syslog crosses the network in clear text, including authentication messages. rsyslog supports two TLS stream drivers: ossl (OpenSSL, package rsyslog-openssl) and gtls (GnuTLS, package rsyslog-gnutls). This example uses ossl; if your release does not ship that package, install rsyslog-gnutls and use gtls instead, and the rest of the configuration stays the same.
Create a small CA and certificates
Do this on an admin host (or use your existing internal CA or FreeIPA certificates). Each certificate needs the host’s DNS name in its subject alternative name.
mkdir ~/syslog-ca && cd ~/syslog-ca
# CA
openssl req -x509 -newkey rsa:3072 -nodes -days 3650 \
-keyout ca-key.pem -out ca-cert.pem -subj "/CN=Syslog CA"
# One certificate per host (server and every client)
for h in loghost.example.com web1.example.com db1.example.com; do
openssl req -newkey rsa:3072 -nodes -keyout $h-key.pem -out $h.csr \
-subj "/CN=$h" -addext "subjectAltName=DNS:$h"
openssl x509 -req -in $h.csr -CA ca-cert.pem -CAkey ca-key.pem \
-CAcreateserial -days 825 -copy_extensions copy -out $h-cert.pem
done-copy_extensions needs OpenSSL 3, which EL9 and EL10 ship. On EL8, put the SAN in an extension file and pass -extfile instead.
Install the files on each host
sudo dnf install -y rsyslog-openssl
sudo mkdir -p /etc/pki/rsyslog
sudo cp ca-cert.pem /etc/pki/rsyslog/ca.pem
sudo cp $(hostname -f)-cert.pem /etc/pki/rsyslog/cert.pem
sudo cp $(hostname -f)-key.pem /etc/pki/rsyslog/key.pem
sudo chmod 0600 /etc/pki/rsyslog/key.pem
sudo restorecon -Rv /etc/pki/rsyslogServer: add a TLS listener on 6514
A single imtcp module instance can only use one stream driver mode, so the simplest approach is to switch the server to TLS only. Replace the module(load="imtcp") and input() lines in /etc/rsyslog.d/10-remote-server.conf with:
global(
DefaultNetstreamDriver="ossl"
DefaultNetstreamDriverCAFile="/etc/pki/rsyslog/ca.pem"
DefaultNetstreamDriverCertFile="/etc/pki/rsyslog/cert.pem"
DefaultNetstreamDriverKeyFile="/etc/pki/rsyslog/key.pem"
)
module(load="imtcp"
StreamDriver.Name="ossl"
StreamDriver.Mode="1"
StreamDriver.AuthMode="x509/name"
PermittedPeer=["*.example.com"])
input(type="imtcp" port="6514" ruleset="remote")sudo firewall-cmd --permanent --add-port=6514/tcp
sudo firewall-cmd --permanent --remove-port=514/tcp
sudo firewall-cmd --reload
sudo semanage port -l | grep 6514 # expect syslog_tls_port_t
sudo rsyslogd -N1 && sudo systemctl restart rsyslogx509/name makes the server check that each client certificate is signed by your CA and that its name matches PermittedPeer. Use x509/certvalid if you only want to check the signature.
Clients: forward over TLS
sudo tee /etc/rsyslog.d/90-forward.conf >/dev/null <<'CONF'
global(
DefaultNetstreamDriverCAFile="/etc/pki/rsyslog/ca.pem"
DefaultNetstreamDriverCertFile="/etc/pki/rsyslog/cert.pem"
DefaultNetstreamDriverKeyFile="/etc/pki/rsyslog/key.pem"
)
*.* action(type="omfwd"
target="loghost.example.com" port="6514" protocol="tcp"
StreamDriver="ossl" StreamDriverMode="1"
StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers="loghost.example.com"
queue.type="LinkedList" queue.filename="fwd_loghost"
queue.maxDiskSpace="1g" queue.saveOnShutdown="on"
action.resumeRetryCount="-1")
CONF
sudo rsyslogd -N1 && sudo systemctl restart rsyslog
logger "TLS logging test from $(hostname -s)"Confirm the session is really encrypted from the client with openssl s_client -connect loghost.example.com:6514 -CAfile /etc/pki/rsyslog/ca.pem -cert /etc/pki/rsyslog/cert.pem -key /etc/pki/rsyslog/key.pem , which should end with Verify return code: 0 (ok).
Differences between the distributions
- RHEL, AlmaLinux, Rocky Linux, Oracle Linux, CentOS Stream: same package names, paths and SELinux policy; the commands above are identical. Oracle Linux with UEK behaves the same, because rsyslog is in user space.
- EL8: older rsyslog (8.2102 era) and OpenSSL 1.1.1. The configuration works, but per-input certificate options (
streamDriver.CAFileon aninput()) need rsyslog 8.2310 or later, which arrived in RHEL 9.4. - EL9 and EL10: newer rsyslog and OpenSSL 3. If FIPS mode is on (RHEL 9.2 and later), TLS 1.2 clients must support Extended Master Secret or use TLS 1.3.
- Minimal and cloud images: some ship without rsyslog and rely on the journal only; install it as in Step 1. Do not try to forward with
systemd-journal-remoteand rsyslog at the same time unless you mean to.
Troubleshooting
- Nothing arrives: on the client, run
sudo journalctl -u rsyslog -eand look foromfwdornetstreamerrors; test the port withnc -vz loghost.example.com 514(packagenmap-ncat). - Server will not bind:
sudo ausearch -m avc -ts recent -c rsyslogdshows SELinux denials. Label custom ports withsemanage port -a -t syslogd_port_tand custom log directories withsemanage fcontext -a -t var_log_t "/srv/logs(/.*)?"plusrestorecon -R. - Remote messages also land in /var/log/messages: the input is not bound to the
remoteruleset, or the ruleset is missingstop. - TLS handshake fails: look for
not permitted to talk to peer(the name in the certificate does not matchPermittedPeer) or certificate expiry; check withopenssl x509 -in /etc/pki/rsyslog/cert.pem -noout -subject -ext subjectAltName -enddate. - Queue growing on clients:
ls -lh /var/lib/rsyslog/fwd_loghost*shows spooled data; it drains automatically once the server is reachable. - Wrong hostnames in file paths: rsyslog uses the hostname inside the message. Set the client hostname with
hostnamectl set-hostname, or switch the template property tofromhostto use the connecting address instead.
Related posts
For older examples of per-service log files, see Configure ISC DHCP Logging. Another server build in this series is Configure Postfix and Dovecot on RHEL, AlmaLinux, Rocky and Oracle Linux, and the firewalld commands used above are collected in the firewalld cheat sheet.