Oracle Infrastructure Series, part 6. Previous: part 5, HAIP vs bonding for the private interconnect. Next up, part 7.
Every RAC client connects to one name, the Single Client Access Name (SCAN), and that name has to resolve correctly before Grid Infrastructure will install. Get it wrong and the installer fails its prerequisite check, or worse, it installs and clients fail over badly later. This post builds the DNS side on Oracle Linux 9 with BIND: one SCAN name with three A records, a matching reverse zone, round robin, the optional Grid Naming Service (GNS) alternative with its delegation, and how to verify everything with nslookup, dig, srvctl and cluvfy. It targets Oracle Grid Infrastructure 19c; the 23ai and 26ai differences are noted along the way. The addresses come from the plan in part 2, planning the RAC cluster.
How SCAN works
The SCAN is a single DNS name that resolves to up to three IP addresses (SCAN VIPs). Clusterware runs each SCAN VIP, and a SCAN listener bound to it, on one of the cluster nodes. A client resolves the name, gets the addresses in rotating order, connects to one SCAN listener, and that listener redirects the connection to the local listener on the node running the least-loaded instance of the service. Nodes can be added or removed without ever touching a client tnsnames.ora.
client --- "rac-scan.example.com" ---> DNS 10.10.10.5 / 10.10.10.6
| returns 10.10.10.31, .32, .33 (rotated each query)
v
SCAN VIP + SCAN listener (one of three, any node)
| redirect to least-loaded instance's local listener
v
node VIP 10.10.10.21 (rac1) or 10.10.10.22 (rac2) ---> instance / service
Three SCAN VIPs on two nodes: one node runs two of them. That is normal.Oracle recommends three SCAN addresses regardless of the number of nodes, so that the loss of one SCAN VIP never leaves a single point of failure and the load spreads. The SCAN name must be resolved by DNS for that to work: a hosts file can hold only one address for the name, and Oracle supports that only as a fallback for tiny test clusters (the installer warns about it). Two valid approaches exist: static addresses in your own DNS (this post, section by section), or GNS, where Clusterware assigns the SCAN VIPs from DHCP and answers for the cluster subdomain itself.
Prerequisites
- The lab layout from part 2: nodes
rac1andrac2(10.10.10.11 and .12), node VIPs .21 and .22, SCANrac-scan.example.comon 10.10.10.31, .32 and .33, all in 10.10.10.0/24 with gateway 10.10.10.1, domainexample.com. - Two DNS servers:
10.10.10.5(primary,dns1) and10.10.10.6(secondary). The same two addresses were given to the nodes asipv4.dnsin part 4, bonding the public network. - A SCAN name of 15 characters or fewer, using only letters, digits and hyphens, and not the same as the cluster name. The three SCAN addresses must be on the public subnet, unused, and must not answer a ping before installation (Clusterware plumbs them later).
- Port 53 open (UDP and TCP) on the DNS server:
firewall-cmd --permanent --add-service=dns && firewall-cmd --reload. See the firewalld cheat sheet. - Nodes prepared as in part 3. The cluster nodes only need
/etc/resolv.confpointing at the DNS servers withoptions timeout:1 attempts:2.
Step 1: Install BIND on the DNS server
On the primary DNS server (dns1, Oracle Linux 9):
# dnf install -y bind bind-utils
# systemctl enable --now named
# named -v
BIND 9.16.23-RH (Extended Support Version) <id:...>If you already run an internal DNS server, skip this and just add the records in the next steps. For a broader walk-through of BIND itself, see the older posts Install Bind/Named/DNS on CentOS and Configure Internal DNS Server; the zone syntax is unchanged on Oracle Linux 9.
Step 2: Configure named.conf
Edit /etc/named.conf so the server listens on the lab address, answers the cluster network, and declares the forward and reverse zones. Keep the default rrset-order (cyclic) so round robin works:
options {
listen-on port 53 { 127.0.0.1; 10.10.10.5; };
directory "/var/named";
allow-query { localhost; 10.10.10.0/24; 192.168.0.0/16; };
allow-transfer { 10.10.10.6; };
recursion yes;
allow-recursion { localhost; 10.10.10.0/24; };
forwarders { 10.10.10.1; };
dnssec-validation no;
/* no rrset-order statement: BIND default is cyclic (round robin) */
};
zone "example.com" IN {
type master;
file "example.com.zone";
allow-update { none; };
};
zone "10.10.10.in-addr.arpa" IN {
type master;
file "10.10.10.rev";
allow-update { none; };
};Step 3: The forward zone with three SCAN A records
# cat /var/named/example.com.zone
$TTL 3600
@ IN SOA dns1.example.com. hostmaster.example.com. (
2026110801 ; serial (YYYYMMDDnn)
3600 ; refresh
900 ; retry
1209600 ; expire
300 ) ; negative-cache TTL
IN NS dns1.example.com.
IN NS dns2.example.com.
dns1 IN A 10.10.10.5
dns2 IN A 10.10.10.6
; cluster nodes (public)
rac1 IN A 10.10.10.11
rac2 IN A 10.10.10.12
; node VIPs
rac1-vip IN A 10.10.10.21
rac2-vip IN A 10.10.10.22
; SCAN: ONE name, THREE addresses = round robin
rac-scan IN A 10.10.10.31
rac-scan IN A 10.10.10.32
rac-scan IN A 10.10.10.33The private interconnect addresses (192.168.10.x and 192.168.11.x) do not belong in DNS; they live in the Clusterware profile. Keep the TTL on the SCAN records modest. Oracle does not require a particular value, and the three addresses rotate on every query regardless of TTL, but a very long TTL makes clients cache one ordering for hours.
Fix ownership and check the syntax before loading anything:
# chown root:named /var/named/example.com.zone /var/named/10.10.10.rev
# named-checkconf
# named-checkzone example.com /var/named/example.com.zone
zone example.com/IN: loaded serial 2026110801
OKStep 4: The reverse zone
Oracle requires the reverse lookup of each SCAN address to return the SCAN name. Without PTR records, the cluvfy SCAN check warns, and some tools and the installer prerequisites complain. Create one PTR per address, plus PTRs for the nodes and VIPs:
# cat /var/named/10.10.10.rev
$TTL 3600
@ IN SOA dns1.example.com. hostmaster.example.com. (
2026110801 3600 900 1209600 300 )
IN NS dns1.example.com.
IN NS dns2.example.com.
5 IN PTR dns1.example.com.
6 IN PTR dns2.example.com.
11 IN PTR rac1.example.com.
12 IN PTR rac2.example.com.
21 IN PTR rac1-vip.example.com.
22 IN PTR rac2-vip.example.com.
31 IN PTR rac-scan.example.com.
32 IN PTR rac-scan.example.com.
33 IN PTR rac-scan.example.com.
# named-checkzone 10.10.10.in-addr.arpa /var/named/10.10.10.rev
zone 10.10.10.in-addr.arpa/IN: loaded serial 2026110801
OK
# systemctl reload namedStep 5: The secondary DNS server
On dns2 (10.10.10.6) install BIND and declare both zones as slaves of the primary. SELinux and the file ownership are already correct for /var/named/slaves:
zone "example.com" IN {
type slave;
masters { 10.10.10.5; };
file "slaves/example.com.zone";
};
zone "10.10.10.in-addr.arpa" IN {
type slave;
masters { 10.10.10.5; };
file "slaves/10.10.10.rev";
};
# systemctl reload named
# ls -l /var/named/slaves/
-rw-r--r--. 1 named named 702 Nov 8 16:20 10.10.10.rev
-rw-r--r--. 1 named named 788 Nov 8 16:20 example.com.zoneRemember to raise the serial in both zones on the primary every time you edit them, then rndc reload; the secondary pulls the change. A forgotten serial bump is the commonest reason one DNS server answers differently from the other.
Step 6: Verify round robin with nslookup and dig
Run these from a cluster node and from a client. Repeat the query several times: the order of the three addresses must rotate.
$ nslookup rac-scan.example.com
Server: 10.10.10.5
Address: 10.10.10.5#53
Name: rac-scan.example.com
Address: 10.10.10.32
Name: rac-scan.example.com
Address: 10.10.10.33
Name: rac-scan.example.com
Address: 10.10.10.31
$ nslookup rac-scan.example.com | grep -A1 Name | grep Address | head -1
Address: 10.10.10.33dig shows the TTL and the exact answer ordering, and lets you target each DNS server separately:
$ dig +noall +answer rac-scan.example.com @10.10.10.5
rac-scan.example.com. 3600 IN A 10.10.10.31
rac-scan.example.com. 3600 IN A 10.10.10.32
rac-scan.example.com. 3600 IN A 10.10.10.33
$ dig +noall +answer rac-scan.example.com @10.10.10.5
rac-scan.example.com. 3600 IN A 10.10.10.32
rac-scan.example.com. 3600 IN A 10.10.10.33
rac-scan.example.com. 3600 IN A 10.10.10.31
$ for i in 1 2 3 4 5 6; do dig +short rac-scan.example.com @10.10.10.6 | head -1; done
10.10.10.31
10.10.10.32
10.10.10.33
10.10.10.31
10.10.10.32
10.10.10.33Now the reverse lookups. All three must return the SCAN name, and the node and VIP names must resolve both ways:
$ for ip in 10.10.10.31 10.10.10.32 10.10.10.33; do dig +short -x $ip; done
rac-scan.example.com.
rac-scan.example.com.
rac-scan.example.com.
$ nslookup 10.10.10.21
21.10.10.10.in-addr.arpa name = rac1-vip.example.com.
$ getent hosts rac-scan # what the OS resolver (and so Oracle) actually sees
10.10.10.33 rac-scan.example.com
10.10.10.31 rac-scan.example.com
10.10.10.32 rac-scan.example.comA word of caution on client-side caching: if systemd-resolved or nscd is on a client, the rotation seen at the application can look sticky even though the DNS server rotates. On the RAC nodes themselves, keep /etc/nsswitch.conf as hosts: files dns and do not run a local caching resolver.
Step 7: Install Grid Infrastructure and read the SCAN configuration back
In the Grid Infrastructure installer, enter rac-scan.example.com as the SCAN name, port 1521, and leave GNS unticked. The installer resolves the name, finds three addresses and builds three SCAN VIPs and three SCAN listeners. After root.sh completes, check what Clusterware stored (as the grid owner):
srvctl config scan
$ srvctl config scan
SCAN name: rac-scan.example.com, Network: 1
Subnet IPv4: 10.10.10.0/255.255.255.0/bond0, static
Subnet IPv6:
SCAN 1 IPv4 VIP: 10.10.10.31
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 2 IPv4 VIP: 10.10.10.32
SCAN VIP is enabled.
...
SCAN 3 IPv4 VIP: 10.10.10.33
SCAN VIP is enabled.
...
$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node rac2
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node rac1
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node rac1The word static after the interface shows the addresses came from DNS. With GNS it reads dhcp. Two SCAN VIPs on one node and one on the other is the expected layout for two nodes.
srvctl config scan_listener
$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
Registration invited nodes:
Registration invited subnets:
Endpoints: TCP:1521
SCAN Listener is enabled.
SCAN Listener is individually enabled on nodes:
SCAN Listener is individually disabled on nodes:
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
...
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
...
$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node rac2
...
$ lsnrctl status LISTENER_SCAN1 | grep -E "Listening|Service "
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.10.31)(PORT=1521)))
Service "orclpdb" has 2 instance(s).Each SCAN listener should list the database services with an instance count equal to the number of running instances. The databases register with the SCAN listeners through the remote_listener parameter, which Clusterware sets to rac-scan.example.com:1521 for you. Check it with show parameter remote_listener.
Step 8: Check with cluvfy
$ cluvfy comp scan
Verifying Single Client Access Name (SCAN) ...
Verifying DNS/NIS name service 'rac-scan.example.com' ...
Verifying Name Service Switch Configuration File Integrity ...PASSED
Verifying DNS/NIS name service 'rac-scan.example.com' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verification of scan was successful.Before installation you can run the same family of checks as part of cluvfy stage -pre crsinst -n rac1,rac2 -verbose, which also tests the SCAN name resolution from every node. Run it from the unzipped grid home as the grid owner.
Option: GNS (Grid Naming Service)
With GNS, Clusterware runs a small DNS responder (the gnsd daemon) for a delegated subdomain. You give the cluster a subdomain such as cluster1.example.com, one fixed GNS VIP, and a DHCP range on the public subnet. The SCAN VIPs, node VIPs and SCAN name are all created automatically from DHCP, which makes adding nodes very easy. The cost is a DHCP server on the public network, a delegation in your DNS, and one more moving part. Most production sites use static SCAN addresses; GNS pays off in large or dynamic environments.
parent DNS (example.com)
cluster1.example.com. NS gns.example.com. <- delegation
gns.example.com. A 10.10.10.40 <- GNS VIP (glue, static)
|
v queries for *.cluster1.example.com go to the GNS VIP
GNS (gnsd, runs on one node, fails over)
rac-scan.cluster1.example.com -> 3 DHCP-assigned SCAN VIPs
rac1-vip.cluster1.example.com -> DHCP-assigned node VIPThe delegation is two records in the parent zone. Add them to example.com.zone, bump the serial, and reload:
; delegation of the GNS subdomain
cluster1 IN NS gns.example.com.
gns IN A 10.10.10.40At install time choose Configure GNS, enter the subdomain and the GNS VIP, and the installer builds the GNS resource. The SCAN name then becomes <cluster_name>-scan.cluster1.example.com. To add GNS to an existing cluster, or to inspect it, use srvctl (as root for the add):
# srvctl add gns -vip 10.10.10.40 -domain cluster1.example.com
# srvctl start gns
$ srvctl config gns
GNS is enabled.
GNS VIP addresses: 10.10.10.40
Domain served by GNS: cluster1.example.com
$ srvctl status gns
GNS is running on node rac1.
$ dig +short racdemo-scan.cluster1.example.com
10.10.10.101
10.10.10.102
10.10.10.103The DHCP range must be on the public subnet, must not overlap your static addresses, and the DHCP server must not hand those leases to ordinary hosts. Firewalls must allow UDP and TCP 53 from the clients and the parent DNS servers to the GNS VIP. Newer releases also support a shared GNS that one cluster serves for several clusters, plus a way to configure it from a client cluster; the delegation idea is the same. Check the GNS chapter of the Grid Infrastructure guide for your release before using those options.
19c versus 23ai and 26ai
- The SCAN and GNS concepts, the DNS requirements and the
srvctlandcluvfycommands above are unchanged in 23ai and 26ai (26ai replaces 23ai as the long-term release). - In 19c the installer is
gridSetup.shfrom the unzipped grid home. The 23ai/26ai installer has the same SCAN page; expect small wording differences in the screens and, in some releases, thesrvctloutput order. - SCAN VIPs can also use IPv6. If you add IPv6 records, add AAAA records for the SCAN and the matching
ip6.arpaPTR records; a mixed family needs both resolving consistently. - Oracle states the SCAN should resolve to three addresses for any cluster size; fewer is allowed (one or two), and you can later change the number with
srvctl modify scan -scannameafter DNS changes.
Changing the SCAN later
If you add a fourth address or change the three, update DNS first, wait for the secondary to transfer, then tell Clusterware to re-read it (as root, from the grid home):
# srvctl modify scan -scanname rac-scan.example.com
# srvctl config scan
# srvctl modify scan_listener -update
# srvctl stop scan_listener ; srvctl start scan_listenerClusterware compares what DNS returns to what is in the OCR, then adds or removes SCAN VIPs. Do this in a quiet window, because stopping a SCAN listener drops connections that are still being routed through it (established sessions are not affected).
Troubleshooting
- Installer: “SCAN name resolves to only one IP” or PRVG-1101/PRVF-4664 messages: DNS returned fewer than the expected addresses, or the nodes are reading a hosts file entry first. Run
getent hosts rac-scanon each node; check/etc/nsswitch.confand remove the SCAN from/etc/hosts. - Round robin does not rotate: someone set
rrset-order { order fixed; }in named.conf, or a cache sits between the client and BIND. Query the authoritative server directly withdig @10.10.10.5. Different results between the two servers mean the secondary has an older serial. - PRVG-1101: SCAN name “rac-scan” failed to resolve from one node only: that node has a wrong
resolv.conf(NetworkManager overwrote it). Setipv4.dnsandipv4.dns-search example.comon the bond0 connection (see part 4) and runnmcli con up bond0. - Reverse lookup failures: missing PTR for one address, or the reverse zone name wrong. For 10.10.10.0/24 it must be
10.10.10.in-addr.arpa. Test withdig -x. - Zone does not load:
journalctl -u named -eshows the file and line. The most common causes are a missing trailing dot on a fully qualified name, a wrong file owner, or an SELinux label (restore withrestorecon -Rv /var/named). - Timeouts of 5 seconds before every connection: the first DNS server in
resolv.confis down and the client waits for the default timeout. Addoptions timeout:1 attempts:2and test each server by itself. - SCAN VIP does not start: the address is in use elsewhere, or sits outside the public subnet or the VIP/subnet mask is wrong. Check
srvctl config networkand ping the address from a third machine before the VIP is plumbed (it must not respond). - GNS: names under the subdomain fail: check the NS delegation and the glue A record with
dig +trace racdemo-scan.cluster1.example.comand confirm UDP and TCP 53 reach the GNS VIP; also check that the DHCP server is leasing addresses on the public subnet.
Oracle Infrastructure Series: this is part 6. Previous: part 5, HAIP vs bonding for the private interconnect. Earlier: part 4, bonding the public network, part 3, preparing Oracle Linux 9, part 2, planning the RAC cluster and part 1, the overview.
References: Oracle Grid Infrastructure Installation and Upgrade Guide 19c for Linux (SCAN and GNS configuration requirements), the Clusterware Administration and Deployment Guide 19c for srvctl and GNS, and the 26ai installation guide.