Solaris 10 JumpStart Server Setup: setup_install_server, add_install_client, rules and Profiles

Solaris Admin Series, part 18. Previous: part 17, Solaris 10 sysidcfg. Related: part 16, Solaris 10 Software Groups (the cluster value your profile picks), part 14, Installing Solaris 10 on SPARC and part 15, Installing Solaris 10 on x86 (what the installer does when nobody answers for it), part 13, Solaris Serial Console Access (watching a network boot on the console) and part 2, Solaris on SPARC vs x86 (platform groups such as sun4v and i86pc).

JumpStart is how Solaris 10 shops installed tens or hundreds of servers without sitting at a console. A client boots from the network, finds an install image over NFS, reads its identity from sysidcfg, matches itself against a rules file and installs exactly what the matching profile says, then reboots into a finished system. This post builds a working JumpStart server from scratch: the install image with setup_install_server, a JumpStart (profile) directory with rules, a profile and check, a per-host sysidcfg, and the client definition with add_install_client. Then it boots a SPARC client with boot net - install, verifies the result and walks through the failures you will actually hit.

Applies to: Oracle Solaris 10 (examples from 1/13, Update 11; the same commands work back to the early updates). JumpStart installs only Solaris 10 and earlier. Solaris 11 and 11.4 are installed with the Automated Installer (AI) instead; a Solaris 11 system can still host a Solaris 10 JumpStart server, with a few changes covered in the Solaris 11 section near the end.

The pieces of a JumpStart setup

Four roles are involved. On a small network one server plays all of them; on a large one they are split.

RoleWhat it providesBuilt with
Install serverA copy of the Solaris 10 DVD (the net install image) shared over NFSsetup_install_server
Boot serverThe network boot image, RARP/bootparams answers and TFTP for clients on its subnet. Needed per subnet unless you use DHCPsetup_install_server -b (only when the install server is on another subnet)
Profile (JumpStart) serverThe JumpStart directory: rules, rules.ok, profiles, begin and finish scripts, checkmkdir, sample files, check
sysidcfg locationA directory holding a file named sysidcfg per host or per classPart 17, Solaris 10 sysidcfg

How a SPARC client finds everything without DHCP: it sends RARP to get its IP address (from /etc/ethers and /etc/hosts on the boot server), loads inetboot by TFTP, asks rpc.bootparamd for its root, install, sysid_config and install_config paths, then mounts those over NFS. add_install_client writes all of that for you; knowing the chain makes the troubleshooting section easy.

Prerequisites

  • A Solaris 10 server (SPARC or x86) with a static IP address on the same subnet as the clients. A SPARC server can serve x86 clients and the other way round; the image you copy decides what you can install, not the server architecture.
  • The Solaris 10 1/13 DVD ISO for the client architecture (sol-10-u11-ga-sparc-dvd.iso or sol-10-u11-ga-x86-dvd.iso). Plan on roughly 4 GB per image on disk; setup_install_server checks the space and refuses if it is short.
  • NFS server software installed (it is in every software group above Reduced Networking) and the ability to share directories read-only with anon=0.
  • For each client: host name, IP address, MAC address and platform group (uname -m output such as sun4u, sun4v or i86pc). On an LDoms guest the MAC comes from ldm list -l on the control domain; on bare metal it is printed by banner at the ok prompt.
  • A tested sysidcfg for the client (part 17), and a decision about the software group (part 16).
  • Console access to the client: ILOM, a serial line or a guest console (part 13).

The example network used throughout:

ItemValue
JumpStart serverjs01, 192.168.10.5, Solaris 10 1/13 SPARC
Clientweb01, 192.168.10.41, MAC 0:14:4f:a1:b2:c3, sun4v (SPARC Enterprise T5220, e1000g0)
Install image/export/install/s10u11_sparc
JumpStart directory/export/jumpstart
sysidcfg directory/export/config/web01

Step 1: Build the install server image

Attach the ISO with lofiadm (or insert the DVD; vold mounts it under /cdrom/cdrom0), then run setup_install_server from the image’s Tools directory. Never run it from your $PATH: each release’s scripts must come from that release’s media.

js01# zfs create -o mountpoint=/export/install rpool/export/install
js01# lofiadm -a /export/iso/sol-10-u11-ga-sparc-dvd.iso
/dev/lofi/1
js01# mount -F hsfs -o ro /dev/lofi/1 /mnt
js01# cd /mnt/Solaris_10/Tools
js01# ./setup_install_server /export/install/s10u11_sparc
Verifying target directory...
Calculating the required disk space for the Solaris_10 product
Calculating space required for the installation boot image
Copying the CD image to disk...
Copying Install Boot Image hierarchy...
Copying /boot netboot hierarchy...
Install Server setup complete
js01# cd / ; umount /mnt ; lofiadm -d /dev/lofi/1

The copy takes a while on slow disks. The target directory must be empty or not exist yet, and it must not be the root of a file system that already holds other data you care about. If you install several updates side by side (s10u10_sparc, s10u11_sparc, s10u11_x86), each gets its own directory and its own Tools.

Share the image read-only with root access mapped (anon=0), because the installer runs as root on the client:

js01# echo 'share -F nfs -o ro,anon=0 -d "S10u11 SPARC image" /export/install' >> /etc/dfs/dfstab
js01# svcadm enable -r svc:/network/nfs/server
js01# shareall
js01# share
-               /export/install   ro,anon=0   "S10u11 SPARC image"

If the directory is a ZFS dataset you can instead use the dataset property, which persists without dfstab. Do one or the other, not both:

js01# zfs set sharenfs='ro,anon=0' rpool/export/install
js01# zfs get sharenfs rpool/export/install
NAME                  PROPERTY  VALUE      SOURCE
rpool/export/install  sharenfs  ro,anon=0  local

Optional: a boot server on another subnet

RARP and bootparams do not cross routers. If clients live on a subnet where the install server has no interface and you are not using DHCP, put a small boot server on that subnet. It holds only the boot image and points clients at the install server:

boot02# mkdir -p /export/boot/s10u11_sparc
boot02# cd /mnt/Solaris_10/Tools
boot02# ./setup_install_server -b /export/boot/s10u11_sparc

You then run add_install_client on the boot server with -s js01:/export/install/s10u11_sparc. DHCP-based booting, which removes the per-subnet boot server, is covered with x86 PXE and WAN boot in part 21.

Step 2: Create the JumpStart directory

The JumpStart directory holds the rules file, profiles and scripts. Start from the samples on the media, which include the check script that must match the release you install:

js01# mkdir -m 755 /export/jumpstart
js01# cp -r /export/install/s10u11_sparc/Solaris_10/Misc/jumpstart_sample/* /export/jumpstart/
js01# ls /export/jumpstart
any_machine     check           host_class      rules           upgrade
begin           finish          net924_sun4c    set_root_pw     x86-class
js01# echo 'share -F nfs -o ro,anon=0 -d "JumpStart dir" /export/jumpstart' >> /etc/dfs/dfstab
js01# mkdir -p -m 755 /export/config/web01
js01# echo 'share -F nfs -o ro,anon=0 -d "sysidcfg files" /export/config' >> /etc/dfs/dfstab
js01# shareall

Keep the sample profiles for reference, but move them out of the way (for example into /export/jumpstart/samples) so the live rules file only refers to profiles you wrote. Everything in this directory must be owned by root: directories 755, files 644, scripts 644 as well (the installer runs begin and finish scripts with sh, so they do not need the execute bit).

Step 3: Write a profile

A profile says how to install: install type, system type, disk layout and software. Part 19 covers every keyword; here are two working profiles, one ZFS root and one UFS root, that are enough for a first install.

js01# cat /export/jumpstart/web_zfs
# web servers: ZFS root, mirrored, minimal software
install_type    initial_install
cluster         SUNWCreq
pool            rpool auto auto auto mirror c0t0d0s0 c0t1d0s0
bootenv         installbe bename s10u11
js01# cat /export/jumpstart/web_ufs
# UFS root on the first disk the installer finds
install_type    initial_install
system_type     standalone
partitioning    explicit
filesys         rootdisk.s0 20480 /
filesys         rootdisk.s1 4096  swap
filesys         rootdisk.s7 free  /export/home
cluster         SUNWCuser
  • cluster SUNWCreq is the Core software group from part 16; add pieces with cluster or package ... add.; part 19 covers adding and removing clusters and packages.
  • pool takes pool name, pool size, swap size, dump size and the vdev list. auto sizes swap and dump for you. A ZFS root needs an SMI-labeled slice, not a whole EFI disk.
  • rootdisk means “the disk the installer picks for root”, which saves you from guessing c0t0d0 versus c1t0d0 across hardware models (see part 4, Solaris /devices, /dev and /etc/path_to_inst).
  • An upgrade profile uses install_type upgrade; leave upgrades to Live Upgrade unless you have a reason.

Step 4: Write the rules file

Each line is rule-keyword rule-value [&& ...] begin profile finish. A minus sign marks a missing begin or finish script. The installer uses the first rule that matches, so put specific rules first and a catch-all last, or leave the catch-all out so an unknown machine never gets wiped by accident.

js01# mv /export/jumpstart/rules /export/jumpstart/samples/rules.sample
js01# cat /export/jumpstart/rules
# rule keywords & values            begin   profile     finish
hostname web01                      -       web_zfs     -
karch sun4v && memsize 32768-       -       web_zfs     -
network 192.168.20.0 && karch sun4u -       web_ufs     -

Common rule keywords (all documented in the JumpStart Keyword Reference):

KeywordValueExample
hostnameClient host namehostname web01
hostaddressClient IP addresshostaddress 192.168.10.41
networkNetwork numbernetwork 192.168.20.0
karchPlatform groupkarch sun4v, karch i86pc
archProcessor typearch sparc, arch i386
modelPlatform namemodel SUNW,SPARC-Enterprise-T5220
memsizeMemory in MB, single value or rangememsize 16384-65536
disksizeDisk name and size range in MBdisksize rootdisk 140000-300000
totaldiskTotal disk space in MBtotaldisk 200000-
domainnameName service domaindomainname example.com
installed, osnameExisting Solaris on disk (for upgrade rules)osname Solaris_10
anyAlways matches; value is -any - - generic_prof -

Begin and finish scripts (the third and fifth fields) are the subject of part 20; for now leave them as -.

Step 5: Validate with check and create rules.ok

Clients never read rules; they read rules.ok, which only check creates. Run it from the JumpStart directory every time you change rules or a profile. Use -p to run the check logic from the install image you will actually install:

js01# cd /export/jumpstart
js01# ./check -p /export/install/s10u11_sparc
Validating rules...
Validating profile web_zfs...
Validating profile web_ufs...
The custom JumpStart configuration is ok.
js01# tail -2 rules.ok
network 192.168.20.0 && karch sun4u -       web_ufs     -
# version=2 checksum=4012
js01# ls -l rules.ok
-rw-r--r--   1 root     root         149 Nov 20 14:40 rules.ok

check validates syntax only. It does not know whether c0t1d0 exists on the client. To test a profile against real disk geometry without installing, run pfinstall -D (dry run) on a Solaris 10 system of the same type; part 19 shows how.

Step 6: Put the sysidcfg in place

Copy the tested file from part 17 into the per-host directory. The client looks for a file named exactly sysidcfg inside the directory you pass with -p:

js01# cat /export/config/web01/sysidcfg
system_locale=C
terminal=vt100
network_interface=e1000g0 {primary hostname=web01 ip_address=192.168.10.41
    netmask=255.255.255.0 default_route=192.168.10.1 protocol_ipv6=no}
name_service=DNS {domain_name=example.com name_server=192.168.10.53
    search=example.com}
nfs4_domain=dynamic
security_policy=NONE
timezone=US/Central
timeserver=localhost
root_password=$5$kR3aQ9sd$Xm3...truncated...
service_profile=limited_net
auto_reg=disable
js01# chmod 644 /export/config/web01/sysidcfg

Step 7: Add the client with add_install_client

The client must be known to the server by name and MAC. Add it to /etc/hosts (or your name service) and pass the MAC with -e, which writes /etc/ethers for you. Then run add_install_client from the image’s Tools directory:

js01# echo '192.168.10.41   web01' >> /etc/hosts
js01# cd /export/install/s10u11_sparc/Solaris_10/Tools
js01# ./add_install_client -e 0:14:4f:a1:b2:c3 \
        -s js01:/export/install/s10u11_sparc \
        -c js01:/export/jumpstart \
        -p js01:/export/config/web01 \
        web01 sun4v
Adding Ethernet number for web01 to /etc/ethers
updating /etc/bootparams
OptionMeaning
-e macClient Ethernet address; added to /etc/ethers for RARP
-s server:pathInstall image (defaults to the image the command runs from)
-c server:pathJumpStart directory with rules.ok; without it the install is interactive
-p server:pathDirectory containing sysidcfg
-dUse DHCP instead of RARP/bootparams; required for x86 PXE; prints the DHCP macros to create (part 21)
-b "prop=value"x86 only, with -e: boot properties such as console=ttya
client platform-groupClient host name and sun4u, sun4v or i86pc

Look at what it wrote. This is the first file to read when a client boots but does not install:

js01# grep web01 /etc/ethers
0:14:4f:a1:b2:c3        web01
js01# cat /etc/bootparams
web01 root=js01:/export/install/s10u11_sparc/Solaris_10/Tools/Boot install=js01:/export/install/s10u11_sparc boottype=:in sysid_config=js01:/export/config/web01 install_config=js01:/export/jumpstart rootopts=:rsize=8192
js01# ls -l /tftpboot | grep -i 0A
lrwxrwxrwx   1 root     root          26 Nov 20 14:46 C0A80A29 -> inetboot.SUN4V.Solaris_10-1
lrwxrwxrwx   1 root     root          26 Nov 20 14:46 C0A80A29.SUN4V -> inetboot.SUN4V.Solaris_10-1

The TFTP file name is the client IP in hex (192.168.10.41 = C0A80A29). add_install_client also makes sure the boot services are running. Check them:

js01# svcs rarp rpc/bootparams tftp/udp6 nfs/server
STATE          STIME    FMRI
online         14:46:02 svc:/network/rarp:default
online         14:46:02 svc:/network/rpc/bootparams:default
online         14:46:03 svc:/network/tftp/udp6:default
online         14:31:10 svc:/network/nfs/server:default

If tftp/udp6 does not exist, the tftp line in /etc/inetd.conf is still commented out: uncomment it and run inetconv, then svcadm enable tftp/udp6. To remove a client later, run ./rm_install_client web01 from the same Tools directory; run it before re-adding a client whose MAC or image changed.

x86 clients

x86 machines boot the network with PXE, which needs DHCP, so x86 clients are always added with -d. The command creates a per-client boot file and GRUB menu under /tftpboot and prints the DHCP options and macro you must define; the DHCP server side is part 21.

js01# cd /export/install/s10u11_x86/Solaris_10/Tools
js01# ./add_install_client -d -e 0:21:28:5e:7a:10 \
        -c js01:/export/jumpstart -p js01:/export/config/db02 \
        -b "console=ttya" db02 i86pc
...
To enable db02 in the DHCP server, ensure that
the following Sun vendor-specific options are defined
(SinstNM, SinstIP4, SinstPTH, SrootNM, SrootIP4,
SrootPTH, SbootURI, SjumpsCF and SsysidCF),
and add a macro to the server named 010021285E7A10,
containing the following option values:
...
  Boot server IP     (BootSrvA) : 192.168.10.5
  Boot file          (BootFile) : 010021285E7A10

Step 8: Boot the client and watch it install

At the SPARC ok prompt (on the ILOM or guest console), boot from the network with the install argument. Without install the client boots the miniroot and stops at an interactive installer.

{0} ok boot net - install
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@2/network@0  File and args: - install
Requesting Internet Address for 0:14:4f:a1:b2:c3
SunOS Release 5.10 Version Generic_147147-26 64-bit
Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface e1000g0...
Configured interface e1000g0
Reading ZFS config: done.
Setting up Java. Please wait...
Extracting windowing system. Please wait...
Beginning system identification...
Searching for configuration file(s)...
Using sysid configuration file 192.168.10.5:/export/config/web01/sysidcfg
Search complete.
Discovering additional network configuration...
Completing system identification...
Starting remote procedure call (RPC) services: done.
System identification complete.
Starting Solaris installation program...
Searching for JumpStart directory...
Using rules.ok from 192.168.10.5:/export/jumpstart.
Checking rules.ok file...
Using profile: web_zfs
Executing JumpStart preinstall phase...
Searching for SolStart directory...
Checking rules.ok file...
Using begin script: install_begin
Using finish script: patch_finish
Executing SolStart preinstall phase...
Executing begin script "install_begin"...
Begin script install_begin execution completed.

Processing profile
        - Selecting cluster (SUNWCreq)
        - Creating ZFS root pool (rpool) on mirror c0t0d0s0 c0t1d0s0
...
Installing Core System Support software group
...
Installation log location
        - /a/var/sadm/system/logs/install_log (before reboot)
        - /var/sadm/system/logs/install_log (after reboot)

Installation complete
Executing SolStart postinstall phase...
Executing finish script "patch_finish"...
Finish script patch_finish execution completed.
Executing JumpStart postinstall phase...
syncing file systems... done
rebooting...

The install_begin/patch_finish lines refer to the SolStart scripts built into the miniroot, not to your rules; they run on every install. Add -v (boot net -v - install) for verbose boot messages while you are debugging. If the client has several NICs, boot through the one whose MAC you registered, for example boot net1 - install after checking devalias.

Verification

After the reboot, confirm the client got what the profile and sysidcfg asked for:

web01# cat /etc/release
                   Oracle Solaris 10 1/13 s10s_u11wos_24a SPARC
  Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
                            Assembled 17 January 2013
web01# cat /var/sadm/system/admin/CLUSTER
CLUSTER=SUNWCreq
web01# zpool status rpool | sed -n '1,10p'
  pool: rpool
 state: ONLINE
 scan: none requested
config:

        NAME          STATE     READ WRITE CKSUM
        rpool         ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            c0t0d0s0  ONLINE       0     0     0
            c0t1d0s0  ONLINE       0     0     0
web01# lustatus
Boot Environment           Is       Active Active    Can    Copy
Name                       Complete Now    On Reboot Delete Status
-------------------------- -------- ------ --------- ------ ----------
s10u11                     yes      yes    yes       no     -
web01# ls /var/sadm/system/logs/
begin.log      finish.log     install_log    sysidtool.log
web01# grep -i -c error /var/sadm/system/logs/install_log
0

Once the client is installed, remove it from the server so a stray boot net - install (or a boot net by a confused colleague) cannot reinstall it:

js01# cd /export/install/s10u11_sparc/Solaris_10/Tools
js01# ./rm_install_client web01
removing web01 from bootparams

Troubleshooting and gotchas

SymptomUsual causeFix
Timeout waiting for ARP/RARP packet repeating on the consoleMAC not in /etc/ethers, wrong NIC, rarpd not running, or client on another subnetCheck banner/ldm list -l MAC against /etc/ethers; svcs rarp; use a boot server on that subnet or DHCP
Gets an IP, then hangs at TFTP or bootparamstftp/udp6 disabled, missing /tftpboot link, or stale entry for an old IPsvcs tftp/udp6 rpc/bootparams; rm_install_client then add_install_client again
Error: unknown client "web01" from add_install_clientHost name does not resolve on the serverAdd it to /etc/hosts or the name service; check getent hosts web01
add_install_client warns the directory “must be shared read-only with root access”Share options wrong, usually a ZFS dataset shared without anon=0zfs set sharenfs='ro,anon=0' or fix dfstab, then shareall
Mount errors such as Permission denied or RPC: Program not registered during bootNFS server down, path not shared, firewall/IP Filter blocking NFSsvcs nfs/server; share; from another host showmount -e js01
Warning: Could not find matching rule in rules.ok and the interactive installer startsNo rule matches, or rules.ok is stale because check was not rerunRerun ./check; test the rule with hostname first; inspect the client with prtconf, uname -m
Interactive identification screens appear-p path wrong or sysidcfg incompleteSee the troubleshooting table in part 17
Profile fails with a disk error such as c0t1d0: not a valid diskDisk names differ on this modelUse rootdisk, match on model, or test with pfinstall -D
Client installs the wrong profileAn earlier rule matched first (often an any rule left high in the file)Order specific rules first; keep any last or remove it
  • Edit rules, not rules.ok. Hand edits to rules.ok break its checksum line and are overwritten the next time someone runs check.
  • Run the check, add_install_client and rm_install_client from the image you install. Scripts from Update 8 against an Update 11 image produce subtle failures.
  • One client, one image: if you move a client to a new image, run rm_install_client first. Two bootparams lines for the same host means whichever answers first wins.
  • JumpStart wipes disks named in the profile with no confirmation. Keep the any rule out of production servers’ JumpStart directory and remove clients once installed.
  • Read logs from the client’s console while it is still in the miniroot: /tmp/install_log, /tmp/begin.log and /tmp/finish.log. After a reboot they are in /var/sadm/system/logs/.
  • Everything over the wire, including the sysidcfg with the root password hash, is plain NFS. Limit the shares to the install network (ro=@192.168.10.0/24,anon=0) or use WAN boot with HTTPS (part 21).

Solaris 11: Automated Installer replaces JumpStart

Solaris 11 cannot be installed with JumpStart, and Solaris 10 cannot be installed with AI. The concepts map closely:

JobSolaris 10 JumpStartSolaris 11.4 Automated Installer
Install imagesetup_install_server (copy of the DVD over NFS)installadm create-service (boot image) plus an IPS package repository
Add a clientadd_install_client (/etc/ethers, bootparams, TFTP)installadm create-client -e mac -n service; DHCP and HTTP, no RARP
Which machine gets whatrules + check = rules.okCriteria on manifests and profiles (installadm create-manifest -c mac=...)
How to installProfile (install_type, pool, cluster)AI manifest (XML: target, software, packages)
IdentitysysidcfgSC profile (sysconfig create-profile, installadm create-profile)
Post-installFinish scriptsFirst-boot SMF service package
Migrationn/ajs2ai -r converts rules and profiles, js2ai -s converts sysidcfg

A Solaris 11 server can still act as a Solaris 10 JumpStart server, which is useful while both releases are in service. Oracle’s Transitioning From Oracle Solaris 10 JumpStart to Oracle Solaris 11.3 Automated Installer documents the differences: install pkg:/system/boot/network (and pkg:/service/network/tftp), run setup_install_server from the Solaris 10 media as above, change the first line of the copied check script to #!/usr/sunos/bin/sh because /bin/sh is ksh93 on Solaris 11, enable svc:/network/tftp/udp6, and make /tftpboot a symbolic link to /etc/netboot if the same server also runs AI. Test it on your exact 11.4 SRU before relying on it.

s11js# pkg install pkg:/system/boot/network pkg:/service/network/tftp
s11js# mkdir -m 755 /etc/netboot ; ln -s /etc/netboot /tftpboot
s11js# svcadm enable network/tftp/udp6
s11js# cd /export/jumpstart && head -1 check
#!/usr/sunos/bin/sh

The AI posts later in the series cover installadm, manifests, SC profiles and client criteria in detail.

What’s next in the Solaris Admin Series

Next up: writing JumpStart profiles in depth, with filesys, partitioning, cluster and package selection, ZFS root pool and bootenv keywords and dry runs with pfinstall -D. After that come begin and finish scripts (part 20) and DHCP, x86 PXE and WAN boot (part 21). Catch up with part 17, Solaris 10 sysidcfg, part 16, Solaris 10 Software Groups, part 15, Installing Solaris 10 on x86 and part 14, Installing Solaris 10 on SPARC.

Official references: Oracle Solaris 10 1/13 Installation Guide: JumpStart Installations (Preparing JumpStart Installations, Creating the rules File, Validating the rules File, JumpStart Keyword Reference), Installation Guide: Network-Based Installations (Installing From the Network With DVD Media) in the Oracle Solaris 10 1/13 Information Library; Transitioning From Oracle Solaris 10 JumpStart to Oracle Solaris 11.3 Automated Installer; and the man pages for setup_install_server(1M), add_install_client(1M), rm_install_client(1M), install_scripts(1M), bootparams(4), ethers(4), share_nfs(1M), js2ai(1M) and installadm(1M).

Leave a Reply

Your email address will not be published. Required fields are marked *