Solaris Admin Series, part 15. Previous: Installing Solaris 10 on SPARC. Related: part 13, Solaris Serial Console Access (installing over a serial port), part 12, Solaris Hostname, Time Zone, Locale and Keyboard (the identity questions the installer asks), part 4, Solaris /devices, /dev and /etc/path_to_inst (reading c1t0d0 and c0d0 device names) and part 2, Solaris on SPARC vs x86 (GRUB vs OpenBoot). Next up: choosing Solaris 10 software groups and minimizing an install.
The SPARC install in part 14 starts at the ok prompt and writes a simple VTOC label to the disk. On x86 the same Solaris 10 installer sits on top of a PC: a BIOS, a Master Boot Record, up to four fdisk partitions, and the GRUB legacy boot loader. The sysidtool and software screens are identical, but the boot steps and the disk layout are not, and the disk layout is where x86 installs usually go wrong. This post walks through an interactive installation of Solaris 10 1/13 (Update 11, the final update) on x86 from DVD, with the focus on GRUB, the Solaris fdisk partition, the VTOC slices inside it, and a ZFS or UFS root.
Applies to: Solaris 10 on x86 (64-bit x86 servers, workstations and virtual machines) only. Solaris 11.4 x86 uses a different installer (Text Installer, Automated Installer, Unified Archives), boots with GRUB 2 and supports UEFI and GPT boot disks; it gets its own posts later. If you are installing on SPARC, use part 14 instead.
Prerequisites
- The Solaris 10 1/13 x86 DVD image (
sol-10-u11-ga-x86-dvd.iso) from Oracle Software Delivery Cloud or My Oracle Support, used under a license that covers your hardware. Since 8/11 only the DVD is shipped; there are no CD sets. - At least 1.5 GB of RAM (Oracle’s minimum for both UFS and ZFS root in 1/13) and at least 16 GB of disk. A ZFS root needs more, because swap and dump live in the pool. With less memory the GUI installer will not start and you fall back to the text installer.
- Hardware that Solaris 10 has drivers for. Check the Oracle Solaris Hardware Compatibility List for non-Oracle servers, and get any Install Time Update (ITU) or driver diskette your storage controller needs before you start. Common supported NICs are Intel (
e1000g,igb,ixgbe) and Broadcom (bge,bnx). - Legacy BIOS boot. Solaris 10 cannot boot from UEFI firmware or a GPT boot disk. On modern hardware enable legacy/CSM boot. The boot disk uses an MBR with a Solaris fdisk partition, and only the first 2 TB of a large disk is addressable by the VTOC inside it.
- The SATA controller mode decided up front (AHCI recommended). Disks show up as
c1t0d0under AHCI and asc0d0(cmdkdriver) in legacy IDE mode. Changing the mode after the install changes the boot device path and the system will not boot. - A console: the local VGA/KVM, ILOM Remote Console, or a serial port (COM1 =
ttya, COM2 =ttyb) through the service processor as in part 13, Solaris Serial Console Access. - The usual answers ready: host name, IP address, netmask, default router, DNS servers and domain, NFSv4 domain, time zone and root password, plus which NIC is cabled.
- A backup of anything on the target disk. An initial install rewrites the fdisk table and the Solaris partition.
How an x86 Solaris 10 boot disk is laid out
Before touching the installer it helps to see the three layers, because the installer screens and every later disk command refer to them:
| Layer | What it is | Solaris device |
|---|---|---|
| MBR and fdisk table | Sector 0 of the disk: boot code plus up to four primary partitions. Solaris needs one partition of type Solaris2 (ID 191, 0xBF) marked active. | c1t0d0p0 (whole disk), p1–p4 (fdisk partitions) |
| VTOC (SMI label) inside the Solaris partition | The Solaris slice table, the same idea as on SPARC but stored inside the fdisk partition, not at the start of the disk. | c1t0d0s0 … s9 |
| Slices | Slice 0 normally holds root (or the whole ZFS root pool), slice 1 swap on UFS, slice 2 the whole Solaris partition (“backup”, never use it for data), slice 8 the 1-cylinder boot slice for GRUB stage files, slice 9 the “alternates” slice on IDE disks. | /dev/dsk/c1t0d0s0 etc. |
The separate x86 boot partition (an extra fdisk partition used before Solaris 10 1/06) is obsolete: since GRUB arrived in 1/06, everything lives in the single Solaris2 partition. Other fdisk partitions (Windows, Linux) can coexist, but on a server give Solaris the whole disk.
Step 1: BIOS settings and booting the DVD
Enter the BIOS/UEFI setup and set: legacy (CSM) boot, the SATA mode you chose, the DVD or virtual media first in the boot order, and the target disk next. Then attach the media (physical DVD, ILOM storage redirection of the ISO, or the virtual CD of a VM) and power cycle. After a memory test and hardware probe, GRUB from the DVD shows its menu:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory)
+-------------------------------------------------------------------------+
| Oracle Solaris |
| Oracle Solaris Serial Console ttya |
| Oracle Solaris Serial Console ttyb (for lx50, v60x and v65x) |
+-------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, 'e' to edit the
commands before booting, or 'c' for a command-line.| GRUB entry | Use it when |
|---|---|
Oracle Solaris | Installing on the local screen, a KVM or ILOM Remote Console (graphics) |
Oracle Solaris Serial Console ttya | The installer output should go to COM1, for example ILOM serial redirection on most Oracle x86 servers |
Oracle Solaris Serial Console ttyb | Output on COM2 (the note mentions old Sun LX50/V60x/V65x, but any box whose SP uses COM2 needs it) |
Editing the boot entry
Highlight an entry and press e to see the GRUB commands. They look like this:
kernel /boot/multiboot kernel/unix -B install_media=cdrom
module /boot/x86.minirootSelect the kernel line, press e again, add options, press Enter to accept and b to boot. Useful additions are - nowin (console text installer), - text (text installer in a desktop session), -B console=ttya (redirect the console yourself), and -v for verbose boot messages when a driver hangs. Esc discards the edit. Edits are only for this boot; nothing is written to the DVD.
Step 2: Choose the installation type
The miniroot boots, probes devices and then shows the installation type menu:
Select the type of installation you want to perform:
1 Oracle Solaris Interactive
2 Custom JumpStart
3 Oracle Solaris Interactive Text (Desktop session)
4 Oracle Solaris Interactive Text (Console session)
(Select option 3 or 4 to install a ZFS root file system)
5 Apply driver updates
6 Single user shell
Enter the number of your choice.| Option | What it does |
|---|---|
| 1 | GUI installer (needs graphics and enough memory). It can only build a UFS root. |
| 2 | Custom JumpStart from a profile server (later in the series). |
| 3 | Text installer inside a minimal desktop session on a graphics console. Supports ZFS root. |
| 4 | Text installer on the plain console. The right choice for serial consoles and remote KVMs that struggle with X. Supports ZFS root. |
| 5 | Load an ITU or driver update from media before installing, for controllers the DVD does not support. |
| 6 | Single-user shell from the miniroot, for repairs or for preparing disks with format and fdisk. |
If you want a ZFS root, and you usually do, pick 3 or 4. The GUI installer only offers UFS. Make the choice promptly: option 1, the GUI installer, is the default.
Step 3: Keyboard, language and terminal type
If Solaris cannot find a self-identifying keyboard (most USB keyboards and all serial consoles), it shows a keyboard layout list. Select your layout (for example US-English) with the arrow keys and Space, then F2. The choice ends up in the keyboard setting described in part 12, Solaris Hostname, Time Zone, Locale and Keyboard.
If you chose option 3 you may see two “is the window legible” confirmations; press Enter in each. Then pick the installer language and, for the console session, the terminal type. On the local VGA console choose PC Console; over a serial line choose DEC VT100 or X Terminal Emulator (xterms), exactly as on SPARC. As in part 14, function keys rarely survive a serial or remote-console session: use Esc then the digit (Esc 2 for F2_Continue, Esc 3 for F3_Go Back, and so on).
Step 4: System identification
The sysidtool screens are the same on both platforms: network connectivity, which interface, DHCP, host name, IP address and netmask, IPv6, default route, Kerberos, name service, NFSv4 domain, time zone, date, root password, the “enable remote services” (Secure by Default) question, and Oracle Configuration Manager/Auto Registration. The screen-by-screen table with recommended answers is in part 14, Installing Solaris 10 on SPARC. The only visible x86 difference is the hardware: typical interface names are e1000g0, bge0, igb0 or rge0, depending on the NIC. If the list is empty, the NIC has no Solaris 10 driver; continue with “Networked: No” and add a supported card or driver later.
Step 5: Install options, license, locales and file system type
Press F2 for a Standard install (F4 is for Flash archives), set Manual Reboot and automatic DVD eject, accept the license, and choose Initial install if you are offered an upgrade (an existing upgradeable UFS Solaris was found). If the disks already hold ZFS pools you will see:
There are existing ZFS pools available on this system.
However, they can only be upgraded using the Live Upgrade tools.
The following screens will only allow you to install a ZFS root system,
not upgrade one.Existing pools are left alone unless you select their disks. Select the geographic regions and locales you need, skip additional products, and then choose the file system type:
Choose a Filesystem Type
Select the filesystem to use for your Solaris installation
[ ] UFS
[X] ZFS
F2_Continue F6_HelpThen select a software group. The options and cluster names (SUNWCXall, SUNWCall, SUNWCprog, SUNWCuser, SUNWCreq, SUNWCrnet) are the same as on SPARC; on x86, prefer SUNWCXall (Entire Distribution plus OEM Support), because the OEM part carries extra drivers for third-party hardware. The next post covers the groups and minimization in detail.
Step 6: Select disks and create the Solaris fdisk partition
This is where x86 differs. The Select Disks screen lists the disks with the space available in a Solaris partition:
Select Disks
Note: ** denotes the current boot disk
Disk Device Available Space
===========================================================================
[X]** c1t0d0 76285 MB (F4 to edit)
[ ] c1t1d0 76285 MB
Maximum Root Size: 76285 MB
Suggested Minimum: 8529 MB
F2_Continue F3_Go Back F4_Edit F6_HelpSample output. Select one disk, or two for a mirrored ZFS root pool (the installer builds the mirror; RAID-Z is not supported for a root pool, and selecting many disks just builds one wide mirror). For ZFS, each disk you select gets the pool on slice 0.
If a selected disk has no Solaris fdisk partition, for example a new disk or one that held Linux or Windows, the installer stops and asks you to create one. The safe choice for a dedicated server disk is the option that uses the entire disk for a single Solaris partition. To keep another OS, choose to customize the fdisk partitions (or press F4 on the disk in Select Disks): you get a table of the four primary partitions where you can delete partitions and create the Solaris one in free space. Changes are only written when the installation starts, but they destroy any data in partitions you delete or overlap.
Preparing the disk yourself (optional)
If the installer will not show a disk, or shows 0 MB, prepare it from the single-user shell (installation option 6, or !/exit to a shell from the installer). fdisk -B writes a default table with one Solaris2 partition covering the whole disk, which wipes the existing partition table:
# format # note the disk name, e.g. c1t0d0, then quit
# fdisk -W - /dev/rdsk/c1t0d0p0 # dump the current fdisk table first
# fdisk -B /dev/rdsk/c1t0d0p0 # one Solaris2 partition, whole disk, active
# fdisk -W - /dev/rdsk/c1t0d0p0
...
* Id Act Bhead Bsect Bcyl Ehead Esect Ecyl Rsect Numsect
191 128 0 1 1 254 63 1023 16065 156232125ID 191 is Solaris2 and Act 128 marks it active (sample values; your geometry will differ). Without -B, fdisk /dev/rdsk/c1t0d0p0 runs an interactive menu where you can create, delete and activate partitions one at a time. Then run format c1t0d0 and label it; a new Solaris partition gets an SMI label with slice 2 and the boot slice 8 by default. Restart the installer with install-solaris, or reboot from the DVD.
Step 7: ZFS settings or UFS layout
ZFS root
Configure ZFS Settings
Specify the name of the pool to be created from the disk(s) you have chosen.
Also specify the name of the dataset to be created within the pool that is
to be used as the root directory for the filesystem.
ZFS Pool Name: rpool
ZFS Root Dataset Name: s10x_u11wos_24a
ZFS Pool Size (in MB): 76285
Size of Swap Area (in MB): 4096
Size of Dump Area (in MB): 2048
(Pool size must be between 6481 MB and 76285 MB)
[ ] Keep / and /var combined
[X] Put /var on a separate datasetThe root dataset name becomes the boot environment name (the default is the release build, s10x_u11wos_24a on x86). Leave the pool at full size unless you need room on the disk for something else. A separate /var dataset keeps logs and core files from filling root. Swap and dump are ZFS volumes and can be resized later with zfs set volsize.
UFS root
A UFS install offers Auto Layout of the slices within the Solaris partition. A reasonable server layout is / on slice 0 (at least 10 GB), swap on slice 1, /var on its own slice, /export/home on slice 7 or wherever space remains, and a small free slice for Solaris Volume Manager state database replicas if you plan to mirror root later. Slices 2, 8 and 9 are reserved and are handled by the installer. Then answer “No” to mounting remote file systems unless you need software from an NFS server.
Step 8: Review the profile and install
Profile
Installation Option: Initial
Boot Device: c1t0d0
Root File System Type: ZFS
Client Services: None
System Locale: U.S.A. (en_US.ISO8859-1)
Software: Solaris 10, Entire Distribution plus OEM support
Pool Name: rpool
Boot Environment Name: s10x_u11wos_24a
Pool Size: 76285 MB
Devices in Pool: c1t0d0
F2_Begin Installation F4_Change F5_Exit F6_HelpCheck the boot device and the devices in the pool, then press F2 (Esc 2). The fdisk table and VTOC are written first, then packages are installed. Logs go to /var/sadm/system/logs and /var/sadm/install/logs on the new system (/a/... while the installer runs).
With Manual Reboot you land at a prompt when the install finishes. If you want to skip Auto Registration so nothing is sent to Oracle on first boot, do it now, before rebooting, as Oracle documents:
# rm /a/var/tmp/autoreg_config
# rebootStep 9: First boot from the disk
Remove or disconnect the media (or put the disk first in the BIOS boot order). GRUB now loads from the disk and shows the installed menu, which times out after 10 seconds:
GNU GRUB version 0.95 (637K lower / 3144640K upper memory)
+-------------------------------------------------------------------------+
| Oracle Solaris 10 1/13 s10x_u11wos_24a X86 |
| Solaris failsafe |
+-------------------------------------------------------------------------+The first entry boots the boot environment; Solaris failsafe boots a small miniroot from the disk for repairs (it offers to mount your root on /a), so you rarely need the DVD for a broken boot archive. On a ZFS root the menu file is /rpool/boot/grub/menu.lst; on UFS it is /boot/grub/menu.lst. An entry for a ZFS root looks like this:
title Oracle Solaris 10 1/13 s10x_u11wos_24a X86
findroot (BE_s10x_u11wos_24a,0,a)
bootfs rpool/ROOT/s10x_u11wos_24a
kernel$ /platform/i86pc/multiboot -B $ZFS-BOOTFS
module /platform/i86pc/boot_archivefindroot locates the boot environment by a signature file instead of a fixed BIOS disk number, so the system still boots if the BIOS disk order changes. Do not edit menu.lst by hand for things bootadm or eeprom can do.
Serial console after installation
If you installed using a Serial Console entry, the installer normally carries console=ttya over to the installed system. Check, and set it if needed, with eeprom, which stores x86 boot properties in /boot/solaris/bootenv.rc:
# eeprom console
console=ttya
# eeprom console=ttya
# eeprom ttya-mode
ttya-mode=9600,8,n,1,-To also see the GRUB menu itself on the serial port, add serial --unit=0 --speed=9600 and terminal serial near the top of menu.lst. Part 13 covers the ILOM side.
Verification
# cat /etc/release
Oracle Solaris 10 1/13 s10x_u11wos_24a X86
Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
Assembled 17 January 2013
# isainfo -kv
64-bit amd64 kernel modules
# cat /var/sadm/system/admin/CLUSTER
CLUSTER=SUNWCXall
# zpool status rpool
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0
errors: No known data errors
# bootadm list-menu
the location for the active GRUB menu is: /rpool/boot/grub/menu.lst
default 0
timeout 10
0 Oracle Solaris 10 1/13 s10x_u11wos_24a X86
1 Solaris failsafe
# fdisk -W - /dev/rdsk/c1t0d0p0 | tail -2
# prtvtoc /dev/rdsk/c1t0d0s2
...
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 2 00 16065 156183930 156199994
2 5 01 0 156199995 156199994
8 1 01 0 16065 16064
# lustatus ; svcs -xv ; ifconfig -a ; netstat -rn/etc/releaseends inX86, andisainfo -kvconfirms the 64-bit kernel; a 32-bit kernel there means an old 32-bit CPU or a GRUB entry forcing it (part 5, Solaris First-Day Commands, covers these commands).prtvtocshows the x86 shape of the label: slice 0 for the pool (tag 2, root), slice 2 the whole Solaris partition (tag 5, backup), slice 8 the 1-cylinder boot slice (tag 1). Sample sector counts.bootadm list-menushows whichmenu.lstis active and the entries in it.svcs -xvprints nothing on a healthy system,netstat -rnshows the default route, andlustatuslists the installed boot environment.
Mirroring a single-disk ZFS root on x86
To add a second disk after a one-disk install, give it a Solaris fdisk partition and the same VTOC, attach it, and install GRUB on it so the BIOS can boot from either disk:
# fdisk -B /dev/rdsk/c1t1d0p0
# prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2
# zpool attach rpool c1t0d0s0 c1t1d0s0
# zpool status rpool # wait until the resilver completes
# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c1t1d0s0Then add the second disk to the BIOS boot order and test it by disconnecting or deselecting the first disk in a maintenance window. Do not skip installgrub: without it the second half of the mirror has your data but cannot boot.
Installing in a virtual machine
- VirtualBox: use the “Oracle Solaris 10 10/09 and later (64-bit)” guest type, a SATA (AHCI) disk and the Intel PRO/1000 NIC (
e1000g0); give it at least 2 GB of RAM. - KVM/QEMU, Proxmox and similar: Solaris 10 has no virtio drivers. Use SATA (AHCI) or IDE disks, an
e1000NIC and SeaBIOS (legacy BIOS), not OVMF/UEFI. - VMware: pick the Solaris 10 64-bit guest type with an LSI Logic parallel or SAS controller and an E1000 NIC.
- Solaris 11 hosts: on SPARC, run Solaris 10 in a guest domain (part 14); on x86 consider a Solaris 10 branded zone on Solaris 11 instead of a full VM. Branded zones get their own post later.
Troubleshooting and gotchas
- The DVD does not boot, or GRUB never appears. The firmware is in UEFI-only mode. Enable legacy/CSM boot, and make sure the virtual media is attached before power-on.
- The installer hangs while configuring devices. Edit the GRUB entry, add
-vto the kernel line to see where it stops (often a USB or storage driver), and try option 5 with an ITU driver for the controller. - No disks found, or the disk shows 0 MB. No driver for the controller, a hardware RAID volume that is not created yet, or no usable Solaris fdisk partition. Check from the shell with
format,cfgadm -alandfdisk -W - /dev/rdsk/c#t#d#p0, and usefdisk -Bif the disk may be wiped. - Only UFS is offered. You started the GUI installer (option 1). Reboot and choose option 3 or 4.
- Garbled installer screens over serial. Wrong terminal type: use DEC VT100 or xterms, never PC Console, on a serial line, and choose the matching GRUB Serial Console entry so the miniroot console is on the right port.
- After install: “GRUB” on the screen and nothing else, or GRUB Error 15/17. The boot loader or the BIOS boot order is wrong. Boot Solaris failsafe (or the DVD, option 6), let it mount the root on
/a, and reinstall GRUB withinstallgrub /a/boot/grub/stage1 /a/boot/grub/stage2 /dev/rdsk/c1t0d0s0. - Panic or “cannot mount root” after a BIOS change. The SATA mode was switched between IDE and AHCI, which changes the device path. Switch it back. Choose the mode before installing.
- Boot archive out of date after a crash. Boot failsafe, accept mounting the root on
/a, then runbootadm update-archive -R /aand reboot. - Disks larger than 2 TB. Solaris 10 can boot from them since 10/09, but the VTOC only addresses the first 2 TB. Use a smaller boot disk or a RAID volume of up to 2 TB.
- Patch it. The 1/13 DVD is from 2013. Apply the current Recommended Patchset (with support) before production use; patching gets its own section later.
What’s next in the Solaris Admin Series
Next up are the Solaris 10 software groups (Core, End User, Developer, Entire, Entire plus OEM) and how to minimize an install safely. After that come sysidcfg for hands-off identification and JumpStart. Catch up with part 14, Installing Solaris 10 on SPARC, part 13, Solaris Serial Console Access, part 12, Solaris Hostname, Time Zone, Locale and Keyboard, part 11, Solaris Run Levels and Init States, part 4, Solaris /devices, /dev and /etc/path_to_inst and part 1, Solaris 10 and 11.4 Releases and Support Lifecycle.
Official references: Oracle Solaris 10 1/13 Installation Guide: Basic Installations (x86: How to Install With the Oracle Solaris Interactive Text Installer With GRUB for ZFS, and the UFS installation procedure), Installation Guide: Planning for Installation and Upgrade (System Requirements and Recommendations, Partitioning Recommendations), the Oracle Solaris 10 1/13 Release Notes and the ZFS Administration Guide in the Oracle Solaris 10 1/13 Information Library; and the man pages for fdisk(1M), format(1M), prtvtoc(1M), fmthard(1M), installgrub(1M), bootadm(1M), eeprom(1M) and zpool(1M).