Solaris Admin Series, part 13. Previous: Solaris Hostname, Time Zone, Locale and Keyboard. Related: part 11, Solaris Run Levels and Init States (single-user and the ok prompt), part 9, Solaris Text Editors on a Bare System (fixing a garbled vi screen) and part 6, Solaris Shells. Next up: an interactive text installation of Solaris 10 on SPARC, step by step, which you will do from exactly the kind of console described here.
Sooner or later every Solaris admin ends up on the console: the network is down, SSH will not start, the box is sitting at the OpenBoot ok prompt, or you are installing the OS. On SPARC servers there usually is no monitor at all, so the console is a serial line, reached through the service processor (ILOM/ALOM), a terminal server, or a null-modem cable from another Unix machine using tip or cu. This post covers how the Solaris console is wired, how to connect to it, how to get out again without sending a break by accident, and how to fix the terminal type and window size so that vi and format menus are usable.
Applies to: both Solaris 10 and Solaris 11 (11.4), SPARC and x86. Differences are called out: OBP variables on SPARC vs eeprom console and the GRUB menu on x86, SAF/pmadm port monitors on Solaris 10 vs console-login SMF instances on Solaris 11, and the keyboard-abort setting (file on 10, SMF property on 11).
Prerequisites
- Root or the root role on the system whose console settings you change (on Solaris 11,
su -or a profile such as Device Management/Service Management). - A way in: ILOM/ALOM network or serial management port credentials, a terminal server port, or a null-modem (crossover) serial cable plus a USB-serial adapter on the machine you connect from.
- The line settings of the port. The Solaris and OBP default is 9600 baud, 8 data bits, no parity, 1 stop bit, no flow control (9600 8N1). Many x86 BIOS/ILOM setups use 115200; both ends must match.
- A second session or a maintenance window before you touch console settings on a remote box: a wrong
input-deviceor speed can leave you with no console at all.
How the Solaris console is wired
| Piece | Solaris 10 | Solaris 11.4 |
|---|---|---|
| Kernel console device | /dev/console, chosen at boot from OBP (SPARC) or eeprom console (x86) | Same |
| Serial ports | /dev/term/a, /dev/term/b (dial-in), /dev/cua/a, /dev/cua/b (dial-out) | Same |
| Login prompt on console | SMF svc:/system/console-login:default (ttymon) | Same service, plus console-login:terma and :termb instances for the serial ports |
| Login on other serial ports | SAF: sacadm/pmadm ttymon port monitor (late updates also ship terma/termb instances; check svcs -a) | svcadm enable console-login:terma |
| Terminal type for console login | SMF property ttymon/terminal_type on console-login | Same |
| x86 console redirect | eeprom console=ttya plus GRUB legacy menu.lst | eeprom console=ttya plus bootadm set-menu console=serial (GRUB 2, 11.1 and later) |
| Break / keyboard abort | /etc/default/kbd KEYBOARD_ABORT, kbd -i | SMF svc:/system/keymap property keymap/keyboard_abort |
Check what you have before changing anything:
# Which devices is the console using? (SPARC)
# eeprom input-device output-device ttya-mode
input-device=ttya
output-device=ttya
ttya-mode=9600,8,n,1,-
# x86
# eeprom console
console=ttya
# Who owns /dev/console and which login services run?
# svcs -a | grep console-login
online Nov_10 svc:/system/console-login:default
disabled Nov_10 svc:/system/console-login:terma
disabled Nov_10 svc:/system/console-login:termb
# svcprop -p ttymon/terminal_type -p ttymon/label console-login:default
vt100
consoleOn SPARC, input-device=keyboard and output-device=screen mean a graphics console; ttya is the first serial port (on servers it is wired to the service processor). The ttya-mode fields are speed, data bits, parity, stop bits and handshake (- for none, h hardware, s software). The default terminal type on the console differs between releases and platforms (sun, sun-color, vt100), so read it with svcprop instead of assuming.
Option 1: the service processor console (ILOM / ALOM)
On Oracle and Sun servers the normal way to the console is through the service processor. SSH to the ILOM network management port, log in as an ILOM user with console rights, and start the host console:
$ ssh root@sp-host01
Oracle(R) Integrated Lights Out Manager
...
-> start /HOST/console
Are you sure you want to start /HOST/console (y/n)? y
Serial console started. To stop, type ESC (
host01 console login:Details that trip people up:
- On many SPARC T-series and newer ILOM versions the target is
/HOST/console; on some older x86 servers it is/SP/console. Useshow /orhelp startif one is rejected. - Exit is ESC followed by ( (shift-9), not
~.. The~.sequence will close your outer SSH session instead. - Only one user gets read-write console access; others are read-only.
stop /HOST/consolefrom another session (orstart -force /HOST/console) takes it over. - To get a SPARC host to the
okprompt from ILOM, useset /HOST send_break_action=break, thenstart /HOST/consoleand answer the prompt (ccontinue,ssync,rreset,oenter OBP). - The console history buffer is handy after a crash:
show /HOST/console/history. - Older ALOM CMT firmware uses
consoleto connect and#.(hash, dot) to escape, andbreak -yto send a break.
The SP console speed is fixed by the SP (normally 9600 on SPARC); you do not set baud rates with this method, which is why it is the least error-prone option.
Option 2: LDoms guest domain consoles (vntsd)
Guest domains under Oracle VM Server for SPARC have no physical serial port. Their consoles are virtual and served by the vntsd daemon in the control domain, one TCP port per domain on localhost:
primary# svcs vntsd
STATE STIME FMRI
online Nov_08 svc:/ldoms/vntsd:default
primary# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 16 32G 2.1% 2.1% 12d 4h
ldg1 active -n---- 5000 8 16G 0.4% 0.4% 3d 22h
primary# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connecting to console "ldg1" in group "ldg1" ....
Press ~? for control options ..
ldg1 console login:Leave with ~. (vntsd) or the telnet escape Ctrl-] then quit. ~# sends a break to the guest, which drops it to its ok prompt. If vntsd is disabled (the default until a virtual console concentrator vcc service exists), enable it with svcadm enable vntsd. LDoms gets a full block of posts later in the series.
Option 3: a direct serial cable with tip
tip is the classic Solaris serial client, and is what you use on one Solaris box to reach the console of another over a null-modem cable on /dev/term/b (or /dev/term/a). It reads named connections from /etc/remote; the stock file has a hardwire entry:
$ grep '^hardwire' /etc/remote
hardwire:\
:dv=/dev/term/b:br#9600:el=^C^S^Q^U^D:ie=%$:oe=^D:Connect with the entry name, or name the device and speed directly:
$ tip hardwire
connected
host02 console login:
# or, without editing /etc/remote:
$ tip -9600 /dev/term/a
connectedAdd your own entries for frequently used ports so nobody has to remember speeds:
# /etc/remote
host02:dv=/dev/term/a:br#9600:
labsw01:dv=/dev/term/b:br#115200:tip escape sequences, typed at the start of a line (after pressing Enter):
| Sequence | Action |
|---|---|
~. | Disconnect and exit tip |
~# | Send a BREAK (on SPARC this drops the remote host to ok!) |
~~ | Send a literal ~ (also how you reach a nested tip or ssh) |
~^Z | Suspend tip |
~> / ~< | Send a local file to / receive a file from the remote side |
~? | Print the list of escapes |
If you are running tip inside an SSH session, both programs watch for ~ at the start of a line. Type ~~. to make SSH pass a single ~. through to tip; a plain ~. kills your SSH session and leaves tip holding the port.
Option 4: cu
cu (from the UUCP tools) does the same job and is often what people coming from other Unixes reach for:
$ cu -l /dev/term/b -s 9600
Connected
host02 console login:
~.
Disconnectedcu uses the same ~. escape to disconnect and ~%break to send a break. It consults the UUCP /etc/uucp/Devices file; if it refuses a line with NO DEVICES AVAILABLE, add a direct entry such as Direct term/b - Any direct. On Solaris 11, cu comes with the service/network/uucp package; if cu or tip is missing on a minimal install, find the providing package with pkg search -r /usr/bin/cu (or /usr/bin/tip) and install it.
From a Linux laptop the equivalents are screen /dev/ttyUSB0 9600, minicom or picocom -b 9600 /dev/ttyUSB0. Disable hardware flow control in the client; a Sun console expects none.
Pointing the console at a serial port
SPARC (both releases)
At the ok prompt or from the running OS:
ok setenv input-device ttya
ok setenv output-device ttya
ok setenv ttya-mode 9600,8,n,1,-
ok reset-all
# or from Solaris, effective at next boot:
# eeprom input-device=ttya output-device=ttyaChanging ttya-mode to a different speed also means changing the service processor or terminal server side; leave it at 9600 on servers behind an ILOM.
x86 Solaris 10
# eeprom console=ttya
# eeprom ttya-mode=9600,8,n,1,-
# vi /boot/grub/menu.lst # or /rpool/boot/grub/menu.lst on ZFS rootIn menu.lst, add -B console=ttya to the kernel$ line of the default entry, and optionally serial --unit=0 --speed=9600 and terminal serial console above the entries so GRUB itself appears on the serial line.
x86 Solaris 11.4
# eeprom console=ttya
# bootadm set-menu console=serial serial_params=0,9600,8,N,1
# bootadm list-menu
the location of the boot loader configuration files is: /rpool/boot/grub
default 0
console serial
serial_params 0,9600,8,N,1
timeout 30
0 solaris-11.4serial_params is port (0 = ttya/COM1), speed, data bits, parity, stop bits. Do not hand-edit grub.cfg on Solaris 11; bootadm regenerates it. To go back to a graphics console use bootadm set-menu console=graphics (or text) and eeprom console=text.
Logins on the second serial port
The console port gets its login from console-login:default. To allow logins on the other port (for example a modem or a terminal server on ttyb):
# Solaris 11 (and Solaris 10 updates that have the instances)
# svcadm enable svc:/system/console-login:termb
# svcprop -p ttymon console-login:termb | egrep 'device|label|terminal_type'
ttymon/device astring /dev/term/b
ttymon/label astring 9600
ttymon/terminal_type astring vt100
# Solaris 10 classic SAF method
# sacadm -l
# pmadm -a -p zsmon -s ttyb -i root -fu -v $(ttyadm -V) \
-m "$(ttyadm -l 9600 -d /dev/term/b -T vt100 -s /usr/bin/login -S n)"
# pmadm -l -p zsmonDo not enable a login on a port you also want to use outbound with tip or cu: ttymon will hold the port and the client reports all ports busy. The label refers to an entry in /etc/ttydefs that sets the speed (sttydefs -l lists them).
Mirroring console messages: consadm
consadm adds an auxiliary console so kernel and syslog console messages also go to a serial port, useful when the main console is a graphics head nobody watches:
# consadm -a /dev/term/a # add now
# consadm -p -a /dev/term/a # persistent across reboots
# consadm
/dev/term/aTerminal settings: TERM, stty and window size
Most “the console is broken” complaints are really terminal settings. A serial line has no way to tell Solaris your window size, and the terminal type has to match your emulator.
$ echo $TERM
sun
$ TERM=vt100; export TERM # sh/ksh
$ setenv TERM vt100 # csh
$ tput cols; tput lines
80
24
$ stty rows 50 columns 160 # tell the tty your real window size
$ stty -a | head -2
speed 9600 baud;
rows = 50; columns = 160; ypixels = 0; xpixels = 0;vt100works everywhere;xtermgives colours and function keys if your emulator is xterm-compatible and the terminfo entry exists (infocmp xtermto check). Solaris 11.4 also shipsxterm-256color.- Set the default for the console login once with SMF instead of in every profile:
svccfg -s console-login:default setprop ttymon/terminal_type = vt100, thensvcadm refresh console-login:defaultandsvcadm restart console-login:default(do this from SSH, not from the console you are logged in on). - If
resizeexists (it comes with xterm; Solaris 11 has it in the X packages),eval $(resize)sets rows and columns from the emulator automatically. - Backspace prints
^Hor^?: match the tty to your emulator withstty erase ^Horstty erase ^?(type the real key aftererase). - Garbage after
cat-ing a binary:stty sane(or, if your keystrokes do not echo, typestty icanon echo icrnl onlcrblind, followed by Ctrl-J), thentput resetorclear.
Put a size guard in root’s profile so a serial login always starts usable:
# ~/.profile (ksh/sh)
case $(tty) in
/dev/console|/dev/term/*)
TERM=vt100; export TERM
stty rows 50 columns 132 2>/dev/null ;;
esacProtecting against an accidental break
On SPARC, a BREAK on the console (from ~#, a terminal server power cycle, or a cable unplugged from some USB-serial adapters) stops the OS and drops to the ok prompt. Typing go resumes it, but on a busy production box that pause can be enough to fence the node out of a cluster. Solaris lets you disable the keyboard abort or switch it to the alternate sequence (Enter, ~, Ctrl-B within half a second):
# Solaris 10
# grep KEYBOARD_ABORT /etc/default/kbd
KEYBOARD_ABORT=alternate
# kbd -i # apply /etc/default/kbd now
# Solaris 11
# svccfg -s system/keymap:default setprop keymap/keyboard_abort = astring: alternate
# svcadm refresh system/keymap:default
# svcadm restart system/keymap:defaultValues are enable, disable and alternate. A disable setting means even you cannot get to ok from the console when the OS hangs; on servers with an ILOM, alternate plus the ILOM break action is the usual compromise.
Verification
eeprom input-device output-device ttya-mode(SPARC) oreeprom consoleandbootadm list-menu(x86) show the intended settings.svcs console-loginis online andsvcprop -p ttymon/terminal_type console-login:defaultshows the type you set.- From the client, press Enter and get a login prompt; log in and run
tty(should print/dev/console) andstty -a. - Reboot once in a maintenance window while watching the console: you should see OBP/GRUB, the kernel banner and SMF messages on the serial line, not on a monitor.
- Disconnect cleanly (
ESC (,~.) and confirm the host did not drop took.
Troubleshooting and gotchas
- Nothing at all on the line. Wrong cable (you need a null-modem/rollover between two DTE ports; Sun RJ-45 serial pinouts are Cisco-style), wrong speed, or flow control enabled in your client. Try 9600 and 115200 with no flow control.
- Garbage characters. Speed or parity mismatch between
ttya-mode/GRUBserial_paramsand the client. - Output stops after GRUB/OBP. On x86 the boot loader is on serial but
eeprom consolestill saystext, so the kernel switched to the graphics head. tip: all ports busyorcu: DEVICE LOCKED. A ttymon login service owns the port, or a stale lock file is left in/var/spool/locks(namedLK.*). Disable the login service or remove the stale lock after confirming withfuser /dev/term/bthat nobody uses it.- Your SSH session died when you typed
~.. SSH took the escape. Use~~.for the inner tip/cu, orESC (on ILOM. - Host is suddenly at
ok. A break got through. Typegoimmediately to resume, then setKEYBOARD_ABORT=alternateas above. vipaints only 24 lines or wraps wrongly. SetTERMandstty rows/columns; see part 9, Solaris Text Editors on a Bare System.- Why the console matters for root.
/etc/default/loginCONSOLE=/dev/consolerestricts direct root login to the console, which is exactly why the console must keep working. On Solaris 11, root is a role by default, so you log in as a normal user on the console and thensu -. - Wrong
input-deviceon SPARC and no console. From ILOM,set /HOST/bootmode script="setenv input-device ttya"and power-cycle, or on older systems use the Stop-N key sequence to reset NVRAM to defaults.
What’s next in the Solaris Admin Series
Next up is the interactive text installation of Solaris 10 on SPARC, done entirely from the serial console you just set up. Catch up with part 12, Solaris Hostname, Time Zone, Locale and Keyboard, part 11, Solaris Run Levels and Init States, part 9, Solaris Text Editors on a Bare System, part 6, Solaris Shells and part 5, Solaris First-Day Commands. ILOM in depth, OpenBoot, GRUB and LDoms (including vntsd console groups) all get their own posts later in the series.
Official references: tip(1), cu(1C), remote(5), stty(1), tput(1), terminfo(5), eeprom(8), bootadm(8), consadm(8), kbd(1), ttymon(8), pmadm(8), sacadm(8) and vntsd(8) in the Oracle Solaris 11.4 Reference Library; Booting and Shutting Down Oracle Solaris 11.4 Systems and the Oracle ILOM Administrator’s Guide in the Oracle Solaris 11.4 documentation; and the equivalent (1M) and (4) man pages plus System Administration Guide: Advanced Administration (Managing Terminals and Modems) in the Oracle Solaris 10 1/13 Information Library.