In the scope of my work at Rails Machine, I manage a lot of servers. Thousands, even. That many servers means that I often have dozens of SSH connections open at a time; some tailing log files, some running top, vmstat, or the likes, and some just waiting at a shell prompt for a rapid fire command.
To make things more complicated, I’m not always in the same place all day. Between my home office, the datacenter, Starbucks, etc, my laptop is open and shut plenty of times a day. SSH timeouts used to be my worst enemy. Enter screen…
Put simply, screen is a window manager for virtual terminals. To create a new window in screen, type C-a, c. That means hold down Control, press a, release Control, and press c. To switch between windows, press C-a, space. You can even switch to numbered windows with C-a, [0-9].
Screen is incredibly useful locally as a replacement for tabbed windows in Terminal.app, but it really shines when run from an always-on server. In my case, I run screen on a gateway/firewall server on the Rails Machine internal network, but this will work wherever you can keep an always-on server with access to all the servers you need. With an always-on screen session:
- I never have to worry about SSH timeouts. If run inside a screen session, I know a command will persist forever. If my SSH connection to the gateway times out, all that’s needed is a quick
screen -D -RRand my previous session is back up and running. - I can access the same screen session from multiple computers. If I run downstairs for a quick sammich and want to check on the progress of a test suite run, I can login to the gateway from my kitchen computer, run
screen -D -RR, and grab control of the screen session from my laptop upstairs. Everything is exactly as I left it.
All of this is fine and dandy, but a while back I started noticing a common pattern emerging in my use of screen: I’d open a new window (C-a, c), then ssh to a server. Over and over again. Two steps. Then, once my session was done, I’d exit the SSH connection, then exit the local shell again. Again, two steps. That’s one step too many, twice. No good. So, after a little research, I created a custom ssh function in my ~/.bashrc:
function ssh() {
eval last_arg=\$$#
screen -t "$last_arg" ssh "$@";
}
Now, when I run ssh user@foo inside a screen session, a new window opens for the SSH connection with it’s title set to ‘user@foo.’ All arguments passed to SSH work as intended, but the screen title only gets the user@host portion of the command. When this SSH session is terminated, the screen window is automatically closed - no need to terminate the local shell containing the SSH process. Nice!
By default, screen doesn’t display the title of all windows, but if you toss this line into your ~/.screenrc, you’ll get a nice status line at the bottom of your screen with the number and title of each open window:
hardstatus alwayslastline "%{gk}[%{G}%H%{g}][%=%{wk}%?%-Lw%?%{R}[%{W}%n*%t%?(%u)%?%{R}]%{w}%?%+Lw%?%?%=%{g}][%{B}%m/%d %{G}%C%a%{g}]"
Put this all together, and you get a screen session that looks like this:
When you need to SSH to a new host, just switch back to the gateway window by typing C-a, 0, type ssh user@host, and a new window will be opened for this SSH connection. The gateway window is left clean, ready to initiate another SSH connection when needed.
As an added bonus, if you toss this in your ~/.bashrc, you’ll get fancy tab-completion on hosts in your ~/.ssh/known_hosts file:
SSH_COMPLETE=( $(cat ~/.ssh/known_hosts | cut -f 1 -d ' ' | sed -e s/,.*//g | uniq | egrep -v [0123456789]) )
complete -o default -W "${SSH_COMPLETE[*]}" ssh
Sick, eh?