check_by_ssc

check_by_ssc is able to follow a chain of SSH connections, rather than a single one like check_by_ssh.

This may be necessary in situations which involve firewalls, or traversal of several conflicting or private IP ranges.

Unlike using ssh-agent and forwarding access to it throughout the chain, (check_by_)ssc makes all connections from the Nagios host, but through tunnels (forwarded local ports) established within the earlier ssh connections.

Neither the user privkey, nor access to ssh-agent ever leaves the Nagios machine.

Check the README for more details, and example_cfg for how to set up your Nagios to use it.

v1.1 (10-Sep-2010): Added port and commandline randomization to combat "Exit Code 255"s.

Example:

Checking whether some port (here, the SSH port) on the last host is reachable still needs differentiation between direct and chained connections

Variants for direct connection:

define command {
    command_name            check-ssh-host-alive
    command_line            $USER1$/check_tcp -H `echo $HOSTADDRESS$ | sed -e 's/:.*//'` \
                        -p `echo $HOSTADDRESS$ | sed -e 's/^[^:]*//' -e 's/^://' -e 's/ .*//' -e 's/^$$/22/'`
    }

define command {
    command_name            check_ssh_connect
    command_line            $USER1$/check_tcp -H `echo $HOSTADDRESS$ | sed -e 's/:.*//'` \
                        -p `echo $HOSTADDRESS$ | sed -e 's/^[^:]*//' -e 's/^://' -e 's/ .*//' -e 's/^$$/22/'` \
                        -w $ARG1$ -c $ARG2$
    }

Variants for chained connection:

define command {
    command_name            check-ssc-host-alive
    command_line            $USER1$/check_by_ssc `echo $HOSTADDRESS$ | sed -e 's/~[^~]*$$//' -e 's/~/ /g'` \
                        -t 20 -C "$USER1$/check_tcp -H `echo $HOSTADDRESS$ | sed -e 's/.*[~@]//' -e 's/:.*//'` \
                        -p `echo $HOSTADDRESS$ | sed -e 's/.*[~@]//' -e 's/^[^:]*//' -e 's/^://' -e 's/ .*//' -e 's/^$$/22/'`"
    }

define command {
    command_name            check_ssc_connect
    command_line            $USER1$/check_by_ssc `echo $HOSTADDRESS$ | sed -e 's/~[^~]*$$//' -e 's/~/ /g'` \
                        -t 20 -C "$USER1$/check_tcp -H `echo $HOSTADDRESS$ | sed -e 's/.*[~@]//' -e 's/:.*//'` \
                        -p `echo $HOSTADDRESS$ | sed -e 's/.*[~@]//' -e 's/^[^:]*//' -e 's/^://' -e 's/ .*//' -e 's/^$$/22/'`" \
                        -w $ARG1$ -c $ARG2$
    }

Otherwise (i.e., for everything where you'd normally use check_by_ssh),

check_by_ssc will handle both - simple examples:

define command {
    command_name            check_ssh
    command_line            $USER1$/check_by_ssc `echo $HOSTADDRESS$ | sed -e 's/~/ /g'` -t 20 \
                        -C "echo OK: SSH login succeeded"
    }

define command {
    command_name            check_remote_cpu
    command_line            $USER1$/check_by_ssc `echo $HOSTADDRESS$ | sed -e 's/~/ /g'` -t 20 \
                        -C "$USER1$/check_cpu"
    }

define command {
    command_name            check_remote_load
    command_line            $USER1$/check_by_ssc `echo $HOSTADDRESS$ | sed -e 's/~/ /g'` -t 20 \
                        -C "$USER1$/check_load -w $ARG1$ -c $ARG2$"
    }

... etc. etc.

Now let's build a chain of hosts:

define host {
    use             generic-host
    host_name           Host1
    alias               A host which we can ssh into directly but which isn't pingable
    address             123.45.67.89
    check_command           check-ssh-host-alive
    }

define host {
    use             generic-host
    host_name           Host2
    alias               A host which we can reach only from Host1 and at a special SSH port
    parent              Host1
    address             123.45.67.89~172.16.12.34:5678
    check_command           check-ssc-host-alive
    }

define host {
    use             generic-host
    host_name           Host3
    alias               A host which is even behind Host2
    parent              Host2
    address             123.45.67.89~172.16.12.34:5678~192.168.12.34
    check_command           check-ssc-host-alive
    }

As said above, checking plain connectivity differs between direct SSH and chains:

define service {
    use             generic-service
    host_name           Host1
    service_description     CONNECT
    check_command           check_ssh_connect!30.0!50.0
    }

define service {
    use             generic-service
    host_name           Host2,Host3
    service_description     CONNECT
    check_command           check_ssc_connect!30.0!50.0
    }

But everything else does not:

define service {
    use             generic-service
    host_name           Host1,Host2,Host3
    service_description     SSH Login
    check_command           check_ssh
    }

define service {
    use             generic-service
    host_name           Host1,Host2,Host3
    service_description     CPU Load
    check_command           check_remote_cpu
    }

define service {
    use             generic-service
    host_name           Host1,Host2,Host3
    service_description     System Load
    check_command           check_remote_load!5.0,4.0,3.0!10.0,6.0,4.0
    }

Don't forget to set up proper dependencies ;-)

define servicedependency {
    host_name           Host1,Host2,Host3
    service_description     CONNECT
    dependent_service_description   SSH Login
    }

define servicedependency {
    host_name           Host1,Host2,Host3
    service_description     SSH Login
    dependent_service_description   CPU Load,System Load
    }