check_xs, Random Monitoring Check Plugins

This is a collection of opinionated monitoring check plugins designed for use with Nagios, Icinga, and similar systems. The aim is not to provide a complete collection of plugins, but rather to fill specific gaps that I personally need.

Other plugins may already exist for these purposes, but they may not work for me, maybe because they are thousand lines of shell script full of bashism and dependencies on non-standard tools (totally random example). That's why all plugins here are written in Go using the same homegrown library and should work on all Unix-like operating systems Go targets.

Plugins

There is only one binary, containg all plugins: check_xs.

check_xs is a collection of opinionated monitoring check plugins
designed for use with Nagios, Icinga, and similar systems.

Usage: check_xs [global arguments]  [arguments]

Global arguments:
  -exit-ok
        Unconditionally exit with OK
  -verbose
        Produce verbose logging output
  -version
        Show version and exit

Commands:
  geohash
        Distance to nearby xkcd Geohashes and Globalhashes
  openbsd_updates
        Syspatches and port updates on OpenBSD
  restic
        Checks the age of last local restic backup
  ssh_key_confirm
        Verify SSH login is possible on another host via a public key
  whois_expiry
        Domain expiration date from whois

Use "check_xs  -help" for more information about a command.

geohash

This plugin checks the distance to nearby xkcd Geohashes and Globalhashes.

Geohashing was introduced in https://xkcd.com/426/, as (a joke) to create daily
geocaches based on a cryptographic hash function. This plugin continues my prior
Prometheus exporter: https://github.com/oxzi/geohashing_exporter

Usage: check_xs geohash -lat 49.4451 -lon 11.0824

Options:
  -dist-crit float
        Critical distance (km) (default 5)
  -dist-warn float
        Warning distance (km) (default 10)
  -exit-ok
        Unconditionally exit with OK
  -globalhash
        Check against the Globalhash
  -lat float
        Latitude of the check reference point (required)
  -lon float
        Longitude of the check reference point (required)
  -timeout duration
        Plugin timeout duration (default 1m0s)
  -timezone string
        Timezone to use instead of the local timezone
  -verbose
        Produce verbose logging output

Performance Data:
  None.

Examples:
  - $ check_xs geohash -lat 49.4451 -lon 11.0824
    OK: No relevant geohashes nearby

  - $ check_xs geohash -lat 49.4451 -lon 11.0824 -dist-crit 25 -dist-warn 100
    CRITICAL: Geohashes nearby: 1x CRITICAL: in 0d at 49.594395, 10.896943 in 21.33km <= 25.00km - 1x WARNING: in 1d at 49.062772, 10.589037 in 55.58km <= 100.00km

  - $ check_xs geohash -lat 49.4451 -lon 11.0824 --globalhash -dist-warn 20000
    WARNING: Geohashes nearby: 2x WARNING: in 0d at 16.991184, 142.899395 in 11242.04km <= 20000.00km, in 1d at -78.701075, 32.053204 in 14317.85km <= 20000.00km

openbsd_updates

This plugin checks that there are no outstanding updates on OpenBSD.

By both checking base system binary patches via syspatch(8) and contributed
applications from ports(7), this check covers most software installed on an
OpenBSD system, except self compiled applications, of course.

Since this check command cannot say which updates are important and which are
critical, it treats all updates as critical. If there is just one update
available, it goes CRITICAL.
There is no logic to allow-list certain updates. This is intentional.

Internally, this plugin calls syspatch(8) and pkg_add(1).

Since the issued commands require root privileges, this check commands needs to
be executed as root.

Usage: check_xs openbsd_updates

Options:
  -no-ports
        Ignore ports to be updated
  -no-syspatches
        Ignore system patches to be updated
  -timeout duration
        Plugin timeout duration (default 1h0m0s)

Performance Data:
  - openbsd_ports_outdated: Amount of updates for ports.
  - openbsd_syspatches_outdated: Amount of available base system patches.

Examples:
  - $ check_xs openbsd_updates
    OK: everything is up to date|[...]

  - $ check_xs openbsd_updates -no-ports
    CRITICAL: 1 syspatch is available|[...]

  - $ check_xs openbsd_updates -no-syspatches
    CRITICAL: 29 port updates are available|[...]

restic

This plugin checks locally configured restic backups.

By both checking the time of the last successful snapshot and utilizing restic's
own "restic check", this check plugin is able to give a brief status report
about a restic backup.

Internally, this plugin just executes restic. Either by using a custom restic
wrapper script as the -restic-bin or passing all necessary arguments via
-restic-args, the underlying restic can be configured.

This plugin might need to be executed with evaluated privileges, for example, as
the user who usually performs the restic tasks.

Usage: check_xs restic

Options:
  -restic-args string
        Additional arguments to be passed to the restic binary
  -restic-bin string
        Path to a custom restic binary or a wrapper script (default "restic")
  -snapshot-age-crit duration
        Latest snapshot age to be considered critical (default 48h0m0s)
  -snapshot-age-warn duration
        Latest snapshot age to be considered warning (default 24h0m0s)
  -timeout duration
        Plugin timeout duration (default 1h0m0s)

Performance Data:
  - last_snapshot_unix: Unix timestamp of the last snapshot.
  - last_snapshot_age: Time in seconds (s) since the last snapshot.

Examples:
  - $ check_xs restic -restic-bin restic-user-backup-backblaze
    OK: Latest snapshot "42aaaaaa" is from 2024-08-03 02:40:02.045653853 +0200 CEST, 21h55m59.071546793s|[...]

ssh_key_confirm

This plugin verifies the possibility of logging in to a remote SSH server by
providing a username and an SSH public key. SSH operates by the client offering
keys to the server, which responds whether it accepts them or not.

See the ssh-key-confirmer repository, which is the base for this plugin, for
more information: https://github.com/benjojo/ssh-key-confirmer

Usage: check_xs ssh_key_confirm -hostname HOST -user USER -user-pub-key PUBKEY_FILE

Options:
  -host-pub-key string
        SSH server's public key to be optionally checked
  -hostname string
        Hostname to be checked (required)
  -port uint
        SSH server port (default 22)
  -timeout duration
        Plugin timeout duration (default 1m0s)
  -user string
        User to login as (required)
  -user-pub-key string
        User's SSH public key (ed25519) to probe (required)

Performance Data:
  - ssh_connection: Time in seconds (s) for the SSH connection.

Examples:
  - $ check_xs ssh_key_confirm -hostname example.com \
        -user username -user-pub-key "$(cat ~/.ssh/id_ed25519.pub)"
    OK: SSH Public Key is present for "user" at "example.com:22"|[...]

  - $ check_xs ssh_key_confirm -hostname example.com \
        -user username -user-pub-key "$(cat ~/.ssh/id_ed25519.pub)"
    CRITICAL: SSH Public Key check failed, Key was not accepted|[...]

  - $ check_xs ssh_key_confirm -hostname example.com \
        -host-pub-key "$(awk '/^example.com / { print $2, $3 }' ~/.ssh/known_hosts)" \
        -user username -user-pub-key "$(cat ~/.ssh/id_ed25519.pub)"
    OK: SSH Public Key is present for "user" at "example.com:22"|[...]

  - $ check_xs ssh_key_confirm -hostname codeberg.org \
        -user git -user-pub-key "$(cat ~/.ssh/id_ed25519_codeberg_org.pub)"
    OK: SSH Public Key is present for "git" at "codeberg.org:22"|[...]

whois_expiry

This plugin checks a domain's expiration date by retrieving it from the whois
entry and alerts when certain thresholds are reached.

Usage: check_xs whois_expiry -hostname DOMAIN

Options:
  -days-critical uint
        Critical amount of days until expiry (default 14)
  -days-warning uint
        Warning amount of days until expiry (default 30)
  -hostname string
        Hostname/Domain to be checked (required)
  -timeout duration
        Plugin timeout duration (default 1m0s)

Performance Data:
  - expiration_unix: Unix timestamp when the domain will expire.
  - remaining: Time in seconds (s) until the expiry.
  - whois_lookup_iana: Time in seconds (s) to perform the initial whois lookup
    against whois.iana.org to get the TLD specific whois server.
  - whois_lookup_tld: Time in seconds (s) to perform the whois lookup at the TLD
    specific whois server.

Examples:
  - $ check_xs whois_expiry -hostname codeberg.org
    OK: Domain "codeberg.org" expires at 2028-07-05 08:54:30 +0000 UTC in 1549.6 days|[...]

  - $ check_xs whois_expiry -hostname codeberg.org -days-warning 2300
    WARNING: Domain "codeberg.org" expires at 2028-07-05 08:54:30 +0000 UTC in 1549.6 days < 2300 days|[...]

Build And Install

The only requirement is a recent version of Go, equal to or greater than the one specified in the go.mod file. After cloning this git repository, you can compile the check plugin.

go build

The plugin installation depends on your monitoring setup. On some systems, all check plugins are located within /usr/local/libexec/nagios/, but this may vary.

CheckCommands for Icinga 2 can be found in contrib/icinga2/commands.conf.

In addition, pre-built binaries are available here on Codeberg for each tagged release. Since this is my personal playground, I am building binaries only for architectures I target. However, this can be changed.

Each released binary is signed with my ed25519 SSH key, same as used to sign git commits and tags. It is available in the allowed_signers file, defined in ssh-keygen(1), ALLOWED SIGNERS.

To use it with git:

$ git config gpg.format ssh
$ git config gpg.ssh.allowedSignersFile allowed_signers
$ git verify-commit HEAD  # or whatever
Good "git" signature for post@0x21.biz with ED25519 key SHA256:plwxRk3c7Mf2Xd4sE1Ar0o5z19ICPSOLWedLEXb2KgI

To verify the release binaries:

$ ssh-keygen -Y verify -f allowed_signers -I post@0x21.biz -n file -s check_xs_openbsd_amd64.sig < check_xs_openbsd_amd64
Good "file" signature for post@0x21.biz with ED25519 key SHA256:plwxRk3c7Mf2Xd4sE1Ar0o5z19ICPSOLWedLEXb2KgI

For people who are not me, the problem might be how to get my public key securely without being MITMed first. But that might be a you-problem unless we know each other and have a secure channel to exchange keys.

Contributing

Although this is a personal collection of check plugins written to satisfy my monitoring needs, contributions are welcome!

The software is licensed under the MIT license, which is included in the LICENSES/MIT.txt file. The authorship is tracked using REUSE. There are parts being reused from other sources, being indicated by a SPDX header, or either being dual-licensed, also indicated through headers and maybe a comment.

To keep things simple, small, and not open the gates to Supply Chain Attacks™, no unnecessary Go dependencies should be introduced. This is not an absolute, just a general rule of thumb.

Patches can be submitted either through Pull Requests here at Codeberg.org or via email to the email address of the initial commit in the git log (sorry for this lame ass spam protection).