Monitoring Private Servers with RapidSpike Connect Anything

The Problem

Monitoring private servers is a major issue faced by modern online companies. Many servers live in private networks and as a security precaution do not have public IP addresses. This makes them extremely difficult to access from the outside world. Unfortunately, this architecture makes monitoring tricky because solutions cannot ‘see’ the servers without special allowances being made.

The Solution

On-server application performance monitoring (APM) solves this problem. However, most APM offerings require large, resource intensive services to be installed. The software often requires deep system access which some people may not be keen on giving.

RapidSpike’s Connect Anything (RCA) solution means that users can write their own scripts to monitor and alert on anything. They can be written in whatever language and scheduled however the users wishes. All control is passed to the user so the RCA script’s configuration is completely customisable and nothing is left to the imagination. The uses of this are virtually endless, but here are a few examples of what RCAs could be written to monitor:

  • CPU usage stats
  • Disk space availability
  • RAM usage percentages
  • Installed and running software checks
  • Database connectivity
  • Network connectivity
  • Number of logged in server users
  • File or directory existence or non-existence

Use Case

Much of our own infrastructure is housed inside private subnets and the servers do not have public IP addresses. Servers cannot be queried directly because access is only granted via a tightly controlled, secure VPN service. We also have software that must be installed and running at all times, which is hard to monitor externally.

Therefore we’ve written Connect Anything monitor scripts that send some basic server stats and software running values to RapidSpike (ourselves!) for monitoring purposes. We schedule them using cron jobs, but they could be ran as part of a provisioning service, as a daemon or as server event triggers.

Example 1

This example sends the load averages and disk usage (‘free’ and ‘used’ values) to RapidSpike, and also checks whether two particular services are running. In production we clone and keep the script in sync from Git with Chef, the RCA ID is then set in an external config file because it’s specific to the server. Other customers may choose to manage their scripts in other ways – this is just how our systems work.

Example 2

Here we’re monitoring a basic web-node. It sends load averages, disk and RAM usage and also runs some essential software checks. This could be extended in a number of ways; to check that Apache has the correct configurations installed and modules loaded, or to ensure an SSL certificate is installed correctly.

Conclusion

RCA monitors allow users to take control of their on-server performance monitoring and get around private networking restraints. Users can monitor as much or as little, as frequently or infrequently as they wish. Granular alert settings enable false positives to be ignored and real issues to be sniffed out.

However, the scope for this product doesn’t just cover server monitoring. Check out our other RCA blog posts to see how it can be applied to other scenarios such as real-life environment monitoring.