Watching server load and usage

Hostctl gives you three views of what the server is doing:

• Server → Overview charts CPU, memory, disk and network over the last month, with a Needs attention list above them.
• Server → Customers → Usage shows which account a busy process belongs to, right now.
• Server → System → Monitoring installs Prometheus, and optionally Grafana, for long-term metrics and dashboards.

Reading the usage charts

Server → Overview has four charts: CPU, Memory, Disk and Network. Choose 24 hours, 7 days or 30 days above them. Hover over a chart, or focus it and use the arrow keys, to read a value at any time. Numbers for the last … underneath gives the current, average and peak figures.

• Memory counts what programs use. Cache the system can free doesn't count.
• Disk is the server's main filesystem.
• Network counts the server's real network interfaces, not containers or VPN tunnels.

The panel takes a reading every minute and keeps 31 days. A gap in a line means no readings, for example while the server was restarting.

Needs attention lists what's worth a look, worst first: a disk or memory running full, failed backups, expiring certificates, waiting security updates, the firewall being off, and more. Each item links to the page where you fix it.

Finding what's using the server

When the CPU chart spikes, open Server → Customers → Usage. It lists the server's processes, busiest first, with the account and domains each one belongs to.

• Refresh snapshot reads the processes again. Nothing updates on its own.
• Find an account or process searches by name, email, domain, Linux user, UID or PID.
• CPU avg. is the process's average over its whole life, where 100% is one CPU core. A long-running process that just got busy can look quieter than it is, so refresh a few times.
• Resident memory includes memory shared with other processes.

A process shown as "System / shared service" belongs to no single account: the web server, the mail server, databases, or older shared sites. The domains listed for a process are its owner's domains, not proof of which site it's serving.

If you're already on the server over SSH, find the same account from a Linux user, PID or UID:

sudo /opt/hostctl/bin/account-owner --pid 1234

--user hc_5 and --uid 1005 work the same way.

Collecting metrics with Prometheus

Prometheus keeps detailed metrics for as long as you need them and can send them to Grafana Cloud. It runs in a container, so install Docker on Server → System → Features first.

1. Open Server → System → Monitoring.
2. To send metrics to Grafana Cloud, fill in Remote write URL, Metrics instance ID and Metrics publish token from your Grafana Cloud stack. Leave all three empty to keep metrics on this server.
3. Press Install Prometheus. Downloading the image can take a few minutes.

Prometheus collects Hostctl's metrics every 30 seconds. To change the Grafana Cloud details later, edit them and press Apply Prometheus settings; applying with the three fields empty turns remote write off. The token is stored on the server and never shown again.

To collect metrics from one of your own containers, choose it under Container scrape targets, give it a lowercase Job name, and press Add target. Only containers that publish a TCP port on the host appear, and Prometheus reads /metrics on that port.

Opening Prometheus and Grafana

Neither is reachable from the internet. Prometheus listens on port 9090 and Grafana on port 3000, on the server itself only. Reach them through an SSH tunnel from your computer:

ssh -L 9090:127.0.0.1:9090 -L 3000:127.0.0.1:3000 you@203.0.113.7

Then open http://127.0.0.1:9090 or http://127.0.0.1:3000 in your browser.

You don't need a local Grafana if you use Grafana Cloud. If you want one, press Install local Grafana, sign in as admin, and add http://127.0.0.1:9090 as a Prometheus data source. The admin password is on the server:

sudo cat /var/lib/hostctl-portainer/monitoring/grafana/config/admin-password

Remove Prometheus and Remove Grafana remove the containers but keep their data, so installing again picks up where you left off.

If the page says monitoring status is unavailable, Docker or Hostctl's container service isn't running. Check that Docker is installed on the Features page, then press Refresh status.