I’ve got a lot of services, some in docker, some in LXC or a VM in proxmox. Currently I’ve got no monitoring service. Recently a service went down and I didn’t notice for quite a while so now I’ve got a bunch of missing data. What monitoring tools do you all use? Looking for something that works with docker and plain Linux CTs/VMs and can notify me if a website is down, docker container crashed, VM is offline, etc.

and as a bonus feature something that I can run on two machines so if an entire machine dies, the other will notice and I’ll still receive a notification.

notification can be anything, email, sms, push, etc.

  • nitrolife@rekabu.ru
    link
    fedilink
    English
    arrow-up
    8
    ·
    1 year ago

    I use:

    1. Monitoring server - prometheus
    2. Alert manager for prometheus - alertmanager. You can write any triggers here.
    3. Web UI for prometheus - Grafana
    4. Exporters for prometheus - node-exporter, blackbox-exporter, mysql-exporter, psql-exporter etc. You can find exporter for everything you need.
    5. Some services native support pormetheus. Docker for example: https://docs.docker.com/config/daemon/prometheus/

    If you whant cluster you can install thanos on prometheus.

    • peregus@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      I’d like to explore Prometheus (I’ve never used it). Right now I use InfluxDB to store some data (ping times, temperature, servers load, etc.) can Prometheus read those values and react if something is off or should I store everything twice?

      • nitrolife@rekabu.ru
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        prometheus use own time series database. you can connect influxdb to grafana and send alarms from grafana, but alertmanager better i think. node-explorer can collect all this data (sensors, VM/PC load etc.)

        • peregus@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          1 year ago

          I’ve never used alerting in Grafana, how do they work? Is it possible to get alert if a ping is higher than xx for a period of time? What are alertmanager and node-explorer? Plugins or standalone tools? Sorry for all the questions! 😁 And thanks for the info!

          • nitrolife@rekabu.ru
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            Grafana sends an email screenshot of the graph when an event is triggered on the graph. You can see alerts part on any graph for understand.