How do you monitor your server containers, disks, load…?

Do you use an easy-to-use web interface? Do you do everything via SSH? Or maybe you’ve got a more complicated setup?

I want to change my setup and I’m looking for new ideas, I’ve been using Cockpit for some years and some of the plugins are really outdated (ZFS for example) and others are completely broken (docker-compose).

  • Concave1142@lemmy.worldEnglish
    3·
    10 months ago

    Zabbix for agent / snmp based statistics.

    Uptime Kuma for up/down states with a webhook notification into Discord so I get instant alerts on my phone when one goes down.

  • SorteKanin@feddit.dkEnglish
    41·
    10 months ago

    Grafana set up to run on the server locally, then I connect to it via SSH forwarding. Then I can view all kinds of metrics in my browser in a neat interface.

  • aksdb@lemmy.worldEnglish
    25·
    10 months ago

    My own server? YOLO

    At work? Grafana, KOBS, Victoria Metrics, Jaeger, OpsGenie, …

    • AlphaAutist@lemmy.worldEnglish
      1·
      10 months ago

      This is the first time I’ve heard of Victoria Metrics. It looks like it has a similar use case as Prometheus, is that correct? If so, what made you or your team choose one over the other?

    • summerof69@lemm.eeEnglish
      2·
      10 months ago

      My own server? YOLO

      I can’t figure out whether there’s a monitoring tool called YOLO or you don’t monitor anything.

      • aksdb@lemmy.worldEnglish
        7·
        10 months ago

        Now I am intrigued to develop one that is called YOLO.

        But just in case: no, I don’t monitor my server. If I notice something not working, I ssh into the machine and check what’s up. I don’t want to deal with another zoo of services for the monitoring part.

    • cmeu@lemmy.worldEnglish
      4·
      10 months ago

      Same been running netdata for years. They’re monetizing now where it used to just be free. Good for them, it’s a great product. And it’s foss

    • Encrypt-Keeper@lemmy.worldEnglish
      2·
      10 months ago

      I love how easy to use NetData is, but when running it on my home servers it destroys their performance lol. Every once in awhile I check in to see if it runs better.

      • ArmoredCavalry@lemmy.worldEnglish
        1·
        10 months ago

        That’s strange, I’ve run it fine on some very underpowered hardware. Are you adding a specific monitoring integration with it, or just out of the box settings?

        • Encrypt-Keeper@lemmy.worldEnglish
          1·
          10 months ago

          Just out of the box. I am usually running it as a container on UnRAID on an x86 machine. It seems primarily to just be a big memory hog when I’ve tried to use it.

          • ArmoredCavalry@lemmy.worldEnglish
            1·
            10 months ago

            Weird! For reference one VM I run on only has 1 GB of memory, and Netdata uses 100-200 MB. Could be something going on with UnRAID though. Definitely some sort of bug I’d think, since normally resource usage should be very low across the board.

  • Dran@lemmy.worldEnglish
    3·
    10 months ago

    How has nobody in this thread said check_mk yet?

    It’s free, you host it yourself. It’s built off of nagios, compatible with nagios plugins, supports snmp or agent based checks. It can email, SMS, slack or discord you when something breaks, you can write your own custom checks in any language that can output to a local console… I could never imagine even looking for something else.

    • corsicanguppy@lemmy.caEnglish
      1·
      10 months ago

      +1 for check_mk.

      It’s got a scriptable config file that begs for automation like mgmtConfig and it does SNMP. For me, that’s it. SNMP->MQTT->SNMP next year.

    • Toribor@corndog.socialEnglish
      1·
      10 months ago

      I started using Checkmk recently after it was mentioned here and I really like it. I’d used Zabbix a bit but was annoyed at how much work it took to get it to do what I want. Checkmk was a lot better right out of the box.

  • CronyAkatsuki@lemmy.cronyakatsuki.xyzEnglish
    1·
    10 months ago

    Used to not monitor at all, but after setting up crowdsec I got completeky into prometheus + grafana system resource and security monitiring on my dedicated hetzner server.

    I also keep uptime kuma on another cheap vps to monitor the states of websites directly for issues and I have seted up watchtower to send ntfy notifications on updates so I can know an update is the one fucking everything up.

    Recently also setup restuc backups so I made it so I also get backup health check logs as ntfy notifications on my phone, which really helps me keep everything runnig.

    What I really need to also do is create prometheus/grafana alerts for additional things to get notifications on my phone for them also ( like when crowdsec starts to randomly not get any more acqusitions, so I have to restart it. Once passed over a week before I looked at grafana and caught that ).

    Note: this is all a hoby, I also don’t host anything at home out of a couple reasons, most important being internet and hardware is expensive af here so it’s simply cheaper for me to play around with vps’s and hetzner dedicated servers.

  • its_me_gb@feddit.ukEnglish
    8·
    10 months ago

    Node exporter on hosts, OpenTelemetry collector to scrape metrics and collect logs, shipping them to Prometheus and Loki, visualising with Grafana.

    Day job is for an observability platform where we heavily encourage the use of (and also contribute) to the OpenTelemetry collector project, hence my use of it.

    • OSH@lemmy.mlEnglish
      1·
      10 months ago

      Similar setup here with additional exporters like cadvisor for container metrics and other components.

      OpenTelemetry is awesome, but still a very fast moving project. Expect therefore more frequent updates and changes compared to more older and established projects.

    • farcaller@fstab.shEnglish
      1·
      10 months ago

      Try VictoriaMetrics. Basically the same feature set as Prometheus, but so much more resource friendly for homelab scale. I store some metrics for 12 months now, because it’s easy.

    • Trincapinones@lemmy.worldOPEnglish
      11·
      10 months ago

      I am running everything in docker compose so I’ve never found a use to it that justifies the waste of power

      • Possibly linux@lemmy.zipEnglish
        0·
        10 months ago

        docker-compose doesn’t scale well and if you run it natively it is a little less secure.

        Virtualization adds 1-2% of overhead at most and gives you way more control of how the hardware is used.

        If you setup is small docker-compose might be easier to manage but as soon as you get more hardware it becomes the limiting factor. I still use docker-compose but now I run it in a VM

        • Toribor@corndog.socialEnglish
          2·
          10 months ago

          I switched from docker compose to pure Ansible for deploying my containers. Makes managing config and starting containers across multiple hosts super easy. I considered virtualizing too but decided it didn’t offer me enough advantages. If I ever have an issue with the host OS I just reinstall using a preseed file and then rerun my playbooks and it’s ready to go.