I just spent a good chunk of today migrating some services onto new docker containers in Proxmox LXCs.
As I was updating my network diagram, I was struck by just how many services, hosts, and LXCs I’m running, so counted everything up.
- 116 docker containers
- Running on 25 docker hosts
- 50 are the same on each docker host - Watchtower and Portainer agent
- 38 Proxmox LXCs (19 are docker hosts)
- 8 physical servers
- 7 VLANs
- 5 SSIDs
- 2 NASes
So, it got me wondering about the size of other people’s homelabs. What are your stats?
You must log in or register to comment.
Dang, how does your isp feel about that many machines talking out to the internet, have they made you pay for business plans yet?
Lol - I’m on unlimited 1Gbps fibre here. So far, they haven’t raised any concerns.
That’s awesome, best of luck it stays that way!
I’ve got one headless cheap desktop PC sitting under my desk.
When I read lists like this, I often wonder, what is this person doing with all these containers and such? Do they actually use all of them regularly?
I’ve got:
1 proxmox machine serving - Openmediavault - 2 shares (jellyfin, general smb shares) Homeassistant Uptimekuma for monitoring Jellyfin
And some misc VMs for trying out things.
1 pi4b - pihole 1 pi3a+ tailscale subnet router / exit node
I often look at lists of things i can host and think to myself “do I need this?”. This br8ngs me back to huge lists of services like this and my curiosity. Do folks actually interact with all these services regularly? Honest question, no shade intended.
Do folks actually interact with all these services regularly?
Regularly, probably not, just depends. If you only spin things us to setup or learn something, no. https://infosec.pub/comment/5234431
Do folks actually interact with all these services regularly?
In my case, yep. I believe in as much separation between services as possible, so each service essentially resides on its own docker host, whether physical or Linux container.
That said, some of my services are stacks of multiple containers. For example. my DNS service is a pair of Pi-hole DNS servers, each running their own Pi-hole container, but each one also running containers for Cloudflare tunnel and telemtry export to Prometheus.
Immich has a stack of 6 containers, Piped a stack of 5. So, out of the 66 containers (that aren’t Portainer agent or Watchtower), it probably condenses down to around half that number (eg. the 25 docker hosts I have, plus a handful or two others).
each service essentially resides on its own docker host, whether physical or Linux container.
This is the way. Multiple simple dedicated systems is so much easier to maintain than a single “do everything” server.
It’s what docker and Proxmox were born to do!
Old laptop, Debian with docker running nextcloud, navidrome, jellyfin, gitea, librespeed, wireguard, dnsmasq, and nginx as a reverse proxy.
I have a NAS and it runs deluge to download torrents, and hosts two very basic websites.
- 33 nomad jobs, most being containers
- 12 physical nomad clients
- 3 amd64 poweredge
- 2 pi4
- 6 Nano Pi r5c
- 1 odroid M1
- Ceph: (nomad orchestrated)
- 8 OSD
- 50TB total raw disk
Ah - I’ve been meaning to look into Nomad. I have plenty of admiration for Hashicorp’s products. How are you finding it?
My day job is a lot of kube/openshift so nomad is refreshing. Having the template blocks are amazing and makes it so that much of what helm gave me is not required. Parameterized jobs are the best once you find a good use case for them!
At my day job, we took a look at nomad and now we are planning to run everything in nomad. It’s just so simple to understand and a joy to use.
I believe they changed some of their licensing from the fallout of their IPO. Just worth noting for the selfhosting crowd. I know terraform is being forked entirely, but I’m unfamiliar with the specifics beyond that.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters AP WiFi Access Point DNS Domain Name Service/System ESXi VMWare virtual machine hypervisor Git Popular version control system, primarily for code HTTP Hypertext Transfer Protocol, the Web LVM (Linux) Logical Volume Manager for filesystem mapping LXC Linux Containers MQTT Message Queue Telemetry Transport point-to-point networking NAS Network-Attached Storage NUC Next Unit of Computing brand of Intel small computers PSU Power Supply Unit PiHole Network-wide ad-blocker (DNS sinkhole) Plex Brand of media server package PoE Power over Ethernet RAID Redundant Array of Independent Disks for mass storage SSO Single Sign-On Unifi Ubiquiti WiFi hardware brand VPN Virtual Private Network VPS Virtual Private Server (opposed to shared hosting) ZFS Solaris/Linux filesystem focusing on data integrity nginx Popular HTTP server
20 acronyms in this thread; the most compressed thread commented on today has 3 acronyms.
[Thread #370 for this sub, first seen 24th Dec 2023, 07:35] [FAQ] [Full list] [Contact] [Source code]
You’ve got like a whole DCs worth of stuff. I’ve downscaled the hardware in my server a lot, but it’s still just a single Threadripper 2970wx with 128 GB RAM and 50 TB of ZFS storage and 50 TB of cloud based object storage in a midtower case. I have like 20 containers running, one is a Caddy webserver which acts as a reverse proxy for all the others.
I love to do things to excess as much as the next geek, but I could never find a reason to run as much as you have.
Honestly, it’s because I like to play. I don’t need PEAP auth for my wireless network, but I run a radius server providing MAC and user auth, anyway.
I hear ya, the answer to “why?” is usually “because I can” 😂
About 8 months ago I had 20x HDDs and 8x NVME drives in my server, totaling 187 TB across three ZFS pools. I could write to the largest pool (2 RAIDZ1 striped vdevs, 6 drives wide) at 250 MB/sec and read from it at over a GB/sec and that was from spinning rust with NVME “special devices”.
What was I doing with all of this? Pirating movies and TV shows and running a media server for my friends and family.
It’s not much, but I’ve got a little LG netbook with an Atom CPU and 2GB RAM running Pi-hole and Syncthing.
My starting point (with this incarnation of my homelab) was my Asrock ION330 nettop box. Then I discovered Raspberry Pis. Then I decided I needed a couple of HP DL360s. RIP my power bill.
One day when I’m all growed up I want to have a better setup. For now I’ve got what I absolutely need.
Yep - fair enough. Admittedly, my homelab is as much for professional development as it is home use, but pretty much everything gets used all the time.
I’m able to get a lot of gear secondhand through my job, so I’ve got:
One 2u Intel server running proxmox in a ‘cluster’ (circa 2013ish. Added RAM and upgraded the CPU/storage.)
One Intel nuc with an i7-7th gen as the other host in the cluster - only one VM is set to fail over between the two if needed.
VMs:
- Plex
- 2x PiHoles (one of these is the failover VM) (these also have a few docker containers like Uptime Kuma.)
- Windows arr box (I know it’s blasphemy but I felt more comfortable doing that stuff in windows)
- anything else I want to mess with because the server really doesn’t run that hard.
Network:
- Sonicwall TZ 300 (incl a perpetual VPN license)
- Unifi 24 port switch (it’s gigabit and POE but doesn’t output enough power for the…)
- single Unifi AP.
All acquired over the last couple years for the low low price of “it was going into the trash anyway”
Nice! There’s nothing better than finding new life for old tech.
Wow I am not in your league
I am currently migrating from a dedicated docker host to a proxmox host with multiple LXC containers.
old host - 23 docker containers, 128GB system drive, 4TB data drive
backup server - 1 docker container, 1TB disk
proxmox - 3 LXC containers, one of which has 3 docker containers. 500GB system drive, 4TB media drive (not LVM)
The plan is to migrate the loads on the old host to the proxmox host. I also have another 4TB drive coming with the intent of setting up a RAID with 2 of the 4TB drives.
I don’t have a homelab ( space contrains ) but I do have 2 vps that I use to host in total 13 docker containers, mail server and an xmpp server.
Edit: My lemmy server is also hosted on them.
What I’m more interesting in is what is it that you selfhost to have so many docker containers?
What I’m more interesting in is what is it that you selfhost to have so many docker containers?
Well, lots of services are stacks of containers - Immich has 6 containers and Piped has 5, for example - so it’s easy for the container count to get up there.
Other “services” are groups of containers/hosts to provide a complete capability - Home Assistant; esphome; Node-RED, for example. Then there’s just the stuff that, due to my desire for loose coupling, are spread across multiple docker hosts/containers - 5 x Sonarr/Radarr instances, for example.
2 Raspberry Pi 4 with a few services running (some directly, some via docker): pihole, pialert, gitlab plantuml, munin, restic rest server, jupyter instance, airsonic-advanced. And an old synology NAS which serves as document and media server
Dude, are you living in your company’s server room?
Lol - not quite. It sounds like a lot, but all of this runs on a couple of HP DL360s, a handful of Raspberry Pis, a nettop box, and a couple of consumer NASes.
“i swear it’s not a lot”
Goes on the describe an infrastructure setup comparable to most medium sized businesses
I love this community!
Well, to be fair, I do use my homelab to play with stuff I may or may not want to use at work. I don’t need PEAP auth for wireless, with a separate RADIUS server and Postgres database. But I have it. 😉
I have a very modest 7 docker containers on a vm on my gaming rig and I have a raspberry pi for my DNS server. Honestly my setup is quite scuffed (in comparison to yours), but it does what I need it to do
Mine’s not as fancy as it sounds - a couple second hand enterprise boxes and a handful of Raspberry Pis, mostly.
