Hello all! Just curious what y’alls typical setup is when it comes to running multiple stacks which require the same “support” containers.
What I mean is, say you want to run two services that both require a connection to a database, would you run two separate DB containers, one for each service and have them connected only to their respective DB “stacks”? Or do you prefer to run a single centralized DB server/service and have your self hosted stacks all communicate with their own databases inside the server?
My recommendation, if practical, is a single, potentially containerized DB server that is backed by storage that provides high availability and redundancy. This is supposing that you are using the same sort of DB (ex SQL, NoSQL, etc) and that you are targeting a smallish number of services that are on-premise.
My reasoning here is that you can treat the DB server effectively as a storage API service and run it via some orchestration service like K8S. This lets you offload your DB stability and data integrity to the FS and/or other low-level stuff that is simple to configure once and only worry about when hardware fails. This in turn greatly reduces DB server configuration and deployment as well as treat them like livestock, not pets.
Now, if you are using a public cloud provider, my view changes slightly. Generally, I’d suggest offloading the DB to a provided service that is compatible with a FOSS alternative so that you can avoid vendor lock-in. This means that you get the HA, etc without having to worry about maintenance and configuration overhead. Just be aware of cost modeling - it’s easy to run up large bills.
One big DB will be a single point of failure. You will not benefit from much more speed, the only thing which is “easier” is backup.
I use different DBs for different stacks. If one hangs up or messed up completly, it won’t affect others
I take the latter approach – a single PostgreSQL database service for all other containers to use. That allows me to concentrate memory/CPU to a single service and optimize for that. I’ve found that a single database service uses less total resources (especially memory) than running separate DB stacks for each service.
I’ve found that a single database service uses less total resources (especially memory) than running separate DB stacks for each service.
This should indeed be the expected result. Each DB server will have a set amount of overhead from the runtime before data overhead comes in. Ex (made up numbers):
storage subsystem=256MBconfig subsystem=128MBauth subsystem=280MBapi subsystem=512MBuser tables=xMB
The subsystem resource usage would be incurred by every instance of the DB server. Additionally, you have platform-level overhead, especially if you are running as VMs or containers as that requires additional resources to coordinate with the kernel, etc.
It’s very much like micro-kernels vs monoliths. On the surface a lean micro-kernel seems like it should be more performant since less is happening during kernel time but, the significant increase in operations to perform basic tasks. For example, if storage access was in userspace, an application would need to call back to the kernel to request communication, which would need to call up to the storage driver, then back… and it becomes a death of a thousand cuts. In a monolithic kernel, the application just tells the kernel that it wants to access storage, what mode, and provide either the input or a buffer receive data.
I don’t wanna go all 90s forum user here, but I think this is the 5th post asking this. Maybe just have a look first if this has been answered already.
On topic: just use a new db with each service.
I think this is the 5th post asking this
Agree, there are a few topics like this. Other ones being “backup software recommendation” threads, “look at my dashboard” threads, “what should I self-host” threads… I just post a comment linking to previous threads like this https://lemmy.world/comment/1761863. Maybe after a while they will learn… On the other hand Lemmy’s search feature is not very pleasant to work with, so I don’t blame OP. Maybe there should be a pinned post about “frequent” questions.
On topic: single DB engine/service, connect each app to this DB service, and let each one have its own database in it. Anything else degrades performance and makes backups more complex.
I’ll usually centralized them. Use less resources than running them separately, and makes backup easier.
Backups harder(well more if them), restaurations easier right?
Separate DB containers for each service. Easier, more secure, and less messy. The overhead impact is trivial.
Multiple. They might need different versions of the database server, they might need to be scaled differently, they might need to be backed up at different cadences, they might need to be moved to different servers…. Etc.
The small marginal resource reduction is just not worth it.
