My home lab has a mild amount of complexity and I’d like practice some good habits about documenting it. Stuff like, what each system does, the OS, any notable software installed and, most importantly, any documentation around configuration or troubleshooting.
i.e. I have an internal SMTP relay that uses a letsencrypt SSL cert that I need to use the DNS challenge to renew. I’ve got the steps around that sitting in a Google Doc. I’ve got a couple more google docs like that.
I don’t want to get super complicated but I’d like something a bit more structured than a folder full of google docs. I’d also like to pull it in-house.
Thanks
Edit: I appreciate all the feedback I’ve gotten on this post so far. There have been a lot of tools suggested and some great discussion about methods. This will probably be my weekend now.
Any chance you wouldn’t mind sharing the SSL renewal doc? Redacted of course. Mine is coming up and I’d like to do it correctly this time. :)
I use WikiJS for documentation. Simple, powerful and has a lot of features
+1 for WikiJS. As a bonus you can have WikiJS back itself up to plain text MarkDown files, so if things explode you can always just read those from wherever.
Another great feature I use is to have WikiJS back itself up into git. If I am going to a place with no internet access I can do a quick git pull and have a complete copy of my wiki including files on my laptop.
That sounds pretty handy.
Git, MinIO, Amazon S3, Filesystem and many more options to for backup🐱
Can’t wait for v3 finally
Git based static site generator, like gohugo or Jekyll.
This is interesting. I already just keep a collection of markdown files… might as well make it an internal documentation site so it’s easier to browse 🤔
I’ve been using Obsidian for a lot of other purposes for a couple years now, so I was comfortable adding my documentation into my existing vault there. I made a couple templates that I fill out for any hardware/software/networking equipment.
Since the app’s selling point is storing all your notes in plain text I wouldn’t put anything security-related in there without some encrypted container. I use KeePass for that part, and keep the file it generates in the same folder as Obsidian so I can link to it within notes. Click the link in the note, KeePass opens the vault and asks for its password.
Mind sharing your template?
Sure.
I left everything in, so no doubt there’s stuff in there specific to my vault you won’t need like metadata - adjust these to your needs or use them as a starting point for something new. There’s no network device template, I usually use the hardware one and just delete the irrelevant bits.
Thanks!
I use obsidian too. It supports mermaid too so you can make your network diagram with it.
This is the 2nd ref I’ve seen to mermaid. I need to check that out.
I love Mermaid, although I don’t think you can currently do network diagrams. I’ve seen Kroki recommended here for doing that, which supports Mermaid plus many similar markup-based diagrammers.
[Edit: added link and more info]
This is the first I’ve heard of Kroki. A quick glance at their site and wow! So many options for markup. I’ll be trying this out for sure
I would not consider Mermaid complete enough for network diagramming. The very basics are possible, but try to describe anything more complicated throws off the placement and makes the pathing whacky.
Straight flow charts are the closest you can get to a network diagram, so if you try to draw a link that travels back up the chart, it breaks mermaid’s brain trying to figure out the order of decision points (network devices).
The allure of text based diagrams is so tantalizing - but if you need them to be functional, it’s not going to happen
There’s an issue tracking the need a new diagram type to handle it.
I made myself a wiki in my helpdesk system - I use osticket to send me various email alerts to so I can track issues I need to fix, and they have a little wiki option.
Then one day that host was down and I needed some info and I was very irritated. Now all of those notes are in my Apple notes backed up in iCloud and searchable on whatever I’ve got handy so if I need info I can get the info
I played with GLPI just long enough to realize that was way more than I wanted or needed. I’d like to track changes but I don’t want to run a full ticketing/chg mgmt system to do it.
I use a combination of netbox for the physical/logical network and server connectivity, and outline for text documentation of the different components.
Woah thanks for the NetBox shout! Gonna check that out
I think I looked at netbox a while back. I may circle back to it for the actual physical layer. If I remember the ipam didn’t include network scanning tho.
Well, whatever you end up using for documentation, print it out and actively maintain an up to date paper hard copy in a 3-ring binder somewhere. That way when all your shit falls over and you have a nonfunctional LAN you can still remember how everything was set up. Don’t ask me how I know…
+1 for hard copy. Hang/tape right on or next to the rack.
deleted by creator
Documentation is not worth much if you can’t access it when needed. So yes, either print it out or store it somewhere else what you can access even if your own hardware is completely dead.
I’m defining my service containers via GitLab and I deploy them via tagged and dockerized GitLab Runners.
If something fails, I change the runner tags for a service and it will be deployed on a different machine.
Incl case of a critical failiure, I just need to setup a Debian, install docker, load and run the GL runner image, maybe change some pipelines and rerun the deployment jobs.
Some things aren’t documented well, yet. Like VPN config…
Ah yes, my router is able to access GitLab as well and pull the list of static routes etc. from it.
As someone not super familiar working in Git I’d love more details about your documentation for your setup. I have most of my containers organized in compose stacks that make sense (eg all the Arrs are in a single compose with the download client) but actually documenting the structure is … well nonexistent.
The thing is it’s not really a “documentation” but just a collection of configs.
I have organized my containers in groups like you did (“arrs”, web server, bitwarden, …) and then made a repository for each group.
Each repository contains at least a compose file and a Gitlab CI file where a aimple pipeline is defined with basically “compose pull” and “compose up”. There are alao more complicated repository where I build my own image.
The whole “Git” management is really transparent, because with Gitlab you can edit directly on the platform in a hosted VSCode environment where you can directlY edit your files and when your satisfied you just press commit. I don’t do weird stuff with branches, pushing and pulling at all. No need for local copies of the repository.
If you want to fulltext search all your repos, I can recommend a “Sourcegraph” container, but use version 4.4.2 because starting with 4.5.0 they have limited the number of private reositories to 1. But this is something for later, when your infrastructure has grown.
I use my gitea instance to backup my compose and config files. I’ve also enabled the Wiki on it to use for documentation.
This seems pretty vanilla based on what other have suggested but I use regular markdown files in a git repo.
For data flows or diagrams, mermaidJS syntax within the markdown file works wonders and when I need to link one document to another or one section to another, you can use the normal link syntax of markdown.
Easy to use, hardly any setup and easily accessible.
I’m using anytype.io, it’s been pretty neat so far.
I run a local MediaWiki appliance from turnkeylinux, super easy ti spin up in proxmox.
Joplin has been my note-taking app of choice. Free (OSS); no accounts needed; simple; local; synchronizes through my Nextcloud instance to Linux, Windows and Android; markdown-based, etc. It’s been a good workflow for me and makes taking and searching through notes quick and pretty painless.
I document my setup, backup and restoration procedures (done rarely enough that it’s good to have it written down), etc. with it.
+1 for Joplin. I have a different setup since I don’t use Nextcloud: Run Joplin server in a docker container and back up the volumes mapped to it (as well as those of other containers) with rsync.
Mix of a Bookstack wiki and various git repos on my self-hosted Forgejo.
Almost nothing haha. Some half-ass notes saved here and there, in a disorganized manner.
My stuff works, but I don’t recommend my approach.
I was going to say my notes are in Joplin, but my more honest answer is basically yours.
This is what I’m trying to get away from.
This is the way.
Same, my stuff continuing to work relies on thoughts and prayers.