Last Updated: 2022-06-13
Machines
My lab consists of 4 machines: my production machine and 3 raspberry pis.
War Rig
My main production machine is a custom built PC that dual boots Garuda Linux and Windows 11. My machines are all named after various vehicles, and since Mad Max: Fury Road is one of my favorite movies and for the general beefiness of the machine at the time of build, it’s dubbed The War Rig. The linked part’s list is a little out of date, with the notable upgrades being:
- I have two vertical 27” BenQ monitors on each side of the MSI monitor
- I upgraded to the Ergodox-EZ, which has been an absolute game changer for me after the initial couple of weeks re-learning how to type
The Windows 11 partition is only used to play a few games, which I definitely could and have played on Garuda, but it never seemed worth the effort to set up all of the creature comforts I already had when I made the switch initially, like setting the DPI of the mouse in the windows only app.
There is no particular reason I am on Garuda and not anything else. I’ve always liked Arch and Arch-based distros, and I stumbled on Garuda one day and thought the defaults in the Dr460nized looked nice and decided to install it. Before Garuda, I was on a base Arch install with KDE.
Raspberry Pis
I have 3 raspberry pi 3s model B+ in a tiny geekpi rack, each running Hypriot, which is just Raspbian that has been optimized for docker. The rack also houses a 6 port PoE swtich and an amazon basics 6 port charging block that powers the pis.
There isn’t much difference between the pis themselves. In the past, I’ve done some work around getting them to work as a kubernetes cluster, but that didn’t end up worth the effort to maintain.
The host names for the pis are all pirate ship names:
- Black Pearl (default hostname for hypriot)
- Wellerman
- Billy Of Tea
Software
What Machine hosts what?
There honestly hasn’t been much thought into which machine hosts which piece of software, and that hasn’t really come back to bite me yet. I have a pretty robust backup system that ensures that each of the important pieces of data gets backed up so that I can have some persistence during a disaster scenario, so there is no real concept of roles that each machine plays. That being said, there are a few considerations I make when deploying new software:
- Does the public docker image support the arm/v7 architecture?
- I’ve gotten around this in the past by using buildx to rebuild the images as arm/v7 to put them on one of the pis, but only if that is something I’ve had a use case for
- Do I have a reason to want to access this service if my main production machine is offline?
- Not very common, but still a consideration
War Rig Services
Portainer Agent
I utilize Portainer as the administrative interface for the majority of my services. To that end, I have an agent installed on all of the machines in my home lab that I intend to deploy containers on.
Webhook Site
GitHub - webhooksite/webhook.site: ⚓️ Easily test HTTP webhooks with this handy tool that displays requests instantly.
⚓️ Easily test HTTP webhooks with this handy tool that displays requests instantly. - GitHub - webhooksite/webhook.site: ⚓️ Easily test HTTP webhooks with this handy tool that displays requests ins...
https://github.com/webhooksite/webhook.site
This is the open source version of the https://webhook.site/ web app.
This is useful for me when creating new automations that hook into various platforms using webhooks. Often, tools will provide the ability to post to web endpoints when events occur, either to generate notifications or perform some standard actions, however finding and understanding the schema of what gets sent to those endpoints is difficult. For example, at work we wanted to be able to create annotations in our Grafana instance whenever an Octopus deployment kicked off. Octopus allows event subscriptions to post to web endpoints, but understanding what needed to happen for that data to be useful for the Grafana API required understanding how the data gets posted out of Octopus, and to gain this understanding I just had the subscription send data to the webhook.site endpoint. From the UI, I was able to understand how the data get sent from octopus, and then understand what changes the automation needed to make to post the data to Grafana.
Tiny Tiny RSS
RSS feeds help me stay up to date of various tech news sites, comic strips, and subreddits. Indeed, I was one of the 4 people who cried when Google Reader was discontinued. For awhile I thought that RSS feeds were going the way of the dinosaur, but recently I found that quite a few site still publish feeds. Using a feed reader also limits the number of ads or sponsored front page content and poorly tuned recommendations that often plague news sites.
String Is
This is the open source version of the https://string.is web app.
Often enough, I find that I need to perform some conversion on text in order to make it useful. It could be something like decoding or encoding base64, turning json data into a csv, getting a timestamp into a different format, or checking a regex expression. String-is provides a swiss army knife for all of those operations.
CyberChef
This is the open source version of the https://gchq.github.io/CyberChef/ application.
CyberChef is similar to String-Is, but with some additional features like the “Magic” tool, which can intelligently decode a string. This comes in handy when practicing security work on HackTheBox or during my OSCP labs/exams. There are also many more tools at your disposal in cyberchef, and some addtional features like being able to take in files and sequence operations in a repeatable manner.
Photoprism
PhotoPrism is my photos management system, and is like an Open Source Google Photos. I am by no means a photographer, but I do like to keep my photos and have some ability to search them and find what I am looking for, and PhotoPrism makes that easy. I have rclone pulling my images from Google Photos into a directory, which PhotoPrism watches for changes and automatically uploads the images.
Paperless
Paperless lets me take in physical documents, performs OCR on those documents, and makes them searchable. Brains are terrible at remembering things, and especially if there is a huge pile of documents that a piece of information could reside in. Paperless lets me turn those giant piles of documents into something searchable and easy to find.
Moodle
I am experimenting with Moodle as a Learning Management System. I want to be able to put out high quality free class-ware for free on the internet. This is bigger than the scope of this document, but I believe that technical skills are some of the greatest super powers that someone can learn on their own, and I believe that the only thing missing from existing platforms is some sense of accountability for work. I think adding a level of mentor/protege or teacher/student accountability will get be a boon to those who are trying to teach themselves technology using the plethora of online tools. Active Experimentation requires critical feedback from someone who understands the subject matter, and that is the missing piece in most online, self paced learning sites.
GitLab
Nothing special here, I host a private GitLab instance for a few personal repositories. I also tinker with the GitLab runners as a CI/CD service, but so far have run into issues getting the runners working on the RPIs. My GitLab instance is also useful for setting up SSO on other open source tools from time to time.
Dashy
I use Dashy as my HomeLab Dashboard. I can’t say enough good things about it, so I won’t.
Code Server
I am blessed enough to work remotely, so I don’t find much use for it, but if I ever needed to work away from my machine on a laptop or something, I can use my code server instance. My instance is already configured with the extensions that I normally use and is themed how I like it, so there is very little friction in using it vs my normal instance of VSCode on my main machine. Who needs Codespaces, amirite?
Bin
This is a pastebin service written in Rust that features server side syntax highlighting, image pasting, file drag and drop, and a neat cli tool. It doesn’t require a SQL server, and runs off of a statically linked binary. It certainly hits all of the wickets for a pastebin service.
AppSmith
This is a new addition to the lab. I work on a pretty understaffed team, and so I enjoy the idea of getting 90% of what you want from a tool with very little effort. There is a place in the world for writing beautiful custom solutions to problems, but sometimes you just need some functionality to exist quickly, and low code solutions like AppSmith can get you over the line with minimal effort.
Wellerman Services
Vikunja
Vikunja is my terribly under-utilized to-do list application. I came across it when searching for a to-do list solution that has a robust API that I could integrate with my Obsidian vault. That work has yet to be done. The app works great, but I really ought to ramp up my use of it.
Uptime Kuma
Uptime Kuma helps me ensure the applications I am hosting that I want to stay up are, in fact, up.
Tandoor Recipes
Cooking is not something I am particularly great at, but I like the idea of being able to manage my recipes and being able to automatically create shopping lists for meals I expect to cook in a week. Plus, it beats flipping through our disorganized binder of Hello Fresh recipes.
Portainer
As previously mentioned, I utilize Portainer as the administrative interface for the majority of my services. This is the actual web service instance, which also administers the docker images on Wellerman.
ChangeDetection.io
ChangeDetection.io is a useful tool to keep an eye on specific sites and to get alerted when they change. This has been useful for me in the past couple of months to monitor the CDC site for any changes to the COVID protocols, or to get updates about vaccines made available for children.
AppRise API
Sending notifications to the right places and at the right times can be a challenge. Apprise is a very robust tool that creates integrations to numerous platforms, and the Apprise API turns that notification ability into an API Service.
Ghost Blog
My wife wanted a blog to augment her youtube channel and didn’t want to have to wait on me to build a custom solution. Ghost makes creating the blog site really easy, and out of the box has all of the features she wanted.
Black Pearl Services
Postgres
Database admins are truly wizards to me. I know just enough SQL to get myself in alot of trouble, and I generally ask for a lot of help whenever I have to manipulate data in a database. I unashamedly use ORMs like sqlalchemy whenever I write code, because writing queries against a database directly is certainly an area of conscious incompetence for me. That being said, I have a Postgres server spun up in my lab, which is used as the back end for a few of my services. Administering users and tables for those services has allowed me to get some good hands on experience in manipulating the data and tables every now and then, but not enough to be very confident.
NocoDB
Now this is something closer to my speed. NoCoDB is an AirTable alternative that can turn databases into smart speadsheets and hooks API endpoints to them. Basically, it the interface of something like Excel, but the data stores in a database.
HealthChecks
HealthChecks monitors scheduled tasks like cron jobs. I am using it specifically to ensure my backups happen on the cadence I set them up for, to clean out old pastes from my pastebin solution, and to keep my RPIs updated.
Portainer Agent
See previous comments
Billy Of Tea Services
Portainer Agent
See previous comments
Whoogle
My self hosted Whoogle search is the default search engine for my browsers. I am not the type of engineer who is totally obsessed with privacy, but it does get uncanny when Google seems to know your thoughts sometimes. Plus, it’s got DDG style bang searches, which is handy.
Nginx Proxy Manager
The public face of my entire lab! I have 3 domains that route to my NGinx proxy manager instance, which then routes all of my traffic to the underlying applications from there. It creates, administers, and automatically renews Let’s Encrypt certs for all of my applications.
Matomo
This is my Google Analytics alternative. There isn’t much I am currently monitoring with it yet, and I am honestly still evaluating it and may switch it out for Plausible or Shynet
Gotify
Gotify came from a desire to have notifications go to my phone for certain events but not require Twilio or any sms integration. Gotify really filled that requirement. I’ve even set up my phone’s notification policies around different levels of priority so that low priority messages don’t wake me up at night. This was a pleasant example of discovering a tool that fit my requirements like a glove.
Notica
This is a self hosted version of https://notica.us/
Notica has a similar use case as gotify, but is generally simpler and more of a one time use kind of thing, which makes it ideal for notifications for long running tasks completing. For example, Gotify might let me know that my hourly backup is late (from HealthChecks), whereas I might use notica to let me know that the a system update is complete and I can use my package manager again or that a long running build has completed.
VaultWarden
VaultWarden is a BitWarden clone written in Rust, which implements all of the features of the BitWarden API including most of the paid services. I use YubiKeys for authentication to most things where I can, so I really wanted to have my web based password manager utilize my YubiKeys like my desktop password manager does (KeePassXC). Another shining example of a tool perfectly fitting a need.