If there is anything more satisfying then putting a computer to good use, it would be doing so with a cluster of computers.
This is the story of how I went from chipping away at archiving my movie collection one DVD at at time to running a bare-metal Kubernetes cluster capable of transcoding dozens a day.
One night a few months back, my husband decided to stream a movie on Netflix, something that we already owned on DVD because getting off the couch to put the DVD in the player is hard. Silly? You bet! Until I remembered that Comcast has recently imposed a 1TB monthly bandwidth cap on our account, and then it felt a whole lot more silly. 🤦♀️
Suddenly I was a woman on a mission: to save all my DVDs so that I could watch them without getting off the couch. A most honorable honorable quest indeed.
All five nodes in my little pony k8's cluster is running full tilt transcoding videos. 🔥 (The master, Twilight Sparkle, not pictured) pic.twitter.com/0SPOQQPQLe
— Carolyn Van Slyck (@carolynvs) August 21, 2017
It took me quite a few weekends, but I finally did it! ❤️ I am now the proud owner of the My Little PonyTM cluster pictured below. It can can simultaneously transcode five HD videos in three hours. In case of a power outage, not an uncommon occurrence around here, everything will reboot and pick up where they left off without a hiccup. If I get a little crazy and buy a few more nodes, they can go from assembled to “work horses” in about 7 minutes.
We’re gonna need some more horsepower!
It goes without saying that a pony-powered cluster is a gift unto itself. But for me, getting there, finishing a side-project, was the best part. I’m going to skip over my exact steps at times; this isn’t a howto. Each step in the story serves a purpose though. If I only showed you the final working setup, you may be tricked into believing that said magical setup had sprung fully-formed from my brainbox, and that is very much NOT the case.
I am not cool enough to do this
I worked on this in fits and starts, with long periods in-between where I did other things like contribute to OSS, be with my family, binge on Netflix, and post pictures of my cats on Instagram. If I was ever going to finish, I needed to change only one thing at a time, ideally completing each piece by the end of the weekend. That way whenever I found more time to work on my project, the only thing I needed to do was identify the next step. I never wanted to be stuck figuring out a chain of “wip: still borked” commits, two stashes, and a handful of uncommitted changes that I didn’t remember making three weeks ago.
A day in the life of Carolyn’s commit history
So here it is. The true story of how I actually finished something:
Next: DIY Netflix