Multistage Docker Builds For Go Services

One thing I like about using Go for services is that it can have a much smaller footprint than other languages. This is particularly useful when building Docker images to run since a Docker container contains everything needed to run a service.

But images sizes can grow quickly. As Go code gets more complicated you can and up with a large number of dependencies and tests. Sometimes additional tools are needed as part of the build process such as for code generation. I also like to have various tools available when debugging build problems. But you can have all of that and still end up with small image by using multistage builds in Docker.

Multistage builds were added to Docker a couple of years ago and work very well. I will refer interested parties to the official documentation, but one issue I have encountered as I have used multistage builds is that sometimes the Go executable I create in a penultimate stage does not execute properly in the image created in the final stage. Typically the reason for this is because I use Ubuntu based images for building (Ubuntu is the Linux flavor I’m most comfortable using) and prefer Alpine based images for the final package to reduce size. So this is perhaps a problem of my own making, but I have found solutions to the common errors I see.

FROM golang:latest as builder 

# Generic code to copy, compile and run unit tests
WORKDIR /src
COPY src/ ./
RUN go test -v


FROM alpine:latest

WORKDIR /root/

# If you don't adde the certificates you get an error like:
# x509: failed to load system roots and no roots provided
RUN apk --no-cache add ca-certificates

# Because musl and glibc are compatible we can link and fix
# the missing dependencies.  Without this we get an error like:
# sh: ./main: not found
RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2

# Copy the go exec from the builder
COPY --from=builder /go/bin/main .

CMD [ "./main" ]

The above Dockerfile is a generic multistage build of a “main” go application. Here are couple things of note:
– The “RUN apk –no-cache add ca-certificates” snippet is necessary to avoid “x509” errors.
– The “RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2” snippet is a bit of a hack to get around the glibc vs musl in Ubuntu vs Alpine. Without it, you get unhelpful, difficult-to-debug errors like “sh: ./main: not found”.

An alternative to the library linking above is to instead set flags when compiling in Ubuntu like “CGO_ENABLED=0 GOOS=linux”.

When using multistage Docker builds for Go services, I’ve found the image for the first stage to be several hundred megabytes, but the image produced by the final stage will often be around a dozen megabytes. You can’t even fit a JVM in an image that small let alone a Java based service.

Meeting Time

My home is typically not quiet. Lately, with everyone home, it means there’s been even more noise. The kids are supposed to practice their various instruments daily which include piano, saxophone, trombone, and clarinet (not to mention the “bonus” ukulele and drum playing). Boy#2 is on a smoothie kick when means the blender is frequently running. Doors slam as kids run out to play in the yard and then come back in yelling about minor injuries. Conversations (calm and animated) abound.

A year or two ago I purchased an “On Air” sign online. The cord exited from the side so I modified it so instead the cord sticks straight out the back. It was intended to be hung by a chain (included), so I added some slots to be able to screw it directly to the wall. I then removed the plug and pushed the cord through a hole I drilled in the wall. The cord was rewired to an extension cord that plugs in to a simple “smart” plug. From there it was simple to hook it into our Alexa home ecosystem.

Now, when I have a meeting and need the house to be quieter-than-normal, I just say “Alexa, turn on broadcasting” (saying “turn on on air” sounded awkward to me) and the sign illuminates. But beyond that, my family is good enough to recognize the glowing red above my door and refrain from high audio behaviors.

When I Grow Up

I don’t know what I want to be when I grow up. In fifth and sixth grade I planned to be a microbiologist, but that phased out as I transitioned into middle school. In college I studied various things including Manufacturing , Music, and Math, Linguistics, Languages, and Literature. I spent some time planning to be a college professor in Comparative Literature, but somehow that turned into my working in the software industry. But still, I don’t think I’ve got everything figured out. So I guess I’m a little jealous of the Potter boy who had Hagrid show up and say, “You’re a wizard, Harry.”

The story of Harry Potter is basically the story of King Arthur. A nondescript, British orphan is informed of what he is to be. A aged, semi-mysterious wizard oversees the education of the boy into his teen years when he comes of age and eventually is able to claim a role in public service. Along the way is various trial, adventure, and snogging where some details vary.

Apparently I am neither a wizard nor the Once and Future King. No messenger has arrived to reveal what I am to be, so instead I am forced to muddle about on my own. What an adventure that has been so far. I still don’t know what I’m going to be when I grow up, but I’m not dissatisfied with where I am now, and curious to see what the future holds.

Judging a Book by its Cover

The Mysterious Benedict Society

I often judge books by their covers and when I saw the cover of The Mysterious Benedict Society by Trenton Lee Stewart I immediately decided that it was a book that my family would enjoy. I was correct.

It is supposedly bad to judge a book by its cover, but I find it to be quite effective and, although I don’t necessarily have a definitive algorithm in mind, there are apparently some “rules” that book covers tend to follow. Over the last few years, I’ve read a few articles about studies where AIs are trained to identify the genre of a book from the cover. I’ll leave an exhaustive search of those sorts of articles as an exercise for the reader, but I will highlight “Does the cover tell you something about a book? Use AI for genre classification” (20 February 2019) by Viridiana Romero Martinez as a fun sampling.

In the case of The Mysterious Benedict Society, the title is certainly a draw, but it’s the fantastic artwork by Carson Ellis which make the book cover (and the covers of the sequels) stand out. The title could be for a murder mystery or horror novel or even a dystopian science fiction book. The art somehow makes it clear that it’s a story about children and for children yet it does not pander to the “norm” (whatever that is). It’s playful and yet dark at the same time. Although not apparent at the first glance, after reading to book one can see the level of detail from the text that went into the visual.

Sometimes I make an incorrect assumption when judging, but I was certainly correct when it comes to The Mysterious Benedict Society and the sequels/prequels.

My Standup Family: Swarming

I work in a fast-paced environment where everyone on the team is assigned several tasks to be accomplished in an allotted time. Many of these tasks are complex, but everyone on the team is a skilled, experienced professional and we do a great job of meeting our individual commitments. Except when we don’t. Sometimes things go wrong. Sometimes a system crashes. Sometimes something that was thought to be easy ends up being complicated. Sometimes someone with a critical task gets sick (or their child gets sick). Sometimes something doesn’t work and nobody knows why. Sometimes during the daily standup someone reports being blocked. When we encounter these problems, we swarm.

Usually swarming begins by getting everyone on the core team into a [virtual] room. If other expertise is required, we will pull in other people. Solving the problem is what is important–I never hear blaming or see finger pointing. Solving the problem becomes everyone’s priority. We all help. Even as we divide out work to reach a solution, we typically keep the video conference going even though people may leave awhile for things like dinner or putting kids to bed. Sometimes we shuffle around assigned tasks so that we can have the right people focused on the problem. Having everyone on the team swarm together takes time and effort, and it certainly isn’t a sustainable pace, but when we solve the problem we all share in the sense of accomplishment (and relief).

My family also swarms. We don’t gather together to fix a “defect” or implement a “business requirement,” rather we swarm together to support an individual. Here’s an example of how we swarm together.

A couple weeks ago my wife bemoaned having married me because, having taken on my last name, it moved her significantly forward in the alphabet. Even though it was only the second week of the term she had to put together a project worth a quarter of her grade (the presentation dates were assigned by last name so had she kept her maiden name she would have had three more weeks). This presentation was on top of the regular coursework and of course she still had to work at her job and do the myriad of things she does to run our home. All of the research got completed and the project was in the works, but my wife expressed during our family standup that the submission deadline was looming and she was stressed. So we swarmed.

In this case “swarming” meant planning and following through how we could support my wife. Dividing out pieces of her project wasn’t a viable option because of the concept of academic integrity, but we could support her in other ways.

The next evening, the kids made dinner. My daughter made bacon (she learned how to make it in the oven from an online video) and also a fruit salad. Boy#2 made scrambled eggs. Boy#3 made toast. My main contribution was cracking the eggs (I can do two at a time). My wife was able to come down, eat dinner quickly with the family, and get back to her schoolwork. After dinner the kids took care of the cleanup (which they usually do anyway).

Part of the project required my wife to use a specific piece of software and this created difficulty. We are mostly a Linux household (including my wife’s laptop) and the software was Windows or Mac only. Fortunately Boy#1 does have a Windows laptop so he and I cleared some space on my disheveled desk and he set the laptop up with mouse, headset, etc. so that she had an environment in which to work.

“Quiet” is not an appropriate word to describe my home. My kids take various music lessons and thus regularly practice piano, saxophone, trombone, or clarinet not to mention the more casual strumming of a ukulele, banging on a drum, or causing odd noises to emanate from my father’s old trumpet. There are also various loud communications, chases, squabbles, and bouts of laughter. The project presentation needed to be recorded meaning background noise was frowned-upon. During the time my wife was “on air”, the family quieted down a few notches, kept activity to the other end of the house, and refrained from conflict.

After our family standup that evening, one of the boys tucked in the girl child while I provided my wife with technical support. The boys found quiet ways to pass the time before going to bed. My wife finished up and submitted her assignment a couple hours before the 11:59pm deadline.

There are many other examples of our family swarming to help each other, but this is a recent one and I like how it shows the kids being involved in family success. I’m proud of my wife earning a perfect score on her project. I’m proud of my standup family swarming to support her.

My Standup Family: Remote

Last October I described how my family started including a daily standup as part of our nighttime routine. This works great when everyone is home, but that’s not always possible. Occasionally I travel for work such as to Chicago last November and Florida last month. But just because we were geographically separated doesn’t mean I couldn’t participate in the family standup–again I looked to how things function for my work team standups.

Everyone on my team works in a different place; mostly we work from home. Of the four members on my core team, we are spread out in three different timezones and four different states. In order for us to have our daily standup, we utilize video conferencing technology which allows us to communicate verbally and with facial expressions and we can also use screen sharing so that we can all see the same thing. Another important factor taken into account is finding a time that works for all of us–my teammates are kind enough to not schedule anything at “o dark thirty” my time and we also take care not interfere with anyone’s family responsibilities such as picking up children from school.

While in Florida, my family used similar strategies to make our family standup work. We don’t have access to enterprise level online meeting tools, but free “social” video chat tools are available that meet our needs. We also had to account for the three hour time difference which we did by moving up our family standup to 8:00pm left coast time which was 11:00pm in Florida. The adjusted time allowed me to participate in the evening activities at the summit I was attending and also get to bed at a reasonable time.

Normally we all sit in the living room, but we had to change that in order to allow everyone to see everyone and also avoid audio feedback from too many mics in the same area. As such, I was of course in my hotel room (mostly lounging on the bed), my wife and Boy#1 were at the kitchen table on one device, Boy#2 and the girl child were upstairs on another device, and Boy#3 was typically in the teen room on yet another device.

It may seem like needless effort and that it would have been fine for me to have missed a few days of family standup, but I believe it was worth the effort and and the value of my attendance was even greater than that of a “normal” standup because there were updates that couldn’t have happened through other informal channels such as around the dinner table. Here are key updates that were shared during these remote standups:

  • Boy#1 announced he finally decided which college he will attend in the fall
  • Boy#2 described on his progress in his exhausting life guard certification course
  • Boy#3 reported on things he was doing with friends and something he was working on on the piano
  • My daughter can make an adventure story from almost any experience (and she did), but I mostly remember her saying she loved and missed me
  • My wife reported her grades earned in her graduate school class and where she was on her final paper
  • I was able to share the experience I had eating dinner beneath the Space Shuttle Atlantis

The Mystery of the Haunted Cul-de-Sac

My cul-de-sac was reported to be haunted; not near my house, but at the other end.  Most of the mysterious events seemed to be in proximity to the abandoned (vacant with a “For Rent” sign out front) house on the street.  I had a carefully neutral response when informed by my daughter of the alleged haunting, but as a whole I was pleased–a haunted cul-de-sac is certainly interesting.  At least as long as it’s the other end . . .

As the unexplained incidents increased, it seemed like we might need to prepare for a Ghostbuster’s style end-of-the-world battle scenario.  But instead of succumbing to fear of the unknown, my daughter and her crony, in true Nancy Drew style, founded a detective agency to investigate.

I don’t recall having read Nancy Drew books except for when her path crossed with the Hardy Boys and they collaborated on a mystery.  For my daughter, however, they have been a source of entertainment–both during the reading and then during the inspired adventures.  With a sleuth bag in which carry detective gear, one is always ready for a mystery.

There was much detectiving and investigation as the girls went up and down the cul-de-sac and deliberated in their office (the gazebo).  Eventually I was informed that all of the mysterious occurrences had a logical explanation.  I think this was a relief to my daughter who replaced fear of the unknown with confidence of comprehension.  However, now she needs another mystery.  Also, I will miss living on a haunted cul-de-sac . . .

Sheldon, Cow Expert!

My family has many happy memories associated with our after dinner reading.  Gordon Korman’s No Coins, Please is one of very few books that has warranted being read aloud twice as a family.  There are many hilarious  passages throughout the text, but probably the most memorable is when one of the characters announces “Sheldon, Cow Expert!”

Those words are perhaps not funnier than the rest of the book, but for some reason the first time those words were read over the dinner table the family was reduced to uncontrollable laughter which continued for some time.  The second time we read this book, despite (or perhaps due to) being anticipated, similar mayhem ensued.  And now that phrase and book are associated with one of may happy moments we have shared.

Although not extremely flexible, we nevertheless find situations where we can quote those now immortal words.  Whenever we do, the family joins together in laughter and any unfortunate bystanders simply stare quizzically–we’ve mostly given up trying to explain.  It’s just to hard to explain who Sheldon is, what a Cow Expert is, why Sheldon is a Cow Expert, why he announces “Sheldon, Cow Expert!” and why it is so funny to my family.

Redirecting stdout to /dev/null in Go

Yesterday was the six month anniversary since I started my new job.  It’s been busy and I’ve learned a lot.  One thing I had to start learning was the Go programming language (aka Golang).  Today I wanted to temporarily suppress the standard out, but searching the Internet didn’t come up with any concise solutions.  What I did was pretty simple, but seems to work.

Basically, I’m using a third party package to parse an ini file.  The code works fine, but it is very verbose and prints out several lines of information that end up cluttering my logs.  My solution was to simply redirect the standard out to /dev/null.  Here’s what my code ended up looking like:

func unmarshal(bytes []byte, info *InfoStruct) error {

    // Redirect standard out to null
    stdout := os.Stdout
    defer func() { os.Stdout = stdout }()
    os.Stdout = os.NewFile(0, os.DevNull)

    return ini.Unmarshal(bytes, info)
}

Hopefully the code is easy to follow.  The “InfoStruct” is just some struct for data that is in the ini file.  The “ini” prefix is for the third party package that I am using.

In my quick searching, I found various information about redirecting logs, redirecting command output, etc.  There were even some examples of redirecting the stdout, but none were exactly what I wanted and most were less concise.

 

Star Wars

Just some random ramblings about my memories of seeing various Star Wars films . . .

The Original Trilogy

The first Star Wars movie I saw was The Empire Strikes Back.  It is also the first movie I remember seeing in a theater.  I went with my parents and older sister to some theater in down town Brussels.  In all honesty, I don’t think I had much idea what was going on, but I do remember the experience of going to a movie as a family.

It was about two more years before I saw A New Hope.  It was on a trip to Berlin with my father and older sister (my younger sister had just been born and stayed home with my mother).  The apartment in which we stayed had a VCR and “A New Hope” was one of the available tapes.  Being older, I was able to better appreciate the movie.

It was the summer of 1983 when Return of the Jedi was released and at the time we were visiting family in Idaho.  My older sister and I went with a few cousins to see the show.  I was at a good age to enjoy the Ewoks and fondly remember the outing to see the show.

The Prequels

My experiences with the prequels are less interesting and less memorable.  Episode I was seen in Arizona while visiting my new sister-in-law.  A couple years later in Colorado I accidentally saw Episode II a week before it was officially released.  I still haven’t seen Episode III even though upstairs there is a Blu Ray disc containing the film (and it has been sitting there for several years now).

Post Lucas (aka The Disney Years)

The Force Awakens had it’s official release date of 18 December 2015, but I managed to get tickets to a 9:30pm showing on 17 December which happens to correspond with Boy#1’s birthday.  So as a birthday present I took Boy#1 and Boy#2 to experience a Star Wars movie in the theater for the first time.

I purchased six tickets for $75.60 to see Rogue One at 8:00pm on 15 December 2016–again one day before the official release.  The tickets were for me, Boy#1, Boy#2, Boy#3, friend of Boy#1, and friend of Boy#2.  The movie was satisfying, but what I remember most was getting to and from the theater.  We had gotten a fair amount of snow (for Oregon at least) and the minivan couldn’t make it up the hill to get out of the neighborhood so we bundled up and walked the two miles to see the show and then walked home with Boy#3 soldiering on without complaint even though he was half-asleep.

In October 2017 I purchased six tickets for the 9:30pm showing of The Last Jedi on 14 December–Boy#2 had a band concert that evening so the later show time was requisite.  So after the concert, the entire family zipped home for Boy#2 to change from his concert apparel and then we all headed to the theater.  This was the first opportunity my daughter had to experience a Star Wars movie in a theater; she fell asleep.

The release of Solo corresponded with my daughter’s birthday–she was not overly keen to see the movie, so we didn’t.  I expect I will see it sometime, but for now I remember not seeing the film.