Updating Homebrew formulae when your software gets a new version
To get bioinformatics software on my Mac I rely on the Homebrew package manager. Unfortunately, the Homebrew-science tap, where most bio software lives, doesn’t always update right after a new version is released. While you could wait for someone to update it for you, it’s faster and more fun to do it yourself, and you also get to learn about contributing to Homebrew!
I’ve written a brief tutorial / account that goes over my experience updating the muscle aligner from version 3.8.31 to 3.8.1551. It also covers some things I commonly experience when updating Homebrew formulae, and assumes some basic familiarity with the macOS Terminal.
NOTE: This is the full tutorial which you may find useful, but for uncomplicated revisions (“Basic Steps” only) you can use
brew install hub && brew bump-formula-pr --strict muscle with
If you don’t have one already, sign up for a GitHub account! All Homebrew development is done over GitHub.
Install hub via
brew install hubin the Terminal.
cd $(brew --repo homebrew/science)
Fork the Homebrew-science repository with
hub fork. If this is your first time using
hubyou will be prompted for your GitHub username and password.
Create a branch to work on the updated formula with
git checkout -b muscle-3.8.1551.
muscle.rbin your preferred editor.
brew edit musclewill default to
vim. If you have Sublime Text or TextMate installed it might open in those editors. You can change this to e.g.,
VISUAL=nano brew edit muscle.
muscle.rb, set the
urlto the URL of the latest release. In the case of
muscleI also had to update the
Back in the Terminal (outside of your editor), run
brew fetch muscleto get the latest release.
brewwill complain about a hash mismatch as the formula still contains the old hash. Don’t worry about this – just copy the provided hash into the
muscle.rbin your editor.
The formula has a
revisionfield, so remove that line as it’s a new version.
brew install -vsd --git muscle. Hopefully the new release compiles!
v turns on verbose mode so you can see what’s happening,
s says to build from source instead of using binary bottles,
d enables debugging so you can enter the build directory in case anything goes wrong, and
--git turns the build directory into a git repository so you can make changes and put them in to the formula as patches to fix minor build problems.
When things go wrong
It didn’t build! Homebrew errors out with a message
No such file or directory - src/globalsosx.cpp. If we drop into a shell (thanks to the handy debug option) we can see pretty readily that the
src/ directory no longer exists in this updated version of
muscle. Exit out of the debugging shell by pressing
Looking at the
muscle formula, we can see that the the error is caused by a patch that fixes build failures on newer versions of macOS:
def install # This patch makes 3.8.31 build on OSX >= Lion. # It has been reported upstream but not fixed yet. inreplace "src/globalsosx.cpp", "#include <mach/task_info.h>", "#include <mach/vm_statistics.h>\n#include <mach/task_info.h>" # This patch makes 3.8.31 build on RHEL 7.x # It ONLY affects Linux (in an "if Linux" clause in the 'mk' script) # It is unnecessary to create a static binary inreplace "src/mk", "LINK_OPTS=-static", "LINK_OPTS=" cd "src" do system "make" bin.install "muscle" end end
Usually when adding patches, you would also report the bug and submit any relevant patches to the upstream software developers. Here we can see that the bug was reported, so let’s see if it’s been fixed by removing all the patches. We should also change the
cd "src" do line as the
src/ directory no longer exists.
def install system "make" bin.install "muscle" end
brew install -vsd --git muscle to see if this fixes things. Looks like there’s still a build error:
ld: library not found for -lcrt0.o clang: error: linker command failed with exit code 1 (use -v to see invocation)
Build errors can be exotic and difficult to debug, but we’re in luck here: if you examine the
Makefile provided by
muscle (by dropping into the debug shell and running
cat Makefile) you can see that the developer of
muscle added a handy tip for macOS users:
# On OSX, using -static gives the error "ld: can't locate file for: -lcrt0.o", # this is fixed by deleting "-static" from the LDLIBS line.
Let’s include this patch in our new version of
muscle. Homebrew provides a function
inreplace that lets you make simple changes to files. For more complicated patches, it’s better to run
brew install --interactive --git muscle and follow the instructions to include a full diff file into the formula.
Remove all instances of
-static and try building it again with
brew install -vsd --git muscle.
def install # Fix build per Makefile instructions inreplace "Makefile", "-static", "" system "make" bin.install "muscle" end
When things (finally) go right
Success! Test that everything went well with
brew test -v muscle and check that the style and syntax of the formula are correct with
brew audit --strict --online muscle.
test works great, but
audit complained and said:
homebrew/science/muscle: * Stable: version 3.8.1551 is redundant with version scanned from URL
version line and rerun
audit to ensure that there are no further issues.
Submitting your changes
Commit the changes with
git commit muscle.rb -m "muscle 3.8.1551" and push them up to your fork with
git push -u <USERNAME>, replacing
<USERNAME> with your own GitHub username.
Navigate to the Homebrew-science page; you should see a bright banner asking if you want to open a new pull request using the branch you just pushed. Go ahead and click that, run through the pull request template and submit your pull request! (You can also check out my pull request for
You’ll now need to wait while Homebrew’s build infrastructure compiles the changes to your proposed formula on your supported macOS versions (currently Yosemite, El Capitan, and Sierra). A Homebrew maintainer will eventually come along and either merge your pull request or ask you to make some changes. Once your new formula is merged, all other users of Homebrew get to enjoy your work! Pat yourself on the back for helping others out and contributing to a important piece of open-source software.
Note that this will all become much easier when
hub 2.3.0 is released; you’ll be able to run
hub pull-request and all these steps (fork, push, adding pull request template, etc) should be done for you.
There’s a lot of edge cases to building software, which is why package managers exist so regular folk don’t have to deal with these problems. I’ve spent too much time diagnosing arcane linker errors; I contribute to Homebrew so no one else has to needlessly suffer.
If you regularly use scientific software that isn’t already in Homebrew, please consider contributing a formula, or, if your software is too unstable or specialized, starting a Homebrew tap for your own tools. I maintain my own tap that I add to whenever I come across bio software that doesn’t already exist in Homebrew-science. Consult the Formula Cookbook for tips on crafting Homebrew formulae.