whateverthing.com

☰

As a modern PHP developer, I'm quite enamoured with Composer and Packagist, and the practice of using prepackaged libraries (otherwise known as external dependencies) to get stuff done faster. This also helps on my hobby projects, where I'm basically on my own and need all the workload optimizations I can find.

However, using dependencies can burn you.

There are many ways this can happen, so let's explore a few and then look at some potential solutions to prevent dependency disasters.

For starters, you can go overboard. One project I've contributed to, GitLab, is massive. Their latest update, 8.11, contains so many changes that I'm amazed they were able to keep to their rapid release schedule. This comes at a cost, though - their Gemfile is over 350 lines long. Each of the 187 gems can also pull in its own dependencies. A single one of those gems could break and disrupt a release.

Speaking of which, dependencies do break. The infamous "left-pad" accident in NPM earlier this year affected hundreds of projects when a deeply-buried and widely-used dependency suddenly vanished. Breakages come in all forms, though, including just the simple scenario of a small and well-hidden bug unexpectedly breaking your project.

Dependencies can go stale. Recently, while helping upgrade the OpenCFP project to use Silex 2, I ran into a situation where a stale dependency also needed to be modified to work with the new framework. Because the maintainer had gone inactive, I ended up needing to use my own fork to work around the problem. A bit of a hackish approach, but it worked for getting the process in motion. It remains to be seen what the ultimate resolution will be for that issue.

Dependencies can hold you back. For instance, you might need to upgrade a portion of your dependencies to a newer version - but the rest of your stack can prevent you from doing that. Maybe you don't have the resources to launch a full-scale upgrade, or maybe you have invested too much in anchoring customizations to a particular major version of a dependency. No matter the reason, it's always possible to set yourself up to get burned by technical debt like this.

Dependencies can fail to install. Your dependencies might be fine, but your dependency management infrastructure might have a momentary outage at a bad time and leave your application broken.

But Why?

So with all this in mind, why do I continue to use dependencies? Well, that's an easy question to answer:

The tireless work of thousands of Open Source developers is poured into these dependencies, and without their contributions, I'd have to write inestimable amounts of boilerplate code just to do basic things in my projects. Sometimes things break, it's true - but when things work, it's like having the A-Team on your side.

What Would Hannibal Do?

Now that you know the myriad ways dependencies can go awry, it would probably be a good idea to plan for it.

Automated Testing

The first stage of dependency disaster preparedness is automated testing. This doesn't have to be some amazing test suite with 100% code coverage - you can start off with a script that installs your app and verifies one or two critical parts of it. Then, before you do a deployment or a release, you can use tools like Jenkins, Travis CI, or GitLab CI to perform the simulated installation and critical checks. You can even wire it in to your source control system to check for problems on every commit. This will help prevent a "left-pad" incident - and during that incident, this level of testing is what first alerted people to the problem and how widespread it was.

Once you have basic automated testing in place, you can start investing in deeper code coverage of your features. Again, it doesn't have to be comprehensive, but it's always a good idea to ensure that your code's critical functionality is working as expected. One way to accelerate this stage is to require that all new features and bug fixes come with tests. As you backfill your test suite, you gradually reach more "remote" areas of your codebase. This will help prevent the disaster of a small bug in a dependency wreaking havoc in your project.

Deployments with Rollback and Bundled Dependencies

It's also a good strategy to have a deployment method that can roll back to the previous working version in the event of a deployment failure. Capistrano is a great tool for deploying all kinds of projects - not just ones written in Ruby.

For both web services and standalone web apps, it can be a good idea to bundle all of the dependencies into a complete release archive that contains everything needed for the app. Some projects even go as far as including the dependencies in their source control system.

The last thing you want is to be in the middle of a deployment and suddenly GitHub goes down and your dependency manager refuses to install updated packages.

By planning for the failure of the deployment and dependency management infrastructure, you can avoid a few common disaster scenarios.

Vigilance and Code Review

Combatting stale and antiquated dependencies is not quite as easy to solve with engineering tools. A keen eye has to be applied to all incoming commits that interface with dependencies. Where possible, Pre-Commit Code Review is a great practice that can improve the engineering robustness of your whole team by ensuring that the committed code meets quality standards.

Code that integrates too tightly with dependencies can make it difficult to swap out the dependency for a newer version or an alternate implementation that is more active. This creates staleness. The tight integration can be something as simple as using a magic framework feature, or having a custom event listener attached to an event that unexpectedly changes its ID in a newer version (which would cause the custom listener to be silently ignored). In some apps, this can also be demonstrated with complex class inheritance/override scenarios that suddenly break on an upgrade.

Constant vigilance is necessary to maintain solid engineering practices, otherwise your tightly-coupled code will be a big, stale pile of technical debt.

Staleness leads to antiquated dependencies. Antiquated dependencies are scary to update, because you have to pour over all the changes of all the upgraded components in order to understand what has changed. If you have a test suite, congratulations - you're a big leap closer to avoiding this slow-moving disaster. You'll be able to dabble in big upgrades and find out right away which parts of your app have broken.

Quality over Quantity

Another slow-moving disaster that requires vigilance is bloat.

Some applications, due to their scope, require a lot of dependencies. It can be challenging to keep track of every dependency and every dependency's dependencies. Often, the dependency management tool will have a way to list all of the dependencies in a project. It would be a good idea to monitor the list for changes, and check it periodically for dependencies that can be eliminated.

Licensing is also an important thing to consider with so many dependencies. Each dependency might be licensed via Apache, MIT, Public Domain, GPL, or even the WTFPL. Depending on your app or company, one or more of these licenses might actually have legal ramifications. Monitoring the entire list of dependencies then becomes very important, in order to avoid possible legal problems.

Dependencies Rock!

Now that you know some strategies for working with dependencies, you're ready to go forth and reap the benefits of your platform's rich ecosystem.

And if you can't find a dependable library for the problem you're trying to solve, well - go ahead and create one! Chances are that someone will find it useful one day.

Just ... make sure they know how to prevent Dependency Disasters, too. :)

---

Update: About ten minutes after I published this, I noticed that there was another "left-pad" incident on NPM.