This blog was originally my talk given at Wagtail Space US 2020.
The modern web is built on layers and layers of open source software technologies, maintained by both giant companies as well as a handful of dedicated individuals and smaller teams.
There is so much high quality Free and Open Source Software available, that entire end-to-end products that make millions can be built and scaled without spending any money at all on software tools (e.g. Facebook built on PHP, Instagram built on Django).
So why pay for open source, if it can be had for free?
Because Open Source is not all fun and games, your team is too small and inexperienced, you are paying for it whether you know it or not, and the alternative is an apocalyptic hellscape.
Open source software is not all fun and games
There is an obsession with finding the reason that developers are willing to work so hard on open source software for no monetary compensation. Jane McGonigal looked at this in her excellent book Reality is Broken: Why Games Make Us Better and How They Can Change the World. Many others have tried to attribute it in various ways — but there are two main reasons that bubble up:
Fixing your hammer — If you use a tool every day, and could easily fix it to work better, and by extension everyone else, why wouldn’t you?
Working on your favorite thing— If you could even carry things around for one day on the set of your favorite movie being made, why wouldn’t you?
Looking at the desire to create new open source projects, the motivations come down to the same things that drive most invention:
“Existing thing is terrible, I can make a new thing that is better.”
“I like this thing that language x has, I’d love to use it in language y”
When extended to corporations, the main drivers are:
Portfolio — Look at this thing we made! Hire us to make awesome things for you!
Crowd-sourcing—Please test and report bugs on this thing that we are using internally, so we can get a bunch of testing and enhancements for free!
People fall into this mindset that open source software is all fun and games, especially since there are leaderboards:
But none of this really helps once your project gets popular — the requests for enhancements start flooding in, bug reports stop being helpful and start becoming a burden, and people who know better start submitting massive changes to your project, and demanding that it run on Windows.
That’s when a lot of open source projects fail — once they get popular enough to start accruing the slings and arrows of people trying to help. Open source stops being fun and games once it gets to a certain size, and it takes real work, and real hours to coordinate all the contributions. If you are depending on a project as part of your product, you can get in serious trouble if this is not managed well.
Even large, established projects such as openssl can get targeted with hacks so serious they get their own logo:
It’s never a good sign when more people know about the problem than the project.
Here are just a few things that can make maintaining an open source project not fun at all:
But at least with open source software, you aren’t paying for it, and your team can modify it if they find an issue?
Let’s talk about that…
Your team is too small and inexperienced
Turns out, most teams aren’t very large, and there is just a bit of a shortage of really experienced developers — the ones confident enough to make fixes to open source projects.
As this data from Techstars shows, more companies fail once they grow to 5 employees — so you should definitely stick to 4 employees 😉:
Seriously — the entire employee count of most tech startups is less than 10 people. And those people are busy building your product.
Those companies that are ‘winning’ the open source game on that Gitstar ranking leaderboard have a lot more employees than you, and a lot bigger teams than most open source projects you are probably depending on, and a whole lot more money pumping into them. But they also don’t tend to share their most important technologies, and the ones that actually make them money. If you want a product that the big tech companies aren’t sharing, you are going to have to look at smaller teams, and smaller companies.
Unless you happen to have a team who are all experts in every open source tech you are using, and want to spend their spare time fixing that tech? Seems unlikely.
So why not buy software? (Apocalyptic Hellscape time)
The last content management project I worked on with closed-source software ended up costing more than two million dollars. And no, we were not paying the developers on the team too much.
Compare that to the very first open source CMS implementation I led (my fourth major enterprise CMS implementation):
And yes, that was similar initial scope. A huge amount of credit on that project goes to Lindsay Hartfiel who did the front end work on that project with lightning speed.
So if 50% of my last enterprise-scale closed-source CMS project was spent on software, what I really should have done was donate to the open source projects that I used to make that project so much cheaper…
…but I didn’t. And yes, I feel guilty about it. After paying massive amounts for proprietary software, it feels wrong to pay nothing, especially when you see some of these open source projects really struggling.
What about ongoing costs?
Compared to:
And yes, they are technically the same person (me), who has quite a bit of experience with enterprise content management, but I’d only been doing Python and Django for 5 years, as opposed to 20 for Java. And I did do that Wagtail tutorial on my own time before I even started on the project, which I could do for free, without having to wait for a software license, or go to a paid training class.
What about support? The whole idea of paying for software is that there is always someone to call:
Turns out that getting a ticket open, supplying the massive amount of “how do I reproduce this” data, going back and forth across 11.5 hours of time zones, and spending three days just convincing them there was really a problem, made things take a long time to get resolved. And this isn’t unique — every big software vendor I have worked with has the same issues.
Compare that to:
In both cases, resolution means that the developer can stop looking for an answer — either they got a fix, or got a definitive answer that it’s broken, or not intended to work that way. And yes, I was including the vendor’s ‘community boards’ in this time (and most of the community board posts ended with ‘yes, that sounds like a problem, you should open a ticket’).
But here’s where ongoing costs can really hold your product back — scaling:
On day one, the traffic to your site will be x, and one day 365, the traffic would hopefully be 10x. So with closed-source enterprise software, you need buy that 10x capacity on day one to be ready for that scaling. Why? Because enterprise procurement takes forever, and runs on calendar that likely doesn’t correspond to when you suddenly see a spike in demand, and need another license now. Vendors will try and sell you on their cloud-hosted product to compensate, but then you are paying the vendor for both hosting AND licenses.
Compare that to:
So if open source is so cheap, why are you telling me to pay?
You are already paying for open source
You and your team are giving these companies your time and data every day to support their open source projects:
But even though you are paying you aren’t getting as much benefit as they are.
You don’t believe you are paying?
Are you sure that your team isn’t contributing to open source projects by asking questions about them on Stack Overflow? Submitting bug reports? Answering questions on Slack? Submitting patches? Talking about using that project online? Presenting at conferences? All those things contribute to making that open source product better — which is why those companies make those projects open source.
How to pay for your open source a better way
There are other ways to pay for open source, and get a lot more benefit for your payment. Especially if the tools you depend on aren’t directly supported by those giant tech firms, you can make your voice heard, and get bespoke development, for a fraction of the cost of licensing proprietary software.
The best way is to commission open source work, ideally from the organization that originated the project— to enhance the open source products you depend on. If you commission open source, you directly support making it better. And once it’s better for you, then not only is it better for everyone else, but you also don’t have to keep paying to keep making it better. That means if you can’t afford to update the work you commissioned in the future, it’s the open source project’s job to maintain it — not your job. You also get your organization’s name out there, which can help you find great employees, and you build brand awareness and reputation in tech community.
Donating to bigger projects makes sense, too. Python, Django, and other popular tools need funding to do big projects — like getting CPython to run on Android. They also need money to fund fellows to wrangle bugs, keep things secure, and support a variety of amazing projects. If you can’t afford to commission work from experts to enhance open source projects, donations are the next best thing. Sometimes getting donations approved from finance departments is hard, which is another reason that commissioning work is the best way.
Alternatives
Paying for support — you can always pay for support of open source software. Buying a support contract is sometimes easier to approve than donations or custom work. I’ve had mixed success paying for open source support through third parties, and find it best combined with commissioned work to maintain engagement, and not have the support fade into the background, where they forget who you are until you call in the middle of the night. Having a Slack channel with the support team, and a less formal support process than ‘submit a ticket and pray’ is also worth it.
Hosted open source — you can pay to use open source that is hosted by the organization that created it. WordPress, MongoDB, ElasticSearch, and many other tools offer this option. It’s not a terrible option, and much better that paying for closed-sourced software and hosting, but it does lock you in to forever costs, and can reduce the benefit of infinite free scaling.
Gig economy workers — you can pay folks on sites like Upwork or Fiverr to submit enhancements to open source projects on your behalf. This can be cheaper than paying the organization that created the open source project, and if that is a bigger project, may have a higher chance of getting prioritized. However, most gig workers won’t know the software as well as the creators, and there may be a lot of friction getting new features accepted. It can also be hard to judge the quality of the code. I had several gig workers build enhancements to open source software, with very mixed results. You spend a lot of time being their boss, to the point where it’s cheaper to hire an agency for anything but the smallest projects. These workers are also representing you to the tech world in both their code and behavior with the open source project you are enhancing — so vetting them is very important.
Hire an agency — you can also contract with a development firm to build the enhancements and submit them as changes to the open source project. This can be a great option, especially if the organization that maintains the open source project is willing to work with the agency closely. I got bids from several firms before I settled on hiring the organization that makes the Wagtail CMS — Torchbox — to do the enhancements we needed.
Philosophy time
All this comes together to support this ‘upward spiral’ of open source contribution, that makes things better for both you and the open source tools you depend on.
This upward spiral takes you from an open source creator, to an open source sponsor — where you are commissioning open source software from others.
Consume — You start out as an open source consumer, using other organization’s projects.
Contribute — Then you start contributing, from submitting bug reports, answering Stack Overflow questions, and submitting pull requests to the projects you use the most.
Create — Then you create your own open source projects, often as modules for other projects (e.g. my django-termsandconditions project). These are usually small one offs and helpful projects that you want to get open source benefits from.
Commission — This is the highest point of the spiral, where you are sponsoring other projects to enhance them to meet your product’s needs, perhaps even creating new, sponsored work along the way.
You may end up at different points of this spiral in your career as a developer, or the lifecycle of your organization. One year, you can get approval to commission work, and the next year, you can’t. That’s fine — the goal is to find the right contribution level for you and your organization at any given point in time.
Conclusion
Open source projects need help to get better and stay secure.
Chances are you don’t have enough folks to spend their time both building your product and enhancing the tools you use to meet your needs.
You are already paying for open source, just not in a way that gets you direct benefit.
You can commission enhancements to open source software through the organizations that create and maintain them, or through gig workers or agencies that can submit enhancements on your behalf.
All this is 1000x cheaper than paying for closed source software, and allows for infinite scaling, and much faster developer problem resolution.
I hope this opened your mind to why you might pay for your open source! If you have questions or comments, feel free to leave those below, or contact me!
Tim L. White is a veteran software engineering manager and architect, with 30 years of experience building complex systems, and particular experience with Enterprise Content Management.
