Open Source is Hard

I'm a very lucky guy, in that for the last 17 years or so, I've been gainfully employed as a software developer. Or, to put it another way, for 17 years, I've managed to reliably get people to pay me to play with computers all day.

That's pretty awesome.

You know what else is pretty awesome? Open source software (OSS). Software that folks like me have developed, and then given away for other people to use. Throughout my career as a software developer, I've been using OSS in one form or another. This has included stuff like the Apache web server, Cygwin, Linux, FreeBSD, gcc, and a whole host of other neat stuff. I spent five years working for a company that was pretty much built on top of OSS, and my current company is also heavily invested in the same (FreeBSD, specifically).

I say all that as a prefix, just to let you know where I stand. I like OSS, it's really amazingly cool, and it's been my bread and butter for a good portion of my adult life.

That said... OSS is also hard.

Not necessarily hard to use, mind you, though there are software packages that are... um. Less polished than others. Not even hard to customize, change, or even fix when something goes wrong. That's what my employers pay me to do, after all - I'd be doing it one way or another, with open source or without; and if my choice is between spending 3 days tracking down and fixing a bug in some OSS package, vs. spending six months writing something from scratch that will have new and unique bugs, well, I know which one I'm picking.

Here's the rub: in a lot of cases, there's a desire to give something back- specifically, to contribute code to a project that you've been using. After all, someone's given you something for free - it would be downright churlish to not reciprocate, don't you think? So a lot of developers who use OSS are more than willing to contribute back to the projects that they work with.

There's a lot of motivation to do so, to. Personally, there's professional credit - being able to point at a project like the Linux kernel, and say, "I contributed to that." In terms of my employers, there's always been a simplicity motive as well. If we fix a bug in or add a useful feature to something we're using, then contribute it back to the original project, we're actually gaining something. We now have software that does what we want, the way we want, and we don't have to maintain it ourselves. When the next version of the OSS software is released, it will (hopefully!) have our changes in it, and that's that - we can just download it, install it, and use it.

Here's why I'm writing: some people really don't understand what goes on in the gap between "Gee, we'd like to contribute this change to project Foo" and the point where the project Foo maintainers actually take the code and put it into their software. Just so you can get an idea, here's a short example...

Let's say I find a bug in project Foo. It's been giving us fits at work for a while, and we finally tracked it down. After some research, experimentation and testing, I've got a fix for the bug, and we build a version of project Foo that we can use without the hassles we were dealing with originally.

Now I've got to convince the Responsible People in the company that the stuff we did for project Foo wasn't part of the company's core intellectual or business property. In other words, it has to be something that's more or less completely useless to my current employers in order for them to consider giving it away. Makes sense, right? As time goes on, and more and more people come to understand OSS and how they can benefit from it, making your case for contributing gets easier and easier... but it still needs to be done.

OK, I've convinced the boss to let me contribute. Now I need to figure out who at project Foo I should contact in order to talk about giving them the change. Believe it or not, this isn't always easy, especially if you're looking at a larger project with a number of maintainers. In that case, just figuring out who to talk to can take some time. Once you get past that, though, you have to figure out how you're going to contact them - finding email addresses for some maintainers is not a simple proposition.

Some projects bypass this sort of thing by having a bug system in place, where external users can file bug reports against a project, and include their code changes as a suggestion on how to solve the problem. While this solves the initial contact problem in a lot of cases, it can raise it's own problems. Some projects have bug databases, but don't allow just anyone to file bugs in them. Other project have bug databases that apparently go largely unused - submitting a bug there is something like tossing a random pebble into a bottomless pit. It will be a long, long time before anybody even notices it's there.

In any case, you hit the next problem - communication. Whether it's by email, or a mailing list, or a web forum, or comments in a bug tracking database, you've got to interact with the people who maintain the project. You;ve got to convince them that the bug you've fixed is, indeed, a bug. Unless you're really skilled or lucky, or your change is truly trivial, you'll probably end up modifying your contribution at least a little bit - to match the programming style of the project, to clean up some code that Joe Developer didn't feel comfortable with, to add some comments and/or documentation of your changes... whatever. You're probably going to end up going back and forth a couple of times, even if you tried hard to get all those things right in the first place.

Not to mention that you're probably going to be asked to work agains the latest and greatest version (head of line) of the code for project Foo. You may have fixed a bug in Foo 1.0, but they're getting ready to release Foo 1.1. The changes you made to Foo 1.0 may require some reworking in order to fit in with the other changes that have already gone in to make Foo 1.1 possible. And if project Foo has a complicated build process, it may take you a while just to get to the point where you can build the old, buggy software yourself.

Overall, though, working against head of line isn't all that hard. However, before someone will accept your code, they probably want some reassurance that you've actually, you know, tested it. Depending on the bug, the software, and the environment it needs in order to run, that may mean anything from "run this command, and you'll see that it's fixed" to "Setting upa virtual machine, installing and configuring a base operating system, installing and configuring support software, and then getting it into precisely the right state to show that it doesn't crash now when it used to crash before."

So. Identify problem. Generate a fix. Convince boss to let you give it back. Locate maintainers. Convince maintainers that there is a bug. Submit suggested changes. Set up a build environment to build the new software with your changes. Set up a test environment to verify that your changes fix your problem, and don't create new ones. Get feedback on your changes. Change your changes, then change the changes to your changes, until everyone is satisfied with what you've got. Contribute change to the project, and bask in the glow of a job well done.

As you can see, there's actually a lot of work that goes on after the "generate a fix" step. Remember that the next time you're wondering why "someone doesn't just fix it" - the real fix is just the first step in a long process. If you're a project maintainer, ask yourself what you can do to help minimize that time and effort commitment on the part of those who really want to contribute.

Then bask in the glow of a job well done.

No comments: