Open Source Software and the Curse of Plenty

Overcome big budgets to successfully deploy open source business intelligence projects.

This article originally appeared on the BeyeNETWORK.

Maybe you saw this blog entry in July: “The curse of plenty: what El Mariachi tells us about writing software” by Neil Davidson. “El Mariachi” was Robert Rodriguez' 1992 directorial debut, a feature length film reportedly produced for the minuscule sum of $7,000. By the 1990s, average feature film budgets were hitting $50 million; big-budget movies now routinely cost hundreds of millions today. How did Rodriguez do it? And what does that have to do with business intelligence and open source software?

Movies and Software

Davidson noted that Bill Buxton pointed out the similarity of producing software and producing movies. It is an apt metaphor: both commercial movies and commercial software products are usually big, complicated and expensive enterprises; both are usually undertaken by highly skilled executives and backed by deep-pocketed investors; and both are driven by two sometimes conflicting goals: to produce the best, most incredibly great product possible – while generating the best return on investment possible.

An 8-figure budget buys a lot for a movie: A-list stars, endless and spectacular special effects, exotic locations and non-stop wall-to-wall advertising and promotions. But to make a profit, you've got to fill a lot of seats in a lot of theaters. If you spend $250 million to make a movie, you can gross a quarter of a billion dollars – and still not make a penny. But if you've only spent $7,000 on that movie, practically every nickel in revenue becomes profit.

How did Rodriguez do it? In his words, “You refuse to spend. Think of a creative way to get around your problem.” For examples of these creative ways around problems, you can read the trivia section at the El Mariachi entry in Internet Movie Database site.

Here are the lessons I learned from reading about the making of El Mariachi:

“Good enough” is good enough. Rodriguez used plastic water pistols that looked enough like real guns on screen. This is a venerable cinematic tradition; for example, the “light saber” used in the first Star Wars movie was actually part of a photographer’s flash unit; the “ice cream” you see in most old movies was actually mashed potatoes, since real ice cream would have melted almost instantly under the hot lights used then.

Turn your critics into your friends. Rodriguez gave two local newscasters parts in the movie to get them to stop talking trash about his filming in their town.

Improvise. This means, do more with less, and do more with what you've got or can get for little or nothing. Rodriguez wore many hats: writer, director, producer, special effects, camera operator. For moving shots, Rodriguez shot while seated in an old wheelchair; for high-speed scenes, he shot from the back of a pickup truck. He used a simple cassette player to do the sound.

Waste not, want not. To save on gas, Rodriguez shot most of his exterior scenes within the same two-block stretch of street. When actors flubbed their lines in the middle of a scene, rather than re-shoot the whole thing, Rodriguez would change the camera angle and pick up from right before the mistake. He managed to do most scenes in one take, saving time as well as film (his biggest cost).

Rodriguez said that he originally intended to shoot the movie to “practice” making a movie, and his intention was to release it direct to video. And the finished product shown in movie theaters was actually spruced up with about a million dollars' worth of post-production – still a pittance in Hollywood terms.

But how does this help you build better software solutions? And just what does it have to do with open source software?

Software and Budgets

Building a commercial software product may be like putting together a major motion picture, but getting any software project done will have similarities to TV/film productions. The number one attribute of any successful project is having a project vision. The driving force behind any project has to be a vision of what the finished product will look like. You can't just decide to make a movie or write a piece of software. For a movie, you need a script; for a software project, you've got to have a set of requirements.

With that vision, you can take the next steps: come up with a plan for achieving that vision, get financing, assemble a team and all the tools you'll need to reach your goal, and then go to work. For most software projects, as for movies (or any other big complicated projects), the size of the budget determines who you can recruit to do the work, the resources you'll have at your disposal (such as software and hardware), and how much time you expect the project to take.

And that's the sticking point: if you have a multimillion dollar budget, you'll find yourself under pressure to spend all that cash: on lots of staff, lots of hardware and lots of software, whether you need it or not. The budget begins to drive the project, and how often is there any real incentive for saving, rather than spending, any of that budget?

On the other hand, how often do corporate IT projects get the funding needed to “do everything right”? When you're a superstar IT executive, maybe you can get Hollywood-scaled budgets for your projects, but what if you're an up-and-coming young project manager and you want to make a big splash? In either case, you want to maximize the impact you can deliver within whatever budget you have. As we learn from the “El Mariachi” story, it's easy to solve problems by spraying them with the “money hose,” but that's not the only way to solve problems.

So here are the lessons applied to software:

“Good enough” is good enough. Do you really need professionally designed animations, graphics, splash screens, logos? Pop-up help prompts and tutorials? Maybe you do, but if your budget is limited, you should probably skimp on those extras and concentrate on the functional aspects of the project. Remember, if you deliver a compelling application that lacks those “extras,” you can always add them in “post-production,” but once you deliver an application, you can't get refunds on the beautifully rendered modules that turn out to be unneeded or wrong.

Turn your critics into your friends. Let's face it, this is harder to do with software than with movies. You can't give your critics walk-on roles in a software project – or can you? What do you have that you can give away, and what might your critics want that you can give them? Can you create an advisory committee for the project? Can you bring your critics on-board in some other way, perhaps by asking them to work on the project or by helping them out with resources for some other task or project they're working on?

Waste not, want not. This is a tricky one for software development, because what may seem wasteful may actually produce worthwhile returns in productivity – and what seems prudent may actually not be. It all depends on your project, your team, and your organization. Can your development team get by with cast-off computers or do they really need the latest hardware (with dual-monitors)? Giving your people the tools they need is important. However, it may not make sense to spend your budget on software licenses that you may not need until the project is complete, just to get a volume discount.

Improvise. The overarching philosophy when working with limited resources should be to make do with what you have – and, more importantly, to make positives out of negatives. A small team means you can't do everything at once –but it does mean you can focus on what's important and do it first. A small team can also respond more quickly to changes in focus.

Over the decades since the dawn of the data processing era, we've see again and again how huge projects with huge budgets are delivered late (if ever), with huge budget overruns. Open source software and a new attitude toward producing software solutions might just be the answer.

Budget Taming with Open Source Software

So how does open source software fit into the picture? When we talk about business intelligence (BI) software these days, we're still not talking about shrink-wrapped off-the-shelf products: you can't just buy a box of BI in the morning, install it at lunch and start getting results in the afternoon. You've got to assemble a BI stack, from operating systems up through end-user applications, all of which require significant investment of time and effort to customize and create useful applications; many approaches call for investment in new hardware as well.

So, one more time, let's go through the lessons of El Mariachi, as they apply to developing with open source software for business intelligence applications:

Good enough is good enough. Let's face it: open source software is, in many cases, just as “good” as proprietary commercial software. A significant portion of Oracle's product line is open source, and commercial open source vendors such as JasperSoft, Red Hat and MySQL now offer enterprise quality software, with plenty of documentation and other no-cost resources for developers. The result is that you should be able to find open source (and no-cost) software that is, at the very least, “good enough.” Even proprietary vendors, from Microsoft on down, are feeling the pinch and  offering no charge trialware that could be good enough for the early stages of development.

Turn critics into friends. Unlike in the movies, where you can get goodwill with walk-on parts, you've got to be able to make a business case for your development choices. For software, this means building the business case that using open source software is going to result in a finished project that is at least equivalent to (if not better than) the same project done using proprietary software – and that the overall expenditure, both up front and ongoing, will be lower. With open source software, you may even be able to make the case that open source software will reduce costs and improve profitability at all stages – a good way to make friends at the top.

Waste not, want not. Open source software reduces waste in many ways: it may enable you to extend the productive lifetime of existing hardware, or just avoid having to pay licensing fees for software that you're “trying out.”

Improvise. One of the often-overlooked benefits of open source software is that the source is open: that means you can do whatever you want with it, including modifying it. Have you discovered an almost-perfect open source solution? There's no reason in the world you can't modify the source code to get the software to doexactly what you want it to do. Try that with commercial software.

Open source software is no more a universal panacea than commercial and proprietary software, but open source software does provide an excellent resource for anyone who wants to bring in more projects on time and under budget.

Dig Deeper on Open source business intelligence




Start the conversation

Send me notifications when other members comment.


Please create a username to comment.