Three Prongs and a Shelf Full of Parts

Recently, I was in a meeting with another technologist and two managers, none of whom were pointy-haired.  We were working on a problem that, in outline, sounded easy to resolve.  After recapping the problem description (yep, still sounded "easy") the manager who was responsible for the troubled area dropped his recommendation on us: he needed 6 months to research a solution, pilot it, negotiate a contract, get hardware approved and ordered, and implement it.  In the most respectful way possible, the rest of us called bullshit and tried to pick apart the one-page schedule he'd brought.  In the end, while we could have nickel-and-dimed him, the broad strokes were unassailable.

The other technologist left the meeting, spent half an hour between a search engine and our internal documentation store, and scoped a solution by taping together elements we already had on our corporate shelf.  His approach was implemented in a week.

Holy cow!

Collaborative tools -- email and wikis and databases -- make it possible for firms to be well run at larger sizes.  Coase talks about this in his 1937 Nobel Prize-winning paper The Nature of the Firm, and he hadn't seen nothin' yet.

But as companies get bigger, they secrete a nautilus shell of protective practices that slow down projects.  Anyone who has ever interacted with a big company -- working at one, selling to one, servicing one -- has seen the endless paperwork, review committees, and RFPs.  On the face of it, these things suck, so why do companies do them? Companies hate to make the same mistake twice.  (Which reminds me of a darkly funny story about monkeys.)  Companies hate mistakes more than they hate soul crushing paperwork, so they'll erect a mountain of paperwork to head off a molehill of mistakes.

The problem I'm describing lives in the intersection of these two tendencies.  Collaborative tools increase the possible size of companies, but big companies demand lengthy product evaluation and contract negotiations... which raises the costs of getting these tools in the first place.

This is not a completely original line of thought, and I'll pause here to acknowledge a debt to Joel Spolsky who got this far with camels instead of monkeys.  Joel's solution (fly under the minimum-noticeable-mistake barrier) is appropriate to the problem of selling to big companies.  My problem is watching the big company I work for piss away an unaccounted fortune trying to avoid technology purchase mistakes.  So I need a different solution.

The solution I've come up with takes three prongs.  They're not steps, they're not tips, they're prongs, and you can't buy a fork without getting all the prongs at the same time.  That said, you gotta write things in some order, so here's the order you'd want if you were building a brand new IT shop.

1. Stock your Shelf with a few flexible Parts.
2. Know thy Parts in depth.
3. Duct Tape holds the universe together.

Of course, that order doesn't help you at all if you're not building a new shop -- and your team has been around since OS2 Warp looked promising.  So I'm gonna describe them to you backwards, because you have projects you have to start tomorrow, and you're about to stop reading, and I need to tell you something right now that at least slows down the death spiral.

3) Duct Tape holds the universe together.
Lets pretend that you actually have requirements from your customer for that project you're starting tomorrow.  First things first, get a big fat Sharpie and strike out all the places where they describe the solution instead of the problem.  You're the expert here, if you wanted pushy ignorant consumers second guessing you, you'd have gone into medicine.

Look at the requirements again, in that inky sea of redaction.  This has two advantages: (a) it'll be considerably shorter, and (b) it'll be possible to spot places where this problem sounds awfully like things you've already solved.  Maybe it's about tracking documents as they change, maybe it's about working better with distant teammates, maybe it's about shepherding a workflow -- but it's going to sound eerily familiar.

Now try to sketch out a solution as a centaur-style amalgamation of Parts you already have.  "It's like our project management tool, if the data entry was more like our wiki tool." "It's like our free-busy tool, but with the search capabilities of our web-based personnel directory." "It's like every CRUD database application ever written, except with a cornflower blue style sheet."

How could you make that centaur sketch a reality?  We're living in the age of the Mash Up, and even the very worst Enterprise Bloatware seems to come with an API of some kind.  This is where the duct tape comes in. 

I don't mean duct tape in the negative sense, I just mean to evoke a substance that can bind two of anything together.  Of course the binding code needs to be itself robust, documented, and supportable.  It needs to be created by a professional, and cared for as a new Part itself.  After all, the binding is the product that you're producing now, instead of buying new Parts.

But in order for the binding to succeed, you have to understand the two (or more) surfaces that you're bringing together, and that brings me to my previous point...

2) Know thy Parts in Depth
For every Part you're taping together, you need an expert that knows, in his bones:

• What is this Part excellent at?  What is it bad at?
• How big can this Part scale? 
• What demands does this Part place on supporting systems like storage and network?
• How do I make this Part fault tolerant?
• What APIs does this Part have?
• How much care and feeding does this Part require?  Is it brittle?

These are all deep questions, and they're not answerable by scanning the Part's web site.  Some of them would be answered in professional training, but some of them can only truly be grokked by a live-in expert.  A Part without the tender loving care of a full time expert is not likely to be strong enough to be a vital component of what you're building.

And that leads to the first prong...

1) Stock your Shelf with a few flexible Parts.
It's not enough to say "we buy the most excellent parts that meet our budget."  Everybody does that.  In order to have the expertise that you need in Prong 2, to build the solutions that you're dreaming up in Prong 3, you have to have a manageable few Parts.

How few?  Well lets start with the maxim "One tool for each job."  If you have several different Parts that perform exactly the same function, migrate all the responsibility to your favorite, and kill all the others.  Even MBA's know this stuff; it's an unalloyed good and you'll get the easiest bonus of your career.

The hard part is when you have two tools that do similar jobs, or a tool that is excellent at one thing and mediocre at a jillion other things.  I agree with the MBA's that there is inherent value in having fewer tools, but I say that out of love for engineers, not Cost of Ownership.  Fewer tools will let the same staff achieve greater mastery.

But under what criteria would you give up (or avoid in the first place) an excellent point solution for a good-enough extension of a system you've mastered?  For one thing, consider how completely your expert believes he can cover the true requirements, at this time.  Resist the urge to buy a point solution that can do really shiny things that you don't need right now.

Flexibility is also an important criteria for picking a generalist over a specialist.  A good Part is broadly configurable, with API surfaces that are easy for your duct tape to adhere to.  Over time, inflexible specialist systems will always be overtaken by programmable generalist systems, it's a question of speed versus acceleration.  Moreover, picking the flexible Part today increases the breadth of tasks you can take on in the future with Prong 3.

Why is this a good idea?

The chief advantage of this approach is to speed up the delivery of solutions to new problems.  Taping together Parts already on your shelf does not wake your big company's fear of risk, and it flattens your learning curve.  This is where the money is, this is how you sell it to your boss.

A subtler good is that you create more experts, instead of a legion of mediocre staff looking after a gob of systems they don't deeply understand.  Experts are more edifying to work with, and they breed more experts.

Also, solutions built in this way get designed explicitly to solve the problem at hand.  They're not grown in some focus group, they're not jealous of market share or cannibalizing existing products, they're here to serve you.  They take on some of the flavor of Situated Software, and that's got its own set of virtues.

I'd also like to make explicit that the Parts you choose should reflect the character of your team.  Maybe your shelf contains Twiki, Zimbra, and MySQL; but it could as easily be SharePoint, Exchange, and SQL Server.  I dig Open Source, but it's not fundamental to this model.

So try it out!  Especially the Sharpie on the requirements doc, that's just good clean fun.