A catalyst is an agent that affects the outcome of a process by providing a (normally alternative) mechanism involving a different execution path and lower activation energy.

This diagram (courtesy of Wikipedia), shows the concept visually.

Normally, you hear about catalysis in chemical or biochemical processes (also known as enzymes in the former realm) and they have a vital role there, but here I would like to draw the attention to the general notion of it: anything that “enables” things to happen, not because they are part of the result, but because they allow it to happen more easily (thus increasing its likelihood of success).

One type of catalyst that I talked about previously is a standard: here the catalytic action is created by separating the concerns and allowing independent evolution (while focusing coordination management costs).

But there is another type of catalyst that is often overlooked: the human one.

There is a general tendency to believe that things either happen or don’t depending on how good the starting idea is and how good its execution is. My experience tells me that the presence of one (or more) human catalysts is always a huge factor in the success (or failure) of any activity.

Then why is it that so very few organizations make such positions explicit in their management hierarchy?

Interesting debate between Joel Spolsky and web standard advocates (read, for example, Sam’s reply). This reminded me of a question that Mitch Baker asked on her blog about “standards and interoperability” that I had in my to-do list to answer.

Joel’s main point is that the “idealist road” on standards is a myth and that people will just see it as a bug, complain and downgrade (or switch to something else). He’s referring to the decision of the IE team to default IE8 rendering mode to “standard”, which will break many IE-specific functions that people did in the past to make web sites work with IE6 and IE7.

He explains the problem of interoperability in a “many to many” situation (many independent producers and many independent consumers) and concludes (my words) that decentralized environments plus entropy result in a mess and there is nothing you can do about it. You have to learn to deal with it.

And that is something I agree with.

Call it like you want (”browser wars”, “ontology harmonization”, “schema alignment”, etc) they are all various facets of the same problem: contract maintenance in a decentralized interoperability scenario.

To allow independent parties to work in parallel and without central control, you have to “separate their concerns” and you can’t do that if you don’t create/specify a “contract” between them that defines the way in which they need to interact.

Joel rightly explains that such “contracts” (a.k.a. “standards”) are inherently poor or incomplete in describing the bridge between two different concern islands.

The situation is bad enough when concerns and contracts are fixed in time, but it gets insanely complicated when such concerns and contracts need to be adjusted over time. “separation of concerns in motion”, we could call it.

The problem is a nasty one: because the “many to many” environment is completely decentralized, nobody has enough power to replace all three parts of the equation at the same time (consumer, producer and standard). This is, most of the time, a feature, allowing nobody to have enough control to tilt the power more on one side than the other.

But when the standard needs to evolve, even just slightly, you can’t do it without work on both sides.

This always reminds me of something that Ben told me years ago: you can’t craft lenses (thus polish glass) with a single abrasive powder, you need at least two different ones. Why? because no matter how fine and well operating, a single abrasive has specific properties and will create specific patterns on the surface. You won’t see the patterns but the result will be not perfectly smooth. Two different ones will likely remove each other patterns. The more different abrasives you use, the smoother the surface will become.

Which is why both email and the web are such an incredibly messy yet magnificently interoperable systems: their standards are constantly under the action of many different abrasive agents (on both sides of the production/consumption barrier), each one using the surface of interaction with the rest and therefore polishing it by usage/friction.

The result is incredibly polished (where for “polish” I mean “able to allow a huge variety of players on both the producer and consumer side of the interoperability equation”), but also incredibly inertial.

But here is where I think Joel’s analysis gets poor: he doesn’t consider the effect of feedback in the system.

Everybody that tried to build automatic controls knows very well that you can’t design a stable system without some sort of feedback. For interoperability systems, friction is the feedback (where for friction I mean “some interoperability breakdown that has overall negative impact on the system”).

Usage of the contact area between producers and consumers creates patterns, some good some bad. The bad ones cause friction to develop on that surface, both parties natural tendency is to change things to reduce such friction.

Note that this does not mean that the surface will ever be polished, but that there is a natural tendency to follow the “gradient of friction”. Also note how there is no coordination between the various players, so both sides will independently try to reduce friction, not necessarily resulting in a overall positive action.

Joel’s main point is that it’s better to look for a local minimum of that friction gradient, while his detractors are saying that it’s tolerable to “climb up” the friction gradient if getting out of this valley will result in a finding a lower minimum later.

Both sides are are just expressing their subjective views about the world and about their natural search algorithm in a complicated solution space: the first one could be called pragmatic and the second idealist; the first is always guarantee to find a local minimum, but it’s not guaranteed to obtain the best possible one, the second could achieve much better results but might take a long time to find it and sustain a higher level of friction in the meanwhile.

In a search space that is so complicated and has so many players, it is almost guaranteed that using a local optimization process for friction will not yield any long-term dramatic improvements. Joel justifies this saying that even in another (lower) friction minimum, there will still be friction anyway, so why bother.

The other camp doesn’t say that friction will go away, but that braver explorations of the solution space (ones that might actually *increase* friction in the short term) might lead to lower minima later.

Who is right? Which search algorithm do you subscribe to? Which one is optimal? And for whom? For the producers or for the consumers?

There are no objective answers for these questions (each individual had very specific ways they use to ‘explore’ solution spaces in search for an optimal one, it’s probably part of how they perceive the world), but there are things that one can extract from such a debate that are generally useful:

1. immovable standards fail to adapt to the friction feedback loop and therefore die and get replaced. In order for a standard to survive in a stable the system, it needs to continue to change. This is *very* counterintuitive, especially for the kind of people that enjoy working on standards. This disconnect between the mentality of the people that work on standards and its need to be easily evolved in order to survive is a constant source of trouble (see XHTML vs. HTML5, XSLT vs. CSS, WS-* vs. REST, just to name a few);
2. standards, themselves, don’t reduce any interoperability friction whatsoever: it’s the usage ecosystem they catalyze that ultimately polishes the contact surfaces and thus reduces the friction. Again, people that enjoy working on standards have a really hard time understanding this, mostly because they focus on improving the surface polish other than improve its catalytic factor (of which, normally, they know nothing and just stumble upon).
3. standards (and their maintenance efforts) should be judged for their ability to catalyze stable polishing activities around the contact surface rather than for the qualities of the surface they were meant to describe. A good polishing process with a rough surface will always end up more polished that a well polished surface with a poor polishing process around it. This is where claiming that Postel’s principle often leads to deployment problems feels like the fool that when pointed to the stars looks at the finger.
4. successful standards have a ‘good enough’ description of the interaction surface that first enables the catalysis between the separated concern areas and a ‘good enough’ evolution process that enables developed friction to polish the surface, even when scratches are created by its use from one side or the other. Improving the polish of the interaction surface won’t do any good to the standard if its evolution process is suboptimal.

In short, a standard is a process, not a specification.

Failing to express that makes most of Joel’s points moot, like judging a dancer by looking at a few pictures of her performance.

When I joined MIT, in January 2004, very few people knew about the SIMILE Project, even if a few suspected that my presence would change that.

Now, a little more than 4 years later, SIMILE is known inside and outside academia, has produced software that is used by thousands of people, in many different environments and, most of all, has pioneered many innovations in data integration, data visualization and the relationship between open development practices and academic environments.

I’m extremely proud of the work we have done. I’ve had the unique and wonderful opportunity to work with amazing people, bringing all sort of different skills, experience and points of view to the table. Both a humbling and empowering experience, and I consider myself incredibly fortunate of having had the opportunity of belonging to this group and to this truly unique and world-wide recognized institution.

But Phase 2 of SIMILE is coming to an end and it’s time for me to re-evaluate my position and my aspirations.

While the academic environment is a wonderful opportunity for vast and deep research and for mental stimulation, it is not the place where it’s easy to make the rubber meet the road and get real traction. I’ve tried to change that, merging my decade-long Apache experience with academic dynamics and its funding; I’ve had the fortune of having wise funders and wise bosses, but I sense that I’ve personally peaked and that it’s time to re-evaluate where to invest my energy.

My contract with MIT expires Jan 2009 and there is still plenty of job to do around SIMILE and friends so I’m in no hurry and also my visa status won’t change before the end of the summer so I can’t even change jobs if I wanted to before that happens… but what some of you already know it’s now public information: I’m officially on the market for a new job.

So what would I want to do next?

First of all, I bet that the future of IT is in data not in software, so I won’t be following Ted in a job that tries to port a particular language on a particular platform, no thanks, not interested.

Second, we just bought a house in Los Angeles and quite simply, I’m not going to move. I’ve worked remotely for MIT for the last 2 years and 6 years with Apache before that, so I’m used to work remotely. I’m not afraid of traveling (flying makes me even more productive at times), but relocation is not an option.

Third, I’m way more productive if I’m passionate about my job, so if you think you can lure me into a job that I won’t feel passionate about with a higher salary, don’t: you won’t get what you’d pay for and we’ll both be unhappy.

That said, it doesn’t mean that I will work for peanuts if I liked the job, that’s part of the reason to leave academia: the job needs to be something that I feel passionate about, something that I want my name publicly associated with and something that I feel I can invest time and energy to make a strong impact and feel proud about. And I need to be properly compensated for it. In this order, but all pieces are important.

I already have several offers on the table a few of which very promising, but no done deal yet, so if you think you have a job for me and it meets that above requirements, send me an email and we’ll take it from there.

And if you worry about the future of SIMILE and its software, don’t worry: it’s not going away anytime soon, although there are already discussions on how to move forward and ’spin off’ some of the most used software in more neutral locations. But that’s another story and this is not even the right place to discuss it.