Jeffrey Greenberg on Technology Strategy

A number of startups have contacted me with plays to compete with Craigslist. Craigslist is dominant in providing localized exchange of goods and services. They are free, they have a huge brand and audience, and (from the outside) they appear to be a relatively efficient organization. The competing startups are either trying to out Craigslist Craigslist or they are going after parts of the whole by focusing on a particular commerce area, like rentals. Those that are going whole hog had better have not only superior features for entire range of commerce that Craigslist covers, but have to be just as consistent as Craigslist is in their usability. And Craigslist’s minimalist design, while it doesn’t do much, it’s easy to get and it’s consistent across the site. It’s a simple form for posting all types of requests — almost always the same no matter what is being exchanged. For a company to take it to a higher level of use and functionality a significant investment is required, and so far I don’t see any threats to Craigslist of this sort.

A better strategy I’m seeing now is from the startups that are those focused on just a particular commerce segment and doing it better than Craigslist. In the long run, they have a better chance in that they are better and free to the consumer. The “free” part of course is a tough business decision at any time, and especially now, but against Craigslist you have little choice. A couple of interesting players in this arena are: 11squarefeet which focuses on a new sub market of renting: office and desk subleasing, and okCupid which is free online dating done extremely well. Okcupid is alot of work and thus must have relatively high development costs so it remains to be seen if they can exist, but they can give Craigslist free dating a run, let alone Match, eHarmony, et. al. (Since an automated dating site (chemistry.com) matched me up with my ex-wife (really), I’ve been more than cynical of these…).

It remains to be seen whether an entrenched player like Craigslist can be removed by the strategy de jour: a mashup + better usability + a market-focus = success. Or do we really need to start investing in harder technology that raises the bar of service and efficiency. An example of such investment I’ve seen is Siri which attempts to up what can be done by using bleeding edge technology. This is an example of a number of AI-ish plays that are seeking to leverage advances in NLP . We may be reaching the end of Web2.0 when the strategy du jour is no longer so easy to succeed with. With that said, Craigslist, by being community focused, comprehensive, and free, is a tough combination to best by any means.

Testing javascript heavy, web frontends remains a hard problem in software development. Everyone wants to do it in order to be agile, but few if none can really do an ideal job of it. Some key reasons they want to automate are:

• allow developers to quickly smoke test their work before they commit it
• support extensive, comprehensive functional testing

The reasons it’s hard to do are twofold 1. the frontend in a new product development is always in flux, and 2. there’s no great tool for quickly creating robust test scripts that can be automated. In particular, while there are some recorders for the gestures on the frontend, they usually fail in certain ways that demand one to further generate custom test scripts for that tool to run. It’s extra work that often becomes outdated on the very next build.

Generally, good agile practice demands lots of test code to support quick and reliable bug finding and fixing early in the development cycle, and not sometime later. So good teams will make some compromises to their agile test strategy to get as close as is reasonable to ideal without it becoming non-agile. The typical strategies are:

• create unit tests for all infrastructure, including going up into the javascript client as far as possible, but not automating user interaction
• create testable javacript widgets (look and feel) that can be instantiated into test pages that can be quickly looked at by a developer or tester

Some tools that help automated javascript and RIA testing are:

• http://windmill.osafoundation.org/trac/wiki/BookChapter-1-Install
• http://windmill.osafoundation.org/trac/wiki/BookChapter-4-3-JavascriptTests

• Provides multi-platform testing and recording
• Downside: recorder doesn’t handle all circumstances so programming is required, and test scripts will end up with dependencies on DOM structure

• Dojo is a javascript library with testable widgets, and a testing infrastructure for scripts
• Provides an experimental testing infrastructure, DOH, with a recorder

All three provide recorders. though Selenium is probably the most mature. They all suffer from issues in recording where the auto-generated scripts can often fail, require knowledge of XPATH to sort things out, and are not easy to incorporate into development processes so these scripts are maintained easy.

The key thing here is that it’s tough to incorporate these tools, which are themselves less than ideal, into workable processes that developers want to do and that support the flexibilty product designers need on the frontend. The upside is that if a working compromise can be found, the development will ultimately go faster & cheaper, because issues can be found sooner.

The long and the short of it is that if you want to do an agile process, it takes time and planning but the results can pay off. In the end, simplicity on the frontend pays off not only with users (typically) but also in development costs and time.

For one of the startups I’ve been involved with we used Trac as a quick way to support “development”. But Trac is very limited and problematic. We choose because it’s free, and a number of successful opensource projects use it. It works pretty well for supporting developers and an open environment, but as we’re running a startup with a rag-tag fleet of programmers, testers, project and product management it’s really not sufficient for all of them. Out of the box, Trac is best for developers and developer run projects. But if you’re a product company, then you’ve got alot of other things to manage other than what the bugs are and which developer is working on them. Because that’s really all you get. When you have people in other roles than programmer and technical manager, you’re beyond the capability of Trac: there simply not enough fields and capabilities to support different kinds of development roles and their interactions.

The key to performing a technical due diligence is an understanding of the business goals that are driving the audit. To perform the audit well requires discipline to listen and the knowledge to ask the right questions that will uncover what is going on. A simple methodology is helpful as a guide to obtain information without bias, to remember to ask all the questions that should be asked to get a fuller picture — you don’t want to dwell on one sore spot or mine only one’s particular interest. Knowledgeable listening and experienced objectivity is key to not getting lost in the expectations and projections of the sponsor rather than the reality of situation. In the end, the client is happiest when the truth is told, even if it’s not what they want to hear. Sometimes it’s worse than the sponsor expects and sometime not.

Kinds of Technical Due Diligence

Due Diligence for an investment interest is different than that for a CEO or division head who is worried about the performance of an organization. All organizations have problems, but some matter more than others with respect to the goals of audit.

Sometimes the outcome of a due diligence has been that I am put in charge of fixing it, but that is not my intent in taking on a due diligence. I am after providing the best insight into what is going on, what is working and what isn’t, and what isn’t so clear, and what of all that really matters to the business and what’s to be done about it.

I’ve been on both the buyer’s side and the sellers side of M&A conversations and in many other angles that drive a due diligence assessment. I know what the various players are looking for and what they are wanting and avoiding. This (advice from someone being assessed I find problematic in that it’s more about defending oneself than learning. A decent due diligence should be constructive for all concerned. It certainly depends on the situation, sometimes the agenda is loaded and sometimes it is merely exploratory… and in the end transparent truth is best, for it serves the greater good.

Acquisition Perspective

In an acqusition driven audit, the team being evaluated often has much at stake, and defensiveness can come into play. However, it is best to work closely with the auditor, because that constructive play can be the best card to play. A team that represents itself as knowledgeable and effective in an audit is not only being the better corporate citizen but also serving it’s own best interests.

Investor Perspective

Buyers are after ensuring their investment is all they expect. They are looking to avoid buying problems or at least to understand them enough to price them. This requires a bit of digging since the Buyer isn’t incented to display their issues. The role of the assessor is to dig in this case, but in this case there is a win-win in the assessment. The buyer already sees value in the company, so the assessor has the opportunity to help identify with the team being assessed the challenges ahead and how to address them. The question is can they or what will it take, and with what impact. If they are worth buying they will be able to face these challenges. The assessor should be able to help the seller and enable and identify a successful relationship between investor and invested company

Seller’s Perspective

Are looking to maximize value. They need to be showing their real issues rather than hiding bad surprises for later, and they need to show how they are constructively addressing these issues.

Team in Trouble

This is always tough because these teams can be embattled. Change is imminent. Due Diligence means that something isn’t working and things have to change. But the upside is that aim is for mutual success. People in this situation typically are doing their best to fix things or to survive a tough situation. It’s not a happy occasion, but there’s lots of pride at work, and that can be a powerful factor. Engineers like to ship real working code, and when it’s not happening, there can be many factors at play. The assessor must stay focused on the business goals, and is looking also to enable the values of the business. At times this might seem cold to a team, as in the icy ‘it’s just business‘, and at other times it will seem senstive to the considerable work and care teams typically put into their work. Nonetheless, a profitable and successful enterprise has to focus on the bottom line (be it a single, double, or triple bottom line), so the truth must be spoken. Like in hiring, if something doesn’t fit, it’s best to change it rather than hope to change the person… it’s better for all concerned, eventually. But often, changes to process is what is needed, validation of what the team already knows and helping them get empowered to act is often the cure. I will often not only make recommendation to the team, but to the senior management as well who are often part of the problem.

We’re working with a rag-tag fleet of development folks: programmers, admins, designers and so forth. Amongst the challenges is that as we bootstrap our development it is a huge challenge for the folks we’re working with to be as committed as we are. For many of them this work is part-time and they squeeze into their busy lives. On top of that, they are distributed all over the place so that keeping everyone in sync and focused is a challenge. And the remoteness itself, without having worked together, makes it very hard for us to develop a solid team feeling.

We stress respectful communications. And we really can’t over communicate in this situation, have too many meetings. Not yet anyway.

Some of the relationships work really well at a very high level of professional commitment, and others are not so engaged and we move on. We’re looking for folks who passionately involve themselves.

I’m reflecting on the scale and speed of changes in my work of late. During this period, I’ve been managing a team of 200 people and a very large IT budget, with all the issues of a large organization, enterprise scale technology, and the pressure of a market leading position. There’s the politics, the multiple agendas, and battle against crustification inherent in such an organization. And there’s the pleasure of a large, well-funded team accomplishing many many things at once. And that’s been fantastically fun.

More recently, I’ve been working with really small teams, including my own startup, where I’m am doing what has to be done just as before. But now it’s the writing of code, working with loosely connected remotely distributed developers with the blistering urgency and minimal resources of a startup. And I’m enjoying that immensely too. And each of these experiences are informing the other. Each are different sorts of challenges related to scale, none better than the other.

The Problem: Web apps often require immediate communications between users mediated by the website, rather than directly connecting users to each other. That website communication via HTTP requires the browser to poll the servers, servers cannot push data to browsers. An example is a chat application: one user wants to send a message to another, so their browser app sends it to the server, but the recipient doesn’t know that there’s a message there on the server to be picked up. The server can’t push it to the recipient’s browser. So the problem is how can the recipient recieve messages quickly and reliably?

The solution is a set of hacks that are collectively known as ‘Comet”. Comet actually a combination of techniques intended to work portably across browses as efficiently as possible, and to work efficiently on the server-side as well. It’s basically, a bunch of ways to do efficient polling. And the reality of it is that at this time there are a bunch of technical challenges to implementing them.

On the Server Side:
The assumption of most http (web) servers is that a browser connection is quick. Most web implementations use interpreted languages which take up alot of memory (ram). So with quick connections of bloated memory, you don’t use up much memory. But if a connection is to be long-lived, then web server will have many more bloated interpreted apps to run at once and will run out of memory sooner. So you get a scaling problem.

You either need smaller apps that take less memory (so you’d have to forget about the entire “P” in the LAMP stack, or you need a server that can run a more efficient server to handle push efficiently.

There is a class of servers out there know as CometD for such long-lived connections. The state of Cometd servers is experimental at best. They are basically bodies of server code which you have to hack. There are no standards, most don’t have plug-ins or any clean way to isolate your code except through coding discipline… There’s no mod_cometd for instance for apache etc. So you’ve got to hack your own. It’s not rocket science, but it’s certainly neither enterprise quality or highest-level maturity, open source quality. It’s the first round of sketches… You have fill in here and there, but you can get the general ideal working.

CometD really only specs a transmission protocol known as Bayeux. There’s no server or client standard, only the protocol. The Bayeux protocol describes the transmission of data in a publish/subscribe model. There is no further control or information for the server to something other than broadcast messages from a publisher to a multiplicity of subscribers. Great!, but say you don’t want just anyone to subscribe to your channel… well the protocol isn’t going to help you directly. So a basic CometD server that supports Bayeux does nothing other than route messages, anything else you’re going to have to provide through modifications to the server. And as I noted there’s no standard here, not nice plugin to modify the server… You have to start hacking. So be it.

So:
- no security: there’s currently no means to prevent anyone from subscribing or publishing… You have to hack that in yourself.
- no connection to your App or DB: you have roll your own. There’s no pre-built mechanism like a callback that you can hook into on certain server events. You have to read the server code and roll your own system These shortcoming are being addressed more or less in open-source and in commercial server implementations. Less for now.

Check these out as they’re the best way to get push done:

• http://meteorserver.org/
• http://cometdproject.dojotoolkit.org/documentation/cometd
• http://www.lighttpd.net/
• Erlang *

* From Facebook Engineering blog: For Facebook Chat, we rolled our own … epoll-driven web server (in Erlang) that holds online users’ conversations in-memory and serves the long-polled HTTP requests. … Why Erlang? In short, because the problem domain fits Erlang like a glove. Erlang is a functional concurrency-oriented language with extremely low-weight user-space “processes”, share-nothing message-passing semantics, built-in distribution, and a “crash and recover” philosophy proven by two decades of deployment on large soft-realtime production systems.

On the Client side, there are Comet libraries that implement the Bayeux protocol to CometD servers, but given the state of the protocol, you’re only getting a starting point for significant elaboration. Dojo supplies one of them, authored by Alex Russell, who coined the term “Comet” and who is one very smart and able technologist.

I recently attended the Linux and Next Generation Data Center show in San Francisco. I saw some cool stuff and I’d like to hear from you about how it’s going with some of these products.

I’ve started to work with the Taproot Foundation as a way to help non-profits build some technology they need. Taproot tries to the smooth the process by automating project management and bringing in best practices for working with clients. So far i’m impressed. As I go along I’ll report on this.

My role for them is account director, where I own the relationship with the client, determine their broad strategic technology needs and create and staff projects for them.

What is the difference between Connectivity and Intimacy?

I heard Evan Williams talk about his company Twitter a few weeks back on Jerry Michalski’s YiTan conference call. Evan is struggling over what to do with Twitter. To say the least, Twitter has buzz. What Twitter does is to automate the ‘shout-out’ (as in, “I just want to do a shout-out to my buddies”). As they put it, they want to answer the question of “what are you doing right now?” With all the buzz, Twitter is the working definition of a hot startup of our era… But to Evan, and I think to his credit, he sees it as a mere “feature” rather than an application or a business. A feature that many websites might have in the next couple of years… He’s not sure how or whether to make money from it, in and of itself.

What I find so remarkable about this is that most business folk would be looking for the exit, shopping their ‘captured’ audience around for sale to the usual suspects. But Evan is saying this is really just a part of something else and not standalone, even if it has million+ people using it. It’s one thing to use something, and it’s another to find deep value that enriches your life.

My suspicion is that shoutcasting is nothing but a social fad sweeping the tech savvy crowd. And I don’t think it’s going much beyond that. It has usefulness, but it’s not something so rich that one devotes oneself to it. So I think Evan is right to think of shoutcasting as a useful part of something else much more helpful in ones life.

Desire to Connect

Shoutcasting and Twitter are speaking to something substantial that so many of us want. It’s pushing against the massive alienation we feel and the urgent desire to connect to each other.

What I suspect though is that a lot of that can be done on a personal and local scale, in direct and intimate forms, with existing technology. One technology I like a lot is the dinner party. Works great, it’s charming, and a rich experience for every sense. Compare that to a 140 character messages delivered over a little glass screen interrupting a conversation with a friend over lunch.

There’s no reason why you shouldn’t own your own health care record. When you go to the hospital, they act like it’s there’s and all you can get are fragments of it in paper form. But now there’s movement afoot that will allow you to create your own personal health care records that can be used by health care service providers. One such effort is WorldVistA, and there are others working on similar efforts.

The current world of health care is built on data exchange standards of a variety of types. It’s an alphabet soup of esoteric ways to exchange complex information. For medical devices it’s done through a standard known as DICOM, for patient tracking and payment standards known HL7, and documents know as CDA. And there are others standards and lots of custom protocols. Some uniformity within the healthcare has been forced through federal HIPAA standards.

Patients data ought to belong to the patient. It’s our personal medical history. We ought to have a reliable store of our medical information to which doctors and hospitals and other health service providers write and read from. Instead, your record is owned by the hospital system effectively. Except for a few toll gates, you cannot get at your record because it’s buried within the each hospital’s medical information system. If you move from one hospital system to another they have to recreate your medical history rather than copy it. It’s prone to errors and it’s not usually visible to you, which is an absurdity. You can get select versions of your history but not the entire thing.

The challenges of implementing patient data that you own lies more in public policy and politics, than in technology.

Tied into the control and access of you data is a configuration of players. The negative side of this configuration is described by Tom Munnecke, an innovator of large scale medical systems in use around the world, as the the ‘disease industrial complex’. The hospital, insurance, medical device, pharma sectors are all involved in this. Getting them all to coordinate requires policy intervention rather than waiting for free market to recognize there’s an issue.

Meanwhile, the opensource community is directly responding with efforts like WorldVistA.

So you want to have you own data and control it and let the health service providers connect to you. Or you want someone you trust to house this stuff that’s unaffiliated with a particular hospital.

This one is worth tracking.

I heard Tim Brown of IDEO on an MIT Entrepreneur podcast who caught my ear about how innovations in products are often accomplished. He pointed out that products tend to be developed in reaction to things annoy us. But he also noted that the world can be inspiring, not just a source of annoyance, and that by focusing on things that inspire us we might develop products with a corresponding edge. I deeply like the idea of focusing on inspiration rather than annoyance as a powerful source of creation; to resonate with optimism, pleasure, and insight rather than from frustration, difficulty, and challenge.

I’ve been working on a variety of startups and they all need prototypes on the way to built systems. Being rather strategic these days (though I hasten to add that I’ve written operating systems and the like), I took the opportunity to get my hands a bit dirty with a detailed look state-of-the-art development choices for the web.

Besides the backend LAMP backend choices (in which I include RoR) and the final mass awakening to the MVC pattern, the biggest item is how to write effective, dynamic interfaces. (I’ll leave the backend technology choices for another day. ) That solution is currently Javascript and DOM/HTML which has been evolving and improving. Nonetheless, browser implementations remain inconsistent and a developer has to tackle that in creating web pages that will work for all users. You can either roll your own or find a decent library to write browser-portable code. The best practice these days is to use a library.

For me this breaks down into the following requirements:

• Does it come with a baseline set of widgets, and are they:
Extensive
Attractive
Customizable
• Portability across leading browsers: FF, IE, Opera, Safari
Portable widgets
Covers javascript implementation issues
Covers DOM implementations issues
• Web 2.x ish support
Ajax-ish data exchange via JSON etc
• Other pluses
Open Source
Support Obsfucation
Download only parts you want
Easy to use
• Special Requirements
Portable graphics

One way to meet most of these requirements is to use the Prototype library which provides portable extensions to Javascript, and then add to that a Javascript widget library like Sciptaculous or JQuery. There’s other solutions too like Mochikit, Rico, Yahoo’s YUI and others. But you get the whole thing with Dojo, Open Source with an active community. There are definitely differences in the number and quality of widgets in Dojo versus say Scriptaculous and JQuery (which are generally prettier). And you can mix and match, but you’ll start to get into issues with the amount of javascript you are downloading to your customers.

These are best out there: Dojo (most comprehensive), Prototype/Scriptaculous/JQuery, YUI, or combine them. There’s still tons of issues with all of these for the state of the art is pretty primitive but for an initial development these are no brainers. Keep in mind that these are evolving platforms. A large refactoring of Dojo is going on as I write this.

The recommended browser debugging platforms are: Firefox with the Firebug debugger, and for IE use JSLint and the Microsoft debugger tools and you’ve got what you need. (Avoid Venkmann debugger, Firebug is far better).

Graphics
Graphics are the bleeding edge these days. The only portable library with any substantial support for graphics is Dojo. Dojo’s implementation is an abstraction that is built on top of SVG and VML (The canvas tag is not used which is particular to Safari). Apps requiring primarily basic line and geometric rendering can avail themselves of some other libraries, though, broadly speaking, these currently tend not to support text, gradients, and other fancier curves. A good library of this ilk people are using is by Walter Zorn.

Highly graphical web pages are coming but are subject to development issues due to the terrible performance of VML under IE. So while graphics are inevitable, the devil is in the details: you’re going to face a lot issues. One hopes that Microsoft will prioritize fixing this aspect of IE rather high. This is an area where Microsoft could innovate and challenge FF (which will be happy to do battle here I’m sure).

There’s a slew of next generation web-based mind mapping tools coming. “TheBrain is working on a java/ajax/html combination solution that performs as well as their admirable native version while adding new features. Harlan Hugh and his crew deserve kudos for their Windows interface solution that intrigued so many folks. This interface has some really nice features but also some significant limitations in terms of what you can map out, and it requires Java, which is a negative by requiring users to deal with Java downloads. The original interface is a variant of the Hyperbolic Browser in which both are mimicing the psychology of attention. The main idea there is that our minds focus on a central area while things in the periphery provide influence. Similarly these products present information with a central focus and a limited view of the connected items. It’s all very exciting to see this happen. But the interfaces remain abstract, complex, and, difficult.

But there are other contenders that are working to extend the idea of thought diagramming in ways that consumers may be able to appreciate. One is Kayuda, that allows for public and sharable mind maps. While this product marries mind maps with social networks, it is rather primitive, and I doubt there’s any IP to protect. They are also rolling out features in an incremental fashion. This is an example of when not do incremental rollout: when you lead the market. Instead they ought to do some usability testing and spare us their learning curve.

Notably, so far though, there’s no public map on Kayuda worth looking at. I suspect that like TheBrain, the interface issues make it too tough.

A number of others use Flash technology to get around the bleeding edge of web/ajax solutions. Besides fluid and high performing graphics Flash also makes it possible to do some things like voice chat. If you look at Thinkature you can see a web/ajax/flash hybrid. And there are a bunch of pure Flash solutions out there as well.

This looks to me like a solution looking for an audience. Mind maps are more of a feature than a product is my feeling.

The innovation in this area will be in developing interfaces that don’t reduce the complexity and understandability of the thing you want to represent. UML is not the way to go, plainly. And more importantly it’s where you choose to use mind maps that will count most So there’s no obvious winner yet and there’s unlikely to be one in this space. I await to see this as a feature in other apps instead.

The interesting thing from these is to see what happens as folks learn how to do more full-blown graphics on the web without using Flash or Java. It’s important from the perspective that solutions that mix technologies are prone to problems, all of which add nothing the primary problem of building a great app. Flash is great, but proprietary, and Java is not the best way to deliver interfaces to users. An opensource browser/ajax/html solution would be better, but it doesn’t cover everything.

I’ve been on the Internet since 1979. In 1979 you could draw a map of nearly all the machines on the internet on a piece of paper. So I’ve had the somewhat rare perspective of seeing something that was just a pretty idea evolve into something essential to our everyday lives.

“That’s all very interesting, but show me how we can make money on this” is what a venture capitalist told me in 1990 at a computer show in NYC. I had just said to him that the most interesting thing about the show was not any of the systems or software on display but the fact that most of the systems were connected to each other through the Internet and how engineers were exchanging email. This was a time when few had a computer, let alone a modem, and the software made it inconceivable that most folks could get on the Internet. And if they did there was no Web to see. It was not the time to make money on the Internet

Now in any hotel lobby I’m overwhelmed by the number of folks who are staring into little glowing rectangles and communicating wirelessly to people around the world, built on the same technology.

So the last 12 years are completely watershed as we’ve gone from near zero internet usage on the way to global acceptance. And we’re just figuring out this stuff still. And as we do it, we have our cultural moments and fads, like an art movement. Globally speaking, we don’t know what we want or need but we are figuring it out. We create businesses and non-profit ventures to meet these needs as we recognize them. Some of these ventures are seen as vital and others less so. And I’m reminded of the conversation with VC when it was too early to really to find value. Some things are funded and others not, and it reflects both the zeitgeist and our ability to deliver something practical.

Invention is sometimes characterized as creating something from nothing, but another view is that we find the doors to knowledge that we can break into. The doors have been there forever and are now weak enough that they cannot bar our entrance. That’s how I see these last years, where the implications of an underlying technology are continuously being discovered and we kick our way through rotting doors into new places, feeling our way. But they are the doors that are the easiest to kick in.