Accelerator data... the debate heats up

Not long ago we wrote about the www.seed-db.com database that Jed Christiansen created. In the few weeks since its launch in July 25, it's become pretty clear that the seed accelerator world is starving for data based on (a) how fast bloggers picked up on it, and (b) the lively debate that it's initiated (more about that a bit further down). That can only be a good thing because there's one thing that everyone agrees on: the degree of success of the seed-accelerator concept at this point in time is just slightly clearer than mud. The intent of this post is to summarize and comment on the debate thus far.

Christiansen has done the industry a big service by putting in a significant amount of sweat into gleaning useful data from what is undoubtedly fairly slim pickings. Even better, he's been extremely forthright about where the data are weak and where he's had to apply guesswork. Net net, we have a starting point of sorts to think through what state the industry is in. And it didn't take long for people to take a deep dive into the database and to draw conclusions. Surprise -- conclusions are divided so far. That's a good thing... this debate is overdue.

The debate started short hours after the word got out on The Accelerator Gazette and elsewhere. Here, for example, is a bit of the early discussion on Hacker News (see link at the end of this post):

-'ChuckMcM' (sorry.. I don't have real names) got the ball rolling by asking what probably everyone was wondering: was there some way to compute a more complete economic impact of an accelerator including rents, services, taxes and so on? That's a question that kept coming up in one form or another in the following weeks.
-Later, 'mani' at akmanalp.com pointed out that only Y-Combinator and TechStars (Boulder) had encouraging results. That also became a recurring theme.
-Then 'imranghory' who runs another site that gathers accelerator stats weighed in, saying; "I run http://seedtable.com which similarly provides analytics built on-top of Crunchbase and what I found from looking at the financial numbers is that Crunchbase just isn't reliable enough to do analytics on. Rounds are frequently incorrect (typos in numbers, orders of magnitude wrong, foreign currencies treated as if they were USD, etc.) or missing." (emphasis added by me)

So, within a few hours, the key recurring themes emerged: (1) does the data available really indicate the full economic impact? (2) a few sites (and particularly Y-Combinator) stand in stark contrast to most other accelerators, and (3) the source data has a lot of indeterminate flaws.

Later the same day, David Cohen on the TechStars blog weighed in with some supportive comments and picked up on the latter themes. Here's part of what he said: "Of course, there are issues with this kind of data. Just be aware of what you’re looking at and focus on what matters. The average funding (or better yet, median and/or mean) are better indicators than “total funding,” which is more of a quantity metric than a quality metric. It’s also important to remember that exit and funding dynamics are indexed by time. ... . Some of the companies in these metrics are barely out of the program, and fundraising takes time. Obviously exits take even more time." So David moved the conversation forward by pointing out the funding data should be 'normalized' using averages instead of totals and by pointing out how time had to be considered in any analysis. Both good points.

Things really began to get interesting when people tried to draw conclusions from the seed-db data. Grasshopper.com and
Jared Konczal of The Kaufmann Foundation came up with pretty different conclusions.

On the www.grasshopper.com site, a blogger wrote (can someone let us know who the author was?):"We found the impact of the seed accelerators most intriguing given the current economic state of the country, especially in regards to job creation." The writer said the data showed:

- 1436 companies accelerated
- 69 exits for $ 979,458,100
- $1,176,787,411 funding
- 3389 jobs created

and then he/she got to the punch line: "One thing that really caught our attention is that, unlike current job creation plans, these jobs haven’t really cost much to create.
- The accelerated companies received $1,176,787,411 in funding.
- $979,458,100 was earned back as a result of acquisitions.
"Thus, the successes of these companies have resulted in the creation of even more jobs at little to no cost."

Oh really --little or no cost? That was an important conclusion but one that wouldn't go unchallenged for long. Jared Konczal wrote an opinion piece on www.forbes.com (also picked up by the Huffington Post) entitled "Evaluating the Effects of Accelerators? Not So Fast".

In his thoughtful analysis, Konczal picks up on a key point-- a point that came up very early in the discussion: the Y-Combinator data skews the total data set a huge amount. With Y-Combinator data in, one could conclude that grasshopper.com might be right. But, with Y-Combinator data removed, a very different conclusion emerges. In his words: "to put it in terms of jobs, the accelerators have spent more than $130,000 for each job created, which does not equate to little-to-no cost. " He's referring to the whole set of accelerators with Y-Combinator removed.

Konczal is more careful in his analysis and in his conclusions than the grasshopper.com blogger -- he digs deeper into the data that is available and he goes to great lengths to warn against making any sweeping conclusions based on the data in its current state. He adds a some zingers into the debate, too. For example: "Maybe these startups would have turned out the same without the accelerator's intervention. In addition to encouraging accelerators to submit data on the companies they do select, why not also keep a list of companies they don't select?" There's too much good thought in his article to cover here... you owe it to yourself to read it yourself.

So there you have it. A lively debate is underway. Weak analyses are being challenged. Important new lines of debate are emerging. Thank you, Jed (and seedtable.com and others) for providing the raw material. This won't be resolved soon.

links to sites and articles mentioned in this post:

http://www.seed-db.com/
http://news.ycombinator.com/item?id=4291075
http://seedtable.com/
http://www.techstars.com/techstars-thoughts-on-seed-db-com/
http://grasshopper.com/blog/2012/08/seed-accelerators-have-created-nearl...
http://www.huffingtonpost.com/jared-konczal/evaluating-the-effects-of_1_...
http://blog.jedchristiansen.com/