Getting actionable intelligence out of data flows and stores is one recipe for agility. Here is team-building advice from pioneers in the new field of data science.
Say you are just embarking on the journey of putting together a data science team -- to get more value out of warehoused data, to explore big-data analytics, to mine transactional or social data, or in general to advance your chops as a data-driven company. You might profit from the experience of the man who co-invented the designation "data scientist." DJ Patil is now Data Scientist in Residence at Greylock Partners. As he was putting together the data science team at LinkedIn, in 2008, Patil met with Jeff Hammerbacher, then the data chief at Facebook. Together they came up with the title "data scientist" to describe "those who use both data and science to create something new."
Where do you find data scientists? As is often the case in a brand-new field, universities are not turning out people ready-made for the role. The ideal data scientist, according to Patil, possesses these characteristics: deep technical expertise in some (usually hard) scientific discipline; curiosity; cleverness; and the ability to tell a story (with data). A computer science background is not among Patil's criteria.
The demand for data scientists far outstrips the supply currently, and this condition will probably worsen for the next few years at least. You're going to have to go looking for suitable candidates: "people with diverse backgrounds who have histories of playing with data to create something novel," according to Patil. A creative way to find people who can solve your toughest problems is to run a competition. Kaggle and TopCoder are two places to set these up; a more established data science group might institute its own challenge, as Netflix did.
In the Organization
While Patil was trying to figure out how to organize his team and how it would relate to the rest of LinkedIn, he interviewed his counterparts at Yahoo, Sun, Google, eBay, and other companies. He was discouraged to hear many iterations of a similar tale: a data scientist arrives at a key conclusion or insight, only to be told by a line product manager, "That's nice, but it's not on our roadmap." So Patil set up LinkedIn's data science team as a full product group, responsible for design, implementation, and maintenance of data products. Eventually, data products began showing up all over the company. "That's how you know when you've won," Patil concludes.
A perspective on a smaller scale comes from Kurt Schrader, VP of Engineering at Intent Media, who writes about building a data science team at a startup. The knowledge he passes on sounds hard-won. He reinforces DJ Patil's dictum that you should give a newly hired data scientist "90 days to knock the company's socks off." He leads off with the assertion that you have no idea what your data science team is going to do, in part because you are probably thinking about the problems in the wrong way. He proposes that "you need to work on keeping everyone without a math background away from the data science team," because constantly explaining what they do and how they do it will not be not the best use of their time.
A data science team that can generate insights about the business, and data products that can move the business in new directions, is coming to be a necessary part of any company that strives for agility in this era of big data.