Data Science Project Success

In a previous post, "Data Science vs Software Engineering Projects," I point out some of the key differences between traditional projects and data science projects. One of these differences is in the deliverables. With traditional projects, the deliverables are often tangible — a new product, a specified increase in revenue or reduction in costs, adoption of a new information system, and so on. With traditional projects, organizations can set objectives and milestones and measure progress. Determining whether a team succeeded or failed in achieving its mission is relatively easy; either they delivered the goods, or they didn't.

With data science project success, the deliverables are less tangible — less quantitative and more qualitative. Data science teams engage in discovery with the goal of growing an ever-expanding body of valuable organizational knowledge and insight. How do you measure that? Their purpose is to provide knowledge and insight that enables executives to formulate more effective strategies and tactics and enables everyone in the organization to make better, data-driven decisions. How do you set a timeline for that?

Although setting objectives and measuring output are not conducive to data science, organizations still need a way to measure data science success. After all, the executive team needs proof that the data science team is earning its keep — that the return on investment is worth it.

Chalking Up Wins

At one point in his illustrious career as inventor, Thomas Edison noticed that one of his assistants was becoming discouraged by their failed experiments. Edison, on the other hand, remained upbeat. He assured his assistant that they hadn't failed. With each failed experiment, they were learning something new. To Edison, even a failed experiment was a success.

Retrospectively, we measure Edison's success by his many useful inventions, most notably the incandescent light bulb, the phonograph, and the motion picture camera. What we don't remember are his many failures, including the electric pen, the magnetic iron ore separator, and the concrete piano. When confronted about his many failures, Edison replied, "I have not failed. I've just found 10,000 ways that won't work."

Success in data science should be measured the same way Edison measured his success — by the learning his team captured and by the innovations it delivered. Data science teams often encounter dead ends. Experiments may not support the team's hypothesis. Insights gained may be of little to no value to the organization. However, as long as the team learns from its failures, those are a measure of success, as well.

Many organizations struggle with this approach. More goal oriented teams may look over at the data science team and wonder, “What is that team doing?” Or even worse, “What does that team do again?” And if the data science team was formed as an experiment to see what it could come up with, and it fails to deliver anything of value in whatever window of time it was granted, the team may be disbanded before it even has a chance to produce.

Improving the Odds of Acceptance and Success

Here are a few suggestions for improving your data science team's chances of acceptance and success:

• Embrace community and transparency. If you isolate the data science team, it won't receive the input it needs or have the opportunity to communicate the value it delivers. Others in the organization may start to question the team's value to the organization.
• Be bold. Focus on big problems and big opportunities by asking compelling questions. If the team is not ambitious in its exploration, it will be less likely to deliver valuable intelligence to the organization.
• Schedule regular storytelling meetings to share what the team has learned with the rest of the organization. Cover the questions that the team is working on, share a few recent insights, and encourage others in the organization to ask questions and discuss their challenges.
• Celebrate wins. When the team delivers valuable intelligence, publicize the value. When others see the value of data science, demand for the team's services will build along with acceptance and appreciation of the team.

An Example of What Not to Do

I once worked for a university that hired a group of “unstructured data specialists.” The Provost wanted a data science team that looked for new insights. The team operated out of an office near the administrators who hired them. Few others in the university knew what the team was doing or were even aware of its existence. Due to the lack of clarity and understanding of the team's purpose, nobody in the university would make the time to meet with the research lead. Without input from the people who were most responsible for executing the university's mission, the team struggled to ask compelling questions and, hence, was unable to deliver any valuable insights.

The team's chances of success would have been greatly improved had it been placed in a location closer to the faculty instead of the administrators and steps taken to inform the faculty of the team's purpose and its potential for bringing value to the university. Making the team more transparent and accessible would have encouraged others to bring their questions and challenges to the team. Regular storytelling meetings would have enabled the team to share its knowledge and insights and celebrate its wins, thus publicizing the team's value.

If you’re the project manager on a data science team, work hard to make sure that the team is sitting with everyone else. Some of your best inspirations might come from people dropping in and asking questions. The better connected the team is with the rest the organization, the easier it will be to come up with compelling questions that lead to flashes of innovation.