What to Consider when Transitioning from Data Scientist to Data Team Leader
High-performing data scientists may at some point consider moving into a leadership role. Sometimes the company will offer a managerial promotion or at times, data scientists gain more and more responsibility with success and eventually grow into management positions. There are, of course, pros and cons to making the switch.
Some data scientists lament the move away from hands-on analytical and technical tasks, while others enjoy leading a team they used to work on for years. As data science demands grown, more companies are seeking out managers on data science teams; in fact, 57% of enterprises now have a Chief Data Officer, and 24% are considering creating a Chief Data Officer position.
A leadership position in data science means getting out of the trenches and running all aspects of the team. Since this position is relatively new, responsibilities can vary. By 2021, however, Gartner predicts the office of the CDO “will be a mission-critical function comparable to IT, business operations, HR, and finance in 75% of large enterprises.”
Managers of data science teams must establish trust, all while creating the right environment for work and synthesizing information. It’s a fundamental shift in work function, as individual work value no longer relies on technical abilities. For those considering a shift to a management position in data science, here are some aspects to keep in mind.
Mindset shift
According to Gartner, the top challenges to the success of a CDO are cultural challenges to accept change (40%) and poor organization-wide data literacy (35%). Businesses typically try to balance time, cost, and quality, which is referred to as the “TCQ triangle.” Most data scientists focus on quality: creating quality models that work. They don’t necessarily have to worry about the time it takes or the amount it costs. Those that move from the team to a team leader must now consider all of those aspects, which can be a stark change. Managers here can struggle wanting to prioritize quality when time and cost are crucial factors now as well. This requires a mindset shift to incorporate new goals and timelines and a customer-centric approach to finding and managing work for the data science teams.
Understand the business side, not just the technical elements
Data scientists obviously have a lot of technical knowledge, but leaders must also understand the business side of projects as well. The average CDO’s time is spent on value creation and/or revenue generation (45%), cost savings and efficiency (28%), and risk mitigation (27%). A leader’s job is to help other people do their jobs well in order to achieve goals. Most data science leaders must work with the business side to understand project goals, objectives, outputs, and come up with a realistic timeline and budget. It means breaking down data science concepts and pushing back on unrealistic wants. This can be a challenge for data scientists used to working only in a technical capacity. Most importantly, being a strong facilitator and communicating in the language of the business will make or break a data science manager.
Adding on new skills
As with any position change, new skills are necessary to achieve success. Becoming a data science team leader means adding on more soft skills. Some data scientists find this challenging, as they’re used to spending most of the day writing code or performing analysis. Soft skills needed include communication and listening to ensure team cohesiveness and happy employees.
Leaders also need to learn how to delegate and not try to do all tasks themselves, even if they are the most skilled at a particular element of the project. Data science team leaders also typically have a hand in team structuring and hiring. Figuring out who does what best and putting them in appropriate roles is part of the job, as is scaling up a data science team. While HR usually helps to find candidates, managers have to figure out needs and assess a candidate’s place on the team.
Part of the full data science lifecycle
Individual data scientists do not always see the full data science lifecycle from start to finish, but managers do. Therefore, they must understand each part of the lifecycle to be able to better budget a project and set a timeline. Leaders will need to know why data cleaning, for example, may take weeks or months, why moving to production won’t happen overnight, and why projects might fail. Before taking on a leadership role, spending time navigating, and understanding the entire data science lifecycle is crucial.
A new way of working
Leaders in data science try to tackle the challenge of planning a project’s timeline and final outcome. This task is near-impossible given the obstacles that occur during a big data project. Managers often have to self-correct and continually reassess goals, budgets, and timelines, a process that is not for everyone. Through 2019, Gartner says that 90% of organizations will hire a CDO, but only 50% of these hires will be considered a success. This failure rate can, of course, be attributed to many things. However, it does imply that those moving from a data science team role to a leadership position should carefully consider all changes that come with the shift. For those that can adopt business and soft skills necessary to be a leader, this move is certainly a step in the right direction.
For advice on how to structure a data science team for success, read our tips here.