Managing the Abstraction Mix

I was recently talking to a friend who, through a dramatic turn of events, has found herself as the senior-most data leader at her company. Where she previously managed a small team, she is now abruptly responsible for supporting and growing the careers of every individual contributor in the organization, roadmapping and coordinating across three separate data teams, and generally trying to create a sense of stability for the everyone else during a chaotic time. It’s the type of situation that people hear about and wince, remarking half-heartedly that it’s a huge opportunity while knowing full well that it’s going to be rough.

I’ve experienced versions of this myself, and in general, it’s not terribly uncommon for data leaders to end up in this position, whether they mean to or not. The data field in its current form is a new and immature field, and one of the many ways it shows is that there aren’t a lot of great examples of what mature data organizations look like. The best we have often come from FAANG or other big tech companies, and their teams are sized and shaped very differently from the smaller ones that most data leaders find themselves responsible for.

Big organizations allow for specialization. This is more apparent in individual contributor roles, with large organizations having dedicated career ladders for data analysts, scientists and ML engineers instead of one generic “data science ladder”, but it’s true for leadership roles as well. At most companies, data teams are relatively flat, and while there are many things to love about flat organizations, one of the most difficult parts about running one is that they blend different types of leadership. That blend creates a particular form of difficulty for data leaders by requiring them to constantly context-switch between different levels of abstraction.

The Leadership Abstraction Levels

If you look at the typical tech company org chart, you might wonder why there are so many managers, especially if there is more than one layer of management. Some companies end up this way because someone wanted to build an empire, but as with most things, management layers are usually created for a good reason. Managers are responsible for making sure their team is coordinated and moving in the right direction, and when organizations grow, this becomes a more complex and time-consuming job. This coordination work takes different forms, and over time these different forms of coordination become big enough jobs that you need people to specialize in them full time.

The first form of coordination is front line management, which is generally demarcated with titles like “manager” or “senior manager.” These people directly manage individual contributors, have usually done the job they're overseeing, and probably have direct knowledge of most of their team's daily activities. They’re primarily responsible for helping their teams produce consistent, high quality work according to specifications. This is the most essential form of management and will happen in every organization. All people leaders start as front line managers.

The second form of coordination is second line management, which is usually titled with “director” or “senior director” titles. Second line managers are managers of managers, although they sometimes also manage a few high level individual contributors. Because they’re primarily interfacing with line managers, they spend more time thinking about productivity at the level of teams than individuals, and this pulls them up a level of abstraction. They’re responsible for enough people that they can’t have direct knowledge of what everyone is doing, so they necessarily need to shift their focus to driving execution.

The third and final form of coordination is departmental management or running a function, which is typically done by vice presidents or folks with “chief” in their title. For this type of coordination, a manager is pulled up a level of abstraction from the execution-oriented second line managers and even further up from the craft-oriented front line managers. They start thinking along longer time horizons and in broad contours, focusing on the strategy and vision of their function or department. They need to focus on identifying and diagnosing the problems their group is best positioned to solve, and then they work with the second line executors to build a feasible roadmap to address those problems.

Much in the same way that switching from individual contributor to manager is a different job, front line to second line management or second line to departmental management can be a big switch too. Leaders at each of those levels are accountable for different things, work with different people, and have to rely on different techniques to operate.

Data teams are full of abstraction mixing

This brings us back to flatter-than-FAANG data organizations. As mentioned above, the data teams of most companies are a lot smaller than engineering organizations at big tech companies, which means that they rarely have as many management layers. And even when they have some, they’re not as cleanly separated.

You might expect that smaller team sizes would mean that managing data teams is inherently less complex, but another key difference between data teams and, say, engineering teams is that data teams have far more heterogeneous structures. Data managers are more likely to be responsible for jobs they've never done and have team members involved in multiple (possibly unrelated) domains of the business.

This is especially true if the data manager’s team has an embedded structure, where ICs' daily work is more determined by a cross-functional partner than it is by their direct manager. Being removed from day-to-day details, it quickly becomes difficult for data team managers to give meaningful or detailed feedback about quality of work being done in an area of the business they're not deeply knowledgeable about. And this gets even worse if they end up managing multiple functions (analysts and analytics engineers, for example).

Data line managers typically have broader scopes than their equivalently-leveled peers in other functions, crossing multiple domains and career ladders. There are too many details for it to be reasonable for data managers to keep pace, so they are often better served by focusing on execution, pulling them up a level of abstraction earlier than an engineering line manager would be. Then combine that with the fact that many data team leaders are also the most senior person in their function, meaning they’re also on the hook for the strategy and vision of the data organization at their company on top of front line and second line managing the data team.

Even if most data teams are smaller than big tech data teams, this is still a lot of complexity because it’s three very different ways of thinking. When you consider that many data leaders are also new to people leadership in general, it means this blend of responsibilities is regularly forced on to people who are unprepared and unsupported to tackle this kind of challenge.

If you aren’t a strong swimmer and don’t have any life rafts, it’s easy to drown in the management abstraction mix.

Techniques for managing the mix

If you lead a data team, chances are you are experiencing some sort of management abstraction mix right now. I hope you’re hanging in there! I don’t know if I can say that it always gets easier, because clean separation of abstraction levels requires an org size that most data teams will realistically not reach any time soon (if at all). That doesn’t mean it’s impossible to master the mix, however. I’ve had plenty of practice at it myself over the years, and although I wouldn’t say I’ve mastered it, I have started to identify some techniques that have helped me get better at managing it.

The first and most foundational is building awareness of which abstraction level you’re operating in. It can be hard to do it in the moment, so start by reflecting after the fact on conversations you have with your direct reports, your stakeholders, your manager, etc. Who you’re talking to can be a useful cue (you’re more to talk execution with your manager than talk shop on methodology), but it’s not perfectly one-to-one either. A conversation with a direct report could easily bounce between craft to execution, or start with a high level vision before zooming in to coaching them on improving a visualization in an analysis that works toward that vision.

The goal in doing this is to recognize situations and styles of conversation that put you in certain mindsets, and it can be as formal or informal as is helpful for you. This practice will help you identify when your brain is shifting gears, and it can make it easier to recognize when you need to play the role of a collaborator in solving a meaty problem, a focused project manager who’s looking to remove blockers, or a big picture thinker who’s trying to imagine what’s possible.

Once you’ve learned to recognize when abstraction level shifts are happening, another technique you might want to try is structuring your schedule in ways that reduce the need to context switch. If you can, try to group meetings and focused working time blocks that happen in the same abstraction level together. That might mean reserving time to review one of your report’s pull requests right before or after you have a 1:1 with them. Or it could take the form of reserving mornings or afternoons a few days a week to chase down the status of in-flight Jira epics and read updates about what peer teams are doing. Personally, I like to use my Mondays for admin and execution (second line management), Tuesdays for coaching (front line management), and Fridays for writing, strategy and synthesis (departmental management).

Now, as a people manager your calendar can feel more like something that happens to you than something you have control over, so don’t expect to do this perfectly. If your calendar seems impossible to tame, I highly recommend focusing only on grouping or arranging recurring meetings and tasks. Things that require you to context switch will still slip through the cracks, but these groupings will give your week a rhythm that will make it much easier to switch into the right abstraction level at the right time.

Speaking of creating consistent structure for yourself, you should also look for ways to standardize and templatize things you do regularly. This is not groundbreaking advice, of course. Leaders of all sorts have to think about a lot of things, and consistent structures help them get to the more differentiated and meaningful parts of their work quicker. But the “look for” part is what I really mean to emphasize here—swamped data leaders should shamelessly copy formats and frameworks that work for them, rather than trying to invent them from scratch. Even if they don’t fully match the way your team operates or is structured, grab execution review templates you see on a blog, introduce a project planning frameworks you used at your last job, copy meeting formats from teams you think are run well—whatever seems like a decent fit for your organization’s needs. If you’re stretched thin, it’s much easier to adapt an existing structure and then course correct to your needs than it is to derive one from first principles.

The final technique I recommend for managing your abstraction mix is to rely on delegation as much as you can. This one might feel tough, because many management tasks require broader organizational context that the average member of your team might not have. But delegation is a reliable way to create space for yourself and opportunity for your team, and I promise you will be pleasantly surprised by the way your team rises to the challenges they’re presented with. There’s definitely an art to delegating well, but as with all things, it’s a skill you can get better at with practice.

If you’re not sure where to begin, delegate small things and work your way up. Start by asking more senior members of your team to be the dedicated reviewers for more junior members’ analyses, then have one of your managers run an org-wide meeting or all hands, then task a tech lead to write a strategy and roadmap for a quarter long project to build an important new ML model deployment service. Keep your eye on them to make sure they have clear milestones and guardrails to keep them on track, but lighten your touch as time goes on. This is how the next generation of leaders is made, and it will create clearer abstraction level separations for both of you.

—

More companies are building data teams, more people are entering the profession, and more data leadership roles are needed than ever before. Data team management is harder in many ways than leadership roles at equivalent levels of other functions we commonly collaborate with (engineering, product, design, etc.), and the fact that us data leaders are often forced to be three kinds of leader at once is under-recognized and under-discussed. It’s brutal, but it is possible to survive and even get ahead if you learn to manage the abstraction mix.