Don't do it for the glory
I talk to a lot of current and aspiring data leaders in both my day job and the more extracurricular parts of my professional life, and I frequently get the impression that when folks step into leadership roles, it’s not that they’re looking for responsibility as much as they’re looking for respect. So many data people have had the experience of presenting their work to a business person and having it minimized or discarded. Or perhaps they’ve suggested to their peers or manager that their team adopt some new framework or tool, only to be ignored or told no. If you have this experience of someone in a position of power gatekeeping you enough times, it’s not unreasonable to assume that the thing that will change this dynamic is becoming one of them yourself. Perhaps once you are in a leadership role yourself, you’ll finally be taken seriously.
I won’t beat around this bush—if this is your motivation to become a data leader, you shouldn’t do it.
Now to be clear, my motivation to become a people leader was not much better. I was driven by spite, so irritated by a string of incompetent managers that I figured I should try it because even if I was a bad manager, I’d still be better than them. I did prove more organizationally effective than my predecessors, but I was quickly humbled by many other parts of the job because a leading data team was much more difficult than I expected. I've been thinking about why that is for a while, and even though I’ve gotten better at identifying and dealing with the challenges of the role, I don’t think data leadership is any easier to do.
Even though data teams are easier to bootstrap and build than ever, most of the folks you work with at your company won’t understand how data teams work, nor do they want or need to. Unless they used to be an analyst themselves, they’ll think of your team’s job primarily as writing SQL queries, or more generally, as making data appear as fast as possible. They won’t care about how it gets to them, and if they ever appreciate the importance of data quality, it will probably only be because they’re inconvenienced by insufficient quality.
This makes your team and its work feel invisible. No one says a thing when things are running smoothly, but when the team can’t keep up with demands placed on it, suddenly you have a dozen voices saying they aren’t getting enough support. Even if the business people1 you work with flag this need in a friendly way that ultimately advocates for better resourcing your team, it’s very easy for it to feel like they’re blaming the team for not doing its job. If you can set aside your feelings about this, it’s a great opportunity to educate the stakeholder about what your team does and why certain projects take the amount of time they do, but the number of data team customers always grows faster than the data team. Once you educate one stakeholder, there will always be someone else new and uninformed to take their place. Leading a data team means you will constantly repeat yourself.
Much in the same way that your company won’t understand how your team works, they won’t know where you yourself fit in. If the most senior data person doesn’t report into the CEO, where should they sit? Marketing, product, the CTO, the CFO? No one can agree on this, not even the data community itself, and your company will probably not make this decision in an especially principled way. As a data leader, your reporting line won’t be decided based on what’s best for you or the data team. It will probably be decided by the same organizational inertia that causes large orgs to get larger, or as a way of rewarding someone that the CEO trusts by giving them a new challenge or broadened scope, or simply by who made the first pitch to hire a data team. Where the data team sits in the org will inevitably make conversations about budget and org design and even calibration in performance management awkward or difficult.
It also means that you will nearly always occupy lonely spots in the org. You probably won’t have a lot of peers doing the same type of work as you, if you have any at all, and you probably won’t clearly be a part of any other teams of cross-functional leaders. Unless you and your team have a very well-defined scope (for example, an analytics manager being paired with a group product manager and their direct reports being paired with the product manager’s direct reports), you will have all sorts of customers with contradictory incentives, making it hard to clearly be a part of any team. You won’t be top of mind for your stakeholders when you think you should be. They’ll forget to invite you to conversations you should have been in, or they’ll decide that involving you and your team would be too much friction when they need to move as fast as they do.
You will feel left out. You will feel undermined. You will feel like your stakeholders are being reckless and shortsighted. We data people love neat categories and clean abstractions, but in a leadership role you will constantly be forced to deal with whatever messiness your company imposes on you. You’ll warn others about the consequences of doing something that’s obviously bad to you, they’ll do it anyway, and you’ll have to clean up the mess that’s created as a result. You’ll have tons of projects and goals you want to pursue, but you’ll never have time to get to. You’ll have to compromise on your bar for quality for the sake of speed. Your team will experience all of this too, and since they’re orderly data folks like you, it will stress them out. Them being stressed will stress you out, and you in your lonely spot in the org may not have colleagues to commiserate with2.
I could keep going, but I think I’ve made my point. Being a data leader is not for the faint of heart. I wish it weren’t this way, and while I do believe it’s possible to improve your circumstances for the better, you need to know what you’re getting into if data leadership is something you want to do.
Accept that all the above will happen to you. It’s not a matter of if, it’s a matter of when, and it won’t simply be a rough patch you need to push through—it will happen over and over again. When it happens to you, you should not view it as a personal failing. It’s part of the job, and certainly a frustrating one, but it happens to every data leader at every company, even when they present a rosy picture externally. Remember that their team was successful even in the face of all the challenges and indignities that data leaders and data teams experience, and that your team’s successes will happen that way too. Even if progress feels painful, it’s still progress. It still counts.
Relatedly, recognize that success doesn’t always come in the form you want or look like you expect it to. Do your stakeholders find problems with the data by looking at a dashboard your team built? Frequent firedrills are frustrating (and you of course need to get your team’s heads above water), but it’s also a sign that you’ve created something legitimately useful. They’re looking at that dashboard all the time! The same goes for creating a model or table that someone starts using for an unintended purpose, which you only learn about when they howl about when you do something to break their downstream application. It might feel like having autonomy to build what you want to build taken away, but you could do what companies do all the time—pivot from your original plan and pave a desire path.
The surest way to feel like you’re failing is fixate on the short term. Data work rarely has a clear end point—there’s always a follow up question, or something you didn’t consider, or more you could explore. When the data community talks about what data teams do, we talk about enabling better decisions, conveniently forgetting that decisions are not discrete, one time events any more than the projects data teams do are discrete, one time tasks. In the real world, decisions and data are infinite games—there are no rules or right answers or final states that can be reached if you just make the right moves. The only way to win is to stay in the game.
If you want to do this, you need to banish us vs. them thinking from your mind. No matter how annoying or frustrating the stakeholders of your team are, don’t forget that you still work for the same company. You might not always see eye to eye and you may sometimes have conflicting incentives, but at the end of the day you both still want your company to succeed. Even if it’s hard, try to work and empathize with them. It will help you understand how your team can productively fit into the bigger picture of the company, and perhaps you can even turn your difficult stakeholder3 into an ally by working with them instead of against them.
You also need to be willing to change your mind about what your role is and what your team should do. You don’t need to start completely from scratch, but you should pay attention to how your company reacts to the way you run your team. It can be hard to get verbal feedback on what you could do better, but you can get a pretty regular signal on what works by paying attention to what the rest of the company notices and engages with. It doesn’t mean you need to shift everything your team does to being about those specific activities, but it does mean you should think about why they resonate. If that’s what works, why? How can you make more of your team’s activities like that?
Doing “better” is something you need to have a holistic view of. Companies require forward motion to stay alive, and while you should of course make your team a part of that forward motion, don’t forget to look backwards too. What have you done previously that felt like success or improvement? How long did it take, and how hard did you have to fight for it? It’s easy to forget how hard something was to achieve after enough time has passed. Use this backward-looking to calibrate your patience, and to remind yourself that your circumstances are never permanent. You might not be where you want to be just yet, but you’ve made progress in the past and you will again. You are probably making progress even now, even if it’s going slower than you want it to or if it’s requiring you take a different path than you planned.
Most of your career will be spent between valleys of despair and peaks of accomplishment. Part of why those extreme experiences are so vivid is that valleys and peaks are the places where you have the best visibility of the path that led you to that point. When you remember those high points and low points, your brain is compressing the whole trek up until then into that one moment. That’s not to say that the peak or trough moment doesn’t matter, as much as it is that the build up is what makes the peak moment so sweet and that the build up can be appreciated too.
Learning to enjoy the work of the climb—to enjoy effort before you’re rewarded—is easier said than done, but if you’re working in data, you probably like something about the day to day tasks and responsibilities of your job. If you want to stick with it, you need to make a point of enjoying those. Get excited about applying and honing your craft. Feel satisfaction at accomplishing more with less, including when the thing you have less of is less time than you’d prefer. Celebrate milestones outside of the big mountain peaks, even or especially if they’re milestones that are only relevant to you.
If you want to be a data leader, don’t do it for the glory. If you want to stay in the game, cultivate self-compassion, embrace flexibility, and learn to love the process.4
Feel free to substitute “business people” with your preferred stakeholder type
If you find yourself in this kind of lonely position, I highly recommend checking out the Locally Optimistic slack’s #leading-data-teams channel.
You will absolutely encounter assholes and bad actors at some point in your professional life, but remember that most people are not this way. I’ve consistently found that with a few years distance, former colleagues who I thought were absolute tyrants are more reasonable than I felt they were at the time.
This post may seem like a stark contrast to Elbows of Data, but people have interpreted that piece as being about aggression when I intended for it to be about being assertive. To-may-toe, to-mah-toe, perhaps, but in any case, both of these pieces call for a long view of career success, and being patient and being assertive can be very complementary