S2 Ep. 3: How Do You Know If Your Product Org is Truly Great?
Subscribe: Apple Podcasts | Google Podcasts | Spotify | RSS
Hope Gurion: Product leaders can benchmark their teams’ performance against their desired outcomes, but how do you benchmark your team against the best performing product teams? What is the evidence that reveals your team is actually great at their craft and positioned to succeed? In this episode of Fearless Product Leadership, I asked 6 seasoned product leaders to answer this question “How do you know if your product org is actually great?”
Welcome to the Fearless Product Leadership podcast. This is the show for new product leaders seeking to increase their confidence and competence. In every episode I ask experienced and thoughtful product leaders to share their strategies and tactics that have helped them tackle a tough responsibility of the product leader role. I love helping emerging product leaders shorten their learning curves to expedite their professional success with great products, teams and stakeholder relationships. I’m your host and CEO of Fearless Product, Hope Gurion.
CEOs and new product leaders often lack the experience to gauge how effective or high performing their product orgs are. Sometimes they want to have their teams assessed, and while I can see skills or knowledge deficiencies, only a leader living with the team every day and observing their actions, relationships and results can really assess how well their team is performing. As a leader of multiple product teams in your org you may find you have different teams at different points on the greatness spectrum. So how do you find the evidence to assess the overall greatness of your product org?
In this episode you’ll hear a clear pattern of the characteristics of great product orgs:
They interact with customers regularly.
They have high trust within the product teams and great relationships with stakeholders.
They deeply care about and understand the why behind their work.
They have a bias for action.
They embrace objectively measuring the impact of their work.
Fearlessly tackling the question “How do you know if your product org is actually great?” are:
Nicole Brolan, Chief Product Officer at SEEK
Nate Walkingshaw, Chief Product Officer at Pluralsight
Sean Murphy, former VP of Product at Target and VP of Product & Engineering at CustomInk
Seth Roe, VP Consumer Product at The Weather Channel
Amanda Richardson, CEO, CoderPad, former VP of Product at HotelTonight and Prezi
Al Ming, VP of Product & Design at CNBC
Nicole Brolan, Chief Product Officer at SEEK, kicks us off with the 3 pieces of evidence she looks for to reveal how healthy her product organization is: consistently challenging the WHY, regular customer interaction, and connecting product behavior to business success metrics.
Nicole Brolan: Yeah, so the question is irrefutable evidence of having a great product org. I’ve got a few answers to this and have actually kind of watched our product or evolved at a time. I think one is about the culture that you witness within the product org, and for me that is around: Do you have people who are agitating for change? Do you have people that are challenging the status quo and are asking questions about, “Why are we doing this?” I don't think we're approaching this is the right way.
I think to have a successful product org, you need to have product leaders and product managers that are really not just being told what to do they're really challenging you know the underlying business reasons for why we're doing what we're doing. So, as the leader, sometimes you sit there and go this is driving me crazy, because every day you're coming up and challenging everything; but I think that's a sign of a fantastic product org.
I think the second piece for me is around customer contact. So, a fantastic product org is one where you see visible evidence of the product teams interacting with customers regularly. So, you can hear your regular insights, you can see in presentations how the insights have translated into therefore what they're building, and I think that's a really telling sign of a fantastic product org.
Then, I think that the third element is having a product org that's really connected to the business metrics that are trying to move. So, I think having a product org that when prompted can actually tell you why they're doing something and how that connects to the outcome and the customer behavior that they're trying to shift. I think when you see all of those things in place, for me, that's that starts to point that you've actually got a pretty healthy product organization.
Hope Gurion: Next, Nate Walkingshaw shares 2 key attributes of great product org: first, it behaves a lot like a close-knit family and second, the individuals each have a clear, measurable sense for how their work impacts customers.
Nate Walkingshaw: If I were to describe to you like irrefutable evidence of a great product or healthy product organization, it probably sits across a series of different attributes to me. I'm going to use the word culture, but it's this is not culture theater, this is real, authentic, human interaction and relationship building; I think is irrefutable evidence and the way that it looks to me is that when you watch a healthy product organization, the description I want to use is they act a lot like my family behaves when the doors are closed. There isn't fear, there isn’t shame, there isn't guilt, there's challenging of ideas; when somebody has an opinion, they're not afraid to actually share exactly the way that they think and see the world and know that they're not going to get judged by the way that they share their thoughts and ideas.
The way that it looks to me is like there isn't judgment from the teams, you allow the person the space to be who they are. That to me, is a really genuine and authentic way that I think human beings should live and work together. I've been really committed to try and foster that culture of Pluralsight. It is very hard, it is very hard to do, it is very emotionally draining to do, but the fact is that I have like an insane group of leaders that philosophically believe the same approach. So, you would have to you would have to believe that this this will make it to the end of the row of every single employee here; every single employee when they drive into work, knows why they matter, they know what work they do and they can see the impact of that work. Then, when they do you deliver the impact of that work, they can feel a sense of purpose and meaning in it.
That's kind of the second indicator of culture to me, is that people have a very clear idea of what they do every single day and how it applies to the customers that we're trying to solve these problems for. If I were to go like on the on the like quant side, the Holy Grail metric for me and it'll sit there forever, is retention. I have three metrics, I've led to three metrics my whole career, which is likelihood to buy, likelihood to stay, NPS. It's pretty simple, when you and I walk into a store, we buy a product off the shelf, did I like what I paid for that product? When I used, it did I feel surprised and delighted and did it do the job that I intended it to do and you know what I go back and repurchase that item again based on the happiness curve that it gave me? You really don't need to get scientific around it, that's it, it's three metrics. You could segment it out and do cohort analysis and do retention curve analysis, yes, we do all those things and we are obsessed with doing those things, but you know the goal is likelihood to buy, likelihood to stay, NPS.
Hope Gurion: Next, Sean Murphy describes what evidence he looks for, to ensure he has a great product org: a team that has a bias for action, will take multiple swings at an important problem and encourages counter-thinking.
Sean Murphy: Hope, it is such a good question. What are the signs of a great product organization and I think there's no one-size-fits-all, we've all seen different teams work together in different ways, that are productive. Here are some of the things that I look for: so, one is more doing and less philosophy; if there's anything I've noticed from product organization, there's a lot of navel-gazing going on. There's a lot of philosophy and the way to do things right and the way to do things wrong and basically what you need to do, is the product teams need to just try and do some things and then periodically pause and reflect to say, “hey is this working? Are we going in the right direction? What would we change?” don't need to get more complicated than that. Two is, another thing that I love to see, is when teams are lunching together. When you are spontaneously seeing teams: the product, the design, the engineering, the operations, whatever, coming together for lunch and breaking bread together, I know good things are coming. Another practice I really like to see, is when teams have set aside time to take two or three or four swings of the bat is that a problem. Organizations that are just on to the next thing, on to the next thing, all the time, the business pressures are there to do that; to just deliver, but a healthy organization has the respect and the command with their peers and their partners that they can take two or three attempts at things. The last thing I really love to see is is counter thinking, where a team has a confidence and the trust in each other, where they can propose going 180 degrees from where the existing plan is and to try something else and the whole team is willing to explore that thoughtfully. So, those are some things that I look for in signs of really successful product teams.
Hope Gurion: Amanda Richardson reveals her 3 tell-tale signs of a great product org: one, it has strategic importance to the CEO, two, it conducts regular user testing and three, the engineering team can tell you what matters to customers and why.
Amanda Richardson: I think there's some very simple clear signs around whether an organization is great, product organization is great and by the way I think there are different flavors of great products organizations, depending on what you need and where your company is, but these are great product organizations that I would want to work with.
Number one, product reports to the CEO. I do think that's critically important, that shows that the CEO cares about what the product is, and cares about how the strategy is coming to life.
Number two, there's regular user testing. Some sort of process, some sort of way in which hopefully, multiple teams but certainly the product managers and the designers are working with users on a regular basis; that could be a beta testing program, that could be a user testing dot com account that they use often, that could be regular trips to Starbucks to find users and do some ad hoc testing. Whatever it is, surveys don't count, regular ways to do user testing, I think is the second criteria.
The third criteria that's really important to me is that you have an engineering team that understands the product. So, this is where you can go up to engineers and ask them how features work, you can go up to engineers and ask them why features work and how users would use it. If the engineers are just punching out code and they don't really understand why or why it matters or they're not invested in the problem that's being solved, I think the product team will never be successful. So, must report to the CEO, must talk to users and must have engineers that care about the product.
Hope Gurion: Next, Seth Roe reflects that the greatest product orgs he’s worked with have 3 attributes: high trust, inclusion of diverse perspectives and real purpose to their work.
Seth Roe: The way I think about irrefutable evidence of a great product org, basically revolves around feeling connected and belonging to a team who has your back. Now, every great team, every great organization, every great family, the there’s always key elements to it.
The first one for me is trust. I ultimately have to be building authentic relationships with folks, who when we have tough times or tough conversations, we're in it for each other and it’s built on loyalty and trust. So, that’s the biggest thing.
The other thing I love about our team and why I feel like it's a great product org is the inclusiveness and diversity. If you told me 20 years ago that I'd be standing around a table with different ages, different genders, different countries, ethnicities, different political points of view, whatever it is; to me, the diversity is ultimately the thing that always leads to us making the choices that I believe in the most. Whenever I feel like our teams lack that diversity, is when I kind of see them tripping over themselves a little bit. So, that that's always a very important part.
Then, what I what I ultimately feel like as part of that team is the most influential ideas, the most meaningful ideas and the most successful ideas I've honestly seen in our org, have always come from the bottom up. It's very hard not to be top down in an outcome driven organization, and I think that's something that we talk about in the product councils (Collaborative Gain: The Councils) a lot and just with product colleagues over beers. It's always about growing our team skills and at the end of the day, if we can lead them in the right direction and we end up achieving the things that we're trying to achieve, that that ultimately is a successful product org.
I will say the the only other thing I would add to that, is we try to give our teams a real purpose. I'm very fortunate to work for a brand that's truly delivering a product that's built on incredible science and information to everyone in the world. Anyone can access it, it's available for everyone at any time of day and although a lot of people just look at it in America as like basic weather information that's very much a commodity, that's simply not the case everywhere else in the world. It's very, very, very important to these folks and it's very important for the people that are ultimately funding our mission in America to know that, but that purpose really helps our teams kind of come to work and drive the passion that that we all have for this. So, some of the other elements in that are just making sure the teams have a growth mindset, that we're embracing failure and allowing them to experiment in meaningful ways; not everything is is always going to work, as you know.
The final piece for me is just making sure that the teams are are having healthy debate, we try to structure our teams around triads, so there's product representation, tech and design representation and we want our teams to have independent decision-making, but I also want the teams shared decision-making in the sense that it's not always product that's running the show, it's not always product that's making the final decision. Based on that, I think we all know we we come here to to drive outcomes and the teams they have to achieve positive outcomes or else we're going to shuffle them up and try a different approach. It's kind of a long answer, but it's all about belonging to a team or as you’re back making sure that team has real purpose and you know they're consistently achieving positive outcomes for the organization.
Hope Gurion: Finally, Al Ming who has led multiple product teams at various companies points out that the evidence of a great product org can be seen in how they interact with each other, their stakeholders and with customers and shares the one revealing answer they never give as the reasoning behind their work.
Al Ming: I think one of the things that is the real irrefutable evidence of a great product org is reflected in the ways that your product teams work together and engage with stakeholders and engage with customers. One of things that I always tell new product managers when they come on board, is that I want them to be the ones who are the champions of the why of it all, that there is a passion and intellectual curiosity that drives them, that they use to motivate their teams, about wanting to know why things work, wanting to know not just what to do or how it performed but why it works. That same curiosity that as a kid, makes you open up a clock and break it. I want them to have that about customers, about industry, about technology, about user experience patterns. That passion almost crystallizing on the edges, helps them to be driven, helps them to ask the right questions, helps them to be experimental and customer focused, because they genuinely want to know what's happening and they want to be able to be that resource to the rest of the organization. So, that stakeholders and executives they know they can come to a organization the Internet's that development team and that's where they go first, that's the value that we bring to the organization; is that understanding of not just what's happening but why it's happening and that we have a level of thoughtfulness that goes into things. No one answers, “Because you told me to” as a reason that we are doing something, they answer because the customer is doing this, because we have this hypothesis, because we believe this to be true and we tested it and we found out and it is true, because of the trends that are happening in the market, because of the partnerships that we have. All of these things tied together and there's always that slice of product as a broker between many different places, but the strength of a product organization comes from that deep knowledge and insight and practical result of knowing what to do, why we're doing it and and bringing that to the organization.
Hope Gurion: CEOs and new product leaders often ask me to help them assess the greatness gaps on their teams and while I can diagnose absence of expected practices and behaviors and prescribe corrective actions, there are intrinsic motivations and characteristics that underlie healthy, high functioning product teams that can only be diagnosed by the teams themselves. Only the members of the teams themselves can see these qualities day to day so it’s important that you encourage open and sometimes private conversations about healthy team dynamics to have a truly great product org. Some of the healthy team dynamics that are present in truly great product orgs include:
The teams deeply care about making high quality decisions with the best available information
The team cares about its members, how they balance their work load, and ensuring they have the right resources/skills to accomplish their goals
They feel psychologically safe to disagree and actively encourages debate and rigor to make important decisions
They believe in the meaning and attainability of their metrics
They make and keep commitments to one another
They have fun and genuinely like spending time together
They would choose to be a part of this team and product org again
If you’re not sure about the healthy dynamics of your team, I love the process that the product team at Canva uses and I’ll include a link to it in the show notes: How and why we do Team Health Checks at Canva
If you’re a product leader feeling like your product org isn’t as great as it could be, I’d love to be of help. Contact me on Linkedin or Twitter or schedule an initial consultation with me using the Contact Me page at fearless-product.com.