It’s mid 2019. I’m in Bristol, leading a squad of 5 engineers working remotely across 4 countries.

We’re responsible for maintaining and developing the internal tools and systems used by over 100 Community Managers in 70 countries across the World.

The Community Managers are responsible for keeping the company’s ~100 million monthly users safe and engaged in our consumer product (a recipe sharing website and app).

They are specifically charged with reviewing all the user generated content that’s published on the service making sure it complies with our community guidelines (no inappropriate content, no recipes that would be culturally offensive in the country / region in which it was published, and no recipes that could pose a health and safety risk, etc).

They are also tasked with keeping recipe authors engaged with the service and encouraging them to share more recipes.

I’ve joined the squad as their first dedicated Product Manager and I’ve got a lot to learn about the tools they have developed and maintain and about how the Community Managers use them.

I’ve spent the first few weeks of working for the team interviewing Community Managers from across the world. I’ve spoken with the country heads who lead these teams locally and I’ve spent time with my squad of engineers understanding where they are in terms of both how they work and what they’re working on.

Throughout this process I learnt a number of things:

• the Community Managers are an extremely passionate, dedicated bunch who care deeply about their local users, their experience of using the service and how our product works
• their passion and dedication has lead to each Community Manager having very close relationships with a handful of our top authors across the world. They consider those authors their friends, and those authors consider them friends too
• it was obvious that while the work of monitoring and reviewing content was an important aspect of keeping the community healthy, it was when Community Managers engaged with and encouraged authors by commenting on their recipes or sharing messages directly that they really shone at getting those authors to return and add more recipes
• the tools the team had built for the Community Managers were functional and had evolved over time but, little consideration had been given to the experience of working with them on a daily basis.

I also realised that a lot about how both the Community Managers and the tools worked was not scalable.

The 3 other product squads working on the consumer product side of the company were all focused on significantly growing the numbers of people who published recipes at least once a week.

The company was not keen on having to hire significant numbers of additional Community Managers to scale with the increasing number of people publishing recipes, nor would they have been able to recruit them fast enough if the product grew at the target rate.

Therefore my squad had to focus on how we could make sure the tools we were designing and building for Community Managers would allow them to continue spending as much time as possible engaging and encouraging authors and reduce the time they spent doing the “admin” work, like reviewing recipes.

To complicate matters we had to do this without interrupting any of their day to day work.

We broke this challenge down into two parts:

1. part 1 involved looking for efficiencies in the workflow Community Managers had to follow when reviewing content and dealing with complaints about content
2. part 2 involved looking for ways of redesigning the tools to support Community Managers getting to know more authors just as thoroughly as they currently knew their existing top authors without having to invest many hours in the process.

We developed a plan.

In order to be able to test our ideas for new processes, workflows and user interfaces, we would run a Beta trial of the new version of the tools in parallel with the existing Community Management tools.

The Beta would be only available to Community Managers in one region to allow us to work closely with them to monitor their use of the tools and get their feedback directly as and when we released updates.

By running the Beta in parallel with the existing Community Management tools, making it easy for Community Managers to switch from the “traditional” version to the Beta version and back again, we were able to release features to Beta one at a time and often, without having to worry about replacing every part of the tools in one go. Which meant we could build, test, learn, repeat relatively quickly.

To monitor the impact of our Beta I created a satisfaction survey that we ran with the testing group every week, starting before the beta went live and continuing throughout its development. This survey allowed us to gather both qualitative and quantitive data about the Community Managers experience of using the tools and track if we were improving things for them.

We also monitored activity within the tools so we could see what impact the changes we were making had on their productivity.

To help keep us honest, share our progress with the rest of the Community Managers around the globe, and to help make sure other squads within the company were aware of what we were working on, I wrote a weekly blog post that was published on our intranet. It highlighted the changes we’d made each week, explaining the design decisions behind them, and reported any new feedback and analytical data we’d gathered from our Beta users.

Having to report progress each week like this helped the squad focus on delivering something of value on a weekly basis.

It also helped us give a bigger voice to the Community Managers within the company. By sharing the progress we were making and how crucial our Beta testers were in helping us make that progress, we were able to share their insights and ideas about how the product should work with a wider audience.

The goal was to develop the Beta to a point where the core features of the existing Community Management tools were redesigned and fully tested before allowing Community Managers from all regions and countries to opt in to accessing them (still making it available in parallel to the traditional version and allowing them to switch back and forth at will).

This would be the ultimate measure of success for us as a squad. If we were able to design and build the right thing, delivering it in the Beta, Community Managers from around the world would choose to use the new version over the old version.

Once we had the core features tested and, importantly, adopted by the majority of Community Managers globally we would work on redesigning or incorporating the “non-core” features into the Beta and work towards permanently switching all Community Managers over to the new designs and switching off the original.

My time at the company ended before the Beta was complete. But over the course of the time I worked with the team we were able to introduce:

• a new global dashboard view that showed Community Managers three levels of information at a glance: global activity, regional activity and the activity related to the users and content they personally had been communicating with / reviewing
• a completely redesigned workflow for recipe moderation that allowed Community Managers to see all of the recipes that were in each stage of the review process and who was reviewing them, all in one view. This could be filtered to show just the recipes they were responsible for reviewing, or those any member of their team were working on, helping the regional teams work together more effectively
• a streamlined recipe review process that required viewing one screen instead of six separate screens to approve / reject a recipe
• a redesigned author’s profile view that allowed Community Managers to see a news feed style view of an author’s recent activity on the service in one stream, along with an “activity summary” that showed an author’s last 12 months activity in one easy to see chart – making it much easier for them to see how active an author was and to get to know what types of recipes they liked to publish and share
• a redesigned messaging view that showed all communications between an author and any Community Manager. This allowed any Community Manager to pick up a conversation where a team member had left off, helping them share workload and always respond to authors, fully aware of other conversations that had already taken place with other Community Managers.

All of those changes were introduced in the Beta while the existing tools continued to be used in parallel. By developing them this way we were able to test with live data, as part of our testing team’s day to day work, which really helped us hone in on delivering the right features to help the Community Managers do their jobs.

Over the course of the project we were able to increase Community Manager satisfaction with the tools from “good”to “very good”and increase their feedback about how the tools were meeting their needs from “they sometimes meet my needs”to “they always meet my needs”