During my term at Zynga, I was on the Words with Friends team where I had the opportunity to focus on creating experiments and features targeted at user growth and acquisition. I learnt about the ways that data is used to drive decisions on the team and track progress of features and experiments.
My work was largely split into the following umbrellas:
Ideating and launching hypothesis-driven experiments.
I worked closely with our Analytics team to create custom datasets with rules and launch A/B tests directed at specific audiences in order to drive the player base of the Clubs Daily Puzzle feature within the game. Once each experiment was launched, I continuously monitored it and conducted readouts on the progress of specific metrics that were being targeted, as well as the impact on top line metrics.
Writing feature requirements for both features existing on the backlog and pain points that I identified.
During the term I wrote product specs for three new features; the first two were targeting localization and player retention respectively. I collaborated closely with design and engineering to determine optimal surfacing behaviours. The third product spec I wrote was for a problem I had identified in the user acquisition flow; for this product spec I integrated new taxonomy and collaborated with both our internal engineering team and a third-party integration teams to ensure feasibility of the improved feature.
Developing an understanding of the player experience to better identify areas of opportunity.
My understanding of the player experience was developed through secondary research of the player journey and primary research by conducting calls. The first project I worked on was to revamp the FAQ page of the game, which involved developing a very deep understanding of what the game mechanisms were as well as areas that players could get confused. I also had the opportunity to run a user interview with one of our top players to better understand their typical game play experience, what drives them, and some of their pain points.
Conducting investigations to identify causes of unusual changes to metrics and track progress of experiments.
Investigations are a very important part of the PM role on WWF. Whenever there was any unusual behaviour with regards to top line metrics or the monitored metrics in experiments, it was important that the root cause was found. During my term, I handled ad-hoc investigations, and in my final week, being on call allowed real-time investigations, gave me insight into operations, and helped identify potential release conflicts.
Learnings
Here are some of my key learnings from the term:
1. How to run successful A/B tests to drive impact
When working in such a large ecosystem with a proven player base, A/B tests are a great tool in understanding the impacts of experience changes or feature additions. I found that successful A/B tests had a very clear structure, needed to be monitored routinely, and could be communicated effectively. It was important to determine what key metric was being tracked, what other metrics could be impacted, and which ones had to be monitored to mitigate adverse effects.
2. The numbers can be wrong - know how, why, and when
When conducting investigations, I found it to be a very useful first step to validate the data. Was the cause of a metric decline actually a system outage? Was there any changes in tracking behaviours? Understanding why the data was behaving a certain way was not just limited to changes in user behaviors but potentially also changes in data behaviours.
3. Creating a framework for feature prioritization
After my first time presenting my ideas for new features to my manager, she prompted me to think more critically about them from a framework oriented lens. I found that evaluating them across the RICE was very useful: this framework allowed for a consideration of impact on the business, while balancing the effort it would take to accomplish. By doing this, I was able to present the most beneficial and realistic ideas to the team.
SPECIAL THANK YOU
The winter term I had was filled with so much learning - every single day I learnt something new, and it was in large part due to the incredibly smart team I worked with. A huge shoutout to my WWF PM team - Kayla, Deanna, Rebex, Maitry, Stuart, Rebecca, Ian, and Liam - thank you for taking the time to teach me about the incredible world of data and everything that can be done when leveraging it. Lastly, my experience would not have been half as cool without the great people I met at the Toronto office!
L-R: Posing in front of the Zynga sign in the Toronto office, the setup in front of the Harry Potter team
L-R: Winter social event with the Toronto office, celebrating Lunar New Year with sweet treats
