Deen
Leveraging User Data and Working On A Project Long-Term
The Brief
The Subject
Deen, an Islamic resource application. This was not a client, but rather an internal project that was not-for-profit. I worked with Deen for the entirety of my time at Rayn (nearly two years).
The Problem
In Deen’s case the problem was about working backwards; the project had dived headfirst into providing a service for a generalized “Muslim” population. Now they wanted to understand their users in order to plan the strategy for the app moving forward.
The Process
Local Usability Testing
International Users - The Survey
And the Usability Study
And the Focus Groups
Data-Based Segmentation Approach
Other Side-Quests - A/B Testing
Other Side-Quests - Team Building Activities
Local Usability Testing 📑
One of the first things we did with Deen was to conduct a local usability study. The biggest chunk of the app’s audience (close to 75%) came from Pakistan. We wanted to ensure a diverse blend of people being testing, so our testing cohort consisted of shopkeepers and delivery drivers, university students, office workers, and the elderly. The study was built around all of the core features of the application, and users went through the tasks and shared their thoughts with us during the process.
International Users - The Survey 📝
We wanted to understand our international user base, and understand how they chose to interact with the app. In order to do this I developed a two-pronged approach. The first was a survey to get baseline information on users and their habits/preferences. For this, we tried to reach users through multiple different ways including through social media posts, but the most effective by far was via push notification which redirected to a Google Form.
The survey was built to explore how users approached religion, what their religious goals were, and how frequently they engaged with religion. I coded the results to try and get a sense of where trends lay for different groups and to assess whether I could see distinct groups emerging.
We were able to begin to think about what our user base looked like, based on the results from the survey.
And The Usability Study 🔎
Since we had a lot of email opt-ins for future research, I also decided to do a quick usability study with some of them which I did through Maze. I used the same study I had done with our local audience and the UX designer on the project helped me put together the prototypes I needed for the test.
This study echoed the results we saw from the local study and re-affirmed what we already expected by way of simple app fixes and what users really wanted from the application, whilst also bringing to light some interesting discoveries such as communication differences - some users were simply not aware that certain features existed within the application and this highlighted to us important communication gaps where iconography and visuals could be more clear and indicative.
And the Focus Groups 🎧
The final step was to conduct focus groups in the hopes that a more organic discussion would foster new discoveries. Because we received email opt-ins from all over the world, in order to get a clearer understanding of our users, I grouped focus groups by region. The greatest population of users were South Asian users (predominantly from Pakistan, Bangladesh, and India) and the second greatest came from Africa (mainly Kenya, Nigeria, and Ghana), so those were my two main focus group sessions.
Of course, as I’m sure every researcher ever has experienced, people can be fickle. I did have a lot of no-shows but I worked with what I had and we still had a great discussion in my smaller-than-hoped-for groups.
The focus groups helped us understand what dedicated users of the application wanted to see in the app moving forward, what their app usage looked like in a more exploratory way, and how we could plan ahead for the future with this feedback while still staying in line with our vision.
Data-Based Segmentation Approach 📊
Our second approach towards understanding user behaviour on the app was data-based; leveraging existing user data spanning across a year from Ramadan 2022 to Ramadan 2023, to analyze user trends, drop off, and feature-specific activity.
We chose Ramadan in particular because that was one of our busiest months for the app, being the holy month for Muslims. So we were interested in analyzing how users behaved in the year following Ramadan and up to the point of the next Ramadan. How did feature use and average session rates vary, and how did overall user activity (DSLA - days since last active) on the application get impacted.
This exercise was really exciting because I got a chance to work so closely with the data team to figure how to get exactly the kinds of insights we needed and could work with.
Our analysis was based on session rates since the app did not collect any user specific data. Based on these session rates, I developed “usage types”, based on where I saw specific, variated patterns of use emerging.
This activity helped us understand what we needed to focus on moving forwards, in order to keep users engaged with the application. The majority of our users were exploratory users in the month of Ramadan, who did not uninstall the app but their usage significantly decreased following Ramadan. Pulling them back into the app or rather reminding them of the app with targeted, specific features was essential. The primary blocker for increased engagement with the application was a lack of motivation and thats what the app wanted to target moving forward.
Sidequests - A/B Testing ⚖️
The exciting part about working on a long-term project is that alongside longer-term strategy work, there are also multiple day-to-day exercises to keep you engaged with the constant improvement of the application, such as regular A/B testing for small design choices, iterations, copy, and iconography improvements.
Other Sidequests - Team Building Activities 👨🏼🤝👨🏽
One thing I noticed during my work with the team on this project was that there was a tendency to work in silos, which was only heightened by the fact that everyone worked in different office locations.
So as we brainstormed ideas for different ways to engage users in higher activity Islamic months, one of the things I set up was a team-wide exercise. This not only brought fresh perspectives to the table, but also made everyone feel like they were part of the app’s decision making process. It was not only productive and collaborative, but also really fun for everyone and received great feedback from the team.
