Many recent articles and blog posts have addressed the use of the hamburger menu on mobile websites and apps. We here at Catalyst Group have written about it in the past as well (read One Hamburger, Hold the Menu on the use of the hamburger menu on desktop). Although the hamburger menu’s popularity continues to grow, there is much debate over its usage due to the belief that users don’t understand its functionality.
The Catalyst researchers wanted to explore this belief in greater detail. In the past year, we conducted nearly a hundred individual research sessions in which a mobile website or app’s navigation used a hamburger menu. From this firsthand experience with so many users, we started to notice some trends regarding who typically understands the hamburger menu and who does not.
Some users get it immediately. Some are curious and want to interact with it, even though they have no idea how it might serve them. In the words of one user, “I want to see what that up there does.”
One of the main things we noticed was that older users were less familiar with the icon, and did not always know what would happen if they were to click on it. But, what does this observation mean? Is this an actual phenomenon, or just an observation with nothing to back it up? We decided to dig deep into our research files and do some analysis to answer the big question: Do users understand the hamburger menu?
Two of us from the Catalyst research team independently pored over research session notes and labeled each participant who interacted with a hamburger menu on a mobile or tablet app or website. Participants were labeled according to whether they “understood” or “did not understand” based on behavior they exhibited regarding use of the hamburger menu and expectations of what it might do. When our labels did not match, we discussed the participant’s behavior and came to a consensus about how the participant should be labeled. Our analysis only included participants who interacted with a hamburger menu when it was in the scope or context of a research session’s tasks.
We also noted which research project each participant was a part of, as well as their age, gender, ethnicity, household income and recruitment criteria for the particular project (when available). We ended up analyzing a total of 93 individual participants.
Overall, 24 participants did not understand the hamburger menu and 69 understood it. Although most did understand it, the fact that over a third of intended users (34.8 percent) had trouble with it should be a warning sign to those who are considering housing the bulk of their navigation within a hamburger menu.
We decided to look into the hypothesis that we had all along: Do younger users understand the hamburger menu better than older users? Our research confirms that yes, they do. The graph below shows users from each age group separated out into those who understood and those who did not.
For younger users, there was a greater proportion of those who understood the hamburger menu. For the purpose of this research question, we defined “younger users” as those ages 18-44 and “older users” as those age 45+ (in our data, the oldest person was age 64). We found that 80.6% of younger users understood the hamburger menu as compared to 52.4% of older users. A statistical analysis backs up the validity of this finding. We can say with certainty that the difference between the two groups is statistically significant.*
When results prove statistically significant, it means they were caused by something other than chance. This isn’t to say that age causes a person to be more likely to understand or not understand the hamburger menu; it is possible that a factor that is correlated with age, such as tech savviness, is the real predictor of understanding.
After validating our original hypothesis, we wanted to see if there were any other indicators of a person’s likelihood of understanding the mobile hamburger menu navigation. We analyzed gender, but did not find any significant gender differences in our data. We also found no significant differences between field testing participants (recruited by Catalyst researchers off the street) and those who took part in lab-based research at the Catalyst facility.
So, what does this all mean? Hard data shows older users experience the hamburger menu differently, meaning that companies need to be cautious about using the hamburger menu in mobile navigation. If you expect that a significant portion of your users are going to be in their 40’s or older, consider an alternate navigation convention.
* For those who want the numbers, this is what we did: A two-sample z-test between proportions was performed to determine whether there was a significant difference between the two age segments with respect to the percentage that was found to understand the hamburger menu. The z-statistic was significant at the .05 critical alpha level, z = 2.5962, p=.00932. Ideally, we would have recruited participants according to the exact same standards and for the sole purpose of this study rather than compiling data ad hoc, but that was not feasible for this activity. Since the 93 participants came from 9 independent research studies, we cannot make perfect comparisons since different recruiting standards were used. Also, some of the mobile website and app designs may have been higher quality than others, making the navigation easier to understand.