Short-form video platforms are revolutionizing content consumption, but how do different platforms shape user experience and engagement? This project aimed to compare the user experience (UX) and user engagement (UE) of two popular short-form video platforms—TikTok and YouTube Shorts. Using surveys and statistical analysis, we explored the relationship between UX and UE, examining how user age and time spent on the apps influenced their experience and engagement.
Timeline
it was 1 semester joined project for Research Quantitative class and Cognitive Ergonomics class under the Human-Computer Interaction masters program.
Research & Planning
We started by understanding the core users of both platforms, focusing on their behavior, preferences, and needs. This was done through a literature review, analyzing existing studies on user engagement in short-form video platforms.
Define
Based on our initial research, we defined two core hypotheses:
Hypothesis 1: There is a consistent relationship between UX and UE.
Hypothesis 2: User demographics, like age and time spent on the app, affect both UX and UE. These hypotheses shaped our questionnaire design and overall research goals.
Ideate
We developed and piloted a questionnaire to evaluate key aspects of UX (e.g., trust, novelty, adaptability) and UE (e.g., focus, perceived usability). This iterative process allowed us to fine-tune the design to ensure accuracy and clarity.
Prototype
A between-subject study was conducted where we collected data from 51 participants using platforms like Reddit, Instagram, and WhatsApp. The survey was designed to be user-friendly, anonymous, and engaging to ensure participant focus.
Test & Analyze
We conducted a two-sample t-test to compare UX and UE between TikTok and YouTube Shorts. Key findings included:
TikTok had a superior UX, largely due to its aesthetics and focus on user rewards.
UE was close between both platforms, with TikTok slightly ahead.
Contrary to expectations, age and time spent did not significantly impact UX/UE, challenging existing research.
TikTok Wins in User Experience:
Our analysis revealed that TikTok significantly outperforms YouTube Shorts in terms of overall user experience (UX). TikTok users reported higher satisfaction across various dimensions, including aesthetics, trust, and stimulation. This suggests that TikTok’s design choices, particularly its emphasis on visual appeal and interactive features, resonate more with users.
Mixed Results for User Engagement:
While both platforms kept users engaged, TikTok had a slight edge. Users were more likely to stay focused and return to the app due to its dynamic content and engaging features, such as duets and challenges. However, engagement levels were close between TikTok and YouTube Shorts, indicating that both platforms succeed in keeping users interested.Hypotheses Tested:
Hypothesis 1 - Partially True:
There is a relationship between user experience (UX) and user engagement (UE), but it is not as strong or consistent as initially expected. For instance, factors like efficiency and stimulation were linked to higher engagement, but other UX elements did not have the same impact.Hypothesis 2 - Not Supported:
Contrary to previous research, neither age nor time spent on the platform had a significant effect on how users experienced or engaged with TikTok or YouTube Shorts. This finding challenges existing assumptions about demographic influences on UX and UE.
Sample Size: Our sample size of 51 participants was relatively small, limiting the generalizability of the results.
Study Design: The between-subjects design may not have captured the full range of user preferences, suggesting future research should explore within-subjects designs for more nuanced insights.
This project reinforced the value of empathy in understanding user needs and iterative testing in refining research tools. We also learned the importance of challenging assumptions with data, as our findings on age and time contradicted established literature.