User research to investigate information behavior impacted by human biases

HIGHLIGHTS

When people search for information, they constantly judge whether content is relevant. However, confirmation bias can distort this process, leading users to engage less deeply with information that challenges their beliefs.

This project investigates how cognitive effort changes during relevance evaluation, and how individual differences such as confirmation bias, familiarity, and curiosity influence user attention and decision-making. Using eye-tracking and pupillometry, we measured real cognitive load during health-related information evaluation tasks.

Role: Lead UX Researcher (Experimental Design, Cognitive Measurement, Eye-Tracking Analysis)
Timeline: January 2021 – December 2023
Collaborators: Dr. Gavindya Jayawardena, Dr. Jacek Gwizdka

UX PROBLEM

Users often misjudge relevance, especially when content conflicts with prior beliefs
Biased relevance judgments can lead to shallow evaluation, missed information, and poor decisions
Most systems do not account for users’ cognitive states during relevance assessment

RESEARCH GOALS

Understand how cognitive load varies during relevance judgments
Examine how confirmation bias affects the depth of evaluation
Identify user traits (familiarity, curiosity) that increase or reduce mental effort
Translate findings into design guidance for bias-aware information systems

RESEARCH APPROACH

Study Design

Type: Controlled, within-subject eye-tracking experiment
Participants: 32 users (balanced by high vs. low confirmation bias tendency)
Setting: UT Austin Information eXperience Lab
Apparatus: Tobii TX-300 eye-tracker

Task Flow

Participants

Read a health-related task scenario
Evaluated a web document’s relevance (relevant, moderately relevant, irrelevant)
Completed post-task surveys measuring familiarity, curiosity, and workload (NASA-TLX)

Each participant completed 18 relevance evaluation trials (health-related content).

Key Measures

Cognitive Load: Low/High Index of Pupillary Activity (LHIPA) derived from pupil dilation
Behavioral Judgments: Perceived relevance ratings
Self-Report: Familiarity, curiosity, perceived workload
*Lower LHIPA values indicate higher cognitive load

KEY FINDINGS

Cognitive Effort Increases When Relevance Is Unclear

Users showed higher cognitive load when perceived relevance conflicted with actual topical relevance
Evaluating “ambiguous” or mismatched content required more mental effort

Confirmation Bias Reduces the Depth of Evaluation

Users with stronger confirmation bias invested less cognitive effort
This suggests that biased users disengage earlier when assessing information relevance

Familiarity and Curiosity Shape Attention

Moderate familiarity produced the highest cognitive load
Higher curiosity consistently led to deeper engagement and more effortful evaluation

UX & PRODUCT DESIGN IMPLICATIONS

Relevance is not binary. Ambiguity drives effort, and systems should support users most in these moments

Biased users may appear confident but are often under-engaged

Interfaces should:

Signal uncertainty clearly
Encourage deeper inspection when relevance judgments are likely biased
Adapt support based on user traits like familiarity and curiosity

IMPACT

This study provides empirical evidence that cognitive effort during relevance evaluation is shaped by bias and user characteristics. The findings inform the design of:

Search interfaces that counteract shallow relevance judgments
Decision-support tools that detect and respond to disengagement
UX strategies for reducing bias without increasing user workload

Relevance, Confirmation Bias, and Cognitive Load