UX Research Methods

CS 535

Introduction to basic qualitative and quantitative methods—such as interviews, contextual inquiry, and design of experiments—to study user experience (UX) with computers and develop actionable insights. Both generative and evaluative UX research will be covered.

qualitative vs quantitative UX methods

Course Details

Tuesday and Thursdays: 5:00 – 6:15 PM, at CDRLC 1406

Tuesdays: 2 – 3 PM, at CDRLC 5417; and by appointment

  • An introductory course in human-computer interaction or user interface design (e.g., CS 422 or CS 522). Or equivalent practical experience.
  • An aptitude to learn and apply social science concepts to computer science applications is required.
  • Some familiarity with basic statistics. 

Students are responsible for familiarizing themselves with the syllabus. The instructor is responsible for being responsive to the diverse needs of the enrolled students and making necessary modifications to this syllabus, which is to be treated as a living document. Blackboard will be used to post course grades. Piazza will be used to post announcements, FAQs, discussion points, course-related general queries, and detailed instructions when deemed necessary. The course webpage will contain links to readings, general instructions, and class schedules. Post general doubts publicly on Piazza and private matters anonymously (to classmates). Students are responsible to keep themselves updated with class announcements.

Except for extenuating circumstances and emergencies, please avoid using emails for class-related doubts/inquiries. Use Piazza.

Course Outcomes

  • Familiarity with basic qualitative UX research methods, like surveys, contextual inquiry, and 1:1 interviews.
  • Familiarity with basic quantitative UX research methods, like log analysis, hypothesis testing, and meta-analysis.
  • Design controlled experiments and identify independent and dependent variables.
  • Use parametric and non-parametric statistics to analyze experiments.
  • Understand how to use data for generative and evaluative computing research.
  • Identify and operationalize different facets of usability and user experience.
  • Understand, assess, and use appropriate metrics to measure user experience.
  • Synthesize the “so what” and surface fresh insights and opportunity areas from UX research.

Textbooks

You are not required to buy any of these books. Either they are available online via the UIC library or relevant chapters will be provided.

Topics

  • Introduction to UX research
  • Structuring the research inquiry
  • Interviews and focus groups
  • Qualitative coding
  • Design of experiments
  • Parametric statistics
  • Contextual inquiry, work and experience modeling
  • Mixing methods
  • Log data and non-parametric statistics
  • Interaction analysis
  • Surveys in HCI
  • Mediation analysis

Schedule

All deadlines are EOD and CST

Week  1

Introduction to UX research

1/13

Read these chapters of Norman’s Design of Everyday Things if you haven’t taken any HCI classes before.

1/15

Structuring the research inquiry (Part 1)

Week  2

1/20

Structuring the research inquiry (Part 2)

1/22

**Class cancelled**

(due to extreme weather warning)

Week  3

Interviews and focus groups

1/27

Read Chapter 8 (Interviews and focus groups) from Lazar’s Research Methods in HCI

1/29

Paper discussions

Pradhan, A., Lazar, A., & Findlater, L. (2020). Use of intelligent voice assistants by older adults with low technology use. TOCHI27(4), 1-27.

Sun, Y., Jang, E., Ma, F., & Wang, T. (2024). Generative AI in the Wild: Prospects, Challenges, and Strategies. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-16).

1/28: Weekly reflection due

Week  4

Qualitative coding

2/3

Read Chapter 11 (Analyzing Qualitative Data) from Lazar’s Research Methods in HCI and One size fits all? What counts as quality practice in (reflexive) thematic analysis? by Braun, V., & Clarke, V. (2021).

2/5

Paper discussions

Yuan, Y., Riche, N., Marquardt, N., Nicholas, M. J., Seyed, T., Romat, H., … & Hinckley, K. (2022). Understanding Multi-Device Usage Patterns: Physical Device Configurations and Fragmented Workflows. CHI

Mahdavi Goloujeh, A., Sullivan, A., & Magerko, B. (2024). Is It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-13).

2/4: Weekly reflection due

Week  5

Design of experiments

2/10

Read Chapter 3 (Experimental Design) from Lazar’s Research Methods in HCI and Chapters 1 and 2 from Field’s Discovering Statistics using R

2/12

Project Discussions

Short project topic presentations and discussions

Week  6

Parametric statistics

2/17

Read Chapter 4 (Statistical Analysis) from Lazar’s Research Methods in HCI and Chapters 9 and 10 from Field’s Discovering Statistics using R

2/19

Paper discussions

Mahmud, S., Alvina, J., Chilana, P. K., Bunt, A., & McGrenere, J. (2020). Learning through exploration: how children, adults, and older adults interact with a new feature-rich application. CHI

Chattopadhyay, D., & Bolchini, D. (2014). Touchless circular menus: toward an intuitive UI for touchless interactions with large displays. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces (pp. 33-40).

2/18: Weekly reflection due

Week  7

Midterm project presentations

2/26

Interaction analysis

Read Chapter 13 (Measuring the human) from Lazar’s Research Methods in HCI and the paper on Interaction Analysis by Jordan and Henderson

Week  8

Midterm Exam

3/5

Paper discussions

Sakhnini, N., Yu, J. E., Jones, R. M., & Chattopadhyay, D. (2020). Personal Air Pollution Monitoring Technologies: User Practices and Preferences. In International Conference on Human-Computer Interaction (pp. 481-498). Springer, Cham.

Li, J., Cao, H., Lin, L., Hou, Y., Zhu, R., & El Ali, A. (2024). User experience design professionals’ perceptions of generative artificial intelligence. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-18).

3/4: Weekly reflection due

Week  9

Contextual inquiry, work and experience modeling

3/10

Read Chapters 7 (Building Experience Models) and 8 (Traditional Contextual Design Models) from Holtzblatt’s Contextual Design.

3/12

Paper discussions

Jones, A., & Thoma, V. (2019). Determinants for successful agile collaboration between UX designers and software developers in a complex organization. International Journal of Human–Computer Interaction35(20), 1914-1935.

Torres, C., Sterman, S., Nicholas, M., Lin, R., Pai, E., & Paulos, E. (2018). Guardians of practice: A contextual inquiry of failure-mitigation strategies within creative practices. Designing Interactive Systems Conference (pp. 1259-1267).

3/11: Weekly reflection due

Week  10

Mixing methods

3/17

Read Chapters 1 (Basic Characteristics of Mixed Methods Research) and 3 (Steps in Planning a Mixed Methods Study) from A Concise Introduction to Mixed Methods Research by John W. Creswell

3/19

Paper discussions

Tuncer, S., Brown, B., & Lindwall, O. (2020, April). On pause: How online instructional videos are used to achieve practical tasks. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-12).

Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020, April). Novice-AI music co-creation via AI-steering tools for deep generative models. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-13).

3/18: Weekly reflection due

Week 11

3/24—3/28

Spring Break

Week  12

Log data and non-parametric statistics

3/31

Read Chapter 12 (Automated data collection methods) from Lazar’s Research Methods in HCI and Chapter 15 from Field’s Discovering Statistics using R

4/2

Paper discussions

Sidenmark, L., & Gellersen, H. (2019). Eye&head: Synergetic eye and head movement for gaze pointing and selection. In Proceedings of the 32nd annual ACM symposium on user interface software and technology 

Matviienko, A., Müller, F., Schön, D., Seesemann, P., Günther, S., & Mühlhäuser, M. (2022). BikeAR: Understanding cyclists’ crossing decision-making at uncontrolled intersections using Augmented Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems.

4/1: Weekly reflection due

Week  13

Project discussions

4/7

Attendance is required. I will check in with students about their projects.

4/9

Paper discussions

Yu, J. E., Parde, N., & Chattopadhyay, D. (2023). “Where is history”: Toward Designing a Voice Assistant to help Older Adults locate Interface Features quickly. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems.

Grønbæk, J. E., Knudsen, M. S., O’Hara, K., Krogh, P. G., Vermeulen, J., & Petersen, M. G. (2020). Proxemics beyond proximity: Designing for flexible social interaction through cross-device interaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.

4/8: Weekly reflection due

Week  14

Survey

4/14

4/16

Paper discussions

Beck, D., Jung, J., Park, J., & Park, W. (2019). A study on user experience of automotive HUD systems: Contexts of information use and user-perceived design improvement points. International Journal of Human–Computer Interaction35(20), 1936-1946.

Elimelech, O. C., Ferrante, S., Josman, N., Meyer, S., Lunardini, F., Gómez-Raja, J., … & Rosenblum, S. (2022). Technology use characteristics among older adults during the COVID-19 pandemic: A cross-cultural survey. Technology in Society.

4/15: Weekly reflection due

Week  15

Mediation analysis and Meta analysis

4/21

Read Parts 1 (Introduction)  and 2 (Effect size and precision) from Borenstein’s Introduction to Meta-Analysis

4/23

No Class; Debaleena @NSF

Week  16

Project discussions

4/28

Attendance is required. I will check in with students about their projects.

4/30

Final project presentations

5/7: 3:30 – 5:30PM

Assignments

 

Please review the course’s collaboration policy before starting to work on assignments.

Mid-term project (20%): Research proposal, research design, and conceptualization of a chosen UX/HCI research question.

Final project (30%): Data collection, analysis, results, and discussion of a chosen UX/HCI research question.

Weekly reflections (30%): 

  • The Assignment: On weeks with assigned research papers, you must write a reflection (max. 500 words) connecting the reading (all the assigned papers) to the topics covered in class to date.

  • Submission: You must submit your reflection to Gradescope by the due date to demonstrate timely completion.

  • In-Class Review: Bring a physical copy to class. I will review your work specifically for conceptual correctness and understanding of the material.

  • Grading Criteria: Your official grade for this component is based on effort and completion.

Midterm exam (10%): The midterm exam will cover all course material through Week 6.

Final exam (10%): The final exam will be comprehensive, covering all material from the beginning of the course through the end of the semester.


Course policy on AI tools

You are welcome to use AI tools to support your learning in this course. When submitting assignments, you must acknowledge any AI tools you used to generate content or code. A simple citation of the tool(s) is sufficient—no explanation of how or to what extent you used them is required.

AI-generated code may be incorporated into your final projects, provided that you clearly indicate the tools used.

Because the boundary between language editing and content generation can be ambiguous, the guiding principle is transparency. If you are unsure whether or how to cite, ask the instructor.

Grading

Midterm exam
Final exam
Midterm project
Final project
Weekly reflections

10%
10%
20%
30%
30%

Letter grade base cutoff

89.5—100: A; 79.5—89.4: B; 69.5—79.4: C; 49.5—69.4: D; 0—49.4: F

Letter grades are determined at the end of the semester. The default cutoffs are provided above. These boundaries may be adjusted downwards if necessary because of the difficulty of the assignments or quizzes, but the boundaries will never be adjusted upwards, so a final average of 90 is guaranteed to be an A. The boundary adjustment is done heuristically, and there are no grade quotas, no grade targets, and no centering of the class on a particular grade boundary.

Attendance

Attending 60% of the class meetings is required. Students who do not meet this course requirement may be penalized up to 10% of their total grade. Students absent during a group presentation will receive a zero for that assignment. Assignment deadlines will be posted (and maybe updated) on the course page.

Policies

Week 2 Enrollment Policy. The official online waitlist for this course expires on Sunday, January 18. Starting Tuesday, January 20, registration will be restricted to “Instructor Approval Required.” If a seat opens up between Jan 20 and the add/drop deadline (Jan 23), you must email me directly to request an override.

Note: Approval to add the course in Week 2 is not guaranteed and generally requires that you have attended the first week of lectures to ensure you are not behind.

Attendance Policy. Class attendance is not always mandatory; however, research indicates that students who attend class are more likely to be successful. You are strongly encouraged to attend every class. Lectures may not be recorded and there may not be slides. If you are unable to attend class, you should consider asking a classmate to take notes for you.

Academic Misconduct. All students should aspire to the highest standards of academic integrity. Using another student’s work on an assignment, cheating on a test, not quoting or citing references correctly, or any other form of dishonesty, unauthorized collaboration, or plagiarism shall result in a grade of zero on the item and possibly an F in the course. Incidences of academic misconduct shall be referred to the Department Head and pertinent University officials and repeated violations shall result in dismissal from the program.

All students are responsible for reading, understanding, and applying the Code of Student Rights, Responsibilities, and Conduct and in particular the section on academic misconduct. Refer to UIC student affairs.

All students are strongly encouraged to read what constitutes plagiarism here and complete this short tutorial here. You must document the difference between your writing and that of others. Use quotation marks in addition to a citation, page number, and reference whenever writing someone else’s words (e.g., following the Publication Manual of the American Psychological Association).

Cheating. Cheating is an attempt to use or provide unauthorized assistance, materials, information, or study aids in any form and in any academic exercise or environment. A student must not use external assistance on any “in-class” or “take-home” examination unless the instructor specifically has authorized external assistance. This prohibition includes but is not limited to, the use of tutors, books, notes, calculators, computers, and wireless communication devices. A student must not use another person as a substitute in the taking of an examination or quiz, nor allow other persons to conduct research or to prepare work, without advanced authorization from the instructor to whom the work is being submitted. A student must not use materials from a commercial term paper company, files of papers prepared by other persons, or submit documents found on the Internet. A student must not collaborate with other persons on a particular project and submit a copy of a written report that is represented explicitly or implicitly as the student’s individual work. A student must not use any unauthorized assistance in a laboratory, at a computer terminal, or on fieldwork. A student must not steal examinations or other course materials, including but not limited to, physical copies and photographic or electronic images. A student must not submit substantial portions of the same academic work for credit or honors more than once without permission from the instructor or program to whom the work is being submitted. A student must not, without authorization, alter a grade or score in any way, nor alter answers on a returned exam or assignment for credit.

Fabrication. A student must not falsify or invent any information or data in an academic exercise including, but not limited to, records or reports, laboratory results, and citations to the sources of information.

Plagiarism. Plagiarism is defined as presenting someone else’s work, including the work of other students, as one’s own. Any ideas or materials taken from another source for either written or oral use must be fully acknowledged unless the information is common knowledge. What is considered “common knowledge” may differ from course to course. A student must not adopt or reproduce ideas, opinions, theories, formulas, graphics, or pictures of another person without acknowledgment. A student must give credit to the originality of others and acknowledge indebtedness whenever: directly quoting another person’s actual words, whether oral or written; using another person’s ideas, opinions, or theories; paraphrasing the words, ideas, opinions, or theories of others, whether oral or written; borrowing facts, statistics, or illustrative material; or offering materials assembled or collected by others in the form of projects or collections without acknowledgment

Interference. A student must not steal, change, destroy, or impede another student’s work, nor should the student unjustly attempt, through a bribe, a promise of favors, or threats, to affect any student’s grade or the evaluation of academic performance. Impeding another student’s work includes, but is not limited to, the theft, defacement, or mutilation of resources so as to deprive others of the information they contain.

Violation of Course Rules. A student must not violate course rules established by a department, the course syllabus, verbal or written instructions, or the course materials that are rationally related to the content of the course or to the enhancement of the learning process in the course.

Facilitating Academic Dishonesty. A student must not intentionally or knowingly help or attempt to help another student to commit an act of academic misconduct, nor allow another student to use his or her work or resources to commit an act of misconduct.

Right to revise. The instructor reserves the right to make changes to this syllabus as necessary and, in such an event, will notify students of the changes immediately.

Grievance Procedures. UIC is committed to the most fundamental principles of academic freedom, equality of opportunity, and human dignity involving students and employees. Freedom from discrimination is a foundation for all decision-making at UIC. Students are encouraged to study the University’s Nondiscrimination Statement. Students are also urged to read the document Public Formal Grievance Procedures. Information on these policies and procedures is available on the University web pages of the Office of Access and Equality.

Recording and Copyrights. Audio/Video Recording: To ensure the free and open discussion of ideas, students may NOT record or share classroom lectures, discussions, and/or activities without the advance written permission of the instructor, and any such recording properly approved in advance can ONLY be used solely for the student’s own private use.

Copyrighted Material: All material provided through this website is subject to Copyright and Fair Use laws. This applies but is not limited to class/recitation notes, slides, assignments, solutions, project descriptions, etc. You are allowed (and expected!) to use all the provided material for PERSONAL use. However, you are strictly prohibited from sharing the material with others in general and from posting the material on the Web or other file-sharing venues in particular.

Course Evaluations. Because student ratings of instructors and courses provide very important feedback to instructors and are also used by administrators in evaluating instructors, it is extremely important for students to complete confidential course evaluations online known as the Campus Program for Student Evaluation of Teaching evaluation. You will receive an email from the Office of Faculty Affairs inviting you to complete your course evaluations and will receive an email confirmation when you have completed each one.

For more information, please refer to the UIC Course Evaluation Handbook.

Results for the “six core questions” will be published on the UIC course evaluation website.

Copyright © 2024 Debaleena C., Ph.D.