UX Research Methods

CS 594

Course listed as Empirical Methods in HCC

Introduction to basic qualitative and quantitative methods—such as interviews, contextual inquiry, and design of experiments—to study user experience (UX) with computers and develop actionable insights. Both generative and evaluative UX research will be covered.

qualitative vs quantitative UX methods

Course Details

Tuesdays and Thursdays: 2:00 – 3:15 PM, at ERF 2068

By appointment

  • An introductory course in human-computer interaction or user interface design (e.g., CS 422 or CS 522). Or equivalent practical experience.
  • An aptitude to learn and apply social science concepts to computer science applications is required.
  • Some familiarity with basic statistics. 
Students are responsible for familiarizing themselves with the syllabus. The instructor is responsible for being responsive to the diverse needs of the enrolled students and making necessary modifications to this syllabus, which is to be treated as a living document. Blackboard will be used to post course grades. Piazza will be used to post announcements, FAQs, discussion points, course-related general queries, and detailed instructions when deemed necessary. The course webpage will contain links to readings, general instructions, and class schedules. Post general doubts publicly on Piazza and private matters anonymously (to classmates). Students are responsible to keep themselves updated with class announcements.

Except for extenuating circumstances and emergencies, please avoid using emails for class-related doubts/inquiries. Use Piazza.

Course Outcomes

  • Familiarity with basic qualitative UX research methods, like surveys, contextual inquiry, and 1:1 interviews.
  • Familiarity with basic quantitative UX research methods, like log analysis, hypothesis testing, and meta-analysis.
  • Design controlled experiments and identify independent and dependent variables.
  • Use parametric and non-parametric statistics to analyze experiments.
  • Understand how to use data for generative and evaluative computing research.
  • Identify and operationalize different facets of usability and user experience.
  • Understand, assess, and use appropriate metrics to measure user experience.
  • Synthesize the “so what” and surface fresh insights and opportunity areas from UX research.

Textbooks

You are not required to buy any of these books. Either they are available online via the UIC library or relevant chapters will be provided.

Schedule

All deadlines are EOD and CST

Week  1

Introduction to UX research

1/10

Read these chapters of Norman’s Design of Everyday Things if you haven’t taken any HCI classes before.

1/12

Structuring the research inquiry

Week  2

Class canceled due to faculty strike

Week  3

Interviews and focus groups

1/24

Read Chapter 8 (Interviews and focus groups) from Lazar’s Research Methods in HCI

1/26

Paper discussions

Pradhan, A., Lazar, A., & Findlater, L. (2020). Use of intelligent voice assistants by older adults with low technology use. TOCHI27(4), 1-27.

Mentis, H. M., Madjaroff, G., & Massey, A. K. (2019). Upside and downside risk in online security for older adults with mild cognitive impairment. CHI

1/25: Weekly reflection due

1/27: Project group signups due

Week  4

Qualitative coding

1/31

Read Chapter 11 (Analyzing Qualitative Data) from Lazar’s Research Methods in HCI and One size fits all? What counts as quality practice in (reflexive) thematic analysis? by Braun, V., & Clarke, V. (2021).

2/2

Paper discussions

Yuan, Y., Riche, N., Marquardt, N., Nicholas, M. J., Seyed, T., Romat, H., … & Hinckley, K. (2022). Understanding Multi-Device Usage Patterns: Physical Device Configurations and Fragmented Workflows. CHI

Dai, J., Moffatt, K., Lin, J., & Truong, K. (2022). Designing for Relational Maintenance: New Directions for AAC Research. CHI

2/1: Weekly reflection due

Week  5

Design of experiments

2/7

Read Chapter 3 (Experimental Design) from Lazar’s Research Methods in HCI and Chapters 1 and 2 from Field’s Discovering Statistics using R

2/9

Paper discussions

Nancel, M., Wagner, J., Pietriga, E., Chapuis, O., & Mackay, W. (2011). Mid-air pan-and-zoom on wall-sized displays. CHI (pp. 177-186).

Foley, M. J., Roy, Q., Huang, D. Y., Li, W., & Vogel, D. (2022). Switching Between Standard Pointing Methods with Current and Emerging Computer Form Factors. CHI

2/8: Weekly reflection due

Week  6

Parametric statistics

2/14

Read Chapter 4 (Statistical Analysis) from Lazar’s Research Methods in HCI and Chapters 9 and 10 from Field’s Discovering Statistics using R

2/16

Paper discussions

Satriadi, K. A., Ens, B., Cordeil, M., Jenny, B., Czauderna, T., & Willett, W. (2019). Augmented reality map navigation with freehand gestures.  IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

Mahmud, S., Alvina, J., Chilana, P. K., Bunt, A., & McGrenere, J. (2020). Learning through exploration: how children, adults, and older adults interact with a new feature-rich application. CHI

2/15: Weekly reflection due

Week  7

Project Discussions

2/21

Attendance is required. I will check in with students about their projects.

2/23

Paper discussions

Sakhnini, N., Yu, J. E., Jones, R. M., & Chattopadhyay, D. (2020). Personal Air Pollution Monitoring Technologies: User Practices and Preferences. In International Conference on Human-Computer Interaction (pp. 481-498). Springer, Cham.

Chattopadhyay, D., & Bolchini, D. (2015). Motor-intuitive interactions based on image schemas: Aligning touchless interaction primitives with human sensorimotor abilities. Interacting with Computers27(3), 327-343.

2/22: Weekly reflection due

Week 8

2/28 

3/2

Midterm project presentations

Week  9

Contextual inquiry, work and experience modeling

3/7

Read Chapters 7 (Building Experience Models) and 8 (Traditional Contextual Design Models) from Holtzblatt’s Contextual Design.

3/9

Paper discussions

Jones, A., & Thoma, V. (2019). Determinants for successful agile collaboration between UX designers and software developers in a complex organization. International Journal of Human–Computer Interaction35(20), 1914-1935.

Torres, C., Sterman, S., Nicholas, M., Lin, R., Pai, E., & Paulos, E. (2018). Guardians of practice: A contextual inquiry of failure-mitigation strategies within creative practices. Designing Interactive Systems Conference (pp. 1259-1267).

3/8: Weekly reflection due

Week 10

3/13—

Mixing methods
Paper discussions

Week 11

3/20—

Log data and non-parametric statistics
Paper discussions

Week 12

3/27—

Spring Break

Week 13

4/3—

Interaction analysis
Paper discussions

Week 14

4/10—

Project discussions
Survey

Week 15

4/17

Mediation analysis
Project discussions

Week 16

4/24—

Paper discussions
Paper discussions

Finals week

TBD

Final project presentations

Assignments

MS students must form a group for projects. Ph.D. students may opt to do a project as an extension of their dissertation research or join a group of peers. The maximum number of students allowed per group is 3.  Students can opt to continue their mid-term project for their final project or choose to do a new separate project.

Please review the course’s collaboration policy before starting to work on assignments.

Mid-term project (20%): Research proposal, research design, and conceptualization of a chosen UX/HCI research question.

Final project (30%): Data collection, analysis, results, and discussion of a chosen UX/HCI research question.

Sign up for projects here.

Weekly reflections (25%): Each week students will submit a reflection on assigned papers and what they learned about them from the topics covered in class. This narrative must not be more than 500 words. In the narrative, students must identify how the concepts covered in class manifest in the research papers, and provide a thoughtful critique. Points will not be assigned for mere summation. The four lowest grades will be dropped out of 10. 

Paper presentations (15%): Each student will present one research paper in this class. The purpose of these presentations is twofold. First, students should summarize and provide an overview of the paper and then lead a class discussion focusing on the research method(s) used. Papers will be chosen by the instructor. Papers scheduled for presentations will serve as assigned readings for that class.

Sign up for paper presentations here.

Class participation (10%): This is a graduate-level seminar course. Students are expected and required to participate actively in class discussions. Students will be asked to critique the mid-term and final project presentations of their peers, which will contribute to their class participation grade.

Grading

Mid-term project
Final project
Weekly reflections
Paper presentations
Class participation

20%
30%
25%
15%
10%

Letter grade base cutoff

89.5—100: A; 79.5—89.4: B; 69.5—79.4: C; 49.5—69.4: D; 0—49.4: F

Letter grades are determined at the end of the semester. The default cutoffs are provided above. These boundaries may be adjusted downwards if necessary because of the difficulty of the assignments or quizzes, but the boundaries will never be adjusted upwards, so a final average of 90 is guaranteed to be an A. The boundary adjustment is done heuristically, and there are no grade quotas, no grade targets, and no centering of the class on a particular grade boundary.

Attendance

Class attendance is not mandatory but strongly recommended. Attending 60% of the class meetings is required. Students not meeting this course requirement may be penalized up to 10% of their total grade. Students absent during a group presentation will receive a zero for that assignment. Assignment deadlines will be posted (and maybe updated) on the course page.

Lateness and Extensions

To give you some flexibility for periods of a heavy workload, minor illness, absence from campus, job interviews, and other occasional (but often predictable) circumstances, you may use limited extensions on course deliverables, called slack days. Each slack day is a 24-hour extension on the deadline. You have a budget of 10 slack days for the entire semester, which you may apply to any combination of individual assignments. You can use up to 4 slack days for a given assignment. Assignments more than four days late will not be accepted.

To use a slack day, just submit the assignment late. You DO NOT need to notify the instructor. When we grade your assignment, we will see that you submitted it late, and dock you the appropriate number of slack days in our records. You are responsible for keeping track of the slack days you’ve used. If you have used up your slack days or exceeded the 4-day limit for a single assignment, you will need an instructor’s permission for more extension.

I expect that almost all your needs for extensions will be handled by slack days. Only truly exceptional, extreme emergency cases will be considered for extensions after your slack days are exhausted. In particular, to receive said extension you will need to convince me that all ten of your slack days were used for reasons that would have justified an extension. So use your slack budget wisely. Finally, we want to emphasize that slack days are for emergencies. Plan to submit every assignment on the real due date, and only use a slack day or two if something unexpected comes up last minute. Do not treat slack days as pre-planned due date extensions. In particular,

if you are already using four slack days for an assignment and email me at the end of your fourth day requesting an extension for a fifth day, I will very likely turn you down.

Policies

Attendance Policy. Class attendance is not always mandatory; however, research indicates that students who attend class are more likely to be successful. You are strongly encouraged to attend every class. Lectures may not be recorded and there may not be slides. If you are unable to attend class, you should consider asking a classmate to take notes for you.

Academic Misconduct. All students should aspire to the highest standards of academic integrity. Using another student’s work on an assignment, cheating on a test, not quoting or citing references correctly, or any other form of dishonesty, unauthorized collaboration, or plagiarism shall result in a grade of zero on the item and possibly an F in the course. Incidences of academic misconduct shall be referred to the Department Head and pertinent University officials and repeated violations shall result in dismissal from the program.

All students are responsible for reading, understanding, and applying the Code of Student Rights, Responsibilities, and Conduct and in particular the section on academic misconduct. Refer to UIC student affairs.

All students are strongly encouraged to read what constitutes plagiarism here and complete this short tutorial here. You must document the difference between your writing and that of others. Use quotation marks in addition to a citation, page number, and reference whenever writing someone else’s words (e.g., following the Publication Manual of the American Psychological Association).

Cheating. Cheating is an attempt to use or provide unauthorized assistance, materials, information, or study aids in any form and in any academic exercise or environment. A student must not use external assistance on any “in-class” or “take-home” examination unless the instructor specifically has authorized external assistance. This prohibition includes but is not limited to, the use of tutors, books, notes, calculators, computers, and wireless communication devices. A student must not use another person as a substitute in the taking of an examination or quiz, nor allow other persons to conduct research or to prepare work, without advanced authorization from the instructor to whom the work is being submitted. A student must not use materials from a commercial term paper company, files of papers prepared by other persons, or submit documents found on the Internet. A student must not collaborate with other persons on a particular project and submit a copy of a written report that is represented explicitly or implicitly as the student’s individual work. A student must not use any unauthorized assistance in a laboratory, at a computer terminal, or on fieldwork. A student must not steal examinations or other course materials, including but not limited to, physical copies and photographic or electronic images. A student must not submit substantial portions of the same academic work for credit or honors more than once without permission from the instructor or program to whom the work is being submitted. A student must not, without authorization, alter a grade or score in any way, nor alter answers on a returned exam or assignment for credit.

Fabrication. A student must not falsify or invent any information or data in an academic exercise including, but not limited to, records or reports, laboratory results, and citations to the sources of information.

Plagiarism. Plagiarism is defined as presenting someone else’s work, including the work of other students, as one’s own. Any ideas or materials taken from another source for either written or oral use must be fully acknowledged unless the information is common knowledge. What is considered “common knowledge” may differ from course to course. A student must not adopt or reproduce ideas, opinions, theories, formulas, graphics, or pictures of another person without acknowledgment. A student must give credit to the originality of others and acknowledge indebtedness whenever: directly quoting another person’s actual words, whether oral or written; using another person’s ideas, opinions, or theories; paraphrasing the words, ideas, opinions, or theories of others, whether oral or written; borrowing facts, statistics, or illustrative material; or offering materials assembled or collected by others in the form of projects or collections without acknowledgment

Interference. A student must not steal, change, destroy, or impede another student’s work, nor should the student unjustly attempt, through a bribe, a promise of favors, or threats, to affect any student’s grade or the evaluation of academic performance. Impeding another student’s work includes, but is not limited to, the theft, defacement, or mutilation of resources so as to deprive others of the information they contain.

Violation of Course Rules. A student must not violate course rules established by a department, the course syllabus, verbal or written instructions, or the course materials that are rationally related to the content of the course or to the enhancement of the learning process in the course.

Facilitating Academic Dishonesty. A student must not intentionally or knowingly help or attempt to help another student to commit an act of academic misconduct, nor allow another student to use his or her work or resources to commit an act of misconduct.

Right to revise. The instructor reserves the right to make changes to this syllabus as necessary and, in such an event, will notify students of the changes immediately.

Grievance Procedures. UIC is committed to the most fundamental principles of academic freedom, equality of opportunity, and human dignity involving students and employees. Freedom from discrimination is a foundation for all decision-making at UIC. Students are encouraged to study the University’s Nondiscrimination Statement. Students are also urged to read the document Public Formal Grievance Procedures. Information on these policies and procedures is available on the University web pages of the Office of Access and Equality.

Recording and Copyrights. Audio/Video Recording: To ensure the free and open discussion of ideas, students may NOT record or share classroom lectures, discussions, and/or activities without the advance written permission of the instructor, and any such recording properly approved in advance can ONLY be used solely for the student’s own private use.

Copyrighted Material: All material provided through this website is subject to Copyright and Fair Use laws. This applies but is not limited to class/recitation notes, slides, assignments, solutions, project descriptions, etc. You are allowed (and expected!) to use all the provided material for PERSONAL use. However, you are strictly prohibited from sharing the material with others in general and from posting the material on the Web or other file-sharing venues in particular.

Course Evaluations. Because student ratings of instructors and courses provide very important feedback to instructors and are also used by administrators in evaluating instructors, it is extremely important for students to complete confidential course evaluations online known as the Campus Program for Student Evaluation of Teaching evaluation. You will receive an email from the Office of Faculty Affairs inviting you to complete your course evaluations and will receive an email confirmation when you have completed each one.

For more information, please refer to the UIC Course Evaluation Handbook.

Results for the “six core questions” will be published on the UIC course evaluation website.

Copyright © 2023 Debaleena C., Ph.D.