User Survey DataDive

AI Project Engine Emergent Skill-Builder Projects
, , , ,
1

UX Research, User Survey and Data Collection for AI Code Assistants

Objective

The goal of this project is to provide students with a comprehensive understanding of user experience (UX) research fundamentals, with a specific emphasis on user interviewing and persona building. However, the main focus of this project is to hone the students’ skills in User Survey and Data Collection, essential competencies in various STEM careers, especially when dealing with AI and machine learning projects.

In this project, students will delve into the world of Generative AI-powered code assistants, such as AWS CodeWhisperer, GitHub Copilot, or Google Duet AI. They will work towards understanding the user’s experiences, motivations, needs, and challenges when interacting with these cutting-edge tools.

Students will have the opportunity to conduct user interviews, collect valuable data, and analyze this information, gaining first-hand experience in qualitative research. The process doesn’t stop at understanding user behaviors. Students will translate their findings into actionable design personas, providing a human face to the collected data. These personas will then serve as structured inputs for subsequent stages involving Machine Learning analysis and Full Stack development.

The end goal is not just to develop an understanding of UX fundamentals but also to leverage these skills to generate data-driven insights. These insights will then inform the design of AI code assistants and the development of a comprehensive web application showcasing the results. In essence, students are not just learning, but also actively contributing to the advancement of AI technologies.


Learning Outcomes

Upon successful completion of this project, students will:

  1. Comprehend the vital role and value of user interviews within UX research, and how they fit into a broader data gathering framework.
  2. Acquire practical skills in planning, conducting, and synthesizing user interviews.
  3. Develop proficiency in transforming user interview data into structured information that could be used for further data analysis.
  4. Understand empathetic design principles and learn how to convert user insights into design considerations.
  5. Enhance their communication skills, especially in articulating and presenting UX research findings to a broader team.

Steps and Tasks

1. Familiarize Yourself with UX Principles

Begin this journey by familiarizing yourself with what UX is all about. Also focus on how UX fits into a broader project framework involving data analysis and application development.

What is UX Research is an article by the Interaction Design Foundation that explains the basics of user experience research and its role in uncovering problems and design opportunities.


2. Crafting the Interview Guide

Students will commence the project by defining clear objectives for their user interviews. The focus should be on understanding user experiences when interacting with AI-powered code assistants, including their needs, frustrations, and motivations.

Using their defined objectives, students will create an interview guide, a roadmap of targeted questions aiming to garner insights linked directly to their goals. Refer to How to Write an Interview Guide for guidance.

:stemaway_bulb: While creating the interview guide and online survey, students should consider how the data gathered will be utilized in the subsequent stages of the project. This may involve creating a clear structure that allows the collected data to be easily organized and analyzed.


3. Conducting Mock Interviews and Distributing Online Surveys

To gain practical experience, students may conduct 3 or more mock interviews either with fellow students, volunteers, or participants found via online forums. Simultaneously, they can disseminate their online survey to a broader audience to gather more user responses. Refer to How to Conduct User Interviews for guidance.

:stemaway_bulb: In addition to conducting the interviews and disseminating the surveys, students should ensure they accurately record and structure the data gathered during these exercises. This process will allow for seamless integration into the data analysis stage.



Steps 4, 5, and 6:

These stages will become more meaningful when dealing with a larger dataset, which you will have access to during the team-based virtual internships. For now, proceed through these steps to gain a foundational understanding of the processes.


4. Data Transcription and Analysis

After conducting interviews, students will transcribe or take comprehensive notes, creating a rich dataset for analysis. They will then employ a systematic approach to analyze this data, identifying recurring patterns, common themes, and key insights that may impact future AI code assistant design and functionality.

The analysis phase will involve categorizing data into several key aspects related to user experience, such as:

  • Usage Patterns: Students will look for clues indicating when and how users interact with AI-powered code assistants. Information such as the frequency of use, duration of interaction, and preferred time of day can provide meaningful context for understanding user habits. This information will not only shape the development of user personas but can also inform the design of user journey maps, offering insights into what a typical user may be doing before, during, and after interaction with an AI code assistant.
  • User Challenges and Frustrations: A critical aspect of the analysis will involve identifying common challenges or frustrations users experience when interacting with AI code assistants. This could include the assistant’s inability to handle complex or multiple queries, latency issues, or lack of integration with other platforms. Understanding these pain points is crucial for developing realistic user personas and identifying areas for design improvement.
  • User Needs and Feature Requests: Students will also pay close attention to users’ expressed needs and feature requests. These might reveal unmet needs or potential enhancements that could significantly improve the user experience. For instance, a user’s request for better integration with other services or a feature allowing the assistant to remember past interactions could hint at a user persona that values seamless integration and continuity in their digital experiences.

This thorough analysis will ensure that the collected data is put to optimal use, informing the design and development of more user-centered AI code assistants. Furthermore, it will provide the foundation for the subsequent machine learning analysis and the development of an interactive AWS web app to visualize the findings.


5. Building Personas

User personas allow us to critically synthesize research findings and construct a fictional, yet realistic, representation of our target stakeholder to guide our UX design decisions throughout the project.

Drawing from their interview analysis, students will construct detailed personas representing different user segments of AI-powered code assistants. These personas should encapsulate demographic information, goals, behaviors, needs, and pain points. Refer to User persona examples for guidance.


6. Creating User Journey Maps

Journey Maps allow us to visualize the process that a person goes through in order to accomplish his or her goal and pinpoint opportunities for design intervention.

For each developed persona, students will craft a user journey map illustrating the persona’s interaction sequence with AI-powered code assistants. This map should highlight significant interactions, emotional touchpoints, and opportunities for user experience enhancement. Refer to Journey map examples for guidance.


Evaluation

Self-Evaluation Criteria

For your self-assessment, please consider the following points:

  1. Research planning: Reflect on the clarity and appropriateness of the research question and the data collection plan you’ve developed. Was the data collection plan feasible and likely to yield valuable insights?

  2. Data collection and handling: How well did you execute the data collection process? Did you comply with ethical standards? Did you face any difficulties during the process, and how did you overcome them?

  3. Data analysis: Were you able to extract meaningful insights from the data? What data analysis skills did you learn or improve upon during this project?

  4. Insight application: Reflect on your ability to make relevant design recommendations based on the analysis. Were your recommendations grounded in the data collected?

  5. Presentation: Assess the clarity and persuasiveness of your presentation of findings and recommendations. Were your findings presented in a way that non-technical stakeholders could understand? Did your presentation effectively communicate the value of your research?

STEM-Away Evaluation for Microcredentials and Virtual-Internship Eligibility

To qualify for STEM-Away microcredentials or to be considered for our virtual internships, your evaluation will include either a live submission or a video recording. This should involve:

  • Demonstration of Key Insight: Present any one key insight. This demonstrates your understanding of UX research principles and your ability to apply them in a real-world context.

  • Interview Guide: You can submit a preliminary interview guide to join the team. Do the best you can with this initial submission. Once in the program, you’ll collaborate with your team to refine and finalize the guide.


Resources and Learning Materials

  • What is UX?
    An article by the Interaction Design Foundation that explains the basics of user experience research and its role in uncovering problems and design opportunities.

  • UX Research Methods by the Nielsen Norman Group.
    A comprehensive guide on UX research methods from the Nielsen Norman Group, a globally recognized UX research consulting firm.

  • Secondary Research
    An insightful article from dscout’s People Nerds blog that emphasizes the importance of secondary research in forming a foundational understanding of the research domain before moving on to primary, generative research.

  • How to Write an Interview Guide
    A detailed resource from the Nielsen Norman Group on crafting effective interview guides for user interviews.

  • Generative Research Guide
    A complete guide from dscout’s People Nerds blog that explains the concept of generative research, its importance, and how to conduct it.

  • Primary vs Secondary Research
    An article from Guide2Research that contrasts primary and secondary research, detailing when and why each type is used.

  • How to Design a Survey
    A guide from Pew Research Center that provides best practices and methodologies for designing effective surveys.

  • How to Conduct User Interviews
    Another valuable resource from the Nielsen Norman Group that shares effective strategies and techniques for conducting user interviews.

  • Affinity Mapping
    A resource by the Nielsen Norman Group that explains what affinity mapping is, and how it can be used to organize and analyze complex data in UX research.


This project aims to set the groundwork for students’ final internship by equipping them with a solid foundation in user interviewing and persona building, and their crucial role in the design and enhancement of a user-centered AI-powered code assistant.


STEM-Away® Recordings