How AI Has Impacted the Daily Lives of UX Professionals in the Discovery Process: Insights from Our Experts Giovani, Rayssa, and Talita from the Internal Systems Team.
Artificial Intelligence (AI) is a topic gaining attention across various domains. Here, the primary question we’ve been exploring is: How can we integrate AI into our daily lives as UX professionals?
This question prompted deep reflection, leading us to reconsider our entire workflow. Recognizing that evolution is constant and rapid, it’s crucial to note that everything we share here is still in development and evolution.
In this text, we’ll delve into our experience using AI in our processes, discussing challenges, key considerations identified during tests, results, and our conclusions. We believe that AI can be a powerful ally, directly impacting the working style of those engaged in Discovery. We are excited to share our initial findings with you.
Implementing AI in Discovery.
With the intrinsic design thinking, we mapped the main pain points in the UX process, formulated hypotheses for possible AI applications, conducted our secondary research, and initiated tests. It’s important to note that a complete Discovery was not conducted; rather, it was a study with tests to implement AI according to our needs. Below, we provide more details about each stage.
Challenges of the Discovery Process
The first stage of our study was to identify the main challenges we face in the Discovery process. Through research, interviews, and activities, we identified the following points:
Desk research: Depending on the context, data collection and analysis can take longer than planned.
Benchmarking: Identifying competitors’ strengths and weaknesses and competitive differentiators can sometimes become a complex stage.
Surveys: They take time not only to create but also to adapt to the tone of voice, minimize biases, and validate with the team.
Data analysis: By far the most costly, studying all the data obtained in the collection requires dedicated time for both analysis and artifact construction.
Based on these points, it’s easy to see that the traditional Discovery process can be time-consuming, costly, and prone to errors (after all, to err is human).
These challenges led us to another question: What is the potential for productivity gain, effort reduction, and savings with the automation of processes using AI? The answer to this question lit up the eyes of many, and we’ll present it to you here, so keep reading
Our Points of Attention Regarding Security and Privacy
Before starting the tests, to ensure the privacy of participants and the integrity of information in our processes, we established some explicit policies:
- Data preprocessing: We removed sensitive personal information, such as names, emails, and phone numbers, before starting the tests.
- AI usage policies in research: We established clear guidelines for the responsible use of AI in data analysis, involving other teams in compliance.
These measures are crucial to protect participant confidentiality and ensure the integrity of our processes; however, if we had an internal tool, this step would not be necessary. Now that we have outlined the foundations, it’s time to explore the tools and tests we conducted.
The AI tools we tested
We decided to redesign our Discovery process and map the stages with templates to integrate intelligence. To achieve this, we identified the most commonly used stages and practices and researched AI tools capable of automating them.
To assess the impact of AI on the Discovery process, we tested specialized and generic AI tools and approaches. Our goal was to identify opportunities to automate tasks and enhance the efficiency of the process.
Questionnaire generation
To automate survey generation, we tested the following tools:
- Pollfish: A specialized tool that enables the generation of complete surveys in a few seconds and features a user-friendly interface (which is its major advantage).
- ChatGPT and Bard: Generic tools that can also be used to generate surveys but require more technical knowledge.
AI can generate complete surveys in just a few seconds, already with the defined tone of voice. The significant advantage is not having to start from scratch, as it is easier to adjust than to create, making the process much faster.
Data analysis
For data analysis, we tested two tools: Luminal and ChatGPT:
- Luminal: A tool specialized in data analysis. It can analyze a large volume of data (and it’s quite substantial, up to 1 million rows!) with an intuitive, user-friendly interface and without the need for technical knowledge.
- ChatGPT: It can also be used for data analysis. In the paid version, it has a feature for importing files in various formats, including Excel, while the free version does not.
To conduct the test with the free version of ChatGPT, we used the Aspose.Cells tool to convert an Excel file into a text file. The process was simple and quick, taking less than 10 seconds to format the data in a way that ChatGPT could generate the analyses.
Luminal and ChatGPT have different features, but both can identify patterns and correlations in large datasets. This helps to better understand user needs and behaviors, leading to valuable insights.
In the analyses we conducted, where the data volume was low, the results obtained were similar. However, it’s important to note that Luminal is a more comprehensive and powerful tool for data analysis, especially when it comes to quantity, while ChatGPT is a more accessible and creative tool.
RTB dynamics
In addition to tests for automating tasks, we also experimented with AI within a dynamic setting.
The RTB (Rose, Thorn, Bud) dynamic is a tool from the Luma Institute that we frequently use. The only challenge we faced was dividing the meeting into two sessions due to its stages.
On the first day, we gathered information about what was working, what wasn’t working, and what could be an opportunity. After filling out the post-its, we needed to group this data by similarity. Then, we concluded by applying an affinity map, meaning we labeled the clusters and generated insights by proposing solutions or improvements.
From experience, we knew that the RTB dynamic was time-consuming. Therefore, we thought about using AI to speed up the process. The idea was to use AI to group the post-its after filling them out, allowing us to continue the dynamic on the same day without sacrificing the quality of discussions and insights.
To ensure that the AI could arrange the grouping as expected, we conducted numerous tests until we arrived at the ideal prompt model for use in dynamics. The model, functional and with quality results, delivered the following topics:
- A brief context (about the RTB dynamic);
- Provide the data and describe what we want to analyze;
- An example with the expected result;
- Limitations on what the AI cannot do (adjust, add, or delete information) to avoid hallucinations.*
According to Alisson Solitto, an Architect here at BRQ and a participant in the Strategic Innovation House,
[…]Due to the probabilistic and generative nature of AI models, they can ‘predict’ outcomes that are not necessarily based on the information in their data but rather on the patterns they identify. These imagined productions are often referred to as ‘hallucinations.’Hallucinations can lead models to categorize information incorrectly, omitting and including others that are not real in our scenario. To correct these hallucinations, we should enrich the prompt by providing more detailed context and making clear the instructions to be followed.
It’s important to note that the result doesn’t always come in the same format; in fact, in these tests, Bard was the one that responded the best.
In addition to saving time and effort in grouping (it was able to group the data consistently with what we would do manually), the AI was able to label, generate insights, and make recommendations for each group. This allowed the dynamic to be carried out in a single day without any loss of quality.
The results of our tests.
As mentioned earlier, one of our major pain points was time; lengthy research processes incur high costs. After studying, testing, and having the opportunity to implement, we managed to alleviate these challenges by utilizing AI, providing speed in the process, generating cost savings, and maintaining the quality of analyses (regardless of the data quantity). Below, we provide a more detailed breakdown of these results.
To assess the impact achieved with the use of AI, we mapped the time required to perform the same processes as in previous research, for comparison.
Previously, each step was conducted by three UX professionals. Now, with AI, we operate individually in the same stages due to the support it provides. This impacts the cost savings associated with the Discovery process.
When we compared the results, we observed a significant impact. On average, AI tools reduced the working time for generating questionnaires and data analysis by 11 and 8 hours, respectively. This represents a productivity gain of over 200% and a savings of over 60%.
A crucial point is that previous research had an average of 30 to 40 participants. If the number of participants were higher, the analysis time would be equivalent. In other words, with the assistance of AI in these analyses, considering scalability, the productivity gain would be even greater.
Conclusion and Next Steps
Artificial intelligence is already a reality, and doubts are common. Every day, new tools emerge, and filtering out what makes sense (or not) to use can be a challenging but worthwhile task. Tests with AI tools have proven that this technology has the potential to enhance the efficiency and quality of processes.
With AI, we can accomplish much more in less time, adding value and quality to our deliveries. It’s not about replacing professionals but amplifying their capabilities and discovering new possibilities. With AI, we can uncover more opportunities and drive the company’s strategy, making a significant impact on business.
Based on the test results, we will continue to explore the potential of AI in our Discovery process. We want to test other AI tools to identify the best solution for our context, both in creation and analysis processes and dynamics.
Here at BRQ, Casa Estratégica, innovation (Generative AI) is one of our pillars. Our goal is to be a reference in the use of AI in business and customer technology as well as throughout BRQ.
We already have some concrete results in this direction, such as ‘ChatIABRQ’ (unofficial name), a still-nascent chat in a private environment that uses OpenAI technology and already allows some tests for simple tasks, such as creating questionnaires, grouping, and analyzing data. The chat is undergoing tests and will soon be available to all BRQ professionals.
As it is personalized, we don’t need to worry about cleaning sensitive data, internal processes, business rules, or any other type of information we are sharing, which further facilitates our steps, generating greater security.
The choice of the best AI tool depends on the specific needs of each team. For teams that have an internal AI tool, such as ‘ChatIABRQ,’ it is an excellent option, but it’s essential to test and evaluate different options to identify the tool that best fits your needs.
Who are we?
We are the UX Chapter, and we operate within BRQ’s Internal Systems, a group of professionals with similar roles. Weekly, we discuss market issues, career development, technology, among other topics, aiming to identify initiatives that add UX value to the company.
__
This article was written by:
Giovanni Alvarenga Silva, Agilist
Rayssa Maria De Melo Viana, UX
Talita Lucianetti, UX
