Scope
An Intelligent Interface for Analysts using Natural Language Understanding (NLU)
Overview
Spring 2021
What is the product? A prototype of an intelligent desktop interface powered by a Knowledge Graphs and Natural Language Understanding (NLU)
Who is the user? Mid-level Intelligent analyst at The Laboratory for Analytic Sciences (LAS)
What is the purpose? The intelligent interface enables the analyst to collaborate with a knowledge graph to understand relevant data and forge useful insights during an investigation.
The project played out in three primary areas:
- Visualizing the knowledge graph interface
- Creating user interactions with the knowledge graph
- Incorporating machine learning capabilities into the user experience in order to alleviate the identified pain points.
My Role
UX Designer
Team
3 Designers: Maryam Nadali, Casey Stanek, Eric Pryor
Duration
9 weeks
Tools
Miro, Figma, After Effects, Illustrator
Client
Solution
The following is the scenario and video of the final solution showcasing a specific path through our desktop program, emphasizing the features of the application and their advantages over current methods of conducting investigations.
Scenario:
Kari has been working on a Request for Information (RFI) regarding the 2015 Bundestag hack. She received new information this morning from the German federal police that there is a connection between the 2015 phishing campaign and suspect Dmitry Badin. Kari needs to know more about how Badin got involved, and who may have hired him.
What are the problems?
- Searching relavent data is time-consuming
- Intelligent Analysts are using old fashion note taking software similar to OneNote to gather their findings.
- They need to organize their data all the time without making any mistake.
- Intelligent Analysts need to be vigilant during their investigation
- Sometimes avoiding biases during an investigation can be hard for analysts.
Why is it important?
Analysts need to work and find data fast for each investigation to report their findings to the superior levels. The intelligent aspect of the designed interface can help them to find data faster, organize data and help them to avoid making bias decisions during an investigation.
Empathize
Initial KG Research
At the beginning of this project we had Knowledge Graphs (KG) crash course with an expert in Machine Learning.
What is a knowledge graph? “A knowledge graph, represents a network of real-world entities—i.e. objects, events, situations, or concepts—and illustrates the relationship between them. It is made up of three main components: nodes (object, place or person), edges (defines the relationship between the nodes), and labels. This information is usually stored in a graph database and visualized as a graph structure, prompting the term knowledge “graph.”
Research and Interview
Before the user interview, we did extensive research on analyst’s workflow from academic paper and the internet. Based on our assumptions we made a user task flow.
We had one hour interview with three intelligent analysts from LAS. During the interview we walk them through the user journey map and we used card sorting to understand better their work flow and their potential problems.
Define
Persona
LAS analysts wanted us to focus on a mid level analyst. Kari is our ideal persona for a mid level analyst.
As-is User Journey Map
The interview and our research helped me to refine the “as is”user journey map and eventually found out their pain points as a group.
Ideate
Benchmarking of data visualization
Each member of the team looked for data visualization and information-compiling visuals that inspired us.
We used these mood-boards to inspire our initial explorations.
Ideation Workshop
We then had some ideations workshop such as Machine learning cards and What if ideation. This helped us to explore more ideas for our prototype.
Sketches
Each one of us in the group start sketching ideas. I came up with the Natural Language Understanding (NLU) concept where the user can see the text version of the knowledge graph. The system can also create a report for the user.
We then made three different sketches of tasks flows and presented them to our stakeholders for feedback. LAS’s feedback was invaluable in seeing what concepts had the most merit and feasibility. My idea of incorporating NLU into the interface concept generated quite a bit of conversation and positive feedback from our stakeholders.
Below are my sketches of the NLU idea task flow.
Prototype and Testing
High-fidelity Iteration and User testing
The previous draft of sketches positioned our team to hone in on two ideas (NLP concept and Multimodal environments concept) that would best meet the needs of mid level analysts. Given that, we each took one of our most successful, well-received ideas and pushed it further in a high-fidelity direction.
I pushed the NLU concept further, prior to our next conversation with LAS, and numerous elements of this iteration made their way into our final concept.
The video is one iteration of the interactive prototype.
We had several critiques and user testing with our skateholder in each iterations. Through out the process, as a team, we had numerous sessions to discuss combining the strongest elements of our ideas and the feedback we got from LAS to create the final design.
Below, are some snapshots of different iteration of the NLU interface.
Think aloud
Before the final delivery we had a user testing with one of our clients. We ask our user to think aloud while using the prototype. The result of this test helped us to understand what was missing and to refine the final prototype.
Final Delivery
Scenario:
Kari has been working on an RFI regarding the 2015 Bundestag hack. She received new information this morning from the german federal police that there is a connection between the 2015 phishing campaign and suspect Dmitry Badin. Kari needs to know more about how Badin got involved, and who may have hired him.
Scope's Features
The main touchpoint of the prototype are presented below:
To-be User Journey Map
Wider Implications
- Make the hunt for data as simple as possible using ML so the analyst can spend more time analyzing.
- Enables the analyst to intuitively interact and organize data using natural language.
- Incorporating ML into this experience provides rich opportunities for identifying and responding to analyst bias.
Reflection
- In addition to using subvocalization to enhance data security, how else might AI be leveraged to support security?
- Could the AI filter out any non-compliant information to protect the analyst?
Uh oh, you have reached the end of this case study!
Do not worry! I have other projects that are waiting for you to check them out!
Thank you for your time and see you in the next project :)
