Elementary Robotics AI is the machine learning platform for web app. Use the model development to train custom inspections for QA.

My Role

Lead UX/UI designer for creating the flow of no code machine learning platform with ML team, Web team, CTO, CEO.

 

Research, Prototype, User testing, wireframe, info architecture, UI design, 

Results

Shipped the web app with a few main functionalities:
1. Configure the ML model.

2. Label and train the model.

3. Deploy and Evaluate the model.

Handed users the tool to create and train their own machine learning models.

Provide a no-code experience for a broader audience.

Group 196.png

What Is It

Elementary’s easy-to-use software, deep learning AI, and camera systems are built to capture visual data, deliver fast and reliable real-time judgments.

This No-code machine learning platform lets the user capture their inspection routines and label, train, deploy their customized deep learning models.

sidePanelCamera.9acd24fb.png
IR01.85.png
Group 1016.png
09_bg.png

User Types

Inspection managers or leads are the main types of user who interacts with the no-code machine learning platform.

Manager

Their goal is to create a customized inspection routine for each object and train the ML model to catch defects.

 

After the models are deployed to different stations, they need to supervise each station from a distance. 

science-in-hd-i4ABHj811N0-unsplash.jpg

User Flow

1

Capture Data

Configure

Data

2

Label & Train ML Model

3

Deploy

ML Model

4

5

Evaluate

​ML Model

Gray Structure

1. Capture Data

- Create inspection routines and images for various objects with different hardware platforms.

- Use the inspection routine that was created to collect images from a set of objects to provide enough data to label and train the model.

Challenge

Users used to only care about which area contains what defect. So they preferred to see results get organized by areas of interest. But they will assign multiple machine learning models to an area of interest. For example, they might look for a missing cap and some printed text in the same area. Each ML model also has different labeling requirements. They also get trained individually. How do we design the info architecture to meet both needs?

Solution

We guided our user to create and label multiple inspections on one area of interest during input phase.

But we delivered inspection results on the area of interest bases. 

Gray Structure

2. Configure Data

- Create bounding boxes around areas of interest on each position in a routine.
- Classify areas of interest according to the type of defects users are looking for. 
e.g. Barcode, Text, etc.
- Create alignment fixtures to tell the machine which orientation to read the image.
- We named the end result of this step as
Inspections.

This is how users add an inspection to an area of interest.

Challenge

Our customers will configure the threshold to decide how strict they want the AI model to perform. They need some product data to support in making that decision. A ROC chart is used in the machine learning world to show false positives and true positives. But it is a hard concept for our users to understand.

Solution

We converted the ROC value into defective shipped and excess scrap which made the data relatable for our user's production line. When the threshold value goes out of range, it would cause the production line also showed the suggested range for the threshold with a green bg. 

Gray Structure

3. Label & Train ML Model

3. Label & Train ML Model

3. Label & Train ML Model

- Label each tool with the image set that was captured in step 1. (Each label will contain results of Pass, Fail or Unknown. This is an essential step for our users to teach ML model how to find the right defects.)
- Once a certain number of image sets are labeled, users can hit the “train” button.
- Once the model finishes training, an entire inspection routine is created and ready to go.

Gray Structure

Challenge

We found out that the labeling process is focus intense. User often asks how many more data do they need to label. We want to respect the focus that and guide them through the process with less confusion.

Solution

We decided to break down label requirements into 4 small labeling goals. In each goal, we provide a metrics panel to indicate how many labels needed clearly. 

4,5. Deploy & Evaluate

- Deploy different versions of the ML model to inspection stations.

- Users will be able to monitor each inspection routine and see their performance on the production line.

And we launched! Check out the Elementary Robotics for more info about the product.

Gray Structure

Learning

01. The importance of Info Architecture

02. When handing users a complex ML training tool, more guidance and encouragements are always welcomed.
    

03. Accept imperfection. The MVP of the product can barely achieve awesomeness. How to work with the team and provide the best customer value is the key.

Thanks for viewing.😀