Join us on April 7-9, 2021

The first industry event focused on data engineering

Register Today
All Blog Posts

A Data Preparation Solution that Speaks a User’s Language

July 23, 2018

Our design team at Trifacta is committed to making the data preparation process more intuitive, accessible and effective for people who work with data. We know that preparing data can be a complex activity involving a large and diverse set of tasks, often spanning different projects, people and departments. As a design team, our goal is to provide users with a powerful set of functionality to accomplish their data transformations in an intuitive and painless workflow. However, as we introduce more and more functionality, the expanse of all available transformations can be sometimes overwhelming to navigate, especially for new users.

In the latest release, we have introduced a series of features with the specific goal of improving the discoverability of transformations within the product. Familiar entry points and intuitive interfaces within the product now enable users to easily and quickly accomplish their tasks.

The design process to improve our tool has involved three main steps: 
  • First, we have gone through a review and re-organization of our transformations, informed by close engagements with our users. 
  • We have then leveraged this new organization and built new transformation interfaces on top of it, focusing on how users think about and relate to transformations. 
  • Finally, we have introduced new entry points to let users easily access and familiarize themselves with our ever-growing set of transformations.

Creating a taxonomy of data transformation

One of the biggest design challenges with current data preparation solutions is the co-existence of many types of users with different backgrounds, skills and needs. Analysts with established Excel experience have different mental models and they accomplish their tasks in different ways then, someone with a strong SQL background. Our mission at Trifacta is to make data wrangling accessible to a larger group of users who have challenges getting data ready for use. But how do we achieve such a goal? How do we build a product that resonates with and satisfies such different types of users with varying mental models?

Not an easy task. Luckily design provides us with the appropriate tools to help tackle this challenge. User interviews, card sorting and tree testing are some of the valuable techniques we can use to better understand our users and quickly iterate on ideas and prototypes to improve both our information architecture and user interface.

We asked our users to help us to better organize our data transformation tasks and answer questions such as: are the names of our transformations understandable? Are transformations categorized in a user-centered manner? Are transformation titles distinguishable from one another? We provided them with a list of transformations and asked them to associate them with one (or more) categories. We then refined our categories and kept iterating on them until we reached an acceptable consensus on the categories.

The table below summarizes one of the card sorting exercises we conducted. We asked our users to associate data preparation tasks (on the left) with a specific category (on the top). The numbers represent the number of times users associated a task with a category – also, darker the color, more the users who made that association. This and other exercises helped us to define and refine a global categorization of transformation tasks that our users could understand.

An example of card sorting we used in our design process

Task-oriented interfaces

Once we have defined a consistent taxonomy for our transformations, we moved on to the creation of new interfaces that allow our users to easily accomplish their tasks with minimum effort.

At this point, we were presented with a big challenge. As you may know, our data preparation process leverages Wrangle: our domain specific language for transformation we have continually refined over the last six years. Wrangle is essential to our architecture as it abstracts the data preparation workflow our users develop in the application from the underlying data processing framework that executes that workflow. Given we support multiple different processing frameworks and these frameworks are constantly changing due to advances in software/hardware, this singular abstraction layer is critical for consistency and portability. However, the language itself does not necessarily reflect the user’s mental model when thinking about and authoring transformations. Moreover, we know that learning a new language can be challenging for some of our core users.

The challenge we had to face was then to create user interfaces that speak our users’ language and provide them with meaningful, powerful and intuitive experiences, while, at the same time, relying on our specific transformation language. This has been an incredibly exciting challenge for both our design and engineering teams. We have worked closely together and tried several approaches, focusing on our user’s needs and constantly testing and iterating on our ideas. After several attempts (and a couple of failures) we have finally been able to define a system to create interfaces on top of our language. The result is the definition of task-oriented interfaces, based on our powerful language but optimized to solve specific data transformation tasks, focusing on what really matters: our users and their needs.

Here’s an example of how these interfaces work. Imagine the following task a user needs to accomplish:

When you look at the underlying language to solve that task, this is what it looks like:

The language has its specific syntax which introduces concepts that do not necessarily resonate with our user. However, by focusing on the user’s mental model, we can build interfaces that are easy to understand and to use, that map naturally to the user’s language rather than ours:

An interface that closely maps to the language

An abstracted task-oriented interface

 New entry points: Toolbar and Search

Once we have defined our taxonomy and built interfaces to let users accomplish their tasks, we wanted to provide them with easy ways to both access and familiarize themselves with our set of transformations.

We worked on two main entry points: a toolbar and a new search input. Toolbars are a common and effective design pattern to organize and expose common functionalities in many desktop and web tools (e.g. Excel, Google docs, Tableau). Built on top of our taxonomy, the toolbar structures transformations into groups and hierarchies and introduces a new set of icons to quickly access and memorize actions.

Toolbar actions can be populated with column selection in the column browser (we are considering extending this capability to the grid too in the future). The toolbar provides also a good entry point for common actions not necessarily related to transformations (e.g. undo/redo, columns/grid toggles, filters,…)

Besides the toolbar, we’ve also added the ability to search across our complete set of transformations and functions. The new search input is also powered by keywords to help users find what they are looking for (e.g. wrangle language terms, synonyms, and similar concepts).

What’s Next

As you can imagine, much more to come here. The effort to increase Trifacta’s usability and discoverability is never ended and we have a long and exciting roadmap ahead. Please check back next month to learn more about the next iteration of the task framework and toolbar. Check out our new cloud version of Trifacta Wrangler for free.