Class is Now in Session

Presenting The Data School, an educational video series for people who work with data

Learn More

Garbage In, Garbage Out: Why Data Quality Matters

November 1, 2017

Data quality refers to the accuracy and cleanliness of data. It includes examining data consistency, completeness, and relevance. Reliable data quality is required for strategic decision-making when we are working with organizational data in an enterprise. The phrase “garbage in, garbage out” is well known to those who wrangle data. And without good data, you cannot have good analysis. 

From entering, storing, pulling, and analyzing data, quality can become compromised at any point. Data maintenance and quality assurance is an ongoing and often onerous process in the day-to-day lives of data managers. This is especially true in today’s world of semi-structured and unstructured data, where data quality must be maintained across disparate sources and systems.

Garbage Defined, Collected, and Taken to the Curb

What do we mean by garbage in? Great question. When assessing data quality, there are a number of specific fundamentals you want to examine, including data accuracy, consistency, completeness, and relevance, just to name a few.

data quality includes examining data accuracy, consistency, completeness, and relevance

Maintaining data quality involves periodic reviews and scrubbing—or cleaning—the data. Scrubbing includes updating, standardizing, and de-duping information, as well as making sure all appropriate data is present. The ultimate goal is to create a uniformed view of the data, regardless of its original disparities, so that it can be leveraged for quality analysis. The quicker data can be cleaned, culled, and unified, the quicker insights can be drawn from it.

That means for you and your team, better business decisions made.

Trifacta Tools for Trash: Get Serious and Speedy About Data Quality

Every data set needs to be scrubbed to certify its quality, but identifying the specific aspects of the data that need to be cleaned can be very difficult. This is especially true when dealing with semi-structured data or the sheer volume of large-scale data sets.

Trifacta is focused on making cleaning fast and easy. At-scale validation of data quality is performed automatically and visually represented through our profiling feature. With our profiling feature, Trifacta users can quickly assess possible problems, unusual patterns, and required changes.

A subset of the data, or result file, is pulled and displayed along with a data quality bar, which immediately shows users inconsistent or missing data in an easy-to-use, visually appealing interface. Users can then click the empty or mismatched values in the data quality bar and be prompted with transform suggestions to fix or remove the values.

Along with the data quality bar, Trifacta’s results page displays a column histogram, which shows basic counts and percentages of the values in each column of your results file. It also illustrates breakdowns of the most frequent values and outlier values, as well as additional statistics (depending on the data type).

Once users are ready to fix their data, transforms can be executed across the entire data set through a few simple clicks in lieu of time-intensive, complex programming. With Trifacta’s data quality features,  data managers are no longer stuck in the trash. Instead, data managers can get back to what they were meant to do: developing insights and making strategic business decisions using good, quality data. 

To learn more about wrangling data for data onboarding, read our brief, Data Onboarding: A Survivor’s Guide To Combining Unfamiliar, Disparate Data; or download the free Principles of Data Wrangling eBook here.

Related Posts

The Role of Self-Service Data Preparation
in Analytics Modernization

This article was originally published by Datanami on October 5, 2017.  It’s no secret that data is playing... more

  |  October 12, 2017

7 Ways Data Enrichment Boosts Your Business

Data is the lifeblood of contemporary businesses and is used to formulate marketing, sales and customer... more

  |  May 11, 2019

Commit to Clean Data: Prioritizing & Setting Targets

Of all the ways you can spend your time as an analyst—especially up to 80 percent of your time—the... more

  |  July 30, 2018