By introducing new functionality including Active Profiling and Smart Cleaning, Trifacta democratizes the traditionally siloed processes of data quality
ORLANDO, March 19, 2019 Trifacta Inc., the global leader in data preparation, today announced a new set of capabilities specifically focused on making data quality assessment, remediation and monitoring more intelligent and efficient. The new capabilities are designed to help organizations modernize their approach to addressing data quality issues that hinder the success of analytics, machine learning, and cloud data management initiatives. With an increasing need to derive faster insights and predictions from disparate sources of data, organizations can no longer rely on legacy, siloed data quality processes to handle the speed, scale, and diversity of today’s data.
The first new capability, Active Profiling, is a selection model that blends real time visual and interactive guidance with machine learning, helping users discover and interact with data quality issues and resolve them with intelligent suggestions all while sharing live previews to ensure that user validation is built into every step. Second, Smart Cleaning is a set of new features to address data quality issues that arise in formatting and standardization. With Cluster Clean, Pattern Clean, and Reference Clean, users can choose from a variety of different intelligent cleaning approaches to resolve data quality issues with mismatched data formatting and miscategorizations.
“Having an intuitive tool enables us to massage and normalize the data very fast,” said Litty Thomas, director of marketing, Malwarebytes. “Historically it took us days to do it and now with Trifacta we are able to turn it around instantaneously.”
As the volumes and sources of data continue to expand, so do the number of advanced machine learning models and analytics tools available to help organizations maximize the value of their data. The trouble is, machine learning models and analytics tools are only as good as the data that feeds them, and many organizations struggle with data quality issues. The success of today’s machine learning and analytics initiatives requires a new approach to data quality that focuses on increasing the speed, scale, and accuracy of cleaning and standardizing data. As organizations modernize data quality processes for the machine learning and analytics use cases of today, the success rate of these initiatives will rapidly improve compared to the anemic success rates currently seen.
“To improve the speed, scale, and accuracy of data quality processes, they must transition from being completely manual, siloed activities, to collaborative initiatives that are automated by machine learning and driven by the users who know the data the best,” said Wei Zheng, vice president of products at Trifacta. “Trifacta’s expansion into Data Quality with the introduction of Active Profiling and Smart Cleaning will help organizations democratize data quality remediation while maintaining governance. As a result, the efficiency and value of their analytics initiatives will significantly improve.”
Gartner Inc. has determined that 40% of all failed business initiatives are a result of poor quality data and data quality effects overall labor productivity by as much as 20%. Research indicates that organizations believe poor data quality is costing them an average of $9.7 million per year. In order to truly capitalize on the unprecedented business opportunity of machine learning and AI, organizations must ensure their data meets high standards of quality. Their algorithms and analytics tools will not provide value if they run on poor-quality data.
The new features from Trifacta to further support data quality initiatives include:
- Active Profiling
- A new Selection Model creates a seamless experience that highlights data quality issues and offers interactive guidance on how to resolve these issues.
- Column selection provides expanded histograms, data quality bars, and pattern information to offer immediate insight to column distributions and data quality issues. These visuals update with every change to the data and offer instant previews of every transformation step.
- Interaction with profiling information drives intelligent suggestions and methods for cleaning that the user can choose from.
- Smart Cleaning
- Cluster Clean uses state-of-the-art clustering algorithms to group similar values and resolve them to a single standard value.
- Pattern Clean handles composite data types like dates and phone numbers that often have multiple representations. It identifies the datatype patterns in the dataset and allows users to reformat all values to a chosen pattern with a single click.
- Reference Clean allows users to specify a reference dataset or dictionary, which Trifacta uses to match and standardize values.
“Canopy’s ability to utilize raw, diverse data sources to help our customers in financial services succeed with machine learning and analytics initiatives is a real differentiator for us and critical to that is ensuring data quality at every stage,” said Amit Prakash Gupta, CTO of Canopy PTE. LTD. “With Trifacta’s new Active Profiling and Smart Cleaning functionality, we’re able to broaden data quality initiatives to span a greater number of users and in the process improve the speed, scale, and accuracy of our projects.”
Later in 2019, Trifacta will focus on bringing data quality to the automation process. With the rollout of additional functionality to support flow orchestration, monitoring, and alerting, organizations will be able to set data quality specifications and isolate any data that doesn’t meet the data quality standards of the organization. This will continue the company’s strategy of expanding beyond data preparation by adding support for data quality and advancing Trifacta into a modern DataOps platform.
Basic data quality capabilities are included in Trifacta’s core product editions. Pricing and packaging for advanced data quality features has not been released.
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
Trifacta is the industry pioneer and established leader of the global market for data preparation technology. The company draws on decades of academic research in machine learning and data visualization to make the process of preparing data faster and more intuitive. More than 100,000 data wranglers in 10,000 companies worldwide use Trifacta solutions across cloud, hybrid and on-premises environments to support a variety of analytic and operational use cases. Leading organizations such as Deutsche Boerse, Google, Kaiser Permanente, New York Life and PepsiCo count on Trifacta to accelerate time-to-insight and discover opportunities that drive success. Learn more at trifacta.com.
1 Friedman, T., & Smith, M. (2015, October 7). Measuring the Business Value of Data Quality (ID:G00218962). Retrieved from Gartner database.
2 Duncan, A. D., Selvage, M., & Judah, S. (2018, April 10). How a Chief Data Officer Should Drive a Data Quality Program (ID:G00304776). Retrieved from Gartner database.