What’s New in Trifacta 8.7 Release
We are back with a brand new release from Trifacta that delivers more capabilities towards better data engineering. Here are the highlights from the latest 8.7 release from Trifacta. Data Pipelines and Slack Channels We now facilitate two powerhouses to work with each other. You can create tasks from Trifacta Plans, the command center for […]
3 Reasons to Attend Big Data London
There’s a tradeshow on my calendar next week. And there’s not a Zoom link in sight. Virtual conferences have made a huge impact in the past year, and I’m glad to see them continue, but right now, I’m thrilled to see the return of an in-person event: Big Data London. In the words of the […]
Introducing the Trifacta Python SDK
Background In recent years, Python has become one of the most popular object-oriented programming languages. Whether you are a beginner or an experienced programmer, Python’s simple, easy-to-learn syntax enables quick readability and integration with heterogeneous systems. This simple method of programming makes Python very attractive for scripting as well as connecting different components of software […]
View from the Summit: Data vs. Delta (and Other Infectious Diseases)
Here in the United States, it looks like the Delta variant of COVID-19 may drive us back into our pandemic pods instead of back to school. But I have reasons to be optimistic. Fighting infectious diseases is, of course, a health problem, but the key to enabling every stakeholder – from the scientists to the […]
Back to SQL: Data Engineering
As part of growing our massive new Data Science program at Berkeley, it became clear that we needed to target a class specifically for Data Engineering. The goals of Data Engineering are different than Software Engineering. So it was interesting to think through this curriculum and how we would teach it differently than our established database classes.
In this new approach, we ended up emphasizing four steps to SQL for Data Engineering that are atypical of a traditional databases class: data quality, data reshaping, “spreadsheet tasks,” and data pipeline testing.
Good Stuff You Can Learn From Bad Data
Let’s say you’re remodeling your kitchen. You want to replace the old linoleum with beautiful new hardwood floors. But as you rip up the old flooring, you realize the subfloor next to your kitchen sink is rotted through. What do you do? At a minimum, you need to patch the hole. You may need to […]
Transformation: Next Level SQL
When we use SQL for Transformation—the “T” in ELT—the focus changes. In this case, we’re taking many messy and disparate tables and manipulating them into a more usable or common form. To take our example from before, we may be extracting and loading sales data from 17 electronics chains that sold the phones, and our job in SQL is to write transformation queries that integrate that data together.
What’s New in Trifacta 8.6 Release
The summer of 2021 is speeding away, and our journey of innovation continues here at Trifacta. It’s now time to share the highlights from the latest 8.6 release from Trifacta. Flexible Schedule Management Collaboration with flexibility is one of the key aspects of data engineering. You can now experience seamless collaboration with the ability to […]
SQL Pipelines and ELT
ELT is increasingly attractive these days. Modern data warehouses are flexible and increasingly cost-effective, allowing us to store large volumes of data—even messy data that includes volumes of text and images. In this environment, transformations occur in the data warehouse, where the native language is SQL.
Leveraging Data Analytics to Enhance Athletic Performance: Part 3
Content-Based Recommendation Engine I worked for a fitness-training startup that offered personalized recommendations to help people reach their fitness goals. The company wanted to know if applying data analytics and artificial intelligence/machine learning (AI/ML) techniques could answer some of their business questions and enhance trainees’ performance. This is the third of a 3-part blog series […]
Summer of SQL: Why It’s Back
For the first decades of the Millenium, it seemed like the Java-centric approach was the "hot new thing," but SQL has been roaring back. Today, SQL seems to be the focus of every data engineering conversation and popping back up on billboards in Silicon Valley.
The comparison of the two "shops" inevitably leads to the question: which is better? There are pros and cons to emphasizing one or the other.
Leveraging Data Analytics to Enhance Athletic Performance: Part 2
Weighted Association Rules Mining and Graph Analysis I worked for a fitness-training startup that offered personalized recommendations to help people reach their fitness goals. The company wanted to know if applying data analytics and artificial intelligence/machine learning (AI/ML) techniques could answer some of their business questions and enhance trainees’ performance. This is the second of […]
Trifacta at HIMSS21 Building a Healthier World with Clean Data
If we were to pause and reflect on the past 20+ months, one major takeaway would be ‘Health Is Wealth’. As we strive to get back to a normal world, healthcare innovation must expand and accelerate on all fronts for a better tomorrow. At Trifacta, we are privileged to work with healthcare organizations around the […]
Leveraging Data Analytics to Enhance Athletic Performance: Part 1
Gathering Data and Identifying Key Variables If you’ve been watching the 2020 Olympics Games from Tokyo, you’ve likely come to appreciate how much effort is expended in achieving athletic excellence. Athletes are closely coached, monitored, and receive guidance on each performance. Their progress is tracked and recorded, and data plays a big part in the […]
Enabling the Modern Data Stack with the Unified Data Warehouse / Data Lake Architecture
Data continues to be the most relevant entity for any organization. This trend continues to gain steam with recent concepts such as “data is the new oil of the digital economy” and “data is the new software.” Companies are moving towards modern data architectures, especially with the cloud being the catalyst to enable scale and […]
BigQuery Optimization from Trifacta
Behind The Scenes
This is the second blog post in a series on how Trifacta integrates with BigQuery to achieve performance gains with data transformation using SQL queries from Trifacta recipes. In the first part of this series, we discussed how BigQuery Pushdown from Trifacta leverages the scale and efficiency of cloud data warehouses to process data quickly, […]
From Trifacta Recipes to SQL Scripts with Google BigQuery
As a leader in data engineering, one of the key focus areas for Trifacta is to play a key role in the modern data stack. Cloud data warehouses are moving from traditional ETL to modern ELT architectures and Trifacta plays an important role by enabling the “T” in ELT with advanced data transformation, at the […]
What’s New in Trifacta 8.5 Release
Time flies faster when you are having fun. It’s already a month since our last update on what’s new from Trifacta. Our innovation journey continues and we are happy to announce the latest 8.5 release from Trifacta. Below are the highlights. Efficient Data Management One of the key traits of the modern cloud data warehouse […]