Using Excel to wrangle big data is like using scissors to mow your lawn. But in many companies, business analysts use it anyway. Often, it’s what they know best and there’s no time to learn a new tool. Or, perhaps it’s “good enough” to use as their database, for data preparation, and as their visualization layer. So many IT organizations shrug—why bother stepping in to fix what’s not technically broken?
Except, in an increasingly data-driven world, you know that there will come a time when your data begins to push Excel past its breaking point. Beyond that, you’re increasingly concerned about data governance—maintaining version control, tracking transformations—and with Excel, you’re at the whim of each analyst. The good news? When you see that your analysts have reached their fill of Excel, you get to step in to help. Better yet? Provide them with a solution that also enables data governance, so that while they’re wrangling data more efficiently, you’re meeting IT standards.
Below, we’ve listed the three telltale signs that your analysts have officially outgrown Excel—and how to rescue them:
1.) They’re working with higher volumes of data.
Most analysts know Excel can only handle a little over 1M rows and 256 columns. But Pivot tables are also limited in how much data they can report on. So a manual build in Excel can take hours, which means they’re spending more time building than analyzing. Data preparation can account for up to 80% of their time—and it’s their least enjoyable activity.
2.) They’re working with more varieties of data.
Your analysts are encountering data from a growing number of different types of datasets derived from disparate sources. In a 2016 NewVantage Partnership survey of executives of Fortune 1000 firms, integration of the variety of data businesses use is the most cited reason for big data investment. You’ll recognize this particular hell in “Frankenstein” style dashboards with multiple sources cobbled together in Excel or Google Sheets; or possibly consultants being hired to do nothing but prepare data for analysis.
3.) They need to be more aligned.
Often projects need a collaborative approach, but your analysts have found that working on the same file at the same time has been difficult and/or can’t be done. Perhaps using Google Sheets isn’t possible with your compliance requirements. So your analysts are unwittingly reproducing inconsistencies and introducing errors in data preparation. Poor data can cost businesses 20%–35% of their operating revenue per year.
In short, living in Excel Hell is like designing a house with a piece of paper instead of 3D modeling software—sure, it can be done, but there is a lot of wasted effort, as well as imprecision in results.
The Self-Service (but Governed) Solution
So how can you step in and rescue your team? By delivering a self-service data preparation tool that allows them to work with more data in less time. In the past, business units might have sidestepped IT due to a perception of bureaucracy, but now it’s your job to step in and rescue your analysts with a next-gen solution.
The best data wrangling tools will allow your analysts to aggregate and de-dupe multiple sources to get to one source of truth, while including tracking data lineage for IT. The interface should be user-friendly enough that your analysts can leverage the same skills they developed for Access and Excel, but now, in a big data ecosystem.
PepsiCo: From Excel to Trifacta
When PepsiCo was faced with Excel Hell, they were determined to find a solution. Their Collaboration, Planning, Forecasting, Replenishment (CPFR) team has been charged with analyzing performance sales for 10+ retailers to provide guidance, and their biggest challenge was churning out reports fast enough for management to forecast sales. But each retailer had their own way of reporting, requiring analysts to manually build and standardize the metrics in Excel and Access weekly to update the 4-5 required reports.
Previously, analysts downloaded data from internal servers, ran it through an Access database, and then passed it back to the servers. However, PepsiCo’s IT Director knew there was a better way. By introducing Trifacta, Pepisco turned their reporting process into a product; accelerating data preparation by 90%, with higher accuracy. To read more about their challenge and Trifacta’s solution, please visit https://www.trifacta.com/customers/pepsico/ or listen to the webinar.
Want to experience wrangling for yourself? Sign up for free version of our Enterprise product, Trifacta Wrangler!