Start Free

Speed up your data preparation with Designer Cloud powered by Trifacta

Free Sign Up
All Blog Posts

Democratization for Speed in the Public Sector

January 31, 2020

Data Prep Democratization = SPEED

The reality for most professionals working on advanced analytics or data modernization projects is that they are faced with a myriad of challenges that span beyond the actual project work they are responsible for (e.g. Data Scientists working on improving the actual AI algorithms). To address this challenge, organizations are trying to figure out how to enable them with the right supporting tools and infrastructure necessary for success.

This is akin to being asked to redesign, build and implement a new fuel system on a passenger jet – while it is in the air flying. (For all you badass data professionals out there, kudos to you for taking this on as no pilot would ever agree to a fuel system rebuild while in-flight).

While this approach may be deemed impractical by most project (or sanity) standards, it is the reality for many public sector agencies updating their data strategies. Driven by rapid policy evolution combined with accelerated competition from their geopolitical peer/adversaries, nearly all Federal Agencies are demanding modernization through the advancement of analytics to meet their mission needs.

Chief Data Officers are now implementing plans focused on leveraging data as a strategic asset and the vast majority of modernization efforts call out the use of Artificial Intelligence as a key component.

Even for leading Fortune-ranked global companies with large budgets and deep pools of technical resources to draw from (all of which are looking to leverage AI to drive competitive advantage in their respective industries), there exists the challenge of finding the right technical talent (Data Scientists, Engineers etc.) that can deliver on their advanced analytics projects. For Federal agencies limited by security clearances, citizenship and geographic constraints, staffing these projects can seem to be an insurmountable obstacle. It is no secret that appropriately skilled resources are in short supply, even more so for agencies as it is extremely difficult to compete with the perks and pay offered by leading global companies.

What’s often lost on IT leaders working to deliver advanced analytics solutions is that the competition they face is not only in recruiting and staffing their projects with “skilled resources;” fundamentally, it is a race against time. This race can only be won by employing a holistic approach to improving DataOps, including the right tooling that empowers users across the ecosystem to leverage data properly to “do their part” in the process.

Spreading the work out to increase speed! (Think like Ants)

The volume and variety of data collected by enterprises across the public sector is rapidly growing. This growth is outpacing the ability to staff key projects with data professionals that have the technical skills that can work effectively to help leverage data as a strategic asset. When talking about advanced analytics projects this is often the proverbial “elephant in the room.”

Further, it is a well agreed upon statistic across the industry that data professionals, the aforementioned Data Scientists and Engineers, spend 80% of their time cleaning and transforming data. This is often accomplished by manually writing code (R, Python, etc.) to cleanse, structure and integrate data from various sources systems for use in downstream consumption for advanced analytics or business intelligence.

This approach is both error prone and is THE critical bottleneck in the DataOps workflow.

What can government agencies learn from leading companies around the world that have rapidly expedited their operationalization of AI and ML-based technologies to WIN in their industries?

An army of ants can EAT an elephant!

I remember seeing a movie in a high school biology class that outlined this visually. An army of ants devoured an elephant carcass that had become stuck in a dry river bed incredibly fast. It was remarkable – and maybe a little gross.

There’s a great lesson here. The army of ants spread the work out to eat the fallen beast, devouring flesh simultaneously. They were working in parallel, dividing the problem (eating the elephant) across many workers to quickly accomplish something that may seem impossible at first glance. This demonstrates the effectiveness of dividing an enormous challenge into “bite sized” tasks (Dad joke… You’re welcome!) that can be handled by many workers. It’s incredibly powerful.

For public sector programs working to operationalize data as a strategic asset, this analogy is akin to a Kipling story for DataOps…

Think like ants! 

Empowering self-service data preparation for analysts and less technical personnel, means enabling the exploration, profiling, transformation and cleansing of data in a model that allows the exploitation of mission specific data without the need for IT involvement; yet still ensuring critical IT governance, and security policies and practices are followed. This is only accomplished by leveraging the right tool sets that enable the self-service flexibility for business users, while enforcing corporate security and governance standards.

This is exactly how to succeed (with speed) through the democratization of data preparation via self-service.

It is also the critical DataOps paradigm adopted by the world’s leading companies to effectively operationalize advanced analytics – at scale.

Data wrangling (also referred to as data preparation) is a key component of modern DataOps that greatly benefits organizations across the world by spreading the work out across teams, wherein each person works on components of the overall process collaboratively (often referred to as “democratization”).

Democratizing data prep with the right tools will enable less technical resources to transform data for use in downstream analysis; eliminating overhead and greatly expediting the maturity cycle for advanced analytics.

To see exactly how we do this, we invite you to sign up to start using Trifacta for free.