Wrangle Summit 2021 On Demand

You can still experience the best people, ideas and technology in data engineering, all in one place

Get All-Access Pass
All Blog Posts

Introducing the Photon Compute Framework: Powering a Rich User Experience for Data Wrangling

March 29, 2016

Today, we’re used to receiving feedback fast. Google delivers millions of search results in under a second. Twitter refreshes its feed before our eyes. Video games provide us with immersive experiences that react faster than we can. Serious software—even if it’s serious fun—has to be built for speed and provide immediate feedback. At Trifacta, we believe that data wrangling shouldn’t be any different: performance is essential to the user experience we are pioneering.

Wrangling at the Speed of Photon

Now, we’re excited to announce that Trifacta’s great wrangling experience has gotten even better, thanks to the Photon Compute Framework, a technology enhancement at the core of Trifacta’s data wrangling interface.

We developed Photon to provide our users with a richly interactive data wrangling experience on large, in-memory data sets. This is nothing like other products for working with data. Trifacta delivers a wealth of immediate feedback and intelligent suggestions every time you interact with it. For end-users, this immediacy offers tremendous value. It means increased visibility into raw data, with just-in-time suggestions from intelligent algorithms. It also gives users the flexibility to explore and preview their transformations, and experiment without regret. Specially, Photon enables:

  • Increased Productivity
    With Photon, users are never removed from the flow of their work or forced to wait for processing to complete. Whether operating on complete in-memory datasets or big samples of big data, users are able to interactively wrangle data volumes that are orders of magnitude larger than was previously possible.
  • Improved Intelligence
    Like intelligent programs for Chess and Go, Trifacta constantly considers the possible next moves that a data analyst might want to make when wrangling data. Photon’s in-memory engine allows Trifacta to explore this space instantly with orders of magnitude more data and computation than was previously possible, ranking suggestions and presenting them to users with rich visualizations of potential outcomes.
  • Enhanced Technology and Portability
    Photon is engineered with critical in-memory performance features for modern architectures, including multi-threaded parallelism, LLVM compilation, columnar compression and pipelined data processing. Yet it only requires a minimal memory footprint. And despite being a high-efficiency natively compiled binary, Photon runs directly within the browser, as well as in standalone single-node environments.

Better Performance Drives Better Business Practices

Ask any data analyst or scientist and they’ll agree: immediate feedback is critical to making data wrangling effective and efficient. Small roadblocks in the data wrangling experience break an analyst’s chain of thought, and reduce their ability and willingness to experiment and iterate on new or complex data sets. Large roadblocks, like batch jobs, change the nature of work with data entirely—analysts end up having sporadic engagement with their data at best. There is a direct correlation between the speed of end-users’ data wrangling, and the value an organization actually gets out of its data.

Photon enables simple but powerful visual interfaces that help end users wrangle large datasets in-memory and receive instant feedback on their data and transformations. This cultivates lean processes to assess data quickly, make intelligent transformations, and, ultimately, deliver the outputs that an organization needs. The fluid interactivity offered by Photon’s performance promotes best practices for data wrangling organization-wide, which is particularly critical as big data becomes increasingly pertinent to every function.

The improvements offered by Photon are also a step forward for high-performance interoperability. As part of Photon’s development, Trifacta has been collaborating on the design of Apache Arrow with leading open-source organizations including Cloudera, Databricks, Twitter, MapR and Dremio. Arrow is an open-source representation for high-performance compute frameworks to interchange data in memory at the full speed of modern processors. In addition, Photon snaps into Trifacta’s Intelligent Execution architecture to run side-by-side with more resource-intensive distributed computing frameworks like Spark and MapReduce that Trifacta supports for big data processing.  

At Trifacta, we believe that the primary purpose of technology is to empower people.  That’s why we developed Photon: to power a user experience that makes people who wrangle data more productive and creative than ever before.

See Photon in Action

Interested in learning more? Trifacta will unveil Photon at Strata + Hadoop World in San Jose at the session Architecting immediacy: The design of a high-performance, portable wrangling engine. The company has a total of three sessions at Strata + Hadoop World in San Jose from Tuesday, March 29 through Thursday, March 31 and will be offering live demos and consultation throughout the show at booth #831.

Of course, if you’re not attending the show, we’d still love to hear from you! Request a demo to learn more about how Trifacta can drive results at your organization, or try out our unique, award-winning data wrangling experience for yourself, sign up for Trifacta Wrangler.