We are thrilled to announce the 2021 Q2 release of Starburst Enterprise – one of our biggest product releases to date! The new release provides Starburst customers with net new capabilities alongside more advanced connectivity, improved performance, and more robust security.
As always, this major release combines features that have been contributed back to the open source Trino project as well as curated for Starburst Enterprise customers. To experience this latest release first hand, please visit our download site. Some notable features for this release include:
Support for write operations on Delta Lake
Among the net new capabilities, Starburst’s Delta Lake connector now includes support for Data Manipulation Language (DML) to empower data analysts, platform admins, and engineers with Delta Lake write capability. By allowing companies to read and write to Delta Lake, Starburst helps with your path to a Lakehouse architecture. The Starburst Delta Lake connector now supports DML that includes INSERT/DELETE/UPDATE in both AWS and Azure environments. Now, Delta Lake updates can be performed directly within Starburst Enterprise.
Typically, updating and modifying data within a data lake presents major compliance and data quality challenges. It is often time-consuming and can become an operational nightmare for data teams. In order to make any changes, the user essentially had to start from scratch, or move the data to more traditionally data warehouse workloads. Delta Lake solves this challenge by delivering targeted writes, allowing for low lift data quality maintenance.
Starburst now supports a more seamless and accessible solution for data consumers on Delta Lake. This means that Starburst can participate more fully in Change Data Capture workflows (small, incremental updates), giving customers the fine grain ability to quickly modify their data directly in the lake. Some current Starburst customers see this feature as the key to replacing their data warehouse. Our support for targeted writes improves the overall maturity of the data lake and provides the ability to handle workloads typically reserved for a data warehouse.
Support for write operations is the latest feature for the Starburst Delta Lake connector that collectively includes:
- Executing queries on Delta Lake data as you would with normal tables
- Fast, efficient reads of Delta Lake transaction logs, with support for Amazon S3, HDFS, and Azure Storage
- Support for data skipping to enhance performance
- Optimization of queries using Delta Lake file statistics
- Parallelism that distributes processing for better performance
- Table statistics support for the Trino cost-based optimizer
- Improving query performance for optimized and z-ordered tables
- Querying special columns of metadata
- Robust security features including Apache Ranger and Privacera platform integration
- Create tables in Starburst Enterprise that point to Delta Lake tables and create new tables with CTAS statements
- Data Manipulation Language (DML) that includes INSERT/DELETE/UPDATE in AWS and Azure
Starburst Cached Views
Starburst Cached Views adds caching capabilities across different data sources that are transparent to the end user. For example, data from a relational database may be cached on S3. The end user will still issue the same query to the relational source and Starburst will redirect to read from the relational table to the cache in S3. Now, Starburst Cached Views allows customers to incrementally update new rows of data, without needing to execute a full refresh. Starburst Cached Views can now improve read performance up to 20x and update 100’s of times faster.
Starburst Cached Views are important and exciting for our customers. For example, data within a slower database can cache tables to a faster and less expensive system. This way they can benefit from the performance improvement without the cost of keeping their data stored there. As of this release, customers are also able to update their cached data incrementally with new information, without needing to go through the entire process again with all their data.
Starburst Insights SQL Worksheet
Starburst Insights provides a visual overview of important metrics about your Starburst Enterprise cluster for all types of users, from platform administrators to data consumers. From the Insights interface, you can access detailed query history, including single-query statistics and query plans, and cluster performance information from a selected date range.
Starburst now provides Worksheet, an integrated SQL IDE to allow users to explore data without any external tools. With Worksheet, querying a Trino cluster has never been easier. Instead of identifying a tool such as DBeaver or Superset, and a corresponding ODBC or JDBC driver, then downloading, installing, and configuring a connection, customers can simply utilize the Starburst Insights Worksheet to run queries and explore data. Tools like DBeaver, while flexible, are complex to set up, and require local desktop installation and sometimes organizational approval.
Worksheet does not require any installation and has no network latency. Data administrators and consumers are enabled with details in the IDE about queries, schemas, and query history. Furthermore, users can triage and test any queries with the built-in editor and obtain more detailed information such as the query execution plan and its explain plan. While Worksheet cannot replace the benefits and robust capabilities of many business intelligence tools, it empowers data teams and dramatically helps organizations increase the adoption of Starburst Enterprise.
New deployment option on AWS GovCloud
Federal, state and local government and educational institutions require specialized systems approved for high security use cases, and can now leverage Starburst Enterprise in the GovCloud. Starburst Enterprise includes robust out-of-the-box security and partner integrations to allows secure access to data anywhere. Many select organizations, however, require their data be stored in environments that meet strict compliance mandates, safeguards sensitive data, strengthens identity management, and protects accounts and workloads. AWS GovCloud gives government customers the flexibility to architect secure cloud solutions that comply with all of those standards. Now, AWS GovCloud customers can deploy Starburst Enterprise to unify and achieve data access, securely of course, in any location.
New Salesforce and IBM Netezza connectors
Adding to an already existing list of over 40+ connectors, Starburst Enterprise now includes proprietary connectors to Salesforce and Netezza, enabling customers to query data in these sources directly with SQL. These connectors will help our customers that aspire to achieve a complete Customer 360 view. Specifically, the Starburst Salesforce connector will help companies connect critical customer interaction data to other systems, enabling a better understanding of the customer journey, allowing marketers to personalize customer experiences, and sales teams to meet demand. Like all our connectors, the Starburst Salesforce connector and Starburst Netezza connectors achieve this data access and insight without having to copy or replicate data outside of their own systems.
Other connector improvements
In addition to major Delta Lake Connector improvements, we’ve also enhanced other existing connectors. The enhanced connectors for this release are Kafka, adding write capabilities and SSO integration, Hive, improving support for Hive Views, and Snowflake, now offering SSO integration. We’ve also added password credential pass-through for the Starburst SQL Server Connector and Starburst Synapse Connector.
New Tableau connectivity
We’ve introduced a new Tableau TACO-based client. This new integration provides Tableau users with the ability to quickly break down data silos to help them make better, faster business decisions. This integration empowers users with more extensive live queries and easy access to data, no matter where it lives. The combined solution provides self-service analytics and accelerates the time-to-insight for data-driven organizations. It is available for Tableau Desktop, Tableau Server, and Tableau Prep and is built using the Tableau Connector SDK to optimize performance.
Improved Security and Kubernetes documentation
Additionally, we’ve improved our documentation covering security topics including TLS/SSL certificates, as well as our documentation on installing and configuring Starburst Enterprise with Kubernetes.
… And these are just the highlights! The full release notes of the release can be viewed here. And if you’re interested in hearing more please register for our Starburst Enterprise’s Q2 (LTS 356-e) Release Webinar on Thursday, June 17th.
Starburst has continued to improve overall performance by adding caching and pushdown functionality to processing queries, or parts of queries, into the connected data source, resulting in faster queries, and reduced network traffic or load on remote data sources.
Want to see Starburst Enterprise in action? You can download it for free here!
