The Presto Foundation announced two significant new contributions to Presto, the open source SQL query engine for data analytics and the open data lakehouse. IBM is donating its AWS Lake Formation integration for Presto, and Uber is donating its Redis-based historical statistics provider for Presto. These community contributions coincide with the 11th birthday of Presto and the 4th year it has operated under the governance of the Linux Foundation.
Tim Meehan, chair of the Presto Foundation Technical Steering Committee and a software engineer at IBM, said, “With the AWS Lake Formation and Presto integration, data platform teams will be able to seamlessly integrate Presto natively with AWS Glue, AWS Lake Formation and AWS S3 while providing granular security for data. This pre-built AWS Lake Formation integration is the first big contribution back to the Presto community by IBM, and it signals the high level of commitment to Presto.”
“The Redis-based historical statistics provider is the latest contribution from Uber to the Presto project,” said Girish Baliga, Governing Board chairperson of Presto Foundation and director of engineering at Uber. “Contributing back to open source is an important part of our ethos, and this latest contribution is a testament to that. We welcome everyone from the community to get involved and contribute back to the project, no matter the size of their contributions.”
Other contributions to the open source Presto project in the last year have come from Amazon, Confluent, Digital Extremes, Humans.net, NewsBreak, PopSQL, Tencent, TikTok/Bytedance and Visa, in addition to Presto Foundation members IBM, Meta and Uber.
Contribution of AWS Lake Formation Integration for Presto
AWS Lake Formation is a service that makes it easy to set up a secure data lake in a matter of days, providing the governance layer for AWS S3. It provides an extra layer of security when running interactive Presto queries on AWS S3 scalable cloud storage.
The donation of an AWS Lake Formation integration to the open source Presto community by IBM, which recently joined the Presto Foundation through its acquisition of Ahana, automates the configuration of Presto and AWS Lake Formation. The integration simplifies the ability to take advantage of the additional data lake security that AWS Lake Formation provides.
Contribution of Redis-based Historical Statistics Provider
Redis, an open-source in-memory database that integrates with Presto through a dedicated connector, added a history-based optimizer (HBO). This HBO enhances the efficiency and speed of query execution for Presto by using historical stats to generate optimized plans for queries. Uber used Presto’s extensive historical plan optimizer framework and the Redis HBO to build a Redis meta-store provider to fetch and update plan-level historical stats.
Now, Uber has donated this Redis-based HBO back to the Presto community. As a result, Presto users can make informed decisions about query planning and resource allocation based on the historical patterns of queries, leading to improved execution times and resource utilization.
Presto Community Updates
In addition to the two contributions from IBM and Uber, the Presto community is also celebrating additional recent updates. Four new area-specific working groups have been formed to advance that part of the project more effectively. The working groups include Architecture, Optimizer, Native Worker and Security. Community members help determine priorities and roadmap as well as work to deliver on that roadmap.
Community engagement is also consistently rising year over year, as evidenced by a 41% increase in pull requests (PRs), 121% increase in issues opened for discussion and resolution, and 41% increase in the number of reviews.
“As we celebrate 11 years of Presto and 4 years with the Linux Foundation, it’s gratifying to see this community continue to grow in their involvement and commitment to this open source project,” said Ali LeClerc, chairperson of the Presto Foundation Outreach Committee. “We look forward to celebrating our community at PrestoCon 2023 December 5-6, our annual in-person community conference that will highlight all of the work going on in the open source project.”
PrestoCon 2023 Tickets Discounted to $11 Off Early Bird
People can sign up for PrestoCon 2023 and use the birthday present discount (enter the code BIRTHDAY) to get a ticket for $11 off the early bird price (regular price is $50 for early bird and $100 for full price) through the end of the day (midnight PDT) on September 21, 2023. Attendees can register at: https://events.linuxfoundation.org/prestocon/.
About the Presto Foundation
A member of the Linux Foundation, the Presto Foundation is a nonprofit dedicated to advancing the open source Presto SQL engine for data analytics and the open lakehouse. Presto was initially developed at Facebook (now Meta) and is known for its performance and scalability. The Presto Foundation operates under an open governance model that promotes transparency, collaboration and community-driven decision-making. Hundreds of companies use Presto, including Adobe, Bolt, Bytedance, Hewlett Packard Enterprise, IBM, Intel, Meta, Tencent, Twilio and Uber.
View source version on businesswire.com: https://www.businesswire.com/news/home/20230920470086/en/
Contacts
Ali LeClerc, Presto Foundation, ali@prestodb.io
Colleen Martell, Martell Communications, cmartell@martellpr.com, 408-832-0147
