Hi HN, we’re Joe and JD from Hydra (
https://hydras.io/). Hydra is a Postgres extension that intelligently routes queries through Postgres to other databases. Engineers query regular Postgres, and Hydra extends a Postgres-compliant SQL layer to non-relational, columnar, and graph DBs. It currently works with Postgres and Snowflake, and we have a roadmap to support MongoDB, Google BigQuery, and ClickHouse.
Different databases are good at different things. For example, Postgres is good at low-latency transactional workloads, but slow when running analytical queries. For the latter, you're better off with a columnar database like Snowflake. The problem is that for each new database added to a system, application complexity increases quickly.
Working at Microsoft Azure, I saw many companies juggle database trade-offs in complex architectures. When organizations adopted new databases, engineers were forced to rewrite application code to support the new database or use multiple apps to offset database performance tradeoffs. All this is expensive busy work that frustrates engineers. Adopting new databases is hard and expensive.
Hydra automatically picks the right DB for the right task and pushes down computation, meaning each query will get routed to where it can be executed the fastest. We’ve seen results return 100X faster when executing to the right database.
We've chosen to integrate with Snowflake first so that developers can easily gain the analytical performance of Snowflake through a simple Postgres interface. To an application, Hydra looks like a single database that can handle both transactions and analytics. As soon as transactions are committed in Postgres, they are accessible for analytics in real-time. Combining the strengths of Postgres and Snowflake in this way results in what is sometimes called HTAP: Hybrid Transactional-Analytical Processing (https://en.wikipedia.org/wiki/Hybrid_transactional/analytica...), which is the convergence of OLTP and OLAP.
Existing solutions are manual and require communicating with each datastore separately. The common alternative is trying to combine all of your data together into a data warehouse via ETL. That works well for analysts and data scientists, but isn't transactional and can't be used to power responsive applications. With Hydra engineers can write unified applications to cover workloads that had to be separate before.
Hydra runs as a Postgres extension, which gives it the ability to use Postgres internals and modify execution of queries. Hydra intercepts queries in real-time and routes queries based on query type, user settings, and Postgres' cost analysis. Writes and operational reads go to Postgres, analytical workloads go to Snowflake.
Recently committed transactions are moved from Postgres to Snowflake in near real-time using Hydra Bridge, our built-in data pipeline that links databases from within Postgres. The bridge is an important part of what we do. Without Hydra, workloads are typically isolated between different databases, requiring engineers to implement slow and costly ETL processes. Complex analytics are often run on older data, updated monthly or weekly. The Hydra bridge allows for real-time data movement, enabling analytics to be run on fresh data.
We make money by charging for Hydra Postgres, which is a Postgres managed service, and Hydra Instance, which attaches Hydra to your existing Postgres database. Pricing is listed on the product pages: https://hydras.io/products/postgres and https://hydras.io/products/instance.
A little about our backgrounds: Joseph Sciarrino - Former PM @ MSFT Azure Open-Source Databases team. Heroku (W08) and Citus Data (S11) alum. Jonathan Dance - Director @ Heroku (2011-2021)
Using Hydra you can create a database cluster of your own design. We’d love to know what Hydra clusters you’d be interested in creating. For example, Elasticsearch + Postgres, BigQuery + SingleStore + Postgres, etc. Remember - You can experiment different combinations without rewriting queries, since Hydra extends Postgres over these other databases. When you think about databases like interoperable parts you can get super creative!
Is this through FDW's? Would love to hear more about the technical details.
Shameless plug -- I work at Hasura (turn DB's into GraphQL API's) and this seems incredibly synergistic and useful to get access to databases we don't have native drivers for at the moment.
Any chance of an OSS limited version?