The MongoDB Shell, mongosh, is one of the standard tools used to interact with MongoDB deployments. In this post, I will demonstrate using mongosh with Tigris.
You will see that you do not have to create a MongoDB Atlas cluster. m1k.preview.tigrisdata.cloud is a serverless deployment we will use. It has built-in automatic database sharding, meaning that, unlike MongoDB Atlas, you must never worry about sharding or shard keys, and the data distribution is automatically handled.
This looks really cool - if I understand correctly, you've implemented a Mongo compatible database that runs on FoundationDB and auto-optimizes the indexes.
Can you share a bit more about what stage this is at, and how mature the database is? I looked around to see some benchmarks, stability testing, large scale usage, etc and couldn't find much.
One of our major goals in building Tigris was to provide an open source cloud-native alternative to MongoDB Atlas! We love the documented-oriented programming model that MongoDB provides, which makes it easier for application developers to manage their data. But we also think there is a need for an architecture refresh, which requires the database platform to be rebuilt from the ground up.
Our design choices at Tigris, such as composable architecture, focus on strong data consistency, automatic data sharding, and being Kubernetes-native, enable the needed modernization and makes for a much more cost-efficient, resilient, and DevOps-friendly experience.
Check out the blog post below demonstrating performing CRUD operations on Tigris using the official Go driver for MongoDB. Thanks to our partner FerretDB for working with us on enabling compatibility with MongoDB's wire protocol.
I am Co-Founder/CEO at Tigris Data and the author of this article.
Here I talk about the increased complexity of data infrastructure these days. How this idea about a specific toolset or component for a particular use-case has become the norm. And there has been a "gold" rush for infrastructure companies to stake a place in this new landscape, creating new tools and resulting in increasingly complex architectures.
We need to rethink what a data platform should look like. It has to be one that removes boilerplate and gives developers the ability to just be software developers and not DevOps.
I think if you look at traditional databases, yes SQL is the API interface that you use to connect to them. SQL is great for adhoc analysis of data and makes perfect sense when you are writing your data warehouse jobs.
But when you are building an application you would not be writing raw SQL queries, instead, you would be using some kind of a mapping layer (ORM) that can help map the constructs you use during application development to SQL queries. Furthermore, if we look beyond simple CRUD operations and look at other data-related functionality that you might need in your application - search, pub/sub data to an event stream, SQL would most likely not be the interface that you choose because it would be cumbersome.
That's the point I am trying to make, that we need to look at the database technologies from an application developer's perspective and provide APIs that work well in their workflow, something that naturally fits within their code. SQL isn't that API.
In the 1990s there was a fad for replacing mainframe "green screen" applications with "client server" applications that might have been implemented as a desktop application that directly did SQL queries against a database.
It was a notorious failure because of problems with performance and security.
I am the Co-Founder/CEO of https://www.tigrisdata.com/ and the author of this post where I discuss the API-driven approach for databases.
We, as developers, love APIs because it means we can concentrate on business logic and hand over other services to others. For example, if the app needs payment processing support, why build my own when I can integrate Stripe through its API?
We need to bring the same API-driven approach to databases as well. But it is not just a matter of putting an API layer in front of every database we have. We need to consolidate the everyday database operations such as CRUD, Search and Event Streaming down to a single API layer. This can only be achieved through a cohesively built platform.
What would you have liked to see differently in the article? More explanation? More examples? I would have written a more technical article, but it probably wouldn't have been the best fit for ComputerWeekly.
I think a lot of times when people write guest or sponsored articles the usual processes which lead to quality for advertising or editorial aren't followed.
You're selling a technical product which many people don't understand the need for, leaving the details out puts you in a place where you make vague and unsupported statements, the real answer to that is going to be refining your messages over and over until they get good.
Developing back end systems that provide an API over a relational, object or graph db has been a big part of what I do for the last decade. I don't think the coding is particularly difficult but there is a lot of it. I count 135 tables on the system I am working on today and there are more views, foreign tables, and other objects in that database. If there is a special challenge it's that an API like that always has some cross cutting concerns such as authentication, authorization, logging, data integrity and such.
Because it is such a big job it would be rational to say a 20% improvement in development time would be impactful (save two months a year) but it's a tough sell because of the cost and risk of transitioning the system and/or trying something new.
On one level I'm enthusiastic about low/no code products that are ontology driven but most of us have the experience that, someday, a system like that will answer you "you can't get to there from here". So that's more work for you to overcome.
Examples and specific details are important. A business-driven example would be good to see, so would be a more technical example where your product vastly simplifies the code. I'd love to see testimonials by engineers or managers about the troubles they have with the status quo and how your product helps. (There is always the horror story of the API that was implemented w/o proper access controls.)
I am focusing more on newer applications at this point than existing applications that need to be migrated over. What if you build a new app where the interface with your data is simple? You have CRUD APIs that you integrate directly with the app. Your app needs full-text search capabilities, and it comes out of the box. You don't have to spin up ES and figure out the data shipping story. What if you are building an async processing flow and you simply have to define your event handlers? The async pipeline gets built up for you automatically and the event handlers are executed based on your business logic.
If I am able to provide all of this abstraction to you, then suddenly you are not just 20% productive, you are 100% more productive. Because instead of doing DevOpsy stuff you ended up focusing purely on your application code.
That's what we are looking to achieve with Tigris.
HTTP/2—the successor of the HTTP protocol—provides advantages over its predecessor, such as multiplexing and server push. Although it's a significant improvement over HTTP/1, HTTP/2 is it a replacement for WebSockets? In this post, Anshuman Bhardwaj compares HTTP/2 and WebSockets and shows an example of event streaming over HTTP using Publish/Subscribe APIs provided by Tigris.
Apart from the size of the document, there is no limit on the size of the array or the depth of nested data. We plan on substantially increasing the document size limit.
I suppose this is due to the FDB limitations, so this obviously isn't a blobstore nor will it ever be (?). For example, we need to store video and image files which are easily 100KB - 1GB in size. Tigris or FDB are great to store metadata in (and the metadata is just as important to us as anything) but the blob storage is a bit of a problem. Would be interesting to integrate something with, for example, MinIO or S3.
Thank you for explaining your use case. This is one of the features we have in mind to support use cases similar to yours for storing videos, images, etc. But this may not happen this year. It would be great if you could add something here https://github.com/tigrisdata/tigris/issues that will help us track and monitor similar use cases.
You will see that you do not have to create a MongoDB Atlas cluster. m1k.preview.tigrisdata.cloud is a serverless deployment we will use. It has built-in automatic database sharding, meaning that, unlike MongoDB Atlas, you must never worry about sharding or shard keys, and the data distribution is automatically handled.