Very nice, I thought it was a basic wrapper around SQLite but then realized it actually had more features then that, such as it's own admin dashboard. Then I realized you built multiple auth designs into the system, then I realized there was a javascript SDK to interact with it, then I found your beautiful documentation site, then I realized there is an actual product inspiring it's development (https://presentator.io/)
Each of these is another encouragement to use pocketbase. I went from "eh, I'll just keep my own wrapper" to "pocketbase has a lot of promise, I should try it"
Did you intend to say baked-in? Like it's built-in? If so, I feel it's better to just use "built-in" because it's more familiar. The same typo's in the site too and confused me a bit.
You could read more at https://github.com/presentator/presentator/issues/183, but to summarize - I wanted something that could be self hosted with almost no additional setup and no dependency on 3rd party services.
i have no idea what you mean by a "backend". If it's got a "admin dashboard" it's got a UI which means it's got a front-end which means it's not a backend. I'm assuming "go backend" isn't specific to go but just indicating that it's written in go.
it's got a example code block but doesn't explain the point of it. Why would an existing framework with a UI and everything need me to write a `main` method? Why doesn't it have one already? If it doesn't have one why doesn't it. If it does have one why am i not using it? My thinking here is that there are many existing patterns for registering plugins / extended functionality that don't require altering the main method of the thing you're hooking into. Why aren't you using that? Is this maybe intended to be like... a starter template for a Sinatra like thing with a pre-defined set of functionalty? Kind-of like how Django sets up model administration UIs for you?
re your summary here and the site:
i have no idea what "backed-in files" means and I've been doing web-dev for decades.
"convenient Admin dashboard UI" ... to admin / do what?
"simple REST-ish API" a) why "ish" and not actually REST b) ... that let's me do what?
---
in summary.... this looks useful but i don't get what you're actually offering or what it's intended usage scenario is. This is compounded by the fact that you're using a very atypical definition of one word "backend" and a very uncommon term "backed-in files"
It's a lightweight replacement for Firebase. I
That's why it is described as a backend, because it's primary feature is to act as a data store. If don't understand what a project does, and you're unfamiliar with the comparisons given (Firebase, Supabase), the right thing to do would bw to look up the comparisons.
I know you have user management, a database, and do something with files. As noted before the "backed-in files" isn't a commonly used phrase so i'm left with 2 pieces of meaningful info, neither of which is particularly notable because every multi-user app has both of those.
Firebase gives me a list of use cases https://firebase.google.com/use-cases and documentation whose left nav gives me a good insight into the high-level functionality it provides "authentication, storage, hosting, security rules, extensions, machine learning, etc"
I think it's safe to assume you don't provide all of those, but i don't know what subset you DO provide. As a result, if i give you the benefit of the doubt, telling me it's a "replacement for firebase" is not meaningful because i don't know what parts it can replace unless it's just the few listed in which case it's absolutely _not_ a replacement.
I'm not OP; I wasn't aware of the project until I read the HN headline.
"Lightweight replacement" also implies missing features, IMO. This project has - at the minimum - authentication, storage and hosting in a single binary that can be self-hosted. This absolutely can replace Firebase for some use-cases (e.g. on my LAN/homelab)
Aside: your tone is weirdly - and needlessly combative. If you see no value in this project to yourself, that is fine, but it appears to be making you angry for some unfathomable reason.
I’m familiar with firebase and an electron developer and I still don’t understand what this project does. I wish I did, because it seems it’d be interesting to me. Even the app that uses this (Presentator) is described in a way that leaves me grasping for “bit what does it DO?”
There is a bunch of software in this space like supabase, flat backend, and appwrite that attempts to be a generic self hosted backend for apps in a manner similar to firebase by providing data storage and authentication with a rest API in an integrated fashion, usually with a built in admin interface for schema design, user management, etc.
You can also think of them as software like postgrest but with the addition of authentication and other app oriented features, and with a JavaScript client library out if the box.
This program is doing the same thing in a more lightweight way as a single executable using sqlite.
The idea is that by using this type of generic backend software for your database and authentication, you can basically just write the remainder of a typical app or webapp entirely as client side/frontend software, which is something that you can also do with firebase, but this allows you to easily self host it without having to create your own custom backend software.
> Ok, so it's a self-hosted CRUD boilerplate with auth written in Go? Or a self-hosted Notion but with less view/UI customization out of the box?
I guess you could describe it as "CRUD boilerplate" although that could also make it sound like the main purpose is to provide database table/form views like Microsoft Access which isn't really accurate.
It provides a database rest API like PostgREST with authentication so that you can write whatever frontend you want for it.
Since you say that you're familiar with firebase, imagine you're writing an android app using react native or something that uses firebase as the backend to store users' posts (or whatever they are).
The idea is that if you want to self host it instead of using firebase, you just run this type of software on your server and it will provide the basic functions that any client side app would need on the backend: storage for users' data as well as handling accounts.
I don't think it's really that complicated; it's basically just an interlace to a database, but if you look at something like PostgREST that automatically creates a rest api for a database, you would probably think "wow, I could almost create an app just using this as the backend without having to create my own backend, but I would have to handle authentication/registration myself," so these types of platforms just go a little further and handle that as well.
It can be used as a CRUD boilerplate for Go projects (where you extend it), alternatively you may use it as a standalone REST(-ish) API backend that integrates auth and storing & retrieving state from some other app in any language. It's similar to DjangoREST which can bw used in both modes (as a base for a project written in the same language, or purely as data layer interfaced via REST)
> PocketBase could be used as a standalone app or as a Go framework/toolkit that enables you to build your own custom app specific business logic and still have a single portable executable at the end.
It provides an authenticated rest API (with a premade JavaScript client library) to an sqlite database, so you can store users' data similarly to how you might use firebase.
Does it render pages automatically based on Schema? I see your really simple example under examples, and its so minimal I want to think you just get all the tables and render views from that. Am I crazy? If so this is really neat.
Did you ever consider using something like GORM? Because I could see this working with other databases and being extremely useful. I use Django to build really quick CRUD interfaces, but sometimes I need to be able to customize it more, and the admin UI is not as extensible as I would normally want, they even suggest you just build custom pages if its not doing what you need, which is a bummer because Django Admin covers the majority of my use cases.
I actually started with GORM, but its too complex and I ended up replacing it with a simpler query builder package (ozzo-dbx). While the query builder has abstraction for other databases, I'm not planning supporting them at the moment.
No worries, I've seen another project called UAdmin that kind of reminds me of this, my problem with UAdmin was that there were bugs and I didn't quite get why it worked how it worked, but having used Django since I realized what I was missing out on.
Thank you for following up, I did see you had a package for extending the built-in Go package for SQL.
"Warning: When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6). The issue has been marked as "Won't fix" in Chrome and Firefox. This limit is per browser + domain, which means that you can open 6 SSE connections across all of the tabs to www.example1.com and another 6 SSE connections to www.example2.com (per Stackoverflow). When using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100)."
This can lead to some weird bug reports from users.
Yes, I'm aware of this limitation, but PocketBase uses HTTP2 when combined with the --https flag (this is by go's default net/http implementation) so it shouldn't be too much of a concern.
Go's default `net/http` package will serve HTTP2 when a https configuration is provided (also most web browsers don't support h2c, aka. HTTP2 without tls).
The default limit of ~100 connections should be more than enough for most applications (additionally, the JS SDK client maintains a single SSE connection for a page no matter to how many things the user has subscribed, so that also helps).
HTTP/1.1 has a limit, set by each individual browser vendor, for the maximum number of connections between a client and a unique server domain. So, if you exceed 6 simultaneous connections to that server (across multiple tabs and windows), it will move the request to a stalled state (like a queue) until one request is completed.
Best solution today is to move to http2 on your server -- which has an SSL (TLS 1.2) requirement.
Looks like pocketbase implements this. If you use another server, like nginx, you have to enable this for each site.
Wow, super impressed. I have always wanted to build something like this where everything is embedded in 1 single file and I knew Go could be a great way to do it but always wondered how to handle the database part. Along comes SQLlite.
One feedback which will probably deviate from everything in 1 file concept but adding an option to upload files to an S3 URL will be awesome. Perhaps a pull request ? I am thinking of use cases like document management using this and if I can have the actual files in S3 (or S3 compatible), that would be a huge plus.
EDIT: I spoke too soon. You already have an option for S3 as I tested the demo. Very cool.
How much work went into this btw ? The github, documentation everything is well done. I envy work like this :)
By default PocketBase uses the local filesystem, but you could also configure a S3 compatible storage (AWS, DigitalOcean, Vultr, etc.) from "Settings > Files Storage > Use S3 storage".
I've started working on the project ~ 3-4 months ago. The last 4 weeks I've taken a longer break from my day-to-day job so that I could finalize the first release.
My intent was not to point out a „silly question“. I was legitimately interested if there is a reasoning and what a use case for that would be.
Thank you for responding nicely!
This looks like a great solution for rapid prototyping scenarios where you want a basic CRUD backend with as little friction as possible. The API preview feature in the admin dashboard is a really nice touch.
This is cool. I think it would also be neat to have option to have Litestream [1] integration so that users can use this without any persistent volume or worry about back ups
PocketBase uses embedded SQLite in WAL mode, so it's already compatible with Litestream if you want to integrate it.
Or do you want Litestream somehow embedded in the Pocketbase binary so it's part of the "one file" artefact? They're both written in Go, so I guess that could theoretically be possible.
This looks really great. Have you thought about adding Operational Transform or CRDT primitives to your offering? Realtime subscriptions are nice - but they're even better with realtime collaboration. Because you're using SQLite directly, it's easy(er) for you to build in efficient CRDT mutations to your database rows because SQLite's BLOB column supports incremental IO: https://www.sqlite.org/c3ref/blob_open.html
This could allow you to save a CRDT update operation without needing to read the entire CRDT blob into memory, parse it, and then save it all back to disk.
Currently any additional data transformation is left to the developers to extend via event hooks or custom client side handling.
I've worked with CRDT in the past (yjs), but it may not be very useful in PocketBase considering that the application was designed to run on a single server and db writes are practically queued (you can have only one sqlite writer at a time). I'll investigate it further and may consider it for future release.
I think if you integrate Yjs or a OT library first-party, you'll have a really exceptional framework - open source firebase alternative top-to-bottom, ready for "local-first" collaborate own-your-own-data apps. Sounds like my dream stack for small-scale projects or friend/family apps.
Nice but my super anal quip is please don't call this thing "realtime". It doesn't make sense in this context. What is realtime about it? Are there timing guarantees or are timing constraints provided in API requests? Because that's what realtime systems are. Realtime does not mean "real fast". For example, a train track switching system might be getting run by a 1MHz CPU and still be real time. It's not about speed, it's about deadline guarantees.
It’s not a real-time operating system. In this case, real-time means it is built such that a client gets access to updated database values as soon as they change, as opposed to a non real-time database which doesn’t necessarily have a built-in mechanism for reliably on time updates.
You mean like eventual consistency vs immediate consistency? Because a database that reflects its changes at the end of a transaction is called an ACID database not a real-time database.
this looks really great! i had recently been thinking “boy it’d be really great if there were a ‘supabase but sqlite’ available” and here you are! now i just need a clever project as an excuse to use this.
1 question: are the docs in a repo anywhere? i don’t see them in the main repo (i might just be missing them somehow) or anywhere in the org. if they are public somewhere would you be open to a docs polishing PR?
The documentation is not auto generated and is part of the static landing site (built with SvelteKit).
I didn't bother open source it because it is kinda messy and a little complicated to be edited by users not familiar with the codebase, but I'll try to find some time in the future and may publish it.
In the meantime, if you find typos or think that some of the wording could be improved, feel free to open an issue or discussion in the main repo and I'll fix them.
This looks awesome! I will definitely give it a try. Right now I use postgres in a docker compose but its definitely a bit heavier for single machine prototyping.
One thing that I like about my setup though is that there's a clear migration path to horizontally scale. Do you have a recommended way to move past pocketbase should an app get to that point?
I guess there will be differences from use case to use case, but in general migrating from PocketBase to a more common horizontal supported stack is not different from rewriting your application.
Internally, each Collection creates a standard SQLite table that holds the collection records, so migrating the data structure shouldn't be too troublesome. The only thing that may prove difficult to migrate could be reimplementing the access rules and filters.
But in my opinion, when your application reaches that level of growth requiring multiple servers and services, your business use cases mostly likely will have changed several times already from your initial idea.
Perhaps sharding data by geographical location could be one path - e.g. deploy one server in the US and another in Europe, each with its own database.
The whole concept of edge computing - deploying app servers closer to your users for minimal network latency - seems to go hand in hand with using an embedded database like SQLite. (I haven't tried it in production yet, but looking forward to.)
Potentially silly question (I'll go out on a limb anyway).
I would like to create a multi-platform app for non-technical users where their data is transparently self-contained/self-hosted. So, they open a single binary and it presents them with a UI, and the data created from within the app would be stored in their local PocketBase database. However, I wouldn't want the user to have to start a server directly (they wouldn't know what a server is).
So, I think the binary would have to start both the PocketBase server and a front-end server, and then launch a browser (similar to Electron) pointed to their front-end server's port.
Does this sound like a good approach? What could you recommend to orchestrate this seamlessly to make for a simple user experience when launching the app?
Honestly I'd look at using Electron because you can take a hard dependency on the browser capabilities and state. If you rely on the user bringing their own browser you're going to have headaches of people on ancient browsers or just supremely jacked up browsers (messed up browser plugins, proxies, turned off javascript, etc.). Node can easily launch processes and there are even fancy process management libraries like pm2 you can add as a dependency should you need it, so the logic in your electron layer can be very simple and just launch some other processes and ferry requests to the view layer.
Hm, there is nothing preventing you to start multiple services from a single binary.
I'm not sure that PocketBase would be suitable for your use case, but it is distributed also as a Go package (see https://pocketbase.io/docs/use-as-framework/) and you could combine it with any other GUI Go package (fyne, go-astilectron, etc.)
The feature set is pretty similar, but I focused mainly on the free form/nosql-ish aspect of it, and although I added support for server-side JavaScript middleware I didn't include authentication OOTB (although it could be implemented or you can integrate with an OAuth2 system - it can validate JWT tokens).
And yes, LiteStore's web app hasn't been updated in years and overall your project looks much more polished than mine.
So good to see a renaissance of "portable apps", now on the web.
All of this extra complexity associated with multi-tiered multi-microservice arch is not needed for many categories of apps, and especially for small teams.
Congrats! What a lovely project. What a fantastic job!
Question: I am currently evaluating Ory/Kratos as authentication backend for a side project. Ory Kratos is a mature and excellent project. However, it has very little documentation when it comes to self-hosting. Further, I also feel like maintaining code that uses it down the line could is going to be
problematic given its shear size and complexity.
How do you think Pocketbase compares with Ory Kratos?
How are the real time subscriptions with SQLite achieved? Is this a feature of SQLite itself or something that PocketBase itself implements on top of it?
It's implemented in the application, it's not a feature of SQLite, but its very simple and robust - when a record is created/updated/deleted I'm broadcasting the change to all subscribers that are allowed to receive it (the access control is based on the colleciton's list and view rules).
Reading through the code/docs it looks like you can subscribe to individual records (database rows) or collections (tables), is this correct? In your Records.getList method, you have filter functionality that would be nice to subscribe to (ie. subscribe to changes in "demo" collection where totalComments > 10). Subscribing to a specific result set seems like it could scale better than notifying subscribers of all table changes, which then get filtered in the subscribe() callback func in the SDK. Very cool work!
Yes, you can subscribe to individual records or collections.
User defined filters for the subscriptions are not supported, but it's a good idea and I may consider it in the future.
Currently the subscriptions are filtered through the collection's list and view rules - they also act as "admin level filters", aka. filters that are always applied and regular users cannot modify (getList user defined filters are only appended to the search query together with the admin defined filters).
That makes a lot of sense. Thanks for the link. I've not worked with Go before so I couldn't find exactly where the implementation was.
This looks like a very interesting project and great for some of the small client projects which I currently use Firebase. I'll be sure to take a proper look in the future!
Really neat! I’ve been wanting to make something similar for _years_ to be a standardized data modeling testbed as I explore/experiment/prototype some new feature or problem space or even doing ad-hoc fixes. I think this is a much better starting place than spreadsheets/similar.
Good luck with the project! Seems like you’ve got some good interest :-)
I really like this. All my current projects use Go as a backend and I definitely need my own routes and business logic, so PocketBase looks way easier to integrate into my existing code than the JavaScript based solutions.
And it would also give me more time to focus on business logic instead of Auth - i will definitely try this in the future.
Other than some json functions, PocketBase doesn't use too much SQLite specific features, but I don't plan adding support for other databases at the moment.
Please also note that PocketBase is designed to be deployed only on a single server, so it cannot scale horizontally out of the box. Supabase/Nhost are better suited for that.
Well all the UI stuff. It is super easy to add a column and change something fast with the UI when doing rapid development. But that piece is missing when developing locally. I have gone on to create 2 projects (one called local and one prod) and manually applying the changes. Would be nice to apply changes from one project to another. PocketBase here for example provides the UI locally too. That is the real selling point IMO.
The existing admin interface could be used as a very rudimentary content management, but it was really intended for quick internal data explorations and edits. Additionally, currently there are no roles for admins so permission controls within the admin area are not possible.
You could always implement your own custom CMS SPA frontend that interacts with the PocketBase API (that is how I'm planning to use it with the next version of my other open source project - Presentator).
why is this written in Go , are there specific features that shine over Python3? I've been looking to learn Go, it seems lot of network intensive software use Go.
If a selling point is run from "one file", that's quite difficult to do in python. There are things like pyinstaller but you end up shipping the entire interpreter in the bundle.
As I recall, pex doesn't package up the runtime. It essentially packages up your venv, code, and resources and runs them under an installed runtime. It makes use of a relatively unknown bit of python functionality, which is that CPython will try to treat a zip file as a valid python script.
oh wow thanks i didn't know about this. in that case it would be super simple for me to wrap up a FastAPI server with SQLite or even some in-memory database that runs elsewhere, deliver as a single file!
Yeah this kind of thing was common when I worked at Twitter (where AFAIK pex/pants was developed) on an infrastructure team. It's a cool tool that few outside of Twitter seem to be aware of.
Each of these is another encouragement to use pocketbase. I went from "eh, I'll just keep my own wrapper" to "pocketbase has a lot of promise, I should try it"