This looks like a great resource! A bit of a rant (not directed at the person who wrote this, because the link at hand is great):
Writing APIs in Go is not the hard part. There are a ton of options for routing basic URLs to handlers. There are a few areas that are a nightmare, total total nightmare:
* SQL - The raw database/sql package in Go is very low-level. It is a terrible experience to use in a project that makes anything more than a handful of simple queries. I'm not asking for an ORM, but it would really benefit the community if there was a SQLAlchemy style (the non-ORM part of it) library for Go. It should be MUCH easier to make basic queries to SQL and get out results.
* Templating - The built in "html/template" package is just not good enough for a web app when compared to something like Jinja (Python). "html/template" gets points for being simple, and actually its not a complaint against "html/template": I think there just needs to be a more feature-packed templating package like Jinja for Go. "html/template" doesn't support things like inheritance, didn't even support comparison operators until Go 1.2, etc.
These two problems have made it so I've pretty much given up web apps in Go for now. Services in Go are _amazing_. But if you're trying to get a full blown web app (again, not the point of the linked article above), then you're gonna have a bad time.
> "html/template" doesn't support things like inheritance
I don't want to get into a debate about what "inheritance" means, but html/template does support template composition and nested templates, which seem to address the same use case.
> These two problems have made it so I've pretty much given up web apps in Go for now. Services in Go are _amazing_. But if you're trying to get a full blown web app (again, not the point of the linked article above), then you're gonna have a bad time.
I'm sorry to hear that, especially since writing a "full-blown web app" is really not difficult in Go at all, once you know what you're doing.
I've actually been working on Jetpack[0], a sample web application in Go to demonstrate how straightforward this is, once one understands how it actually works. (Disclaimer: it's still a work in progress, so it's not complete/ready yet).
Many of your comments on this thread state that writing webapps in Go (SQL, etc) is not so difficult "once you know what you're doing." Could you elaborate on this? What should one know before attempting to use Go for such things?
Here's one possible example. At the top of this thread, there's a statement about how Go templates don't support inheritance-- embedding one template within another as Django and Jinja do[1]. I personally consider this a requirement, because the old school way of having headers and footers in separate templates is radically inferior.
Go doesn't offer a clear simple way to do this in their builtin template library. But it turns out, it really is possible to do it, as described in this thread on Stack Exchange[2]. I bookmarked that so that I can boil it down into some simple-to-reuse code if I ever bite the bullet and start using Go for the presentation layer on a major website. What they're doing there is a cool technique, but it's not simple to use out of the box.
the problem with html/template is that it's made for Go programmers. A good template language is one that's made for front end devs and designers. html/template tends to be painful in practice because it's too difficult for a non-gopher to use.
html/template is literally nothing more than an "HTML-safe" version of text/template, which is Go's arbitrary text template language. It's meant to be completely a drop-in replacement for text/template, and as such, its design does not give HTML output any special consideration, which an HTML-specific template language would.
I understand the point you're making - I just want to point out that Haml is explicitly designed for HTML, whereas html/template more or less "just happens" to be useful for HTML.
As for ERB, I'm not convinced that it is too different in this regard. I'm not much of a Rubyist, and I find it incredibly frustrating to use, but perhaps that's a matter of personal preference.
{text,html}/template have their warts - I literally started Jetpack because of an unusual html/template quirk I ran into that was tricky to debug. However, I don't think it's any worse than other template language in this regard (my main points of reference are Jinja2, ERB, Haml, and Scribble).
It is context aware, which is actually really awesome! It understands if it is in a tag, in a script, in quotes, in double quotes, in source, in css, in style and lots of stuff I am forgetting and does the custom escaping required to make it safe for that context. That is freaking magical.
The html/template package seems to address the use case, but it comes up short in several ways. One example is defining a site template with sections that are optionally specified in a page specific template.
> I'll release it once I'm a little more confident in the interface.
Release it now! I've said the same thing in the past about some code I was working on. Add a little readme, explicitly say it is a work in progress, and get it out there. At the very least the rush of publicizing something will push your drive to work on this codebase, and you might even gain some contributors. I'd rather use a repo that is obviously a work in progress but has active contributions from the creator over some more 'complete' but stagnant repo any day.
Don't worry, I'm not shooting for anything like perfection. I released another library that this builds on just last week: https://github.com/lann/builder
I'm usually pretty anti-ORM and actually like low-level SQL interfaces (I guess because I did a short stint as an Oracle developer once, once you see what can be done with some custom SQL...). But for a recent sideproject of mine, I really didn't want to create a laborious interface to the database. I wanted something quick and dirty. I got something quick, and it doesn't feel dirty at all. It's called gorp: http://github.com/coopernurse/gorp.
By the way, my sideproject is about getting financial data from sources like Yahoo Finance, Bloomberg, et cetera. You can find it here: https://github.com/aktau/gofinance
We're about to release our Go web framework which, I think, implements all the features you're mentioning and a few more. We're aiming for a release in Q1, but getting everything cleaned up and writing good documentation is taking a bit longer than expected. Right now, our internal framework (codenamed Gondola, but subject to change), includes:
* A full ORM which generates tables from structs and allows relations between models, as well as a simple querying system based on Go functions which joins the tables automatically for you e.g.
memeId := base36.Decode(param)
var meme *Meme
var template *Template
err := ctx.Orm().One(orm.Eq("Meme|Id", memeId), &meme, &template)
Meme declares a FK to Template, so the ORM automatically joins the tables and populates both pointers. Note that the ORM does not require the backends to be relational, so even when ATM we only have database/sql based backends you could write a backend for mongodb or rethinkdb.
* Inheritance based template system, with a declarative assets pipeline supporting compilation (e.g. it compiles coffescript to JS and less to CSS on demand - also pluggable). This is the base template from one of our apps:
This server query 1.10 and 2.0 from Google's CDN, bootstrap and fontawesome from bootstrapcdn.com and bundles all our styles and scripts a CSS and a JS file. Then other templates can extend this one and override its blocks e.g.
(note that base.html could also define its own "main" block which would act as a default if you loaded base.html and it would be replaced by main.html's "main" block if you loaded "main").
* Easy to use signed and encrypted cookies.
* Regular expression based URL routing.
* Functions for easily parsing input parameters.
* A development server which automatically rebuilds the source as it changes and reports compilation errors in the browser itself. While developing, runtime errors are also reported in the browser with complete backtraces and printing the values passed to each function. While running in production, those reports are sent by email to the project administrator(s).
* App-wide and low-level caching system with pluggable backends, currently supporting Redis and Memcache as well as a dummy backend for development.
* Form generator and validator from Go structs, with renderers for both Bootstrap and Foundation (renderers are also pluggable, so you could also write your own. We also support generating forms from multiple objects so you e.g. write an object which generates a captcha and include it in any form with a single line in code (in fact, we provide just that, but a simple numeric captcha a a full-blown recaptcha object). It also does automatic CSRF protection.
* A complete i18n system, which extracts strings from Go code and templates, generates .pot files which can be translated using any editor (or a specialized .po editor if you feel like it) and them compiles .po files back to Go code again. This makes packages with their translations "go-get-able".
* Periodic and background tasks which can either be scheduled or just started from any web request.
* A configuration package which takes a struct and fills it by parsing an optional configuration file as well as command line flags.
* Support for administrative commands, which can be executed from the command line or started remotely.
* Support for pluggable apps. e.g. we do have a "users" app which provides user registration, authentication and social integration. This can be included in any other app with a couple of lines of code (one line for the import, another line to tell our main app to include it). Included apps can do a lot of stuff, like inserting code in every template rendered by the parent app, including external assets like images, styles or scripts and even rendering their own templates contained inside your parent's app base template.
* Last but not least, a minimal overhead (in the range of 100ns per request) profiling framework which lets you know how much time a request spent in the cache, the orm, the template, e.g... as well as seeing how much each operation took (e.g. you can check how much time a SELECT took and also perform an EXPLAIN on it). Users can also define their own events and profile them by instrumenting their code (it usually requires just a line of code per resource, to indicate its name). You can also profile a live server by using a command line app included in the framework.
We also have helper and utility functions for the more mundane stuff and basically everything you could expect from a web framework. We took inspiration from Django, Tornado and Rails and tried to take the best of them while building this framework (I used to contribute to Tornado back in the day). We've been developing this internally for almost 2 years and it's now around 40KLOC of Go and a couple of K of coffescript, less and html.
> * SQL - The raw database/sql package in Go is very low-level. It is a terrible experience to use in a project that makes anything more than a handful of simple queries. I'm not asking for an ORM, but it would really benefit the community if there was a SQLAlchemy style (the non-ORM part of it) library for Go. It should be MUCH easier to make basic queries to SQL and get out results.
I can highly recommend https://github.com/jmoiron/sqlx which allows you to easily unpack results into structs/maps, or pass in a struct/map and insert it (using struct tags + named queries). If you're writing a CRUD application this covers a lot of the basics. There's also https://github.com/eaigner/jet which is a 'micro ORM' (and very light on the ORM part).
> "html/template" doesn't support things like inheritance, didn't even support comparison operators until Go 1.2, etc.
I grew to like html/template over time, but the lack of comparison operators pre Go 1.2 was a huge sore point and I had I been working on this little project a year ago I'd have been extremely frustrated. The lack of inheritance was a bit of a pain for me too—I ended up splitting my templates into many (!) smaller pieces so I could at least re-use the portions I needed to. Hopefully something "Jinja-esque" comes in time.
There's an (unmaintained) Mustache port and a Mustache-alike in Mandira (https://github.com/jmoiron/mandira) but I've not seen anything like Jinja yet. If there's one thing html/template has going for it is that I can trust it (as much as one can) to be 'safe'.
I wouldn't say web apps in Go should be avoided at all costs, but I would say that you should consider the scope/scale and do a bit of research first. Sometimes Django/Rails or Flask/Sinatra might make it easier on you.
You mentioned "Services in Go are _amazing_", but how do you avoid the first issue you mentioned? I have found so far getting data in and out to be somewhat cumbersome. Also how do you handle data validations and logging?
This looks like great work, so probably this comment is a little off topic, but here goes anyways:
Every web framework I've seen for Golang seems to do a competent job of tackling the easiest part of designing a web framework, which is routing requests to handlers.
Actually, the built-in net/http library already does a serviceable job of handling requests, and if you understand closures, filtering them and maintaining request state as well.
The thing no framework I have seen so far addresses is persistence. SQL persistence in Golang is a bit of a nightmare. It's raw SQL, and the database libraries are fussy.
The thing that makes Rails so easy to write in, especially for newcomers, is ActiveRecord. As far as I can tell, nothing like it exists (in a stable form) for Golang.
I agree with you re: the lack of ORMs in golang. I actually enjoy writing and profiling my own SQL, but if I want to prototype, ORMs can speed up the process.
I will say that some of the parts of SQL in golang that I find most annoying, such as populating structs, are handled very nicely by jmoiron's sqlx library. [1] This is not an ORM, but it's an extension to sql that removes some of the most annoying boilerplate nevertheless.
I find the lack of ORM a feature in a web framework. Personally, I find them frustrating and an extra layer of abstraction that just gets in my way. Then again, I also don't mind writing queries. I usually end up using my own lightweight set of helper functions to manage queries, regardless of the language.
I still have Hibernate flashbacks from back in the day.
I don't think that the argument is that ORMs are bad, but that it shouldn't necessarily fall into the scope of a web framework. I think you could reasonably argue that the primary task of a web framework is to be a meaningful abstraction layer over http requests and http responses, but on the other hand I can also see why someone would disagree. In some sense you could make the same argument against template rendering systems.
Personally, I work a lot with a database setup that is managed primarily through REST and JSON-RPC interfaces, which makes a framework that lacks ORM a perfect fit, and I'd rather use something like Flask than something that sets me up with an ORM library whether I need it or not.
ActiveRecord may be a stepping stone for those just starting out, but for serious, long-living, evolving applications I firmly believe it does more harm than good.
Back when I did Rails contracting, cleaning up code which conflated database access with business logic was quite possibly what was keeping us in business.
Yes, SQL is still spotty in Go. I wrote a medium sized program with it and the learning curve was fairly high. Documentation for the database libraries is mostly nill. Though I'm not sure if it would benefit from an ORM. This is a systems language. I'd rather have it use interfaces to access data from a database than have it deal with it itself. That way I keep it tidy and fast. Which is Go's aim.
I failed to convey my experiences. Which is that the third party drivers are very hit and miss (along with the documentation). Understandably due to how young the language is. But yes, the golang docs are very nice.
I'm not aware of a better ORM anywhere. The simplicity, and the way it separates the model classes from everything else, is sublime. It does pretty much everything activerecord does and lots more, without all the layers of abstraction.
I'm going to chime in, since I've been using Go as my primary language for over a year, and I'm currently working on a major (production) Go project that others might see as well-suited for Rails (in fact, the early prototype was written in Rails[0])
> The thing no framework I have seen so far addresses is persistence. SQL persistence in Golang is a bit of a nightmare. It's raw SQL, and the database libraries are fussy.
> The thing that makes Rails so easy to write in, especially for newcomers, is ActiveRecord. As far as I can tell, nothing like it exists (in a stable form) for Golang.
I wouldn't recommend newcomers to Go use SQL right off the bat. Document-based storage (like CouchDB or MongoDB), or even key/value (Redis, etc.) is much more aligned with Go's concept of structs, and all three of those databases have strong library support in Go. If you're just looking to dabble in Go and just happen to need a database (and it doesn't matter which), I'd recommend playing around with those first. Even Neo4j works very well with Go, though as of January 2013 (when I last used it), the only Go library for Neo4j was third-party, and not very well-documented.
That said, once you know the Go language, using SQL (either with or without an ORM) is straightforward, if not as convenient as ActiveRecord. jmoiron has a good blog post[1] comparing the good, bad, and ugly of Go ORMs, so I won't try to top that here - rather, I'll share my experience.
(As pointed out elsewhere, ORMs are not universally liked as a general concept, so I'm writing this comparison assuming that one actually does want the features an ORM provides - naturally, if you prefer prepared SQL, Go already provides perfect support for that.)
Gorp is definitely not as "magical" as ActiveRecord, but it certainly does the job. A number of the ways in which it differs from ActiveRecord have more to do with the (strongly) static vs. dynamic difference in type systems, though some are specific to Go's architecture.
There are a few odd features here and there that are still missing (for example, I'm actually not sure if there's a wrapper function for "INSERT IF NOT EXISTS" yet, though it's not a huge bother).
I do end up spending more time thinking about the queries that I'm using, compared to Django or Rails. There is some additional verbosity - I have to write some helper functions that I wouldn't otherwise have to. On the other hand, I know exactly what queries are being executed at all times, so it's easy to identify potential bottlenecks.
In short - I agree that ActiveRecord is more convenient than anything that currently exists in Go. On the other hand, it's reasonably advanced enough that relative newcomers to the language[2] shouldn't have a problem with it. So far, the biggest bottleneck in development of this application has not been with the SQL bindings, even though it would be nice to spend a bit less time on that front.
It hasn't had as much time to develop and mature as ActiveRecord have, but those of us using it in production have accepted those tradeoffs.
[0] If you're wondering why we didn't continue it in Rails, we had to scrap the original codebase for a number of other reasons, so given that we weren't keeping any of the original code, there was no logical reason we needed to use the same language.
How about having two statements for running queries, Exec, and Query, but unlike other database engines where you might expect to be able to execute a PQ query that returns values like:
INSERT INTO PERSON (NAME) VALUES ('JACK') RETURNING ID_PERSON, CREATION_TIME
...no, actually, you cannot perform a query for an INSERT statement, even if it returns values.
This isn't a database driver issue even, apparently. It's just the way the database API was defined. Query is not for inserts.
Or the fact that if you run a query (eg. CREATE TABLE blah...) you get a result set that you have to call Next() on to flush the query to the database, even if you close the connection.
...because obviously, running next on a create statement is what you would do.
Or how you have to import a singleton to load the database driver you want.
import (
_ "github.com/lib/pq"
"database/sql" <----- Previous import has side effects.
)
How is that idiomatic go? -__- It's totally a hack.
It's nothing major, but the database/sql library api just doesn't feel right.
IMO having some ORM automagically construct queries for you is a pretty scary idea.
Obviously it's not a good idea to have SQL scattered all over your codebase (we've all seen it).
But including SQL in your code isn't hard if you keep it in one place and abstract it properly.
This post really highlights what I like about Go so far vs other languages -- the evolving community. Like Rails, there is an emphasis on abstraction and productivity. Unlike Rails, the blog posts I've seen so far seem to be centered around explaining how to implement the functionality instead of just wrapping the functionality and providing an easy API. Now that I've read this post, I could download his library, implement my own, or extend the parts I want to customize very easily. I feel more powerful.
I would have taken a different approach for handling unsupported resource methods. I'd love to hear some feedback on whether or not this is considered idiomatic. Instead of this:
type HelloResource struct {
sleepy.PostNotSupported
sleepy.PutNotSupported
sleepy.DeleteNotSupported
}
Something like this:
type HelloResource struct {
sleepy.DefaultResponder
}
DefaultResponder is simply a struct which has all the methods of sleepy.PostNotSupported, sleepy.PutNotSupported, etc, and is defined:
Now, if you want to implement resource methods, you define them on the top-level struct, like so:
type HelloResource struct {
sleepy.DefaultResponder
}
def (HelloResource) Get(values url.Values) (int, interface{}) {
data := map[string]string{"hello": "world"}
return 200, data
}
So now, if you call Get on a HelloResource type, it uses the method defined on HelloResource, not the one defined on it's embedded DefaultResponder type. But if you call HelloResource.Post, it uses the method defined on DefaultResponder, i.e., it returns a 405 error. This is effectively like "overriding" in Java or other object oriented languages and saves a few lines when people are using your library.
What do you think? Again, I'm really just looking for feedback here as I too am new to go. Is this something that would be considered idiomatic, or is the original solution better?
(Apologies in advance for any typos or syntax errors. Didn't check this with a compiler, but hopefully you get the idea.)
You could even go further and assume all user resources are logical extensions of sleepy.Resource:
type HelloResource struct {
sleepy.Resource
}
This makes it clear that you're inheriting everything from sleepy.Resource, so you can provide reasonable implementations for all methods of resources, not just HTTP methods handlers, e.g. getBodyAsJSON(), redirect(resource), etc.
Also, depending on how your mux works, you could store app-level data in sleepy.Resource, to be accessed by all resources:
Nice, easy to use, really straightforward code. The only thing I would have done differently is make sleepy.Api implement http.Handler so that it would easily integrate with other libraries that build upon and play nicely with net/http.
I haven't seen the full picture of Go yet, partially by some examples. I found it makes calling multi-threading functions a lot easier.
I consider Ruby, Python, Go and Julia are the new generation of programming languages aiming to be quickly picked up and made hands-on working.
Can anybody tell me if Ruby, Python, Go supports a full spectrum of functionalities for various network protocols, multi-threading, messaging (sync and async) and concurrent accessing etc. Those kind of functionalities don't only require additional libraries, but also frameworks and algorithms.
Maybe by using Jython, JRuby or something like that, they are able to achieve the complete functionalities like Java.
According to some research, Python evolved from Lisp, and Ruby is closed to Perl. So why aren't those script-based language more popular than Java/C++?
This is great. I sat down to write a quick api in go about three weeks ago but couldn't find a framework that I wanted to use. This looks like it has a decent abstraction and libraries like this really help language adoption.
Why exactly do Resource's methods take variadic argument?
In what scenario can there be more than one?
Also, small remarks about code:
You define type HandleFunc in your package which is identical to http.HandlerFunc. Looks like it can be removed.
Some error logging (rw.Write, json.Marshal) would be useful as well.
I'm glad to read that. May I ask what technology did you guys use?
SockJS' Python implementation works well (well, I haven't deployed yet...), but the Go port isn't that mature IIRC.
The Go runtime is still being optimized. For instance we found that the upgrade to Go 1.2 made a big difference for some of our performance tests. So it's nice to get surprises like that but also shows that the ecosystem is still young.
It doesn't have the same aim as Martini, which provides more of a full middleware manager.
Sleepy is a simple wrapper for RESTful resources that takes a few steps out of the handler work, e.g. internally handles the work of encoding a response to JSON and then sending it.
Nice to see new developers delving in, albeit a little clumsy at first :P
Writing APIs in Go is not the hard part. There are a ton of options for routing basic URLs to handlers. There are a few areas that are a nightmare, total total nightmare:
* SQL - The raw database/sql package in Go is very low-level. It is a terrible experience to use in a project that makes anything more than a handful of simple queries. I'm not asking for an ORM, but it would really benefit the community if there was a SQLAlchemy style (the non-ORM part of it) library for Go. It should be MUCH easier to make basic queries to SQL and get out results.
* Templating - The built in "html/template" package is just not good enough for a web app when compared to something like Jinja (Python). "html/template" gets points for being simple, and actually its not a complaint against "html/template": I think there just needs to be a more feature-packed templating package like Jinja for Go. "html/template" doesn't support things like inheritance, didn't even support comparison operators until Go 1.2, etc.
These two problems have made it so I've pretty much given up web apps in Go for now. Services in Go are _amazing_. But if you're trying to get a full blown web app (again, not the point of the linked article above), then you're gonna have a bad time.