Lessons Learned from Building a REST API on Google App Engine
1. Lessons Learned from Building a
REST API on Google App Engine
Jonathan Altman
Presentation to GolangDC-October 29, 2015
2. Whitenoise Market Webapp
• White Noise by TMSoft (http://www.tmsoft.com/white-noise/) is the
leading sleeping app for iOS,Android, Mac, and Windows
• Customer wanted a way to:
• Allow users to download additional content to the app
• Create a vibrant community for users to interact with each other
• Scale to the large demand of existing users
5. Project
• Build a RESTful API to drive Whitenoise Market’s web front-end
• Angular SPA front end, also built as part of the project
• User authentication with Google or Facebook account—OAuth2
• Role-based authorization
• Implied: customer will use the API from a native mobile client as well
• Golang on Google App Engine, leverage their APIs
6. Sample Calls
• GET /api/items — get all items
• GET /api/item/item_id — get data about the item with id item_id
7. GAE via Golang
• Project was approx. 6 person/weeks 2nd 1/2 2014, including front end
• Customer specification based on their research
• Inherited solid proof of concept app, but no firm API
• GAE golang support was still beta, long term support indeterminate
• Actual GAE API usage calls: outside the scope of this talk (but see
https://cloud.google.com/appengine/docs/go/)
8. Issues
• Package management
• Routing
• REST response formulation/error logging
• OAuth2 support for providers other than Google
• Authorization
• Miscellaneous
9. Package Management
• goapp get not go get
• Not building an exe locally, packages need to be in source tree
uploaded to GAE - feels weird compared to golang philosophy
10. Routing — GAE has choices
• Prefix hostname with module — exposing internals
• Dispatch file: dispatch.yaml — 10 routing rules max
• Roll your own — just start matching URLs in the main dispatch
handler in your golang code
• or…
• and remember: Google Cloud Endpoints were not yet a thing.
Probably the way to go today
12. 3rd Party Router: Gorilla mux!
• http://www.gorillatoolkit.org/pkg/mux
• Gorilla web toolkit has a bunch of other nice parts
• Other 3rd party router libraries probably work fine
• Parameterization, method control
• GAE takes care of a lot of other things Gorilla toolkit provides
r.HandleFunc("/api/comments/{sid}",
handleGetComments).Methods("GET")
r.HandleFunc(“/api/comments/{sid}",
aihttphelper.AuthenticatedEndpoint(HandleAddComment)).Methods("PUT")
13. REST Status/Response Logging
• Standard REST success and error responses
• gorca — https://github.com/icub3d/gorca
• gorca.LogAndMessage: Logs console message and returns short
message plus status code
• gorca.WriteJSON: succesful responses
gorca.LogAndMessage(c,
w,
r,
err,
"error",
"not_authenticated",
http.StatusUnauthorized)
gorca.LogAndMessage(c,
w,
r,
err,
"error",
err.Error(),
http.StatusBadRequest)
gorca.WriteJSON(c,
w,
r,
map[string]interface{}{“status”:
"OK",
"tagAdded":
tagValue})
14. OAuth2 Support - gomniauth
• GAE does OAuth2 authentication…only for Google
• gomniauth does OAuth2 authentication for multiple providers,
including google (https://github.com/stretchr/gomniauth)
• jwt for HTTP Bearer Token — (https://github.com/dgrijalva/jwt-go)
• Accepted pull request in gomniauth allows setting http Transport used
because the GAE runtime replaces net/http’s DefaultTransport with a
context-based one https://github.com/stretchr/gomniauth/pull/23)
15. gomniauth Patch
• You have to fetch a Transport with the current requests’ GAE context,
and pass that to gomniauth before doing authentication
• See https://github.com/jonathana/gomniauth/commit/
3e2e23995b035e26bbd58a0f56cb2b2d61dbe993 for details/usage
16. Authorization
• Separate from authentication. What a user can do, once we know
who the user is
• Wrapper function shown before:
• “Middleware” takes a target function with an extra argument beyond
the normal HTTP request handler for the authenticated user
information, and returns a normal HTTP handler function that does
the authorization check and runs the target function if authorized
• Factory functions encapsulated role info, but could pass in ACL data
r.HandleFunc(“/api/comments/{sid}",
aihttphelper.AuthenticatedEndpoint(HandleAddComment)).Methods("PUT")
17. Authorization Middlewaretype
AiHandlerFunc
func(appengine.Context,
http.ResponseWriter,
*http.Request,
*aitypes.AIUserInfo)
func
generateAuthenticatedEndpoint(h
AiHandlerFunc,
requiredRoles
aitypes.RoleValue)
http.HandlerFunc
{
return
func(w
http.ResponseWriter,
r
*http.Request)
{
c
:=
appengine.NewContext(r)
authUser,
err
:=
AuthenticateRequest(c,
r)
if
(err
!=
nil)
{
gorca.LogAndFailed(c,
w,
r,
err)
return
}
//
401
User
not
authenticated
if
(authUser
==
nil)
{
http.Error(w,
"",
http.StatusUnauthorized)
return
}
//
403
User
not
authorized
(authenticated,
but
no
permission
to
resource)
if
(requiredRoles
>
0
&&
!(hasRole(authUser,
requiredRoles))
{
http.Error(w,
"",
http.StatusForbidden)
return
}
//
User
is
authenticated
and
authorized
h(c,
w,
r,
authUser)
}
}
func
AuthenticatedEndpoint(h
WnHandlerFunc)
http.HandlerFunc
{
return
generateAuthenticatedEndpoint(h,
0)
}
18. Miscellaneous
• Concurrency: ignored as a premature optimization. Issues with
urlfetch.Transport led to concern on runtime support/research time
• GAE API deprecation: not golang specific, but several APIs in use were
deprecated post-project and had to be replaced (blobstore)
• GAE appears to be going to more of an a la carte model where
existing components are replaced with general GCE equivalents
• Google Cloud Endpoints were not available at the time
19. Miscellaneous, cont.
• You’ll be playing with the JSON serialization properties. Javascript<-
>go naming rules mismatch: nobody wants Javascript properties to
begin with capital letters. Also, I tend to prefer map[string]interface{}
over defined structs where I can
• Using appengine.Context. You will need to, almost everywhere,
whether it’s for working with datastore, making outbound http
requests, or logging via its .Infof() call
