Grow your global business

The Shipwire Blog/Open Source: Redis Client for Go

Open Source: Redis Client for Go

There are a number of client libraries for Redis written in Go. They generally take two forms: one which provides a connection type where you can send handwritten commands, and the other which provides a connection type with methods for the many Redis commands. The reply types typically provide a set of methods that return a string, a slice of strings, a slice of bytes, a map, etc.

This leaves out my favorite Go idiom: readers and writers, so I wrote another Redis client for Go that solves this deficiency. This means when I GET a value that’s a 128 MB bulk string, I get back a reader that I can stream back to the client almost as quickly as I receive the data, and without storing any more than my buffer length in memory.

So what does this mean?

This allows for some exciting efficiency, particularly when it comes to bulk strings and arrays, but it does demand some complexity to reconcile the Redis protocol with a FIFO stream. In particular, when you read an array from a Redis response, and the elements are bulk strings, to read the next element, you must read the entire previous element into memory. To solve this, I wrote a few utility IO types to allow response elements to be seamlessly cached. This gives you the flexibility to have the best of both worlds when required.

The library currently supports:

  1. Connection pooling

  2. Pipelining

  3. Pub/Sub

  4. A Print-like API


  • Derek Perkins 11/21/14
    Thanks Stephen, that's really helpful. Can't wait to try it out!
  • Stephen Searles 11/20/14
    Derek: This package is agnostic to the type of http request you're using. A trivial handler that avoids caching would be:
    func getHandler(w http.ResponseWriter, r *http.Request) {
        client, err := redisPool.Conn()
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
        client.RawCmd("get", r.URL.Path)
        b, err := redisClient.Resp().BulkString()
        if err != nil {
            http.Error(w, err.Error(), http.StatusBadRequest)
        io.Copy(w, b)
  • Derek Perkins 11/19/14
    This sounds like it solves exactly a problem that we've been trying to work around. I've got a couple implementation questions: 1) Are you using WebSockets or some other persistent connection to stream data out to the client? 2) How do you avoid caching the entire 128MB response you're referring to if it's just a standard API call? Thanks, Derek

Leave a Comment

Forgot your password?