This document details aspects of the Riak client interfaces and can be used as a general guide for understanding how to interact with Riak and how to implement a compliant client library or application. Below are some high-level recommendations for implementing well-behaved clients.
Hide Transport Details
Although Riak's two interfaces do not have complete feature-parity, client libraries should make an effort to hide implementation details of the protocols from applications, presenting a uniform interface. This will reduce the amount of code changes needed to select a different transport mechanism for operational reasons.
Protocol Buffers API
While Riak continues to have a fully featured HTTP API for the sake of backwards compatibility, we strongly recommend building new clients to use the Protocol Buffers API instead, primarily because internal tests at Basho have shown performance gains of 25% or more when using Protocol Buffers instead of HTTP.
For a general introduction to Protocol Buffers, we recommend checking out Google's official documentation. In essence, using Protocol Buffers involves the following steps:
- Finding a Protocol Buffers message generator in the language of your
choice and converting Riak's
.protofiles to native code.
- Once you've generated all of the necessary messages, you'll need to
implement a transport layer to interface with Riak. A full list of
Riak-specific Protocol Buffers messages can be found on the PBC API
page. The official Python
client, for example, has a
RiakPbcTransportclass that handles all message building, sending, and receiving, while the official Java client takes a more piecemeal approach to messages (as shown by the
FetchOperation) class, which handles reads from Riak.
- Once the transport layer is in place, you can begin building higher-level abstractions for your client.
The drawback behind using Protocol Buffers is that it's not as widely known as HTTP and has a bit of a learning curve for those who aren't used to it. The good news, however, is that Google offers official support for C++, Java, and Python and many other languages have strong community support.
As described in the Eventual Consistency document, there are many scenarios that can result in temporary inconsistency, which may cause the appearance of stale data or sibling objects. To react appropriately to unexpected or unsuccessful results, clients should enable users to retry requests a specifiable number of times before failing to the caller. For example, when fetching a key that might be inconsistent among its replicas, this gives read repair a chance to update the stale copies. Retries also may give the client an opportunity to reconnect to the Riak node (or a different node in the cluster) when the original connection is lost for whatever reason. A number of operations in Riak are idempotent and thus can be retried without side-effects.
In order to give applications the opportunity to recover from conflicting or partitioned writes to a key, Riak can be configured to present the client multiple versions of the value, also known as siblings. It then becomes the responsibility of the application to resolve those siblings in a way that is meaningful to the application domain. Clients should provide a way to encapsulate resolution behavior such that it can be run automatically when sibling values are detected, potentially multiple times in the course of a fetch or storage operation. Without sibling resolution, the number of stored versions will continually grow, resulting in degraded performance across the cluster in the form of extremely high per-operation latencies or apparent unresponsiveness.
Read-before-Write and Causal Context
Riak will return an encoded causal context with every “fetch” or
“read” request that does not result in a
not found response. This
context (which is simply a non-human-readable byte string) tells Riak
how to resolve concurrent writes, essentially representing the “last
seen” version of the object to which the client made modifications. In
order to prevent sibling explosion, clients
should always return this causal context to Riak when updating an
object. Because of this, it is essential that object be fetched before
being written (except in the case where Riak selects the key or there is
a priori knowledge that the object is new). Client libraries should
either provide means for easily performing such update
operations or perform read-before-write operations
automatically. This can reduce operational issues by limiting sibling
explosion. Clients may also choose
to perform automatic sibling resolution on read.
Discourage Expensive Operations
A number of operations (e.g. listing buckets or lising keys), while useful, are extrenely expensive to run in production. A well-behaved client should expose all functionality but provide some sort of warning to the developer when they choose to perform those expensive operations.
In most cases — especially when using the PBC API, which tends to
use small messages — clients should set the
TCP_NODELAY flag on
opened socket connections to Riak, which disables Nagle's
profiles having a minimum of 40 milliseconds often indicate the presence
of Nagle on either end of the connection. If the client application has
significant round-trip latency to the Riak cluster, disabling Nagle will
have little effect, but for well-connected clients it can significantly
Compatibility with Riak 2.0
The release of Riak 2.0 has brought a variety of fundamental changes to Riak that client builders and maintainers should be aware of, including a variety of new features, such as security and Riak Data Types. The sections below will list some of those changes and suggest approaches to addressing them, including some examples from our official client libraries.
Nodes and Clusters
When writing Riak clients, it's important to remember that Riak always functions as a clustered (i.e. multi-node) system, and connecting clients need to be built to interact with all nodes in the cluster on the basis of each node's host and port.
While it's certainly possible to build clients to only interact with a single node, this would mean that your client's users will need to create their own cluster interaction logic. Instead, you should build your client to do things like the following:
- periodically ping nodes to make sure they're still online
- recognize when nodes are no longer responding and then stop sending requests to those nodes (until they come back online)
- provide a load-balancing scheme to spread interactions across nodes (or provide multiple schemes, or an interface whereby users can register their own schemes)
In general, you should think of the cluster interaction level as a kind of stateful registry of healthy, responsive nodes. In some systems, it might also be necessary to to have configurable parameters for connections to Riak, e.g. minimum and/or maximum concurrent connections.
Prior to Riak 2.0, the location of objects in Riak was determined by bucket and key. In version 2.0, bucket types were introduced as a third namespacing layer in addition to buckets and keys. Connecting clients now need to either specify a bucket type or use the default type for all K/V operations. Although creating, listing, modifying, and activating bucket types can be accomplished only via the command line, your client should provide an interface for seeing which bucket properties are associated with a bucket type. More information can be found in our documentation on bucket types and Protocol Buffers.
One of the changes to be aware of when building clients is that Riak has
changes its querying structure to accommodate bucket types. When
performing K/V operations, you now need to specify a bucket type in
addition to a bucket and key. This means that the structure of all K/V
operations needs to be modified to allow for this. We'd also recommend
enabling users to perform K/V operations without specifying a bucket
type, in which case the
default type is used. In the official Python
client, for example, the following two reads are equivalent:
Dealing with Objects and Content Types
One of the trickier things about dealing with objects in Riak is that objects can be of any data type you choose (Riak Data Types are a different matter, and covered in the section below). You can store JSON, XML, raw binaries, strings, mp3s, MPEGs (though you should probably consider Riak CS for larger files like that), and so on. While this makes Riak an extremely flexible database, it means that clients need to be able to work with a wide variety of content types.
All objects stored in Riak must have a specified content type, e.g.
application/octet-stream, etc. While
a Riak client may not need to be able to handle all data types, a
client intended for wide use should be able handle at least the
- plain text
You should also strongly consider building automatic type handling into your client. When the official Ruby and Python clients, for example, read JSON from Riak, they automatically convert that JSON to hashes and dicts (respectively). The Java client, to give another example, automatically converts POJOs to JSON by default and enables you to automatically convert stored JSON to custom POJO classes when fetching objects, which enables you to easily interact with Riak in a type-specific way. If you're writing a client in a language with strong type safety, this might be a good thing to offer users.
Another important thing to bear in mind: all of your client interactions
with Riak should be UTF-8
complient, not just for the data stored in objects but also for things
like bucket, key, and bucket type names. In other words, your client
should be able to store an object in the bucket
If you're using either Riak Data Types or Riak's strong consistency subsystem, you don't have to worry about siblings because those features by definition do not involve sibling creation or resolution. But many users of your client will want to use Riak as an eventually consistent system, which means that they will need to be able to create their own conflict resolution logic.
In essence, your users' applications will need to be able to make
intelligent, use-case-specific decisions about what to do when the
application is confronted with siblings. Most fundamentally, this means that your client
needs to enable objects to have multiple sibling values instead of just
a single value. In the official Python client, for example, each object
of the class
has parameters that you'd expect, like
data, but it also has a
siblings parameter that returns a list of
In addition to enabling objects to have multiple values, we also strongly recommend providing some kind of helper logic that enables users to easily apply their own sibling resolution logic, i.e. some means of paring the list of sibling values down to a single value. What kind of interface should be provided? That will depend heavily on the language. In a functional language, for example, that might mean enabling users to specify filtering functions that whittle the siblings list down to a single “correct” value. To see conflict resolution in our official clients in action, see our tutorials for Java, Ruby, and Python.
Riak Data Types
In version 2.0, Riak added support for conflict-free replicated data types (aka CRDTs), which we call Riak Data Types. These five special Data Types—flags, registers, counters, sets, and maps—enable you to forgo things like application-side conflict resolution because Riak handles the resolution logic for you (provided that your data can be modeled as one of the five types). What separates Riak Data Types from other Riak objects is that you interact with them transactionally, meaning that changing Data Types involves sending messages to Riak about what changes should be made rather than fetching the object and modifying it on the client side.
This means that your client interface needs to enable users to modify
the Data Types as much as they need to on the client side before
committing those changes to Riak. So if an application needs to add five
counters to a map and then remove items from three different sets within
that map, it should be able to commit those changes with one message
to Riak. The official Python client, for example, has a
function that sends all client-side changes to Riak at once, plus a
function that fetches the current value of the type from Riak with no
regard for client-side changes (in fact, reloading the type will
actually wipe out all client-side changes).
One of the most important features introduced in Riak 2.0 is security. When enabled, all clients connecting to Riak, regardless of which security source is chosen, must communicate with Riak over a secure SSL connection rooted in an x.509-certificate-based Public Key Infrastructure (PKI). If you want your client's users to be able to take advantage of Riak security, you'll need to create an SSL interface. Fortunately, there are OpenSSL and other libraries for all major languages. To see SSL in action in our official clients, see our tutorials for Java, Ruby, Python, and Erlang.
Features That Don't Require Client Changes
The following features that became available in Riak 2.0 shouldn't require any changes to client libraries:
- Strong consistency — While adding strong consistency has entailed a lot of changes within Riak itself, K/V operations involving strongly consistent data function just like their eventually consistent counterparts in most respects. The one small exception is that performing object updates without first fetching the object will necessarily fail because the initial fetched object contains the object's causal context, which is necssary for strongly consistent operations. It may be a good idea to add this requirement to your client documentation.
- New configuration system — Configuration has been drastically simplified in Riak 2.0, but these changes won't have a direct impact on client interfaces.
- Dotted version vectors —
While dotted version vectors (DVVs) are superior to the older
vector clocks in preventing
problems like sibling explosion, client libraries interact with DVVs
just like they interact with vector clocks. In fact, our Protocol
Buffers messages still use a
vclockfield for both vector clocks and DVVs, for the sake of backward compatibility.