Alternatives to Nested Structures in Redis?

redis

I keep running into cases where I have more complicated information to store than what can fit into any of Redis' simple data structures. I still want to use Redis, but I was wondering if there are any standard alternatives people use when ideally they'd like to use a nested structure?

Solution 1:

You have basically two strategies:

you can serialize your complex objects and store them as strings. We suggest json or msgpack for the serialization format. This is easy enough to manipulate from most client-side languages. If server-side access is needed, then a server-side Lua script can easily encode/decode such objects since Redis is compiled with msgpack and json support for Lua.
you can split your objects in different keys. Instead of storing user:id and a complex data structure to this id, you can store several keys such as user:id, user:id:address_list, user:id:document_lists, etc ... If you need atomicity, pipelining MULTI/EXEC blocks can be used to guarantee the data consistency, and aggregate the roundtrips.

See a simple example in this answer:

Will the LPUSH command work on a record that was initialized from JSON?

Finally, Redis is not a document oriented database. If you really have a lot of complex documents, perhaps you could be better served by solutions such as MongoDB, ArangoDB, CouchDB, Couchbase, etc ...

Solution 2:

When you need to modify the object, it's very inefficient to serialize the complex object to string, and save the string to Redis. Since you have to fetch the string back to the client side, deserialize it to an object, modify it, serialize it to string again, and save it back to Redis. Too much work...

Now, it's 2019, and there're some new Redis modules which can enable Redis to support nested data structures, e.g. RedisJSON, redis-protobuf.

Desclaimer: I'm the author of redis-protobuf, so I'll give some examples on this module. Also redis-protobuf is faster and more memory efficient than RedisJSON, since it's use a binary format other than text format to serialize/deserialize the data.

First of all, you need to define your nested data structure in Protobuf format and save it to a local file:

syntax = "proto3";

message SubMsg {
    string s = 1;
    int32 i = 2;
}

message Msg {
    int32 i = 1;
    SubMsg sub = 2;
    repeated int32 arr = 3;
}

Then load the module with the following configure in redis.conf:

loadmodule /path/to/libredis-protobuf.so --dir proto-directory

After that, you can read and write the nested data structure:

PB.SET key Msg '{"i" : 1, "sub" : {"s" : "string", "i" : 2}, "arr" : [1, 2, 3]}'
PB.SET key Msg.i 10
PB.GET key Msg.i
PB.SET key Msg.sub.s redis-protobuf
PB.GET key Msg.sub.s
PB.SET key Msg.arr[0] 2
PB.GET key Msg.arr[0]

Please check the doc for detail. If you have any problem with redis-protobuf, feel free to let me know.

Alternatives to Nested Structures in Redis?

Solution 1:

Solution 2:

Related

Recent Posts