Working with Redis Data Types

Written by Dan Sackett on November 13, 2014

After you have an understanding about the types of data Redis can store, you're ready to take on manipulating those data types with the many commands offered.

If you haven't figured out the data types and how they work, check out my post from yesterday for a brief introduction. When you have it under control, your next step to mastering Redis is learning how to get subsets of data, adding and removing from data types in more useful ways, and using set arithmetic. There's a lot of commands that Redis provides fresh out of the box so don't worry if this post overwhelms you. When you get stuck or if you ever need to check the commands when working, you will always be able to use the help command in the redis-cli.

$ redis-cli
redis 127.0.0.1:6379> help @string

This will echo a list of the commands and arguments you can use with "strings". Redis allows auto completion so using the @ symbol will allow you to scroll through all of the help files.

With that out of the way, let's get into the string commands. As a reminder, all commands from here on will be based in the redis-cli module.

Strings

Strings are the simplest data type in redis and are simple key value entries. Let's create two keys to work with.

redis 127.0.0.1:6379> SET key value
OK
redis 127.0.0.1:6379> SET key2 2
OK
redis 127.0.0.1:6379> GET key
"value"
redis 127.0.0.1:6379> GET key2
"2"

Strings use simple SET and GET commands to save and fetch data. We also have other means of setting and getting data though.

redis 127.0.0.1:6379> GETSET key 1
"value"
redis 127.0.0.1:6379> GETSET dj 1
(nil)

We can think of the the GETSET command as a get and update or create command. If the key exists, we will get the key, give it a new value, and return the old value. If the key does not exist, we create it and it returns nil since there is no previous value. How about if we want to get and set multiple values?

redis 127.0.0.1:6379> MSET hello world boom boom
OK
redis 127.0.0.1:6379> MGET hello boom
1) "world"
2) "boom"

Using the MSET command, we can do multiple key value pairs in a row where the arguments will be MSET key value [key value ...]. The MGET command takes multiple keys and returns their values. What about when we want to be safe about updating values?

redis 127.0.0.1:6379> SETNX boom bam
(integer) 0
redis 127.0.0.1:6379> MSETNX hello dan bam boom
(integer) 0

Using the SETNX and MSETNX commands will say "set these values only if the key does not exist". What you'll notice with the multiple set command is that in the presence of a key that is already set, keys that are not set will not be set either. This is a safety feature which allows you to write code that doesn't need to worry about existence checks.

Another common thing to do with keys is add and expiration or TTL.

redis 127.0.0.1:6379> SETEX my_key 5 gone
OK
redis 127.0.0.1:6379> GET my_key
"gone"
redis 127.0.0.1:6379> GET my_key
(nil)

With the SETEX command, we set an expiration in seconds and when that passes, the key and value is deleted. This is great for expiring cache.

Another thing we can do with strings is increment and decrement integer values.

redis 127.0.0.1:6379> GET key2
"2"
redis 127.0.0.1:6379> INCR key2
(integer) 3
redis 127.0.0.1:6379> GET key2
"3"
redis 127.0.0.1:6379> INCRBY key2 10
(integer) 13
redis 127.0.0.1:6379> GET key2
"13"
redis 127.0.0.1:6379> DECR key2
(integer) 12
redis 127.0.0.1:6379> GET key2
"12"
redis 127.0.0.1:6379> DECRBY key2 10
(integer) 2
redis 127.0.0.1:6379> GET key2
"2"

Starting with the INCR and INCRBY command, we will increment an integer value. With INCR, we increase by 1. With INCRBY, we can define the increment amount. Of the two, INCR is used a lot to get the next value allowing consecutive sequences.

Of course, we have the opposite with DECR and DECRBY commands to go the opposite way.

Integer values aren't the only things we can modify, we can also work with normal string values.

redis 127.0.0.1:6379> set hello world
OK
redis 127.0.0.1:6379> GET hello
"world"
redis 127.0.0.1:6379> APPEND hello !
(integer) 6
redis 127.0.0.1:6379> GET hello
"world!"
redis 127.0.0.1:6379> SETRANGE hello 5 " and goodbye"
(integer) 17
redis 127.0.0.1:6379> GET hello
"world and goodbye"
redis 127.0.0.1:6379> STRLEN hello
(integer) 17
redis 127.0.0.1:6379> SUBSTR hello 6 8
"and"

This miscellaneous group of commands are all pretty self explanatory but I'll help out. We can use the APPEND command to add a value to the end of an existing value. We can also use the SETRANGE command to change only a portion of a string value. We can use the STRLEN command to get the length of our string and finally we can use the SUBSTR to get a section of our string.

There's a lot we can do with strings and if you ever get stuck or can't remember the correct command, use HELP @string to see them all.

Lists

Lists are ordered data types allowing us to store multiple values for a single key. Let's create a basic list to play with.

redis 127.0.0.1:6379> LPUSH list 1
(integer) 1
redis 127.0.0.1:6379> LPUSH list 2
(integer) 2
redis 127.0.0.1:6379> LPUSH list 3
(integer) 3
redis 127.0.0.1:6379> LPUSH list 4
(integer) 4
redis 127.0.0.1:6379> RPUSH list 0
(integer) 5
redis 127.0.0.1:6379> LRANGE list 0 -1
1) "4"
2) "3"
3) "2"
4) "1"
5) "0"

We use the LPUSH command to prepend a value onto a list and RPUSH to append to it. Lists allow duplicates so we will always be able to add values without worrying. To see our entire list, we can use the LRANGE command to get a set of values based on start and end indices. We can also remove values pretty easily.

redis 127.0.0.1:6379> LPOP list
"4"
redis 127.0.0.1:6379> RPOP list
"0"
redis 127.0.0.1:6379> LRANGE list 0 -1
1) "3"
2) "2"
3) "1"

We use the LPOP and RPOP much like their counterpart PUSH commands to remove items from the beginning and end of lists. Since lists are ordered, we can do index based commands.

redis 127.0.0.1:6379> LINDEX list 0
"3"
redis 127.0.0.1:6379> LINSERT list BEFORE 3 4
(integer) 4
redis 127.0.0.1:6379> LINSERT list AFTER 1 0
(integer) 5
redis 127.0.0.1:6379> LRANGE list 0 -1
1) "4"
2) "3"
3) "2"
4) "1"
5) "0"

Indices will allow us to use the LINDEX command to get the value based on an index. We can also insert values BEFORE and AFTER indices with the LINSERT command. One thing to note about the LINSERT command is that we actually reference the value and not the index. It's called a "pivot" and still confuses me actually.

Like strings, we can allow Redis to worry about existence of a list.

redis 127.0.0.1:6379> LPUSHX not_a_list 5
(integer) 0
redis 127.0.0.1:6379> RPUSHX not_a_list 5
(integer) 0

Using LPUSHX and RPUSHX will only add to a list if the list exists beforehand. Some other various commands include:

redis 127.0.0.1:6379> LLEN list
(integer) 5
redis 127.0.0.1:6379> LTRIM list 0 3
OK
redis 127.0.0.1:6379> LRANGE list 0 -1
1) "4"
2) "3"
3) "2"
4) "1"
redis 127.0.0.1:6379> RPOPLPUSH list new_list
"1"
redis 127.0.0.1:6379> LRANGE list 0 -1
1) "4"
2) "3"
3) "2"
redis 127.0.0.1:6379> LRANGE new_list 0 -1
1) "1"

Starting at the top, we can get the length of our list with the LLEN command. If we want to trim our list to a specific subset of it, we can use the LTRIM command to make the new list based on indices.

A cool command is RPOPLPUSH which will the the last value from one list and append it to another list. When we run this, notice that it returns the popped value.

The last few commands we can use with lists are repeats in a sense.

redis 127.0.0.1:6379> BLPOP list 5
1) "list"
2) "4"
redis 127.0.0.1:6379> BRPOP list 5
1) "list"
2) "2"
redis 127.0.0.1:6379> LRANGE list 0 -1
1) "3"
redis 127.0.0.1:6379> BRPOPLPUSH list new_list 5
"3"
redis 127.0.0.1:6379> LRANGE list 0 -1
(empty list or set)
redis 127.0.0.1:6379> LRANGE new_list 0 -1
1) "3"
2) "1"

Each of these commands you have seen before, but we prepend them with a "B" which stands for blocking. We set a timeout in seconds for each of them so if there is nothing available to pop, we will wait for the timeout until the command finishes.

As you can see, there's a lot we can do with lists. When you forget the commands, run HELP @list.

Sets

Like lists, sets are unordered. Unlike lists, each member of a set must be unique. Let's make a set to play with.

redis 127.0.0.1:6379> SADD my_set hello
(integer) 1
redis 127.0.0.1:6379> SADD my_set world
(integer) 1
redis 127.0.0.1:6379> SADD my_set dan
(integer) 1
redis 127.0.0.1:6379> SMEMBERS my_set
1) "dan"
2) "world"
3) "hello"

We use the SADD command to add members to our set and the SMEMBERS command to view everything in our set. These kind of makes sense when you think of them as "set add" and "set members". We can also remove members in two ways.

redis 127.0.0.1:6379> SREM my_set dan
(integer) 1
redis 127.0.0.1:6379> SMEMBERS my_set
1) "world"
2) "hello"
redis 127.0.0.1:6379> SPOP my_set
"world"
redis 127.0.0.1:6379> SMEMBERS my_set
1) "hello"

Using the SREM command will allow us to select the item we want to remove. If we want to remove a random member though, we can use the SPOP command which returns the randomly removed member. This is random remember because sets have no order. We can also do some other common commands like we've seen with lists.

redis 127.0.0.1:6379> SCARD my_set
(integer) 1
redis 127.0.0.1:6379> SISMEMBER my_set dan
(integer) 0
redis 127.0.0.1:6379> SRANDMEMBER my_set
"hello"

Using the SCARD command to get the length of the set and the SISMEMBER command to check if a member value belongs to the set. If we want to get a random member, we can use the SRANDMEMBER command. These are all cool, but what really sets a set apart is that you can do set arithmetic on them.

redis 127.0.0.1:6379> SADD my_set world
(integer) 1
redis 127.0.0.1:6379> SADD my_set dan
(integer) 1
redis 127.0.0.1:6379> SADD my_second_set dan
(integer) 1
redis 127.0.0.1:6379> SADD my_second_set boom
(integer) 1
redis 127.0.0.1:6379> SADD my_second_set bam
(integer) 1
redis 127.0.0.1:6379> SMEMBERS my_set
1) "dan"
2) "world"
3) "hello"
redis 127.0.0.1:6379> SMEMBERS my_second_set
1) "dan"
2) "bam"
3) "boom"

Before I get to the set arithmetic, I now have two sets that I'm working with as you can see above.

redis 127.0.0.1:6379> SDIFF my_set my_second_set
1) "hello"
2) "world"
redis 127.0.0.1:6379> SDIFF my_second_set my_set
1) "bam"
2) "boom"
redis 127.0.0.1:6379> SINTER my_set my_second_set
1) "dan"
redis 127.0.0.1:6379> SUNION my_set my_second_set
1) "dan"
2) "bam"
3) "hello"
4) "world"
5) "boom"

If you've worked with sets in Python before or in a math class, then these commands will make sense. The SDIFF command will subtract one set from another. The order that we do the subtraction matters as you can see because the values returned are those in the first set that do not exist in the second set. When we want to find the members that exist in both sets, we use the SINTER command. When we want all of the unique members from multiple sets then we can use the SUNION command.

Sets are very good for handling unique data and running calculations across multiple sets. With each of these commands, these lists are calculated but not stored. We can store them though with a few different commands.

redis 127.0.0.1:6379> SDIFFSTORE my_new_set my_second_set my_set
(integer) 2
redis 127.0.0.1:6379> SMEMBERS my_new_set
1) "bam"
2) "boom"
redis 127.0.0.1:6379> SUNIONSTORE my_new_set my_second_set my_set
(integer) 5
redis 127.0.0.1:6379> SMEMBERS my_new_set
1) "dan"
2) "world"
3) "bam"
4) "boom"
5) "hello"
redis 127.0.0.1:6379> SINTERSTORE my_new_set my_second_set my_set
(integer) 1
redis 127.0.0.1:6379> SMEMBERS my_new_set
1) "dan"

By adding the STORE keyword to the end of the other commands, we say that we want to persist the data. Also, our command now looks like SINTERSTORE new_set old_set1 old_set2. These are very useful if you want to save the state of the calculation to use later.

As with the other sections, we can run HELP @set to see the list of set commands in the redis-cli.

Sorted Sets

While sets are unordered, a sorted set is ordered based on scores that you give to the members. Let's create a sorted set to play with.

redis 127.0.0.1:6379> ZADD my_zset 10 first
(integer) 1
redis 127.0.0.1:6379> ZADD my_zset 50 second
(integer) 1
redis 127.0.0.1:6379> ZADD my_zset 37 third
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1
1) "first"
2) "third"
3) "second"
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1 WITHSCORES
1) "first"
2) "10"
3) "third"
4) "37"
5) "second"
6) "50"

We add members to a zset with the ZADD command which expects an integer score and then the member itself. When view a zset, we use the ZRANGE command much like lists. This is because this is ordered and indices give us expected results. There are also a number of other ways we can return our data:

redis 127.0.0.1:6379> ZRANGEBYSCORE my_zset 0 40 WITHSCORES
1) "first"
2) "10"
3) "third"
4) "37"
redis 127.0.0.1:6379> ZRANGEBYSCORE my_zset 0 40 WITHSCORES LIMIT 1 1
1) "third"
2) "37"
redis 127.0.0.1:6379> ZREVRANGE my_zset 0 -1 WITHSCORES
1) "second"
2) "50"
3) "third"
4) "37"
5) "first"
6) "10"
redis 127.0.0.1:6379> ZREVRANGEBYSCORE my_zset 40 0 WITHSCORES
1) "third"
2) "37"
3) "first"
4) "10"

We can use the ZRANGEBYSCORE command to get a range of items based on their score rather than their indices. With that, we have additional options that we can use with these commands. Namely, we can add with WITHSCORES as we've seen and a LIMIT based on indices. When we want to get the order of members from highest to lowest, we can use the ZREVRANGE command. With that, we also have access to a ZREVRANGEBYSCORE command which will get the highest to lowest members based on their scores.

When we want to inspect members of a zset, we have a few tools too:

redis 127.0.0.1:6379> ZCARD my_zset
(integer) 3
redis 127.0.0.1:6379> ZCOUNT my_zset 10 40
(integer) 2
redis 127.0.0.1:6379> ZRANK my_zset third
(integer) 1
redis 127.0.0.1:6379> ZSCORE my_zset third
"37"

When we want to get the size of our zset, we use the ZCARD command. As well, if we want to get the number of members in the zset between a certain score range, we use the ZCOUNT command. The ZRANK command will get the index based on a member while the ZSCORE will return the score for the member.

When removing members, we can do one by one or a range:

redis 127.0.0.1:6379> ZREM my_zset third
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1
1) "first"
2) "second"
redis 127.0.0.1:6379> ZREMRANGEBYRANK my_zset 0 0
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1
1) "second"
redis 127.0.0.1:6379> ZREMRANGEBYSCORE my_zset 0 10
(integer) 0
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1
1) "second"

When we want to select a specific member to remove from a set, we can use the ZREM command. When we want to remove a range of members based on index, we can use the ZREMRANGEBYRANK command. When we want to remove that range based on the score, we can use the ZREMRANGEBYSCORE command. These few commands will give us flexibility when cleaning up our zsets.

One thing you'll do a lot with zsets most likely is updating a score value:

redis 127.0.0.1:6379> ZINCRBY my_zset 100 second
"150"
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1 WITHSCORES
1) "second"
2) "150"

We use the ZINCRBY command to increase our score by a specific amount. Think of a game with constantly changing scores and how this could be useful.

Since this is a set, we also have a few set arithmetic commands we can use. Let me first create another zset and fill ours again.

redis 127.0.0.1:6379> ZADD my_zset 20 hello
(integer) 1
redis 127.0.0.1:6379> ZADD my_zset 21 world
(integer) 1
redis 127.0.0.1:6379> ZADD my_zset 9000 dan
(integer) 1
redis 127.0.0.1:6379> ZADD my_second_zset 5 jesse
(integer) 1
redis 127.0.0.1:6379> ZADD my_second_zset 5 matt
(integer) 1
redis 127.0.0.1:6379> ZADD my_second_zset 5 chris
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_zset 0 -1 WITHSCORES
1) "hello"
2) "20"
3) "world"
4) "21"
5) "second"
6) "150"
7) "jesse"
8) "15"
redis 127.0.0.1:6379> ZRANGE my_second_zset 0 -1 WITHSCORES
1) "chris"
2) "5"
3) "jesse"
4) "5"
5) "matt"
6) "5"

With those two zsets, we have two commands we can use:

redis 127.0.0.1:6379> ZINTERSTORE my_new_zset 2 my_zset my_second_zset
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_new_zset 0 -1 WITHSCORES
1) "jesse"
2) "20"
redis 127.0.0.1:6379> ZINTERSTORE my_new_zset 2 my_zset my_second_zset WEIGHTS 2 3
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_new_zset 0 -1 WITHSCORES
1) "jesse"
2) "45"
redis 127.0.0.1:6379> ZINTERSTORE my_new_zset 2 my_zset my_second_zset WEIGHTS 2 3 AGGREGATE SUM
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_new_zset 0 -1 WITHSCORES
1) "jesse"
2) "45"
redis 127.0.0.1:6379> ZINTERSTORE my_new_zset 2 my_zset my_second_zset WEIGHTS 2 3 AGGREGATE MIN
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_new_zset 0 -1 WITHSCORES
1) "jesse"
2) "15"
redis 127.0.0.1:6379> ZINTERSTORE my_new_zset 2 my_zset my_second_zset WEIGHTS 2 3 AGGREGATE MAX
(integer) 1
redis 127.0.0.1:6379> ZRANGE my_new_zset 0 -1 WITHSCORES
1) "jesse"
2) "30"

The first command is ZINTERSTORE. Much like a normal set, this will create a new zset based on the values that overlap. One thing to note is that the command takes the form of ZINTERSTORE new_zset num_of_zsets old_zset1 old_zset2. We have to directly specify the number of zsets we want to process basically. As you can see, we can also add some options to our command. The first is the WEIGHTS option which specifies a multiplier. When the zsets are processed, they will be multiplied by their respective weights before being aggregated. By default, it sums the like member scores. You can use the AGGREGATE option though to choose between SUM, MIN, and MAX where MIN and MAX will return the lower or higher of the two resulting scores after they are multiplied by their weights.

Confused?

It might seems odd, but play with a few examples in the redis-cli to really understand it. We can also do unionstores:

redis 127.0.0.1:6379> ZUNIONSTORE my_new_zset 2 my_zset my_second_zset
(integer) 7
redis 127.0.0.1:6379> ZRANGE my_new_zset 0 -1 WITHSCORES
 1) "chris"
 2) "5"
 3) "matt"
 4) "5"
 5) "hello"
 6) "20"
 7) "jesse"
 8) "20"
 9) "world"
10) "21"
11) "second"
12) "150"
13) "dan"
14) "9000"

The ZUNIONSTORE command takes the exact same structure as the ZINTERSTORE command and the same options as well. The only difference being that the resulting zset will have all unique members in all zsets.

If you want to look up the commands, run HELP @sorted_set in the redis-cli to see the reference.

Hashes

Hashes allow us to store multiple key-value pairs within a single key. They are unordered by nature. Let's create one to play with.

redis 127.0.0.1:6379> HSET my_hash dan sackett
(integer) 1
redis 127.0.0.1:6379> HMSET my_hash captain hook peter pan
OK
redis 127.0.0.1:6379> HKEYS my_hash
1) "dan"
2) "captain"
3) "peter"
redis 127.0.0.1:6379> HVALS my_hash
1) "sackett"
2) "hook"
3) "pan"
redis 127.0.0.1:6379> HGETALL my_hash
1) "dan"
2) "sackett"
3) "captain"
4) "hook"
5) "peter"
6) "pan"

Already I'm showing off a few commands. We can set a single entry in the hash with HSET and we can set multiple values at once with HMSET. If we want to get all of the keys in our hash we use the HKEYS command and when we just want the values we can use the HVALS command. We can get all of our keys and values with the HGETALL command. Pretty straightforward stuff compared to zsets if you ask me.

How about if we want individual values based on keys?

redis 127.0.0.1:6379> HGET my_hash dan
"sackett"
redis 127.0.0.1:6379> HMGET my_hash dan captain
1) "sackett"
2) "hook"

The HGET command will fetch a single value while the HMGET command will fetch multiple values based on keys. And what about existence?

redis 127.0.0.1:6379> HEXISTS my_hash dan
(integer) 1
redis 127.0.0.1:6379> HSETNX my_hash dan sack
(integer) 0
redis 127.0.0.1:6379> HGETALL my_hash
1) "dan"
2) "sackett"
3) "captain"
4) "hook"
5) "peter"
6) "pan"

We can check existence directly with the HEXISTS command. If we only want to add an entry if it doesn't exist though, we can use the HSETNX command. Finally, let's look at the few other commands we have available to hashes.

redis 127.0.0.1:6379> HDEL my_hash peter
(integer) 1
redis 127.0.0.1:6379> HGETALL my_hash
1) "dan"
2) "sackett"
3) "captain"
4) "hook"
redis 127.0.0.1:6379> HSET my_hash number 1
(integer) 1
redis 127.0.0.1:6379> HINCRBY my_hash number 20
(integer) 21
redis 127.0.0.1:6379> HGETALL my_hash
1) "dan"
2) "sackett"
3) "captain"
4) "hook"
5) "number"
6) "21"
redis 127.0.0.1:6379> HLEN my_hash
(integer) 3

We delete entries with the HDEL command. When we have integer values in our hash, we can use the HINCRBY command to increment them programattically. Finally, we can use the HLEN command to find out the length of our hash.

If you need a reference, run HELP @hash to see the commands available.

Generic Commands

Wrapping up this post, I wanted to touch on a few of the generic commands that you'll run in Redis.

redis 127.0.0.1:6379> DEL my_hash
(integer) 1
redis 127.0.0.1:6379> SET test_key value
OK
redis 127.0.0.1:6379> EXPIRE test_key 20
(integer) 1
redis 127.0.0.1:6379> KEYS *
1) "test_key"
redis 127.0.0.1:6379> PERSIST test_key
(integer) 1
redis 127.0.0.1:6379> KEYS *
1) "test_key"
redis 127.0.0.1:6379> RANDOMKEY
"test_key"
redis 127.0.0.1:6379> TTL test_key
(integer) -1
redis 127.0.0.1:6379> RENAME test_key new_name
OK
redis 127.0.0.1:6379> keys *
1) "new_name"
redis 127.0.0.1:6379> TYPE new_name
string

Starting from the top, we can use the DEL command to delete any type of key. We can set an expiration on keys with the EXPIRE command specifying seconds. There is also an EXPIREAT command which takes a unix timestamp. When we want to see keys based on a regex pattern, we can use the KEYS command. For keys that you want to persist when they have an expiration set, we can use the PERSIST command. When we want to see the key's time to live (TTL) we use the TTL command. We also can get a random key with RANDOMKEY and we can rename keys with the RENAME command.

One very import command is the TYPE command. When you have a data base of lots of keys, it's important to know what kind of data type it is so you know how to manipulate it. The TYPE command will help you here.

There are a few other commands available here and you can learn about them with HELP @generic in the redis-cli.

Conclusion

Hopefully the exhaustive list of commands above will help you better understand how to work with Redis data types. There are a lot of commands available to you so don't hesitate to use the HELP command to refresh your memory when you need it.

Tomorrow I want to get into a viable use case for Redis.


redis

comments powered by Disqus