Redis Lua Scripting

In this article, we will explore how Lua scripts work. We will also see when Lua scripting can be used for specific use-cases which can not be done in any other efficient way.

Consider a case, where we have a leaderboard that is maintained in a Redis zset, now for our use case, for a given user we want to get that user's rank and that user's neighbors in the leaderboard. On the face, of it, the problem statement looks very easy, lets's visualize our leaderboard and see how it can be done.

The obvious solution is 2 commands back to back right?

• 1st get the user's rank, using zrank leaderboard Heather, which will return 5.
• Then use the above rank, to get the lower and upper bound, let's say we want 3 from the top and 3 below, so we can just use zrange leaderboard 2 8.

While the above looks good, it is buggy, because the leaderboard can change between zrank and zrange Redis calls.

The solution that could come to mind is to use redis-pipeline, however, the pipeline can not be used as zrank's output need to be zrange's input, which can not be used in the same pipeline.

One solution that would actually work is to use take some kind of lock before getting the user's rank and then use zrange and then remove the lock. And before, writing to the leaderboard, we have to check for the lock, and if the lock is there, retry until the lock the removed. The lock approach is to make sure, that no writes happen in between zrank and zrange. While this approach works, we have obviously increased the complexity of the entire operation.

An elegant way to handle our use case is to use Lua scripts. Redis lets us upload and execute Lua scripts on the server and because scripts execute on the server, reading and writing data from scripts is very efficient.

Also, Redis guarantees the script's atomic execution. While executing the script, all server activities are blocked during its entire runtime. Let's start with a simple hello world example.

> EVAL "return 'Hello, world!'" 0
"Hello, world!"

While we won't cover the Lua scripts basics in this article, let's see, how we can solve the problem at hand by writing a simple lua script.

local rank = redis.call('zrank', KEYS[1], ARGV[1]);
local min = math.max(rank - ARGV[2], 0);
local max = rank + ARGV[2];
local ldb = redis.call('zrange', KEYS[1], min, max);
return {rank+1, ldb};

Let's break down the script line by line:

• In the 1st line, we are getting the user's rank, KEYS[1] is the leaderboard name and ARGV[1] is the username for which we want to get the neighbor's information.
• 2nd and 3rd lines are just to get the lower and upper bound for the range cmd. In ARGV[2], we are getting the offset for the leaderboard. For min_value we have a max check so that it does not go below 0.  
• In the 4th line, we are getting the leaderboard.
• In the 5th line, we have initialized an array with the user rank and neighbors and returned it.

While the above works, sending this big script every time from the client is not efficient, thankfully, Redis has APIs with which we can save the scripts on the server and use that instead of calling these large scripts from the client every time. Please note that the Redis script cache is always volatile and is not persisted. The cache may be cleared when the server restarts, during fail-over when a replica assumes the master role, or explicitly by SCRIPT FLUSH. That means that cached scripts are ephemeral, and the cache's contents can be lost at any time.

Redis provides SCRIPT LOAD <script>, which returns the SHA1 for the script, after which EVALSHA <sha1> can be used to invoke the script. Let's have a look.

Conclusion:

1. Since scripts execute in the server, reading and writing data from scripts is very efficient. Data locality reduces overall latency and saves networking resources.
2. Lua scripts can be used where atomicity is needed and pipelines/transactions can not be used. Redis guarantees atomicity for lua scripts.
3. EVAL command is used to run the scripts, however, sending lengthy scripts is not efficient, in that case, we can save the script on the server using SCRIPT LOAD <script>, and then use EVALSHA <SHA> command for script execution.
4. Redis script cache is always volatile and is not persisted.

Resources: