[NoSQL Redis] redis-dump: Backup and restore your Redis data to and from JSON

Posted on 2021-04-17 Edited on 2024-09-12 In Cloud Native , Storage , Redis Views: Word count in article: 4.4k Reading time ≈ 4 mins.

redis-dump

Backup and restore your Redis data to and from JSON.

Installation

1	$ gem install redis-dump

Or of you want to run it directly:

$ git clone https://github.com/delano/redis-dump.git

$ cd redis-dump

$ gem install redis

$ gem install uri-redis

$ gem install yajl-ruby

$ gem install drydock

$ ruby -rubygems bin/redis-dump

Usage

There are two executables: redis-dump and redis-load.

$ redis-dump

$ redis-dump -u 127.0.0.1:6371 > db_full.json

$ redis-dump -u 127.0.0.1:6371 -d 15 > db_db15.json

$ < db_full.json redis-load

$ < db_db15.json redis-load -d 15

# OR
$ cat db_full | redis-load

$ cat db_db15.json | redis-load -d 15

# You can specify the redis URI via an environment variable
$ export REDIS_URI=127.0.0.1:6371

$ redis-dump

# If your instance uses a password (such as on RedisToGo), you
# can specify the Redis URL as such:
# :<password>@<domain>:<port>
# Note the leading colon is important for specifying no username.
$ redis-dump -u :[email protected]:9055

Output format

All redis datatypes are output to a simple JSON object. All objects have the following 5 fields:

db (Integer)
key (String)
ttl (Integer): The amount of time in seconds that the key will live. If no expire is set, it’s -1.
type (String), one of: string, list, set, zset, hash, none.
value (String): A JSON-encoded string. For keys of type list, set, zset, and hash, the data is given a specific structure (see below).

Here are examples of each datatype:

{"db":0,"key":"hashkey","ttl":-1,"type":"hash","value":{"field_a":"value_a","field_b":"value_b","field_c":"value_c"},"size":42}
{"db":0,"key":"listkey","ttl":-1,"type":"list","value":["value_0","value_1","value_2","value_0","value_1","value_2"],"size":42}
{"db":0,"key":"setkey","ttl":-1,"type":"set","value":["value_2","value_0","value_1","value_3"],"size":28}
{"db":0,"key":"zsetkey","ttl":-1,"type":"zset","value":[["value_0","100"],["value_1","100"],["value_2","200"],["value_3","300"],["value_4","400"]],"size":50}
{"db":0,"key":"stringkey","ttl":79,"type":"string","value":"stringvalue","size":11}

Important note about TTLs

One of the purposes of redis-dump is the ability to restore the database to a known state. When you restore a redis database from a redis-dump file, the expires are reset to their values at the time the dump was created. This is different from restoring from Redis’ native .rdb or .aof files (expires are stored relative to the actual time they were set).

Output directly to an encrypted file

For most sensitive data, you should consider encrypting the data directly without writing first to a temp file. You can do this using the power of gpg and file descriptors. Here are a couple examples:

Encrypt the data (interactive)

$ redis-dump -u 127.0.0.1:6371 -d 15 | gpg --force-mdc -v -c > path/2/backup-db15.json.gpg

# Encrypt the data (automated)
$ redis-dump -u 127.0.0.1:6371 -d 15 | 3</path/2/passphrase.txt gpg --force-mdc -v -c --passphrase-fd 3 > path/2/backup-db15.json.gpg

# Decrypt the file (interactive)
$ gpg path/2/backup-db15.json.gpg
Enter passphrase: *******

# Decrypt the file (automated)
$  3</path/2/passphrase.txt gpg --passphrase-fd 3 path/2/backup-db15.json.gpg

Loading data with binary strings

If you have binary or serialized data in your Redis database, the YAJL parser may not load your dump file because it sees some of the binary data as ‘invalid bytes in UTF8 string’. If you are certain that your data is binary and not malformed UTF8, you can use the -n flag to redis-load to tell YAJL to not check the input for UTF8 validity. Use with caution!