Monitoring Redis with in-Transit Encryption Enabled

By Published Amazon Web Services, Databases, Open Source

Introduction to Redis

Redis is an open source, in-memory, key-value data store for use as a database, cache, message broker, and queue. It delivers sub-millisecond response times enabling millions of requests per second for a variety of real-time applications. Redis is a popular choice for caching, session management, real-time analytics, geospatial, chat/messaging, media streaming, and gaming leaderboards. It offers different levels of on-disk persistence and clustering for high availability

Monitoring Redis on Amazon Web Services

AWS Web console gives you very limited options to monitor Redis health and activity. You can navigate to the ElasticCache dashboard and view service level events (Services > ElasticCache > Events). From CloudWatch, you can look at the following counters individually or create a custom dashboard to include them all in it.

  1. BytesUsedForCache
  2. CacheHits
  3. CacheMisses
  4. CurrConnections
  5. NewConnections
  6. EngineCPUUtilization
  7. Network I/O

If you feel the need to interact with the database and get the following questions answered, you need to connect to the database using a client tool:

  1. Who is connected and from what IP address?
  2. What’s the of Redis database?
  3. What key-values pairs are currently stored in the database?
  4. Which commands are being process by the server right now?

In addition, you can also perform the following actions on the database:

  1. Kill a specific client connection
  2. View current configuration settings
  3. Modify configuration
  4. List all databases
  5. Reset a database or all databases.
  6. Create a new key-value pair.
  7. Retrieve a  key

Connecting to Redis with in-Transit Encryption Enabled

To access data from ElastiCache for Redis nodes enabled with in-transit encryption, you must use a client that works with Secure Socket Layer (SSL). However, redis-cli doesn’t support SSL or Transport Layer Security (TLS). A TLS proxy can be used with redis-cli to enable it for connection to Redis over an encrypted link. “stunnel” is of the TLS wrappers which is available for free.

stunnel TLS Proxy

stunnel is a Transport Layer Security (TLS) offloading and load-balancing proxy. It works as TLS encryption wrapper between remote clients and local or remote servers. non-TLS aware daemons running on your system can communicate with clients over secure TLS channels. Like any other SSL protocol, stunnel requires a certificate to use for client to server communication. Since Redis client is usually used internally only on private networks, you can create a self-signed certificate instead of buying one from a certificate authority such as GeoTrust and Symantec.

Installing stunnel

SSH to your Linux EC2 and login using ‘ec2-user” and the SSH key machine was deployed with. Run the following commands in the provided sequence. You can skip the SSH steps and follow only SSL certificate instruction in case you don’t need to apply it to your Redis server install on EC2. If your using AWS’s Redis cluster, you can connect just with the SSL certificate. Remember to update the connect string with your Redis end-point.

sudo yum install stunnel
cat /etc/stunnel/redis-cli.conf
cd /etc/stunnel
sudo vi redis-cli.conf

;######## /etc/stunnel/redis-cli.conf  #################
fips = no
setuid = root
setgid = root
cert = /etc/stunnel/private.pem
pid = /var/run/stunnel.pid
debug = 7
delay = yes
[redis-cli]
   client = yes
   accept = 127.0.0.1:6379
   connect = clustercfg.agill-test-dev-rg.xyz7ia.use1.cache.amazonaws.com:6379
;######## /etc/stunnel/redis-cli.conf  #################

sudo openssl genrsa -out /etc/stunnel/key.pem 4096   #generate key
sudo openssl req -new -x509 -key /etc/stunnel/key.pem -out /etc/stunnel/cert.pem -days 1826   #generate certificate
cat /etc/stunnel/key.pem /etc/stunnel/cert.pem >private.pem #combine both
sudo chmod 600 /etc/stunnel/key.pem /etc/stunnel/cert.pem /etc/stunnel/private.pem   #lock down sensitive files

sudo stunnel /etc/stunnel/redis-cli.conf	#start the stunnel process
sudo netstat -tulnp | grep -i stunnel	#verify it’s running
sudo pkill stunnel    #terminate stunnel process, once done testing.

Installing Redis Client

SSH to your Linux EC2 and login using ‘ec2-user” and the SSH key machine was deployed with. Run the following commands in the provided sequence. You can optionally combine multiple commands together. They are spread out to enhance readability and add comments againt individual commands wherever needed.

sudo yum install wget 	# GNU network utility to retrieve files from the World Wide Web using HTTP and FTP,
sudo yum install gcc   	#GNU C/C++ compilers, 
which gcc && gcc –version		#see if gcc installed ok.
wget http://download.redis.io/redis-stable.tar.gz	      #download Redis client
tar xvzf redis-stable.tar.gz		#uncompress 
cd redis-stable	#go to source direcroty
make			#compile Redis client 
sudo cp src/redis-cli /usr/local/bin/ 	#copy client binary to user bin, to access from anywhere
sudo chmod 755 /usr/local/bin/redis-cli	#update permissions on client binary

Connecting to Redis

sudo netstat -tulnp | grep -i stunnel
sudo stunnel /etc/stunnel/redis-cli.conf	#start the stunnel, if not running. Can be added to initd.
redis-cli -h localhost
telnet localhost 6379

Useful Commands

Note: Don’t run the “flush*” & “kill*” commands unless you absolutely understand the outcome. They are listed for reference only, should there be a need for it. They are not required for monitoring.

ping	     		#You can play ping pong with the server to verify your connection. Will fail without “auth”.
echo "Hello AGill"	#Verify your connection. Will fail without “auth”.
auth 	     	     	#Authentication password, Auth Code. Good practice to launch Redis cluster with this option 
ping		     	#You can play ping pong with the server to verify your connection
echo "Hello AGill"      #Will respond only if connected to the server.
command count   	#Count of all the Redis commands available to you at this prompt
command    	     	#List of all the Redis commands available to you at this prompt
command info 		#list  specific command(s)
client list     	#list of current connections, their IP addresses
client getname  	#Get client name for your connection
client setname AGill-Redis-Client	#Set client name for your connection
client getname  	#Get client name for your connection
client list  	     	#Get client name for your connection
client kill addr       	#Kill a connection, based on “ip:port” returned from “client list”    
client kill id         	#Kill a connection, based on “id” returned from “client list”   
monitor  	     	#Debugging command to live activity. Streams all the requests being processed by Redis.
	  	     	#Ctl+C to cancel monitoring from “redis-cli”
		     	#Issue “quit” command to stop monitor stream running from “telnet” client
dbsize  	     	#Total number of key/value pairs stored in current database
select   	     	#Select a database. New connections use the database “0”. Not available in cluster.
keys *      	     	#list of all the key/value pairs stored in the database
scan 0	     	     	#Start a new cursor to browse through the stored keys. 
    	     	     	#Will return the value for new cursor for next call, and an array of elements.
scan    	     	#Begin from where first new cursor left off
set agill “Hello Handsome!”  #Create a new key.
get agill    	     	#Retrieve a  key.
lastsave.    	     	#Contents flushed to disk. *seconds since 01/01/1970 UTC, 
     	     	     	#UNIX timestamp to datetime:  https://www.epochconverter.com/
flushdb      		#Delete all the keys stored in the current database.
flushdb  async    	#Delete all the keys stored in the current database.
     	     	     	#Delete occur in background in a different thread, without blocking the server.
flushall      		#Delete all the keys stored in all the existing database.
flushall  async	     	#Delete all the keys stored in all the existing database. 
     	     	     	#Delete occur in background in a different thread, without blocking the server.
quit			#Close the connection to the server

Redis is Single Threaded

Due to the single-threaded nature of Redis, it is not possible to kill a client connection while it is executing a command. From the client point of view, the connection can never be closed in the middle of the execution of a command. However, the client will notice the connection has been closed only when the next command is sent (and results in network error).

References and Further Reading:

What’s Redis
Introduction to Redis
Secure Redis: SSL Added to Redsmin and Clients
Amazon ElastiCache for Redis now supports In-Transit and At-Rest Encryption
CloudWatch Mertices for Redis
stunnel
Redis Commands Reference
Single-threaded nature of Redis

Leave a Reply

Your email address will not be published. Required fields are marked *