We are building a system that runs on our cloud and that needs information from our clients network that must not be exposed openly. We have concluded that the only way this could work, is if our clients can install an agent in their network, that would gather the required information and push it to our system through the internet.
This agent application, cannot communicate to our server using a regular RestAPI or webservice, because we have use cases that require us requesting specific information to the specific agent instances (a request to resync data from client 1 should only go to client 1’s agent). We looked into gRPC streams and keep a stream open with every agent and just routes those requests to the stream, but since our backend has multiple instances, it would be hard to identify and reuse a single stream from different instances.
Then something that works for us, is using Redis streams and create a stream per agent, then our backend would only have to write to the specific agent redis stream (it would know the name of the stream as it would the agent/client identifier) from whichever of our instances.
The concern though, is the “over the internet” part.
Redis has TLS support since version 6, allows us to block certain commands (flushall for example) and supports defining users with access to only specific commands and keys/streams…
I understand exposing Redis over the internet or to any other network that not the one serving redis itself, has always been discouraged (AWS ElastiCache only allows access from within the same VPC/subnet for example). However, everything I find about redis security, say something in the lines of:
Do not publicly expose the Redis server
Since Redis has no default authentication, it does not support encryption. All data is stored in cleartext. An attacker can use FLUSHALL command to delete all key-value data sets…
That type of statement holds very true to older versions of Redis that didn’t provide TLS nor access control. But since those are now supported since version 6, am I crazy to go forward with this?
There’s a concern about rate limiting, which is probably something that is not addressed by Redis itself, but that we can possibly find a solution with Kubernetes or network configuration in the cloud.
So, I know this is an unusual architecture and I am not the only one to think about it, but I am just looking for your opinions on whether exposing Redis streams over the internet for my use case, considering the features that Redis now has, is that much of a security risk/ bad idea overall.