network background popup

Nginx with Stream Module Dynamic Upstream CNAME

Oct 12th 2017

In the age on scalable web applications , many organizations turn to cloud-based server hosting to dynamically add additional servers during peak usage, or attain redundancy by having multiple geographic web \-server locations. One of the methods used for this is DNS CNAME resolution. Using this option with various cloud hosting providers that support it can allow the following:

1) Load Balancing.

The CNAME DNS record can be configured to respond to requests with more then one IP Address, allowing load balancing of multiple origin servers, dynamically scaled by the cloud hosting providers DNS service.

2) Global Server Load Balancing.

The cloud hosting DNS can provide different record results in different geographic areas of the world. This can be achieved based on Geo-Location or other methods.

3) Fail-over redundancy.

Since the CNAME record is controlled by the cloud provider, upon failure of one data center, all records pointing to a geographic location which has failed can be automatically switched to the remaining data-center.

Is there a way Nginx can perform Dynamic DNS resolution, for CNAME or other records used as Upstreams/Origins?

Reverse proxies have been a core component of our service since 2007, to say we are experienced in this area is an understatement. Which is why when a customer of ours was having trouble configuring their open-source Nginx to use cnames for their load balancing configuration, they came to us to see if we had any advice or ideas (one of the many benefits being part of a fully managed DDoS mitigation service).

The challenge the customer was facing, and which some of you Nginx administrators may be aware of, is that the open-source version of Nginx does not have a built in dynamic DNS resolver. Essentially it will only resolve domains initially on web-server "start", and "reload", but will not update the record if a DNS record changes during running operation.

After doing some research on various forums and testing in our labs, we identified that in order to use open-source Nginx to dynamically resolve domains, one would have set the domain in a variable, which would then cause Nginx to resolve the domain in the variable dynamically, and according to Nginx's DNS Cache/TTL .

The variable is then used in the "proxy_pass" directive to send the visitor to the correct origin without requiring a reload to be kept up to date.


**There is a problem with using the above workaround for Nginx's "Stream" module; The "Set” directive does not exist.

If you attempt to perform the same method on a TCP Pass-through using Nginx Stream, you will find that since the 'set $variable "value"; ' method is not available within Nginx Stream. The previous method cannot be used.

Is there a way to perform dynamic DNS resolution within open-source Nginx's stream module, or is a 3rd party module that could be used?

Although there is a “stream-lua-nginx” module by Openresty team being developed that could be used for such a purpose, we are not aware of any free 3rd party Dynamic DNS resolution modules that work with Stream. There is however a way to use essentially the same method as used with the Nginx HTTP Proxy, by using the Nginx Stream Map directive.


Above is the relevant configuration file snippet.

**This configuration snippet requires that you have a base nginx.conf configuration already setup. Included in the example are the portions of the configuration that should be present within the Nginx "stream" directive.

***Please keep in mind the following facts:

1) Fail-over / Load Balancing behavior works differently then standard Nginx upstreams.

Instead of using Nginx upstream load balancing or Passive health-checks, Load Balancing and Redundancy should be handled by the CNAME DNS service itself; Nginx "Upstream" directives are not used in this case, so there is no way to mark a server as down. Since there is more then one worker process in any deployed configuration, Round Robin DNS , where a Nameserver lookup returns more then one resulting record can be used to perform load balancing.

2) Each Nginx "Worker" will perform DNS lookups for requests handled by that worker.

This means that if you have 20 worker processes , all 20 will be performing DNS lookups and caching the results , holding the results in memory for the DNS Cache/TTL configured using the Nginx "resolver" directive. You may want to use a local DNS server or caching resolver in order to lower the number of DNS queries made.

Scott Girbav

DOSarrest Internet Security

Senior Network Security Engineer

Added By : Scott Girbav

DDoS Article Categories