DevOps is a broadly adopted approach that merges development and operations teams into one organizational unit. DevOps enables shared responsibility for development projects, with devs and ops teams working together to build applications that not only have the desired functionality, but are also easy and reliable to deploy and operate. Since the inception of DevOps, it was understood that automation was key to making DevOps work.
The achilles heel of DevOps automation is the Domain Name System (DNS) system. DNS is how systems and services discover and connect to each other, and how users connect to newly-deployed software. DNS has been around for decades, and traditional DNS systems were not designed to operate in modern DevOps environments. Issues include outdated or hard-to-use API's, slow propagation times and poor performance at scale.
Below we review key trends driving the need for DevOps automation and why legacy DNS causes critical problems simply impedes DevOps success.
IaC makes it possible to maintain server environments just like software developers build and maintain source code. Both developers and operations staff can create infrastructure blueprints, which automatically sets up servers with a predictable, predefined set of applications and configuration settings. IaC can take the shape of Configuration as Code (CaC) tools like Terraform, Ansible and Chef. It might also involve containerization tools like Docker and Kubernetes, and cloud automation tools from cloud providers like AWS and Azure.
Legacy DNS can become a challenge and a bottleneck for DevOps teams implementing fully automated deployment processes because they lack critical capabilities:
No native APIs - traditional DNS platforms like BIND or PowerDNS, even some commercial services and appliances have no API; API's aren't built for modern application infrastructures. Traditional offerings' were developed “after the fact,” their reliability performance often do not meet requirements of high scale automation.
Restart on DNS updates - BIND requires a reload of the DNS database when DNS records are updated. As a result, DNS updates must be scheduled in specific and infrequent deployment windows, to ensure minimal disruption to live services. This approach is incompatible with a continuous delivery methodology. Many commercial DNS appliances or services are based on BIND.
Slow propagation- DNS propagation times, which are often 30 minutes or more, are too slow to keep up with infrastructure and deployment
Continuous Integration (CI) is a set of tools and coding practices which encourage developers to commit their code into a version control repository very often, commonly multiple times per day. CI servers integrate code from multiple developers and automatically execute a build, creating a new version of the software which can be run and tested. Doing this often improves collaboration and responsibility, increases quality and ensures teams build software that works and delivers value to customers.
Continuous Delivery (CD) is an extension of CI, in which teams automate not just their build process, but the complete development lifecycle, including testing, staging and production environments. In a mature CD environment, it is possible to push a code change from development, to testing, to staging and finally to production, with the click of a button. Test automation ensures that new software versions can be released quickly and with high quality.
CI/CD accelerates software development, such that dev, test and production environments are deployed on a daily, even hourly basis. Such frequent changes to infrastructure require fast DNS propagation. With traditional tools taking up to 30 minutes to propagate even in a local network, DNS becomes a bottleneck in the dev-test-prod lifecycle.
Modern DevOps teams rely on an ecosystem of tools that helps them manage, monitor and report on the systems they operate. These include, apart from Infrastructure as Code tools mentioned above:
Containerization and deployment tool such as Apache Mesos, Docker and Kubernetes
Network and server monitoring tools such as AWS CloudWatch, Catchpoint and Datadog and Monitis
Application monitoring tools such as Monitor.us, Monitis, New Relic and Pingdom
Log analysis and visualization tools such as Grafana, Logstash and Kibana
Alerting and communication tools such as PagerDuty, Slack and HipChat
DevOps teams are used to integrating their toolset with all parts of the environment to enable seamless management, monitoring and alerting. However, DNS can be an outlier - a part of the application stack that doesn’t play well with the DevOps toolsets and processes.
The DNS Bottleneck and Four Ways to Release It
A common way to automate DNS is using DNS/DDI appliances, which allow you to easily deploy DNS anywhere on the network. These appliances can fully manage internal DNS infrastructure - managing DNS server updates, automatically distributing DNS changes to all appliances, and providing a management interface that lets you centrally control DNS on the network. Most DNS/DDI appliances provide an API that lets you automate DNS operations as part of DevOps processes.
Easy to deploy and manage - if you already have it, leverages existing infrastructure
Lets you automate some complex DNS processes
Smart traffic management - for example, ability to distribute traffic based on performance
Traditional DNS network appliances use the open source BIND server for core DNS functions. As a result, they have many of the same limitations as the BIND platform itself. In addition to the challenges related to cost of ownership of appliances, the following should be considered:
API syntax is complex, non-standard and can be difficult to work with
Each API command takes time to execute because CLI calls are made in the background
Cannot execute multiple API commands rapidly, commands need to be “throttled”
Propagation can take longer, as CLI commands are executed on all appliances
While DNS/DDI provides a working automation solution, it is not easy to use and can significantly delay automated processes due to slow response of BIND-based appliances.
Some organizations try to automate DNS by deploying an open source DNS server like BIND or PowerDNS. These servers do not provide a native API, so automation needs to be done via scripting.
Commands and environment is intuitive and well known to many developers.
Major effort required to script automated actions - these servers were built at a time when Infrastructure as Code did not exist.
Slow response to scripted CLI commands, cannot execute multiple API commands rapidly.
DNS propagation times, which can be 30 minutes or more, cannot keep up with frequent changes to infrastructure.
BIND requires a restart when DNS records are updated, which causes service interruption. This means DNS updates cannot be done as part of the DevOps automation flow
Open source DNS solutions require ongoing management of patches and security updates, and have high maintenance overhead - even more so than other open source tools, because DNS is a critical infrastructure and more exposed to attack.
Amazon, Azure and other cloud providers provide native DNS services which are a good choice for automation. Unlike BIND-based solutions, cloud DNS services offer a modern, native API and improved performance.
However, cloud DNS services they are not enterprise-wide solutions. Each cloud provider has its own DNS system, and no cloud provider offers an on-premise DNS solution. The result is a hybrid, enterprise DNS system without the ability to orchestrate processes among heterogeneous environments. DevOps teams must interface with and maintain multiple, disparate DNS systems.
Cloud-native API, high performance (no reliance on BIND).
DNS changes propagate fast.
Some traffic management - for example, ability to route traffic based on load and capacity.
Cloud DNS solutions do not extend easily (or at all) to on premise infrastructure. As long as you have some systems running on-premise, you’ll need to maintain a local DNS system, which does not provide the same automation capabilities.
Each cloud provider has its own DNS solution, with its own API and proprietary features. If you operate on more than one cloud, this creates a steep learning curve and additional integration effort.
If you choose to centralize all DNS activity on one cloud provider, all DNS requests will need to go to that cloud first, then to other locations, incurring high latency.
Smart traffic management features will only work on the same cloud provider. For example, in a hybrid cloud scenario, you cannot use the public cloud’s DNS service to distribute traffic between private/public cloud based on current load.
DNS appliances, self-deployed open source servers and cloud DNS services can all be used to automate DNS as part of DevOps workflows. However, none of these solutions provide support for all cloud and on-premise environments, together with modern integration and automation capabilities.
NS1 provides next-generation, carrier-grade DNS solutions that power some of the biggest Internet services and enterprise applications, including Yelp, Salesforce, and LinkedIn. NS1's Private DNS is now offered as a private DNS solution you can deploy on-premise and in the cloud.
Works on-premise and in any cloud or containerized environment
Easy to use RESTful API with intuitive syntax and high-speed command processing
Integrates with popular products across the DevOps toolset - IaC, monitoring, visualization and alerting (see our integrations)
DNS updates propagated in seconds across global infrastructure, supporting IaC and automated cloud provisioning. No restarts needed for DNS updates.
Permissioned access at the DNS zone level, with public/private key-based API control, and restricted access based on source IP. Comprehensive logging.
Scalability, performance and reliability trusted by the world’s largest online companies.
Advancements in DNS including the ability to communicate via APIs and the power to couple real-time telemetry with automation, give this often overlooked but foundational component of the app delivery stack a critical role in a DevOps-driven world.
In this webinar, we explore ways to reduce technical debt and how you can use DNS as a shortcut to eliminating decades of technical debt in your organization.
DNS—that little-noticed piece of Internet infrastructure—is emerging as a key tool to realizing the promise of cloud computing.
As you build your application environment you've undoubtedly thought about utilizing multiple providers at every level of your stack.
