What is DevOps Automation?
Since the inception of DevOps, it was understood that automation was key to making DevOps work. Instead of building software and “throwing it over the wall” for operations to deal with, DevOps teams needed efficient ways to eliminate manual handoffs, manual deployment processes and configuration issues. Automation was needed at all stages of the dev lifecycle:
Automating builds - including unit tests, integration and UI tests, using continuous integration and test automation suites.
Automating testing environments - setting up realistic testing and staging environments and seamlessly deploying software to them.
Automating production deployments - ensuring that the same software that worked on a staging environment can be pushed to production at the click of a button.
Why Outdated DNS Can’t Keep Up with DevOps
The achilles heel of DevOps automation is the Domain Name System (DNS) system. DNS is how systems and services discover and connect to each other, and how users connect to newly-deployed software. DNS has been around for decades, and traditional DNS systems were not designed to operate in modern DevOps environments. Issues include outdated or hard-to-use API's, slow propagation times and poor performance at scale.
Below we review key trends driving the need for DevOps automation and why legacy DNS causes critical problems simply impedes DevOps success.
The Evolution and Future of DevOps
Despite its benefits, moving to DevOps can be a challenging undertaking, especially for organizations riddled with legacy applications and technology. Many companies fail to reap the benefits because making the shift requires substantial changes to culture, processes and the resulting toolset.
Watch the Webinar
Trends Driving DevOps Automation and the DNS Challenge
Infrastructure as Code (IaC)
IaC makes it possible to maintain server environments just like software developers build and maintain source code. Both developers and operations staff can create infrastructure blueprints, which automatically sets up servers with a predictable, predefined set of applications and configuration settings.
IaC can take the shape of Configuration as Code (CaC) tools like Terraform, Ansible and Chef. It might also involve containerization tools like Docker and Kubernetes, and cloud automation tools from cloud providers like AWS and Azure.
The DNS Challenge
Legacy DNS can become a challenge and a bottleneck for DevOps teams implementing fully automated deployment processes because they lack critical capabilities:
No native APIs - traditional DNS platforms like BIND or PowerDNS, even some commercial services and appliances have no API; API's aren't built for modern application infrastructures. Traditional offerings' were developed “after the fact,” their reliability performance often do not meet requirements of high scale automation.
Restart on DNS updates - BIND requires a reload of the DNS database when DNS records are updated. As a result, DNS updates must be scheduled in specific and infrequent deployment windows, to ensure minimal disruption to live services. This approach is incompatible with a continuous delivery methodology. Many commercial DNS appliances or services are based on BIND.
Slow propagation- DNS propagation times, which are often 30 minutes or more, are too slow to keep up with infrastructure and deployment updates.
Continuous Integration CI/CD
Continuous Integration (CI) is a set of tools and coding practices which encourage developers to commit their code into a version control repository very often, commonly multiple times per day. CI servers integrate code from multiple developers and automatically execute a build, creating a new version of the software which can be run and tested. Doing this often improves collaboration and responsibility, increases quality and ensures teams build software that works and delivers value to customers.
Continuous Delivery (CD) is an extension of CI, in which teams automate not just their build process, but the complete development lifecycle, including testing, staging and production environments. In a mature CD environment, it is possible to push a code change from development, to testing, to staging and finally to production, with the click of a button. Test automation ensures that new software versions can be released quickly and with high quality.
The DNS Challenge
CI/CD accelerates software development, such that dev, test and production environments are deployed on a daily, even hourly basis. Such frequent changes to infrastructure require fast DNS propagation. With traditional tools taking up to 30 minutes to propagate even in a local network, DNS becomes a bottleneck in the dev-test-prod lifecycle.
The DevOps Toolset
Modern DevOps teams rely on an ecosystem of tools that helps them manage, monitor and report on the systems they operate. These include, apart from Infrastructure as Code tools mentioned above:
Containerization and deployment tool such as Apache Mesos, Docker and Kubernetes
Network and server monitoring tools such as AWS CloudWatch, Catchpoint and Datadog and Monitis
Application monitoring tools such as Monitor.us, Monitis, New Relic and Pingdom
Log analysis and visualization tools such as Grafana, Logstash and Kibana
Alerting and communication tools such as PagerDuty, Slack and HipChat
The DNS Challenge
DevOps teams are used to integrating their toolset with all parts of the environment to enable seamless management, monitoring and alerting. However, DNS can be an outlier - a part of the application stack that doesn’t play well with the DevOps toolsets and processes.
The DNS Bottleneck and Four Ways to Release It
Strategy #1 for Automating DNS
Using a Traditional DNS/DDI Appliance
A common way to automate DNS is using DNS/DDI appliances, which allow you to easily deploy DNS anywhere on the network. These appliances can fully manage internal DNS infrastructure - managing DNS server updates, automatically distributing DNS changes to all appliances, and providing a management interface that lets you centrally control DNS on the network. Most DNS/DDI appliances provide an API that lets you automate DNS operations as part of DevOps processes.
Pros:
Easy to deploy and manage - if you already have it, leverages existing infrastructure
Lets you automate some complex DNS processes
Smart traffic management - for example, ability to distribute traffic based on performance
Traditional DNS network appliances use the open source BIND server for core DNS functions. As a result, they have many of the same limitations as the BIND platform itself. In addition to the challenges related to cost of ownership of appliances, the following should be considered:
Cons:
API syntax is complex, non-standard and can be difficult to work with
Each API command takes time to execute because CLI calls are made in the background
Cannot execute multiple API commands rapidly, commands need to be “throttled”
Propagation can take longer, as CLI commands are executed on all appliances
While DNS/DDI provides a working automation solution, it is not easy to use and can significantly delay automated processes due to slow response of BIND-based appliances.
Strategy #2 for Automating DNS
Deploying an Open Source Solution
Some organizations try to automate DNS by deploying an open source DNS server like BIND or PowerDNS. These servers do not provide a native API, so automation needs to be done via scripting.
Pros:
Commands and environment is intuitive and well known to many developers.
Cons:
Major effort required to script automated actions - these servers were built at a time when Infrastructure as Code did not exist.
Slow response to scripted CLI commands, cannot execute multiple API commands rapidly.
DNS propagation times, which can be 30 minutes or more, cannot keep up with frequent changes to infrastructure.
BIND requires a restart when DNS records are updated, which causes service interruption. This means DNS updates cannot be done as part of the DevOps automation flow
Open source DNS solutions require ongoing management of patches and security updates, and have high maintenance overhead - even more so than other open source tools, because DNS is a critical infrastructure and more exposed to attack.
Strategy #3 for Automating DNS
Using the DNS of your Cloud Provider
Amazon, Azure and other cloud providers provide native DNS services which are a good choice for automation. Unlike BIND-based solutions, cloud DNS services offer a modern, native API and improved performance.
However, cloud DNS services they are not enterprise-wide solutions. Each cloud provider has its own DNS system, and no cloud provider offers an on-premise DNS solution. The result is a hybrid, enterprise DNS system without the ability to orchestrate processes among heterogeneous environments. DevOps teams must interface with and maintain multiple, disparate DNS systems.
Pros:
Cloud-native API, high performance (no reliance on BIND).
DNS changes propagate fast.
Some traffic management - for example, ability to route traffic based on load and capacity.
Cons:
Cloud DNS solutions do not extend easily (or at all) to on premise infrastructure. As long as you have some systems running on-premise, you’ll need to maintain a local DNS system, which does not provide the same automation capabilities.
Each cloud provider has its own DNS solution, with its own API and proprietary features. If you operate on more than one cloud, this creates a steep learning curve and additional integration effort.
If you choose to centralize all DNS activity on one cloud provider, all DNS requests will need to go to that cloud first, then to other locations, incurring high latency.
Smart traffic management features will only work on the same cloud provider. For example, in a hybrid cloud scenario, you cannot use the public cloud’s DNS service to distribute traffic between private/public cloud based on current load.
Strategy #4 for Automating DNS
NS1’s Next-Generation Private DNS Solution
DNS appliances, self-deployed open source servers and cloud DNS services can all be used to automate DNS as part of DevOps workflows. However, none of these solutions provide support for all cloud and on-premise environments, together with modern integration and automation capabilities.
NS1 provides next-generation, carrier-grade DNS solutions that power some of the biggest Internet services and enterprise applications, including Yelp, Salesforce, and LinkedIn. NS1's Private DNS is now offered as a private DNS solution you can deploy on-premise and in the cloud.
NS1 Private DNS: DNS Designed for DevOps
NS1 was built around several design principles that makes it an ideal solution for a DevOps environment
Infrastructure and cloud agnostic
Works on-premise and in any cloud or containerized environment
API-first design
Easy to use RESTful API with intuitive syntax and high-speed command processing
Built-in integrations
Integrates with popular products across the DevOps toolset - IaC, monitoring, visualization and alerting (see our integrations)
Fast propagation
DNS updates propagated in seconds across global infrastructure, supporting IaC and automated cloud provisioning. No restarts needed for DNS updates.
Security controls
Permissioned access at the DNS zone level, with public/private key-based API control, and restricted access based on source IP. Comprehensive logging.
Carrier-grade
Scalability, performance and reliability trusted by the world’s largest online companies.
