Delivering Must-Have Visibility for Service Provider Networks
October 13, 2020
High-speed query logging helps with data collection
By David Ayers, Product Marketing Manager, Infoblox
Understanding what’s going on within the network is essential to service providers, and DNS plays a critical role in gathering that intelligence. Clearly, from both a performance and security standpoint, knowing what subscriber devices are looking up in DNS provides valuable information about healthy vs. abnormal operations. DNS query and response logging has been one of the significant ways for a DNS system to produce raw data on what queries are being made. At the same time, reporting is the organization and transformation of that raw data into humanly readable formats. Here’s how partners can deliver much-needed visibility to service providers.
DNS Query and Response Logging
As a DNS server sends queries to other DNS servers, it can keep a copy of these queries for later analysis or aid in troubleshooting DNS-related issues. Also, the information can be processed later to generate reports with information such as what the most queried domain name was in the past hour. System log, or syslog, is typically the mechanism to record this query information, a process known as query logging. Similarly, as a DNS server receives answers, besides sending the responses to the client, that server could also keep a copy of the answer for later analysis or to aid in troubleshooting. It can use the same mechanism as query logging by storing this information in logs, which is known as response logging.
But here’s the problem. While query and response logging provide a valuable source of data for both security and operational analysis, both processes can significantly add more I/O (Input/Output) load to the DNS servers and are typically disabled by default in operational environments. With query logging, every outbound query sent by the DNS server translates to (eventually) another write-operation to the disk. This load is even more burdensome when response logging is enabled since the number of responses could be very large. The speed of disk storage is a limiting factor, with the additional load on I/O resource-limited hosts, because query processing must wait until the log data has been written to disk.
That can be painful, considering that network performance is everything to the subscriber.
High-Speed Query Logging With dnstap
Infoblox Network Identity Operating System (NIOS) is the operating system that powers Infoblox core network services, ensuring the non-stop operation of network infrastructure. In our latest version of NIOS, we’re adding high-speed query logging through dnstap, a relatively new and open standard for how to monitor and look deeper into what DNS servers are doing. It’s meant for large and active DNS server implementations characteristic of network operators. Designed to be fast and lightweight, it provides a quick, flexible method for capturing and logging DNS traffic without significant performance trade-off. Unlike I/O-intensive query and response logging, dnstap operates asynchronously, allowing high-speed query logging with minimal performance loss.
Available on the Infoblox DNS Cache Acceleration solution, dnstap uses Google’s protocol buffer implementation to package log query and response information directly from the DNS server within a flexible binary structured log format, and then stream this data to a receiving server. From there, service providers can leverage Big Data tools to analyze this data (and even combine this data with other sources) to build a complete picture of subscriber usage patterns based on network data.
Digging Deeper: Use Cases for dnstap
The more information that service providers have about their networks, the better they can manage and protect not just the network but also the business. Many service providers and enterprises already utilize Infoblox Reporting and Analytics to take this data and turn it into Actionable Network Intelligence — enabling them to create custom dashboards and reports to troubleshoot network and application problems, uncover security threats, and address compliance requirements.
Service providers are unique, since the network is their business, so they sometimes require more granular data on subscriber usage patterns to facilitate network and product planning. Subscriber data can be likened to gold for service providers, and dnstap can provide a veritable firehose of subscriber behavioral data. Here are a few use cases that come to mind:
● Uncover subscriber impacts. Here’s a great example from recent events: How have online subscriber behaviors changed since social distancing restrictions were put in place? Network data can highlight how subscriber usage patterns have changed and what actions can be taken to ensure optimal network performance. Another example: What’s the impact to subscribers based on a new strain of malware? Again, service providers can determine effects based on malware DNS queries.
● Protect the brand. Service providers are continually seeking ways to combat customer churn in the face of an ultra-competitive environment. With DNS network data, it’s now possible to model subscriber behaviors and identify those at risk to churn. Are subscribers viewing competitor websites? Are they leaving online reviews of their experiences on various properties? Network data with usage profiles is a critical input into developing predictive churn models and a great indicator of the subscriber usage experience.
● Generate new revenue streams. Depending on location, recent changes to regulatory environments may have opened the door for service providers to increase average revenue per user (ARPU) by monetizing the data flowing through their networks. Service providers can identify trends in subscriber network data and leverage this information to develop new product offerings and enhancements and even expand their offerings into newer areas such as IoT. When combined with Infoblox Subscriber Services, service providers can gain and leverage data from the individual subscriber and device level to help develop targeted new services to benefit subscribers and increase revenue.
Infoblox Provides Visibility and Performance
Performance matters more than ever. Service provider DNS caching infrastructure needs to be robust. When subscribers experience slow page loads, buffered video, or suspicious app activity, the overall experience is affected — and loyalty to their current service provider may be questioned. Infoblox has provided carrier-grade security, high availability, and ultra-low latency that helps create a better first-connection impression for subscribers. With dnstap, we’re giving service providers even greater visibility and insight into their DNS servers while retaining the optimal ultra-low latency they expect.
David Ayers, product marketing manager at Infoblox, is a business technology and marketing specialist based in Virginia who has sold, built, and messaged mission-critical cloud and hosted solutions in an evolving industry. With a unique blend of product management, product marketing, and sales experience, Ayers led product marketing activities for healthcare, public-sector clouds, embedded governance, risk, and compliance at Virtustream (a Dell Technologies business) before joining Infoblox. He’s also worked at Verizon, Terremark, SunGard Availability Services, Sun Microsystems, Digex, and Symantec.
DNS Query and Response Logging
As a DNS server sends queries to other DNS servers, it can keep a copy of these queries for later analysis or aid in troubleshooting DNS-related issues. Also, the information can be processed later to generate reports with information such as what the most queried domain name was in the past hour. System log, or syslog, is typically the mechanism to record this query information, a process known as query logging. Similarly, as a DNS server receives answers, besides sending the responses to the client, that server could also keep a copy of the answer for later analysis or to aid in troubleshooting. It can use the same mechanism as query logging by storing this information in logs, which is known as response logging.
But here’s the problem. While query and response logging provide a valuable source of data for both security and operational analysis, both processes can significantly add more I/O (Input/Output) load to the DNS servers and are typically disabled by default in operational environments. With query logging, every outbound query sent by the DNS server translates to (eventually) another write-operation to the disk. This load is even more burdensome when response logging is enabled since the number of responses could be very large. The speed of disk storage is a limiting factor, with the additional load on I/O resource-limited hosts, because query processing must wait until the log data has been written to disk.
That can be painful, considering that network performance is everything to the subscriber.
High-Speed Query Logging With dnstap
Infoblox Network Identity Operating System (NIOS) is the operating system that powers Infoblox core network services, ensuring the non-stop operation of network infrastructure. In our latest version of NIOS, we’re adding high-speed query logging through dnstap, a relatively new and open standard for how to monitor and look deeper into what DNS servers are doing. It’s meant for large and active DNS server implementations characteristic of network operators. Designed to be fast and lightweight, it provides a quick, flexible method for capturing and logging DNS traffic without significant performance trade-off. Unlike I/O-intensive query and response logging, dnstap operates asynchronously, allowing high-speed query logging with minimal performance loss.
Available on the Infoblox DNS Cache Acceleration solution, dnstap uses Google’s protocol buffer implementation to package log query and response information directly from the DNS server within a flexible binary structured log format, and then stream this data to a receiving server. From there, service providers can leverage Big Data tools to analyze this data (and even combine this data with other sources) to build a complete picture of subscriber usage patterns based on network data.
Digging Deeper: Use Cases for dnstap
The more information that service providers have about their networks, the better they can manage and protect not just the network but also the business. Many service providers and enterprises already utilize Infoblox Reporting and Analytics to take this data and turn it into Actionable Network Intelligence — enabling them to create custom dashboards and reports to troubleshoot network and application problems, uncover security threats, and address compliance requirements.
Service providers are unique, since the network is their business, so they sometimes require more granular data on subscriber usage patterns to facilitate network and product planning. Subscriber data can be likened to gold for service providers, and dnstap can provide a veritable firehose of subscriber behavioral data. Here are a few use cases that come to mind:
● Uncover subscriber impacts. Here’s a great example from recent events: How have online subscriber behaviors changed since social distancing restrictions were put in place? Network data can highlight how subscriber usage patterns have changed and what actions can be taken to ensure optimal network performance. Another example: What’s the impact to subscribers based on a new strain of malware? Again, service providers can determine effects based on malware DNS queries.
● Protect the brand. Service providers are continually seeking ways to combat customer churn in the face of an ultra-competitive environment. With DNS network data, it’s now possible to model subscriber behaviors and identify those at risk to churn. Are subscribers viewing competitor websites? Are they leaving online reviews of their experiences on various properties? Network data with usage profiles is a critical input into developing predictive churn models and a great indicator of the subscriber usage experience.
● Generate new revenue streams. Depending on location, recent changes to regulatory environments may have opened the door for service providers to increase average revenue per user (ARPU) by monetizing the data flowing through their networks. Service providers can identify trends in subscriber network data and leverage this information to develop new product offerings and enhancements and even expand their offerings into newer areas such as IoT. When combined with Infoblox Subscriber Services, service providers can gain and leverage data from the individual subscriber and device level to help develop targeted new services to benefit subscribers and increase revenue.
Infoblox Provides Visibility and Performance
Performance matters more than ever. Service provider DNS caching infrastructure needs to be robust. When subscribers experience slow page loads, buffered video, or suspicious app activity, the overall experience is affected — and loyalty to their current service provider may be questioned. Infoblox has provided carrier-grade security, high availability, and ultra-low latency that helps create a better first-connection impression for subscribers. With dnstap, we’re giving service providers even greater visibility and insight into their DNS servers while retaining the optimal ultra-low latency they expect.
David Ayers, product marketing manager at Infoblox, is a business technology and marketing specialist based in Virginia who has sold, built, and messaged mission-critical cloud and hosted solutions in an evolving industry. With a unique blend of product management, product marketing, and sales experience, Ayers led product marketing activities for healthcare, public-sector clouds, embedded governance, risk, and compliance at Virtustream (a Dell Technologies business) before joining Infoblox. He’s also worked at Verizon, Terremark, SunGard Availability Services, Sun Microsystems, Digex, and Symantec.
