2024-08-30 - EMEA - DNSWatch DNS resolution issues
Incident Report for WatchGuard Technologies
Resolved


Event summary:
On August 30th, 2024, between approximately 13:30 UTC and 15:00 UTC, DNSWatch DNS Resolution Service experienced a service interruption in our European region. Although service has been restored, we know that this event impacted some of our customers and prevented them from using DNSWatch.



Event findings:
This was caused by an unexpected overload on our backend DNS resolvers, resulting from an increase in CPU usage. This increase was tied to a necessary software upgrade aimed at patching security vulnerabilities and enhancing system security. The upgrade was thoroughly tested and deployed in a phased manner to ensure stability. However, despite these precautions, it inadvertently placed higher demands on the hardware, leading to the intermittent performance issues experienced. Once the issue was identified, the DNSWatch team acted swiftly to restore normal operations.



We sincerely apologize for the impact on our affected customers, and we know the stability of our services is important to you and your business. At WatchGuard, we will never be satisfied with anything less than perfect operational performance and will continue to do everything we can to drive improvements across our services.
Posted Aug 30, 2024 - 13:30 UTC