2025-11-17 - GLOBAL - ThreatSync Data Gap

Incident Report for WatchGuard Technologies

Postmortem

Event Summary:

Between Friday, November 14th, 2025, at approximately 19:00 UTC and Sunday, November 16th, 2025, at 12:00 UTC, the WatchGuard Cloud ThreatSync Core (TS Core) data pipeline experienced an interruption in data processing. This interruption resulted in a data gap for both TS Core and Total MDR. The missing data has been restored. The event is resolved, and services are operating normally across all regions.

Event Findings:

At approximately 19:00 UTC on November 14th, 2025, an update was deployed to our ThreatSync Core (TS Core) data pipeline which introduced a software bug that escaped our automated testing. By 12:00 UTC on November 17th, a fix was deployed to all regions to resolve the software bug, and data processing resumed for all new events. During this service disruption, all underlying security products (Firebox, Endpoint Security, AuthPoint, ThreatSync+ NDR, and Wi-Fi) continued to enforce protections as designed. While Core MDR telemetry collection was unaffected, Total MDR relies on TS Core for incoming events which caused delays until service was restored. Once service was restored, a gap in TS Core data during the disruption window was identified. In order to maintain reliability and avoid inconsistencies while restoring missing data, we undertook careful preparation, testing, and verification to ensure accuracy and integrity before re-ingesting and processing the data. Four areas of data backfill were completed: Firebox Visibility (November 21, 15:12 UTC), WatchGuard Endpoint (November 22, 15:12 UTC), WatchGuard Wi-Fi (November 24, 07:00 UTC), and AuthPoint (November 27, 05:55 UTC).

At WatchGuard, we strive for flawless operational performance; our teams are implementing improvements to prevent recurrence and reduce re-ingestion times in the future. We sincerely apologize for the impact to our affected customers, and appreciate the opportunity to meet your security needs.

Posted Nov 29, 2025 - 00:03 UTC

Resolved

Our teams have successfully restored Firebox, Endpoint security, and Wi-Fi data into ThreatSync Core. We estimate Authpoint data will be fully restored by Friday, November 28th. For Managed Services customers, our normal MDR processes and SOC reviews occur as data is restored into ThreatSync Core. All systems remain fully operational, and all underlying security products continue to enforce protections as designed. Thank you for your continued patience and understanding.

Posted Nov 24, 2025 - 15:51 UTC

Update

Our team has successfully restored Firebox data into ThreatSync Core and is continuing the recovery process for Endpoint Security data. We estimate Endpoint Security data will be fully restored by Saturday, November 22nd, at approximately 15:00 UTC. All systems remain fully operational, and all underlying security products (Firebox, Endpoint Security, AuthPoint, ThreatSync+ NDR, and Access Point) continue to enforce protections as designed. Core MDR telemetry collection also remains unaffected.

For Managed Services, detections for Endpoint Security were not impacted. As missing data is restored, our normal MDR process and SOC reviews will occur, except for ThreatSync+ NDR-related events. We will provide further updates as restoration progresses.
Thank you for your continued patience and understanding.

Posted Nov 21, 2025 - 15:16 UTC

Identified

On November 17th we identified and resolved a bug in our ThreatSync Core data pipeline that caused a temporary interruption in data processing. As a result, there is a gap in data for ThreatSync Core and Total MDR between Friday, November 14th, 2025, at 19:00 UTC and Sunday, November 16th, 2025, at 12:00 UTC. All systems are processing current data normally and all underlying security products (Firebox, Endpoint Security, AuthPoint, ThreatSync+ NDR, and Access Point) continued to enforce protections as designed. Core MDR telemetry collection also continued. Our team will begin restoring missing data on November 20th, except for NDR events; we will provide updates as this restoration progresses.
For Managed Services, only data for Firebox, ThreatSync+ NDR, and AuthPoint were affected, detections for Endpoint Security were not affected, and as missing data is restored, our normal MDR process and SOC reviews will occur except for ThreatSync+ NDR related events. Thank you for your patience and understanding.

Posted Nov 20, 2025 - 21:48 UTC

This incident affected: ThreatSync:::EMEA (Incident Persistence:::EMEA), ThreatSync:::AMER (Incident Persistence:::AMER), ThreatSync:::APAC (Incident Persistence:::APAC), Managed Services:::AMER (Data Ingestion:::AMER), and Managed Services:::EMEA (Data Ingestion:::EMEA).