Resolved -
The delays in incident persistence have been resolved. During this incident, a small subset of accounts may have experienced gaps in event processing. We are conducting a full post-event review, and a full RCA will be available upon request once complete. We apologize for any impact this may have had on your business.
Apr 28, 05:08 UTC
Update -
We continue to closely monitor the situation. All systems are operating normally, and there has been no impact to service availability at this time.
Our next update will be shared on April 28 at 05:00 UTC, or earlier if any notable changes occur.
Thank you for your continued understanding and support
Apr 27, 23:03 UTC
Update -
We continue to monitor. All systems remain fully operational, with no impact to service availability. The next update will be provided on April 27 at 23:00 UTC, or sooner if there are any significant developments. Thank you for your continued patience.
Apr 27, 20:00 UTC
Update -
We continue to assess the data recovery process. All systems are operating normally with no impact on service availability. The next update will be shared on April 27 at 19:00 UTC, or earlier if warranted. Thank you for your continued patience.
Apr 24, 20:08 UTC
Update -
We continue to investigate the data recovery process while all systems are operating normally. There is no impact to current service availability. Our next update will be shared on April 24 at 20:00 UTC, or earlier if appropriate. Thank you for your continued patience
Apr 24, 08:49 UTC
Update -
A permanent fix has been implemented. Our teams will continue to closely monitor the system. We will continue to investigate the recovery of any data that may have been affected. We'll post our next update on April 24 at 08:00 UTC, if not sooner. Thanks for your patience.
Apr 24, 01:51 UTC
Update -
Our teams are actively working on implementing a permanent fix and continue to closely monitor the system. There are no delays at this time. We will continue to investigate the recovery of any data that may have been affected once the permanent fix has been deployed. We'll post our next update on April 24 at 02:00 UTC, if not sooner. Thanks for your patience.
Apr 23, 19:56 UTC
Update -
Our teams are continuing to actively work on data recovery efforts related to this incident. System stability remains under close monitoring, and we are focused on safely restoring any data that may be affected. We’ll provide our next update at 20:00 UTC if not sooner.
Apr 23, 08:03 UTC
Update -
The delays in incident persistence have been resolved. Our teams are continuing to closely monitor system stability and are working toward a long-term fix to prevent recurrence. We will continue to investigate the recovery of any data that may have been affected. We'll post our next update on April 23 at 08:00 UTC, if not sooner. Thanks for your patience and understanding.
Apr 23, 02:22 UTC
Update -
We are still working hard to resolve delays in incident persistence. As a load-control measure, Botnet incidents have been fully filtered out. Our teams continue to monitor the system closely and are working on implementing a long-term fix. We will work to recover any data that may have been affected once the delays have been fully resolved. We'll post our next update on April 23 at 02:00 UTC, if not sooner. Thanks for your patience.
Apr 22, 20:03 UTC
Update -
We are seeing a recurrence of delays in incident persistence, currently around 10 hours. To help stabilize the system, Botnet Incidents are temporarily not being consumed by ThreatSync. Our teams are monitoring closely and working on a long-term solution, while also investigating the recovery of any data that may have been affected. We'll post our next update on April 22 at 20:00 UTC, if not sooner. Thanks for your patience.
Apr 22, 16:03 UTC
Update -
A fix has been implemented and the delays have been resolved. The service is operating normally at this time, and we are actively monitoring to ensure system stability. Our teams continue to investigate the potential recovery of any data that may have been lost over the last few days. We will post our next update on April 22 at 16:00 UTC, if not sooner. Thank you for your continued patience and understanding.
Apr 22, 01:13 UTC
Update -
We are continuing to actively work on resolving the incident persistence delays. We’ll post our next update at April 22, 03:00 UTC, if not sooner. Thanks for your patience.
Apr 21, 21:08 UTC
Identified -
The team has identified the cause of the delay and is actively working to resolve the issue. Unfortunately, a gap in our monitoring has resulted in incidents for some accounts being lost over the last few days. The team will be working to recover this data once the delay has been resolved. We apologize for the impact this event has caused. We'll post our next update at 21:00 UTC if not sooner.
Apr 21, 14:55 UTC
Investigating -
We are investigating delays in incident persistence. Once we determine the impact and scale of these we'll post our next update in 2 hours, if not sooner.
Apr 21, 12:39 UTC