29 Sep '17, 8pm
Intermittent degraded performance resolved
Last night we experienced a sequence of events that led to intermittent degraded performance across many OpenSRS services. At 1AM UTC we were the target of a sophisticated DNS attack that was followed by an unrelated double failure of core network equipment at our main Canadian data center, caused by an undocumented software limitation. We were able to quickly recover from the equipment failure but continued to experience the DNS attack until 13:10 UTC, when the attack was stopped and systems started responding reliably again. The network equipment failure made it more difficult for us to identify that we were under a DNS attack and impacted our response time.