We would like to share the postmortem summary concerning the 3 separate service incidents on December 17, 18 and 22.
Partial cellular disruption on December 17
There was a partial cellular service disruption beginning at approximately 23:03 UTC. During the service disruption, devices negotiating a new connection were unable to establish service with the Particle cloud. The issue was determined to be an outage on a server provided by our telephony partner. Devices resumed normal service at approximately 02:39 UTC. We are working with our telephony partner to better mitigate the risk through earlier detection and building more resilient service.
Webhooks service disruption on December 18
There was a service disruption of integration and webhook requests beginning 19:33 UTC. During the disruption, some webhook message traffic could not be delivered to integration endpoints. The issue was the result of network availability with AWS services and webhook integration service was restored at approximately 23:38 UTC. We are working with AWS to better understand the root cause of the service outage and will share further information when they become available.
Electron 2G/3G cellular disruption on December 22
There was a partial cellular service disruption beginning at approximately 04:17 UTC. During the disruption, Electron devices using 2G/3G cellular services were disconnected. The disruption was caused by a database outage and services were restored at approximately 06:30 UTC. We are actively working with our telephony partner to mitigate the risks and partner with them on providing earlier detection and delivering more reliable service.