Cellular Connectivity issue

Incident Report for Particle

Postmortem

We would like to share the postmortem summary concerning the 3 separate service incidents on December 17, 18 and 22.

Partial cellular disruption on December 17

There was a partial cellular service disruption beginning at approximately 23:03 UTC. During the service disruption, devices negotiating a new connection were unable to establish service with the Particle cloud. The issue was determined to be an outage on a server provided by our telephony partner. Devices resumed normal service at approximately 02:39 UTC. We are working with our telephony partner to better mitigate the risk through earlier detection and building more resilient service.

Webhooks service disruption on December 18

There was a service disruption of integration and webhook requests beginning 19:33 UTC. During the disruption, some webhook message traffic could not be delivered to integration endpoints. The issue was the result of network availability with AWS services and webhook integration service was restored at approximately 23:38 UTC. We are working with AWS to better understand the root cause of the service outage and will share further information when they become available.

Electron 2G/3G cellular disruption on December 22

There was a partial cellular service disruption beginning at approximately 04:17 UTC. During the disruption, Electron devices using 2G/3G cellular services were disconnected. The disruption was caused by a database outage and services were restored at approximately 06:30 UTC. We are actively working with our telephony partner to mitigate the risks and partner with them on providing earlier detection and delivering more reliable service.

Posted Dec 23, 2020 - 13:15 PST

Resolved

This incident has been resolved.
Posted Dec 21, 2020 - 23:31 PST

Monitoring

Cellular connectivity is recovering and Electrons are coming online. We shall continue to monitor for another hour.
Posted Dec 21, 2020 - 22:30 PST

Update

One of our telephony provider's 3G / 2G service has degraded. We are working with them on resolving this
Posted Dec 21, 2020 - 22:05 PST

Identified

We have identified that the impact is limited to certain devices. We are continuing to investigate this issue.
Posted Dec 21, 2020 - 21:09 PST

Investigating

We are experiencing a cellular connectivity issue; we will provide an update shortly.
Posted Dec 21, 2020 - 20:41 PST
This incident affected: Cellular Connectivity.