Device Service Connectivity Issue
Incident Report for Particle
Postmortem

From 4:15pm to 4:40pm PDT on August 4th, a telephony partner experienced a disruption that caused connection issues for 2G/3G Particle devices. These devices were unable to connect to the Particle Cloud during this period.After the telephony disruption ended at 4:40pm PDT, some devices were stuck in a connection loop trying to connect due to an overloaded backend service in the Particle Cloud.  Also, there were devices that did not achieve a stable connection and reconnected frequently, further increasing the load. Engineers deployed a fix at 5:23pm to reduce load on the backend system. By 5:41pm, the issue fully resolved: new device connections were successful and stable.Particle Engineers are validating a change to the backend system to reduce load in scenarios like this. It will deploy in the coming days without any customer disruption. We will continue to monitor and reduce load on this backend system in order to minimize disruption of the Particle Cloud.

Posted Aug 05, 2020 - 15:25 PDT

Resolved
This incident has been resolved.
Posted Aug 04, 2020 - 19:11 PDT
Monitoring
All wi-fi and cellular devices are expected to be recovering. Our team continues to monitor the situation.
Posted Aug 04, 2020 - 17:45 PDT
Identified
We continue to monitor the incident. Wi-fi devices are also found to be impacted. Particle is continuing to investigate the situation.
Posted Aug 04, 2020 - 17:26 PDT
Monitoring
Devices are coming back online and we will continue to monitor the connectivity issue.
Posted Aug 04, 2020 - 17:05 PDT
Update
We are continuing to investigate this issue.
Posted Aug 04, 2020 - 16:42 PDT
Investigating
We’re experiencing a device connectivity issue that appears to be specific to Electron 2G/3G and E Series 2G/3G devices
Posted Aug 04, 2020 - 16:40 PDT
This incident affected: Cellular Connectivity.