We have identified a remediation strategy that appears to be improving performance back to normal levels. We are applying the fix and monitoring for any other performance regressions.
Posted Jan 30, 2019 - 09:22 PST
Update
The background work associated with this database maintenance is taking longer than expected. We are actively investigating other options to improve the situation.
Posted Jan 30, 2019 - 08:56 PST
Update
We've determined the underlying cause: general degradation to database performance causing requests to timeout.
Our database provider informed us that once the background work initiated via maintenance on their side subsides, we should see performance improve.
Unfortunately, that will take over 8 hours.
We'll keep you posted as we learn more.
Posted Jan 29, 2019 - 18:12 PST
Update
It appears that database provider maintenance changes from earlier are the underlying cause of the intermittent errors with certain types of compile requests (mostly from the Web IDE). We are in conversations with them now to see if they can expedite the recovery process
Posted Jan 29, 2019 - 13:27 PST
Update
We are continuing to monitor for any further issues.
Posted Jan 29, 2019 - 12:48 PST
Monitoring
Compiles triggered by the Web IDE have recovered and we are monitoring the situation.
Posted Jan 29, 2019 - 12:42 PST
Investigating
We are investigating alerts surrounding compiles triggered by the Web IDE.
Posted Jan 29, 2019 - 12:11 PST
This incident affected: REST API and Customer Resources (build.particle.io).