Operations suspended due to AWS outage
Incident Report for Aptible
Resolved
We have enabled all operations platform-wide, in all regions, but will continue to monitor for any regression.
Posted Dec 07, 2021 - 19:12 EST
Update
We are seeing very low error rates from AWS, and are re-enabling operations for:

* All Shared tenancy Stacks
* All Dedicated tenancy Stacks outside of us-east-1

We will continue to monitor the status of us-east-1 as AWS works to finish resolving the issue, and re-enable operations in us-east-1 as soon as it looks safe to do so.
Posted Dec 07, 2021 - 18:30 EST
Monitoring
The error rate has improved, but not enough to re-enable operations. We are continuing to monitor the error rate.
Posted Dec 07, 2021 - 16:46 EST
Update
We are continuing to investigate this issue.
Posted Dec 07, 2021 - 11:05 EST
Investigating
Operations are currently suspended on Aptible while we investigate an AWS outage. The AWS Status Page [0] currently has limited details on the outage, but we have observed that nearly all AWS API calls are failing for resources in us-east-1, as well as some API calls in other regions.

To prevent Aptible from attempting AWS API calls that might fail and leave infrastructure in an inconsistent state, we have disabled all operations. We will be testing the AWS APIs continuously in order to be able to resume operations as soon as AWS is available again.

Resources that are already running are not impacted by this issue.

[0] https://status.aws.amazon.com/
Posted Dec 07, 2021 - 10:50 EST
This incident affected: api.aptible.com and Aptible Deploy.