AWS has marked this issue RESOLVED as of 19:19 UTC, and we have not observed any issues in the last hour.
The issue has been resolved and all services are operating normally.
Posted Sep 26, 2024 - 15:26 EDT
Update
The latest update from AWS indicates that operations created around 17:10 through 17:20 UTC were impacted, which matches our internal metrics. AWS has promised another update by 18:00 UTC, and we will continue to monitor the situation until we are satisfied that it is resolved.
Posted Sep 26, 2024 - 14:06 EDT
Identified
We're again seeing degradation and failure to register new load balancer targets in about about 10% of running operations.
Posted Sep 26, 2024 - 13:52 EDT
Monitoring
Loadbalancer registration appears to be working as expected at this time. We will continue to monitor operations until AWS resolves their service degradation notice.
Posted Sep 26, 2024 - 12:57 EDT
Update
AWS has acknowledged the impact we are seeing and opened an incident:
> We are investigating increased load balancer back-end instance registration times in the us-east-1 Region. September 26, 2024 at 16:21:43 UTC
Since 16:05 UTC, Aptible is observing some recovery, about half of endpoint target registrations are succeeding at this time.
Posted Sep 26, 2024 - 12:37 EDT
Identified
This service impact only applies to resources hosted in the `us-east-1` region.
Customers may notice operations reaching timeout, but at this point all operations are rolling back successfully to the previous state.
Posted Sep 26, 2024 - 12:24 EDT
Investigating
We are investigating abnormally long registration times for new targets with AWS Load Balancers. This may be causing extended operation times for releases (Deploy, Scale, Restart) for services that have Endpoints.