Resolved -
The maintenance is complete. At this time, all app, build, bastion and database instances in shared-tenancy stacks have been replaced, and all shared-tenancy resources are running on instances with patched kernels.
May 18, 22:27 EDT
Update -
We're beginning the scheduled database instance replacement now, and we'll update this status page once the maintenance is complete.
May 18, 18:46 EDT
Update -
All app, build and bastion instances in shared-tenancy stacks have been replaced as of Thursday, May 16.
After continued investigation by our Security Team, we have decided to replace all database instances in shared stacks as well, and will complete this migration *on Saturday, May 18 between 18:00 and 22:00 EDT.* During this period, all shared-tenancy databases will be restarted on new instances, and this will result in a brief downtime (up to 60 seconds) for each shared-tenancy database.
May 17, 20:55 EDT
Investigating -
Today, a new speculative execution attack affecting Intel processors was announced. The attack uses the Microarchitectural Data Sampling ("MDS") approach, and has been nicknamed "ZombieLoad" by the security community.
In order to ensure that customer environments are protected against this vulnerability, our Reliability Team is scheduling the process of re-launching customer containers and utility services (e.g., build instances, ephemeral SSH servers) on new EC2 instances that have been updated to the latest Linux kernel, which contains a mitigation against this attack.
To begin, the most vulnerable instances will be replaced — i.e., instances in shared environments where arbitrary code may be run. These include:
• App instances
• Build instances (where Docker images are built)
• Bastion instances (where ephemeral `aptible ssh` containers run)
The replacement operation should incur no downtime for your apps, but will require that we run `aptible restart` (a zero downtime operation) on each app in your shared environments.
We will continue to update this status page as we proceed with the patching process. We expect to begin replacing instances within the next 18 hours.
Here are a couple relevant links to learn more about this vulnerability:
•
https://www.cyberus-technology.de/posts/2019-05-14-zombieload.html•
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00233.html
May 14, 23:04 EDT