Forge and Digital Ocean - Meltdown and Spectre Vulnerabilities

I got an email from Digital Ocean yesterday about "Meltdown and Spectre Vulnerabilities":

Hello,

DigitalOcean is working to mitigate the industry-wide security vulnerabilities known as Meltdown and Spectre. More information about the Meltdown and Spectre vulnerabilities can be found on our blog. 

This email is to notify you that, as part of our mitigation efforts, we have planned an upcoming maintenance that will affect all Droplets in all regions. During the course of this maintenance, we will reboot physical machines and the Droplets on them. These reboots are necessary in order to apply the patches that mitigate the Spectre vulnerability within DigitalOcean’s infrastructure. 

Users must also apply patches on their own Droplets. To patch and protect your Droplets, we strongly recommend you follow the steps outlined in this article to ensure your Droplet is running an updated kernel, even if the migration is only partial. 

We recommend taking a backup or snapshot of critical data before making changes to a production system, and verifying your recovery plan ahead of all maintenance windows.

If your distribution is not listed as patched, we highly recommend you move your data to a new Droplet running a version that is receiving security updates. 

To simplify the act of patching, we have recently updated Droplets to utilize a GrubLoader. On certain legacy Droplets, this may cause issues if the kernel is not upgraded.

We are aiming to begin this maintenance in our NYC1 region during the week of January 29, with maintenance to follow in our other global regions. Notices, including a list of affected Droplets, will be sent to all affected customers at least 24 hours ahead of scheduled maintenance windows. 

We recognize the disruptive nature of this maintenance and will make every effort to provide as much advance notice as possible. We are exploring ways to reduce the impact of this kind of maintenance in the future. 

If you need assistance planning for this maintenance, or have any other questions, please reach out by opening up a support ticket. 

Thank you,
Team DigitalOcean

They link to the following article which explains more:

https://www.digitalocean.com/community/tutorials/how-to-protect-your-server-against-the-meltdown-and-spectre-vulnerabilities

Does anyone know how this affects Forge servers and if I need to do anything?

ejdelmonico

8 years ago

Level 53

Nope, it's a wait and see issue. Although, Ubuntu has made a few patches as well so you should update each server. My opinion assuming general expectations is that since a droplet is virtual including virtual CPU and virtual memory and virtual drive that the individual server racks will suffer perf issues but our droplets should be fine as long as the individual racks are limited in the number of VM's.

I made a snapshot and updated all of my servers. I use Ottomatik for DB backups to S3. I also have Vultr servers and did the same thing.

Snapey

8 years ago

Level 122

There is some talk of performance degradation because the fix is to stop the optimistic processing of code. Its a hardware issue in the cpu so its not going to be fixed easily.

Unless you are on the limit of performance you probably won't notice though.

1 like

tptompkins

8 years ago

Level 7

Thanks guys.

shiroamada

8 years ago

Level 1

You need to be logged in is as root user.

You need to run the following command and reboot once it’s finished.

apt-get update && apt-get upgrade

Then

reboot

Before you reboot your server, it’s advisable to set proper downtime and inform your users in advance before doing so.

#How to confirm my servers are now safe?

To confirm your kernel has been patched with the latest security fix, you should run the following command:

uname -r

If your kernel version is greater than 4.4.0-109-generic, it’s has been patched successfully.

1 like

click

8 years ago

Level 35

I run Ubuntu 16.04 with Forge & Digital Ocean. In my case only a reboot of the system was required to get the latest kernel: sudo reboot.

I now have 4.4.0-112-generic running.

Zini

8 years ago

Level 1

Of course you should update your servers. Forge is not a server, it is a tool pretty much limited to deploying your code on a server.

Digital Ocean published a guide on how to mitigate (not fix) the issue: https://www.digitalocean.com/community/tutorials/how-to-protect-your-server-against-the-meltdown-and-spectre-vulnerabilities

You can compare your kernel to their list, uname -r.

click

8 years ago

Level 35

@Zini, it is not a strange question because Forge is taking care of the security updates. You only need to manually reboot your machine and it should be updated to the latest version.

Please or to participate in this conversation.