Best Practice using on-demand queues from Amazon and .env files

@alfrednutile, we use AWS with auto scaling groups for several of our projects, and we did it a little differently that you are...

We have Jenkins build new servers using packer with fresh OS updates and fresh code upon commits to the repo, and then save off the server as an AMI to s3 buckets. Then we spin up new workers with the latest AMI behind the LB, and shut the old ones down. We have worked very hard to not have a situation where an update forces a core change that an old worker would not be able to process jobs in the queue during this update window. This allows us to do rolling updates throughout the day.

We have actually gone as far as disabling ssh all together, so there is no remote management of the servers--you have to script out the changes to the OS. We are using Chef to bake the servers.

We have this all working with several L4 projects, and are looking into the best way to do this with an L5 project that we are starting in a few weeks, and our current plan is to have Jenkins drop the .env file on the server before writing it off as an AMI. We already have AMI's for each of our environments, so this works for us.

The reason that we are staying with .env's over just coding the sensitive variables in the config//whatever.php file, is that we actually have the a Jenkins server that belongs to & lives in each of our clients VPC's at AWS, so they are the only ones that have the production & sandbox credentials that gets baked into the AMI, so we never have those permissions.

I hope that idea may be of help to you.

As a side note, we are actually, moving all of this to Docker, as our build process is taking a little longer than we want to bake an entire AMI for each commit, so we are going to be spinning off smaller Docker containers, so that maybe something that you consider too.

@jimmy.puckett thanks! I will take this back to the group and talk about it. We were going docker too but backed off as we work to meet a release date coming up. Also Amazon is releasing a container service soon that might make it easier than the ElasticBeanstalk / RDS setup we spent a week on.

Ya, we have been "playing" with docker, but was not overly interested in it for "production" until AWS announced the container service. Now we are moving that direction. Since all of our servers were already scripted out to build with Chef, we just have to make some slight changes with our deployment steps, and then have packer spit out the containers.