Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

Talinon's avatar
Level 51

Secret Management

Hi all,

I am currently going thru the process of making an application SOC 2 compliant. For anyone unfamiliar, it's a certification that ensures controls are set in place for best practice in securing customer data. One such requirement is Secrets Management.

I am willing to bet that the mass majority of production applications out there, just put their secrets into their .env files and move on with their day. I don't want this to become an opinionated discussion, because everyone's opinion (including my own) is irrelevant in this case. Personally, I see no issues with competent developers using an .env and including it in their .gitignore. All that matters is what the auditor wants. When the Auditor asks us to show them where we keep secrets, and I show him an .env file, we will fail the audit and potentially lose a multi-million dollar client.

From what I've gathered, this is not so much a security issue as it is a secret sprawl issue. Instead of managing secrets in dev/staging/prod .env files, they ultimately don't want developers entering secrets anywhere near the code base, regardless if ignored by version control. This is where a service such as Microsoft Azure Key Vault, or Hashi Corp come in. Instead of a developer entering a secret into their local .env, the application will retrieve the key from a vault. It's about secret key management and clear separation of secrets from the application and code. It also allows specific roles to be assigned within the vault, where developers will only be granted specific secrets. It could be taken as far as assigning specific secrets to each developer, where a leaked key could be traced back to a specific person.

With all that being said, let's all just agree for this discussion that we can't use .env for secrets, especially in prod. Moving onto the issue.

I came up with implementing a Secrets interface for client code, where an implementation (say a Hashi Corp service) would be responsible for retrieving the key from the vault. Obviously, doing this every time the secret is required is expensive and asinine, so it would cache it. On subsequent calls, it checks if its cached and returns the secret to the calling client code, otherwise it renews it with the vault. Simple enough, especially for first party client code.

Now, let's say I need to store the MySQL database password in the vault, removing it from the .env file. When the Laravel Framework first boots up, one of the first things it does is read the config files. This makes perfect sense, as it obviously needs to know configuration before it starts bootstrapping services. This also means that nothing has been registered, so I can't use convenient facades or other bindings. In other words, I can't use \Cache::get('DB_PASSWORD') or \Redis::get('DB_PASSWORD'), etc. This forces me to use a static method that pulls the secrets out of cache (in my case, Redis) using the Predis client directly. If it can't find a critical key within cache, it will abort with an error. This means that a necessary step as part of the deployment process would be to refresh all the keys with the vault before attempting to run php artisan config:cache, so that all expected secrets are available in Redis memory. I don't see any other way of going about this when considering 3rd party vendor packages look for their secrets in the configuration files, which now live in a vault.

All of this essentially means that the secrets will be cached twice. Once by Redis, and then again by the framework when it reads the configuration files, which will then statically pull it out of Redis, and cache it again. It also means that the framework will have no knowledge of secrets with a TTL. To solve that issue, I'm leaning towards just setting up a periodic cron job that will invoke the secrets service to renew all keys, and then proceed to re-cache the config.

Does this sound like a solid plan? I'd love to hear from those who have experience at storing secrets in vaults and outside of the application. Is there a more efficient approach?

Thanks

0 likes
7 replies
LaryAI's avatar
Level 58

To achieve SOC 2 compliance and manage secrets effectively without using .env files, your approach of using a secrets management service like HashiCorp Vault or Azure Key Vault is a solid plan. Here’s a refined approach to address the concerns you’ve raised:

Steps to Implement Secrets Management

  1. Use a Secrets Management Service:

    • Choose a service like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault to store and manage your secrets securely.
  2. Access Secrets at Runtime:

    • Implement a service or utility class in your application that can fetch secrets from the vault at runtime. This class should handle caching the secrets locally to minimize repeated calls to the vault.
  3. Bootstrap Configuration:

    • During the application bootstrap process, before Laravel reads the configuration files, fetch the necessary secrets from the vault and populate them into the environment variables. This can be done by creating a custom bootstrap script or service provider.
  4. Cache Secrets Locally:

    • Use a local caching mechanism (like Redis) to store secrets temporarily. This reduces the number of calls to the vault and improves performance.
  5. Periodic Refresh:

    • Implement a cron job or scheduled task that periodically refreshes the secrets from the vault and updates the local cache. This ensures that any changes in the vault are reflected in your application.
  6. Configuration Caching:

    • After fetching and caching the secrets, run php artisan config:cache to cache the configuration files. This step should be part of your deployment process.
  7. Handle Secret Expiry:

    • Ensure that your application can handle secret expiry gracefully. The periodic refresh task should renew secrets before they expire.

Example Implementation

Here’s a simplified example of how you might implement a service to fetch and cache secrets:

Deployment Considerations

  • Pre-deployment Script: Before deploying, run a script that fetches all necessary secrets and populates them into the cache.
  • Environment Configuration: Ensure your deployment environment has access to the vault and necessary permissions to fetch secrets.

Conclusion

Your plan to use a secrets management service and cache secrets locally is a robust approach to achieving SOC 2 compliance. By ensuring secrets are fetched securely and cached efficiently, you can maintain both security and performance. Adjust the caching strategy and refresh intervals based on your application's specific needs and the sensitivity of the secrets.

martinbean's avatar

@talinon I think your main problem stems from the fact that you seem to think that secrets should be stored in .env files. They shouldn’t. .env files are a replacement for environment variables proper, which you should be using. This then adheres to one of the tenets of the “12 Factor” apps where configuration is kept entirely separate from your application code.

So, in short, use environment variables.

Talinon's avatar
Level 51

@jlrdw I'm aware of that thread. It was the primary reason for me writing a long commentary about the issue, because I know it's a controversial topic. Look at the responses in that thread for support for .env. But again, I'm not arguing that secrets shouldn't be in a .env, the entire point of this thread is to discuss how to keep them out of the .env.

@martinbean You don't need to convince me that secrets shouldn't be in the .env. I have no choice in the matter. You have mentioned in a couple threads to use environment variables, but you don't provide any insight on how you manage it, and the content for this on the web is scarce.

So let's talk about O/S level environment variables for a moment. How do you set them? If you place them within the /etc/environment file, it doesn't get automatically applied to processes, and requires a reboot. Do you reboot the server every time a secret gets rotated?

Do you use a script that gathers the secrets somewhere and uses putenv()? But that won't work, because that's specific to the process. It would mean that every PHP process would need to do that.

Do you put them into the web server configuration, such as SetEnv for Apache. That works for Apache processes, but it doesn't for PHP CLI, which renders it useless as most of these keys are used for background processes.

You can't manually use an export bash command because it only lives in the current session.

So, how do you manage your environment variables? That is what I want to know. If I am to use environment variables, then how do I automate setting the environment variables when they live on an external vault, making them available to processes without a reboot?

I think what I have works.. Lary regurgitated everything I said, and the AI generated code is quite similar to my solution, so it seems he doesn't have any issue with it. I'm just curious about there being better ways.. perhaps a package that handles this? Maybe a python solution that could facilitate this operation?

Let me know how you handle it. Thanks.

1 like
jlrdw's avatar

I never place the main laravel folder in a web accessible folder. I use the actual config files for these things.

Talinon's avatar
Level 51

@jlrdw I don't even know how to respond to that comment... Who said anything about making the main Laravel directory accessible? Please stay on topic.

3 likes
gareth771's avatar

@talinon Sorry to dig up a old thread, but I am tackling the exact same problem as you and would love your insight in to what solution you ended up with. Currently, I plan to load all the secrets on boot via a Provider then as you said, use a cron job to reload to preserve the TTL aspect. Here is the code for that option (the environment variable for region and secret name will come from the task definition of our ECS):

public function boot(): void
  {
    $region = env('AWS_REGION', 'us-east-1');
    $secretName = env('AWS_SECRET_NAME');

    if (!$secretName) {
        return;
    }

    $client = new SecretsManagerClient([
        'version' => 'latest',
        'region' => $region,
    ]);

    try {
        $result = $client->getSecretValue([
            'SecretId' => $secretName,
        ]);

        $awsSecrets = json_decode($result['SecretString'], true);

        foreach ($awsSecrets as $key => $value) {
            Config::set($key, $value);
            $_ENV[$key] = $value;
        }
    } catch (\Exception $e) {
        CustomLogger::warning("AWS SecretsManager fetch failed: " . $e);
    }
}

The other option I considered was to fetch every secret every time, our daily active users is quite small so perhaps it is more feasible but it does make me nervous.

Please or to participate in this conversation.