Within our python code we will use the dotenv library, so we must provide the library to our python environment via

In our Python code we then will use the load_dotenv() function to access the secret:

Eventually you may want to manage multiple secrets, so using a dictionary might be a more comfortable approach:

If you have multiple files with secrets, you can use the dotenv_values function with different file names maybe from different environment us it like this:

Read secrets from environment variables

If you’re working in a local environment, you could use OS environment variables, but keep in mind that everybody who has access to the local console easily can read the environment variables.

Assume you’ve set an environment variable like this on a unix based OS (Linux, macOS):

On a Windows machine it would look like this:

To access the variable from a Python script, we will use the os library.

There is one caveat: this does not work from within a Jupyter Notebook, as the Jupyter kernel cannot read directly from environment, at least not when executed in a IDE like VSCode or PyCharm. For Jupyter notebooks it is easier to use the dotenv approach, otherwise you would have to configure your notebook kernel to be able to read the environment, which is out of scope for this post.

But for normal python scripts (.py) the os.envriron.get() function works pretty fine. Alternative you could use the os.environ class instead, but that might throw an error, when the variable is not known to the environment. the os.environ,get() function just would return “none” if the environment variable is not set:



So far, we could read secrets from file without the necessity to store them directly in out python code, which provides more security to our source code.

This is fine for single development and local execution, but it become tricky if the code is being shared in your team or shall be executed in any kind of pipeline or productional environment.

You would have to deal with syncing the “.hidden” files over the environments, with could lead to a security leakage.

On cloud environment a secure way to handle secrets is to use a cloud based secret manager. All major cloud provider offering such services and we’ll discuss how to use them in out python scripts in a bit.

We will discuss examples for AWS, Azure and GCP separately.

Read a secret from AWS Secrets Manager

I’ll assume that you already have configured your AWS secrets Manager, following the AWS documentation and have a secret named “API_KEY” already stored in the secret manager.

Furthermore your AW Senvironment has to be configured to enable python to authenticate (aws configure..). Have a look into the AWS Python SDK documentation for details.

To retrieve a secret from AWS Secrets Manager, you’ll first need to set up the AWS SDK for Python, boto3.

Python script to retrieve the secret from AWS key vault:

Save the above script to a file, e.g., “retrieve_secret.py”.



Run the script:

This script will print the value of the API_KEY from AWS Secrets Manager. Make your that to replay “<region>” with the region where you configured the secret manager.

Read a secret from Azure Key Vault

A quite similar approach works for a key vault in azure as well. To proceed, you need to configure a key vault in azure and already stored the secret you want to retrieve via script in the vault.

Note down the vault URI, as well as the key name, for the example I’ll use “API_KEY” again..

To retrieve a secret from Azure Key Vault using Python, you would typically use the Azure SDK for Python.

The libraries azure-identity and azure-keyvault-secrets make this process straightforward. We’re using “DefaultAzureCredential()” function to login, which provides a default chained token credential configuration for development. It attempts to authenticate via multiple methods in this order:

• Environment variables: AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID
• Managed identity (if running on an Azure service)
• Azure CLI (if installed and logged in)

One of these functionalities has to be enabled for your environment for the script to authenticate properly. Check the azure python SDK documentation for details if you have any issues.

To get the secret from key vault via script, install the required Python libraries first:

Then set up Azure CLI and log in (this step is optional but provides an easy way to authenticate):

Save the following snippet as “azure-get-secret.py”:



Once executed, the script will then print the value of the API_KEY from your Azure Key Vault. Ensure you have appropriate permissions to access the secret from the Key Vault.

Read a secret from GCP Secrets Manager

As a last example, let’s have a look how to use secrets manager in Google GCP.

To retrieve a secret from Google Cloud Platform’s Secret Manager using Python, you’d typically use the google-cloud-secret-manager library.

Install the required Python library:

Set up the Google Cloud SDK and authenticate:



This command will open a browser window for you to log in. Once logged in, it will create application default credentials that the Python script can use.



The Python script to retrieve the secret:

Replace YOUR_GCP_PROJECT_ID_HERE with your Google Cloud project ID and run the script:



The script will print the value of the API_KEY from GCP Secret Manager. Ensure you have appropriate permissions (roles/secretmanager.secretAccessor) on the secret in the GCP Secret Manager to access it.

Note: Always remember to keep credentials and sensitive data secure and avoid hard-coding them directly into scripts. Use environment variables, managed identities, or other secure methods of storing and retrieving such data.



That’s it. This should give you at least an idea how to store and retrieve secrets depending on your requirements. To utilize key vault in one of the major hypervisor, it must be ensured, that the environment is configured properly upfront running the examples. There are plenty of tutorials available. providing the details, so I skipped these details here.

Conclusion

Secret management is more than just a technical requirement; it’s a foundational element of cloud security. By efficiently and securely handling secrets in Python, developers fortify their applications against potential breaches, data thefts, and unauthorized access.

Each cloud provider, be it AWS, Azure, or GCP, offers a unique approach to secrets management, yet the underlying principle remains: secrets should be protected with the same vigor as one would guard their personal treasures.

As explored in this post, the tools are available and accessible; it’s up to us to implement them wisely.

Happy coding !!