Introduction<\/h2>\n
Developers work on applications that are supposed to be deployed on some server in order to allow anyone to use those. Typically in the machine where these apps live, developers set up environment variables that allow the app to run. These variables can be API keys of external services, URL of your database and much more.<\/p>\n
For local development though, it is really inconvenient to declare these variables on the machine because it is a slow and messy process. So I\u2019d like to share in this short tutorial how to use Pydantic<\/a> to handle environment variables in a secure way.<\/p>\n What you commonly do in a Python<\/a> project is to store all your environment variables in a file named .env. This is a text file containing all the variables in a\u00a0 The following is an example:<\/p>\n Be aware the .env file should remain private, so it is important that this file is mentioned in your\u00a0.gitignore<\/strong>\u00a0file, to be sure that you\u00a0never push it on GitHub<\/strong>, otherwise, other developers could steal your keys and use the tools you\u2019ve paid for.<\/p>\n To ease the life of developers who will clone your repository, you could include an env.example file in your project. This is a file containing only the keys of what is supposed to go into the .env file. In this way, other people know what APIs, tokens, or secrets in general they need to set to make the scripts work.<\/p>\n python-dotenv is the library you use to load the variables declared into the .env file. To install this library:<\/p>\n Now you can use the load_dotenv to load the variables. Then get a reference to these variables with the os module.<\/p>\n This method will first look into your .env file to load the variables you\u2019ve declared there. If this file doesn\u2019t exist, the variable will be taken from the host machine. This means that you can use the .env file for your local development but then when the code is deployed to a host environment like a virtual machine or Docker container we are going to directly use the environment variables defined in the host environment.<\/p>\n Pydantic is one of the most used libraries in Python for data validation. It is also used for serializing and deserializing classes into JSON and back. It automatically generates JSON schema, reducing the need for manual schema management. It also provides built-in data validation, ensuring that the serialized data adheres to the expected format. Lastly, it easily integrates with popular web frameworks like FastAPI.<\/p>\n<\/blockquote>\n pydantic-settings<\/strong><\/a>\u00a0<\/strong>is a Pydantic feature needed to load and validate settings or config classes from environment variables.<\/p>\n We are going to create a class named\u00a0 In the\u00a0 In our case, we setup a\u00a0 You can also have multiple aliases to look for in the .env file that you can specify by leveraging\u00a0 The last attribute\u00a0 The next step would be to actually instantiate and use these settings in your Python project.<\/p>\n Now what use the settings in other parts of our codebase.<\/p>\n You finally have an easy access to your settings, and Pydantic helps you validate that the secrets have the correct format. For more advanced validation tips refer to the Pydantic documentation:\u00a0https:\/\/docs.pydantic.dev\/latest\/<\/a><\/p>\n Managing the configuration of a project is a boring but important part of software development. Secrets like API keys, db connections are what usually power your application. Naively you can hardcode these variables in your code and it will still work, but for obvious reasons, this could not be a good practice. In this article, I showed you an introduction on how to use pydantic settings to have a structured and safe way to handle your configurations.<\/p>\n The post Manage Environment Variables with Pydantic<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n.env file<\/h2>\n
key : value<\/code>\u00a0format. You can use also the value of one of the variables to declare one of the other variables by leveraging the\u00a0{}<\/code>\u00a0syntax.<\/p>\n#.env file\n\nOPENAI_API_KEY=\"sk-your private key\"\nOPENAI_MODEL_ID=\"gpt-4o-mini\"\n\n# Development settings\nDOMAIN=example.org\nADMIN_EMAIL=admin@${DOMAIN}\n\nWANDB_API_KEY=\"your-private-key\"\nWANDB_PROJECT=\"myproject\"\nWANDB_ENTITY=\"my-entity\"\n\nSERPAPI_KEY= \"your-api-key\"\nPERPLEXITY_TOKEN = \"your-api-token\"<\/code><\/pre>\nenv.example file<\/h2>\n
#env.example\n\nOPENAI_API_KEY=\"\"\nOPENAI_MODEL_ID=\"\"\n\nDOMAIN=\"\"\nADMIN_EMAIL=\"\"\n\nWANDB_API_KEY=\"\"\nWANDB_PROJECT=\"\"\nWANDB_ENTITY=\"\"\n\nSERPAPI_KEY= \"\"\nPERPLEXITY_TOKEN = \"\"<\/code><\/pre>\npython-dotenv<\/h1>\n
pip install python-dotenv<\/code><\/pre>\nimport os\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nOPENAI_API_KEY = os.getenv('OPENAI_API_KEY')\nOPENAI_MODEL_ID = os.getenv('OPENAI_MODEL_ID')<\/code><\/pre>\nPydantic<\/h2>\n
\n
!pip install pydantic-settings<\/code><\/pre>\nSettings<\/code>. This class will inherit\u00a0BaseSettings<\/code>.\u00a0<\/em>This makes the default behaviours of determining the values of any fields to be read from the .env file. If no var is found in the .env file it will be used the default value if provided.<\/p>\nfrom pydantic_settings import BaseSettings, SettingsConfigDict\n\nfrom pydantic import (\n AliasChoices,\n Field,\n RedisDsn,\n)\n\n\nclass Settings(BaseSettings):\n auth_key: str = Field(validation_alias='my_auth_key') \n api_key: str = Field(alias='my_api_key') \n\n redis_dsn: RedisDsn = Field(\n 'redis:\/\/user:pass@localhost:6379\/1', #default value\n validation_alias=AliasChoices('service_redis_dsn', 'redis_url'), \n )\n\n model_config = SettingsConfigDict(env_prefix='my_prefix_')<\/code><\/pre>\nSettings<\/code>\u00a0class above we have defined several fields. The\u00a0Field<\/code>\u00a0class is used to provide\u00a0extra information<\/a>\u00a0about an attribute.<\/p>\nvalidation_alias<\/code>. So the variable name to look for in the .env file is overridden. In the case reported above, the environment variable\u00a0my_auth_key<\/em>\u00a0will be read instead of\u00a0auth_key<\/em>.<\/p>\nAliasChoises(choise1, choise2).<\/code><\/p>\nmodel_config<\/code>\u00a0, contains all the variables regarding a particular topic (e.g connection to a db). And this variable will store all .env var that start with the prefix\u00a0env_prefix.<\/code><\/p>\nInstantiate and use settings<\/h2>\n
from pydantic_settings import BaseSettings, SettingsConfigDict\n\nfrom pydantic import (\n AliasChoices,\n Field,\n RedisDsn,\n)\n\n\nclass Settings(BaseSettings):\n auth_key: str = Field(validation_alias='my_auth_key') \n api_key: str = Field(alias='my_api_key') \n\n redis_dsn: RedisDsn = Field(\n 'redis:\/\/user:pass@localhost:6379\/1', #default value\n validation_alias=AliasChoices('service_redis_dsn', 'redis_url'), \n )\n\n model_config = SettingsConfigDict(env_prefix='my_prefix_')\n\n# create immediately a settings object\nsettings = Settings()<\/code><\/pre>\nfrom Settings import settings\n\nprint(settings.auth_key) <\/code><\/pre>\nFinal thoughts<\/strong><\/h2>\n
![\"\ud83d\udcbc\"](\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f4bc.png?ssl=1\")
\u00a0Linkedin<\/a>\u00a0| ![\"\ud83d\udc26\"](\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f426.png?ssl=1\")
\u00a0X (Twitter)<\/a>\u00a0|\u00a0![\"\ud83d\udcbb\"](\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f4bb.png?ssl=1\")
<\/a>\u00a0Website<\/a><\/p>\n