{"id":2969,"date":"2025-04-09T07:02:50","date_gmt":"2025-04-09T07:02:50","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/04\/09\/a-data-scientists-guide-to-docker-containers\/"},"modified":"2025-04-09T07:02:50","modified_gmt":"2025-04-09T07:02:50","slug":"a-data-scientists-guide-to-docker-containers","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/04\/09\/a-data-scientists-guide-to-docker-containers\/","title":{"rendered":"A Data Scientist\u2019s Guide to Docker Containers"},"content":{"rendered":"

A Data Scientist\u2019s Guide to Docker Containers
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

For <\/mdspan>a ML model<\/mdspan> to be useful it needs to run somewhere. This somewhere is most likely not your local machine. A not-so-good model that runs in a production environment is better than a perfect model that never leaves your local machine.<\/p>\n

However, the production machine is usually different from the one you developed the model on. So, you ship the model to the production machine, but somehow the model doesn\u2019t work anymore. That\u2019s weird, right? You tested everything on your local machine and it worked fine. You even wrote unit tests.<\/p>\n

What happened? Most likely the production machine differs from your local machine. Perhaps it does not have all the needed dependencies installed to run your model. Perhaps installed dependencies are on a different version. There can be many reasons for this.<\/p>\n

How can you solve this problem? One approach could be to exactly replicate the production machine. But that is very inflexible as for each new production machine you would need to build a local replica.<\/p>\n

A much nicer approach is to use Docker<\/a> containers.<\/p>\n

Docker is a tool that helps us to create, manage, and run code and applications in containers. A container is a small isolated computing environment in which we can package an application with all its dependencies. In our case our ML model with all the libraries it needs to run. With this, we do not need to rely on what is installed on the host machine. A Docker Container<\/a> enables us to separate applications from the underlying infrastructure.<\/p>\n

For example, we package our ML model locally and push it to the cloud. With this, Docker helps us to ensure that our model can run anywhere and anytime. Using Docker has several advantages for us. It helps us to deliver new models faster, improve reproducibility, and make collaboration easier. All because we have exactly the same dependencies no matter where we run the container.<\/p>\n

As Docker is widely used in the industry Data Scientists need to be able to build and run containers using Docker. Hence, in this article, I will go through the basic concept of containers. I will show you all you need to know about Docker to get started. After we have covered the theory, I will show you how you can build and run your own Docker container.<\/p>\n

\n

What is a container?<\/h2>\n

A container is a small, isolated environment in which everything is self-contained. The environment packages up all code and dependencies.<\/p>\n

A container has five main features.<\/p>\n

    \n
  1. \nself-contained<\/strong>: A container isolates the application\/software, from its environment\/infrastructure. Due to this isolation, we do not need to rely on any pre-installed dependencies on the host machine. Everything we need is part of the container. This ensures that the application can always run regardless of the infrastructure.<\/li>\n
  2. \nisolated<\/strong>: The container has a minimal influence on the host and other containers and vice versa.<\/li>\n
  3. \nindependent<\/strong>: We can manage containers independently. Deleting a container does not affect other containers.<\/li>\n
  4. \nportable<\/strong>: As a container isolates the software from the hardware, we can run it seamlessly on any machine. With this, we can move it between machines without a problem.<\/li>\n
  5. \nlightweight<\/strong>: Containers are lightweight as they share the host machine\u2019s OS. As they do not require their own OS, we do not need to partition the hardware resource of the host machine.<\/li>\n<\/ol>\n

    This might sound similar to virtual machines. But there is one big difference. The difference is in how they use their host computer\u2019s resources. Virtual machines are an abstraction of the physical hardware. They partition one server into multiple. Thus, a VM includes a full copy of the OS which takes up more space.<\/p>\n

    In contrast, containers are an abstraction at the application layer. All containers share the host\u2019s OS but run in isolated processes. Because containers do not contain an OS, they are more efficient in using the underlying system and resources by reducing overhead.<\/p>\n

    \"\"
    Containers vs. Virtual Machines (Image by the author based on docker.com<\/a>)<\/figcaption><\/figure>\n

    Now we know what containers are. Let\u2019s get some high-level understanding of how Docker works. I will briefly introduce the technical terms that are used often.<\/p>\n

    \n

    What is\u00a0Docker?<\/h2>\n

    To understand how Docker works, let\u2019s have a brief look at its architecture.<\/p>\n

    Docker uses a client-server architecture containing three main parts: A Docker client, a Docker daemon (server), and a Docker registry.<\/p>\n

    The Docker client is the primary way to interact with Docker through commands. We use the client to communicate through a REST API with as many Docker daemons as we want. Often used commands are docker run, docker build, docker pull, and docker push. I will explain later what they do.<\/p>\n

    The Docker daemon manages Docker objects, such as images and containers. The daemon listens for Docker API requests. Depending on the request the daemon builds, runs, and distributes Docker containers. The Docker daemon and client can run on the same or different systems.<\/p>\n

    The Docker registry is a centralized location that stores and manages Docker images. We can use them to share images and make them accessible to others.<\/p>\n

    Sounds a bit abstract? No worries, once we get started it will be more intuitive. But before that, let\u2019s run through the needed steps to create a Docker container.<\/p>\n

    \"\"
    Docker Architecture (Image by author based on docker.com<\/a>)<\/figcaption><\/figure>\n
    \n

    What do we need to create a Docker container?<\/h2>\n

    It is simple. We only need to do three steps:<\/p>\n

      \n
    1. create a Dockerfile<\/li>\n
    2. build a Docker Image from the Dockerfile<\/li>\n
    3. run the Docker Image to create a Docker container<\/li>\n<\/ol>\n

      Let\u2019s go step-by-step.<\/p>\n

      A Dockerfile is a text file that contains instructions on how to build a Docker Image. In the Dockerfile we define what the application looks like and its dependencies. We also state what process should run when launching the Docker container. The Dockerfile is composed of layers, representing a portion of the image\u2019s file system. Each layer either adds, removes, or modifies the layer below it.<\/p>\n

      Based on the Dockerfile we create a Docker Image. The image is a read-only template with instructions to run a Docker container. Images are immutable. Once we create a Docker Image we cannot modify it anymore. If we want to make changes, we can only add changes on top of existing images or create a new image. When we rebuild an image, Docker is clever enough to rebuild only layers that have changed, reducing the build time.<\/p>\n

      A Docker Container is a runnable instance of a Docker Image. The container is defined by the image and any configuration options that we provide when creating or starting the container. When we remove a container all changes to its internal states are also removed if they are not stored in a persistent storage.<\/p>\n

      \n

      Using Docker: An\u00a0example<\/h2>\n

      With all the theory, let\u2019s get our hands dirty and put everything together.<\/p>\n

      As an example, we will package a simple ML model with Flask in a Docker container. We can then run requests against the container and receive predictions in return. We will train a model locally and only load the artifacts of the trained model in the Docker Container.<\/p>\n

      I will go through the general workflow needed to create and run a Docker container with your ML model. I will guide you through the following steps:<\/p>\n

        \n
      1. build model<\/li>\n
      2. create requirements.txt<\/code> file containing all dependencies<\/li>\n
      3. create Dockerfile<\/code>\n<\/li>\n
      4. build docker image<\/li>\n
      5. run container<\/li>\n<\/ol>\n

        Before we get started, we need to install Docker Desktop. We will use it to view and run our Docker containers later on.\u00a0<\/p>\n

        1. Build a\u00a0model<\/h3>\n

        First, we will train a simple RandomForestClassifier on scikit-learn<\/code>\u2019s Iris dataset and then store the trained model.<\/p>\n

        \n\t