\u201cI train models, analyze data and create dashboards \u2014 why should I care about Containers<\/a>?\u201d<\/p>\n Many people who are new to the world of data science ask themselves this question. But imagine you have trained a model that runs perfectly on your laptop. However, error messages keep popping up in the cloud when others access it \u2014 for example because they are using different library versions.<\/p>\n This is where containers come into play: They allow us to make machine learning models, data pipelines and development environments stable, portable and scalable\u200a\u2014\u200aregardless of where they are executed.<\/p>\n Let\u2019s take a closer look.<\/p>\n Table of Content<\/strong>s Containers are lightweight, isolated environments. They contain applications with all their dependencies. They also share the kernel of the host operating system, making them fast, portable and resource-efficient.<\/p>\n I have written extensively about virtual machines (VMs) and virtualization in \u2018Virtualization & Containers for Data Science Newbiews<\/a>\u2019. But the most important thing is that VMs simulate complete computers and have their own operating system with their own kernel on a hypervisor. This means that they require more resources, but also offer greater isolation.<\/p>\n Both containers and VMs are virtualization technologies.<\/p>\n Both make it possible to run applications in an isolated environment.<\/p>\n But in the two descriptions, you can also see the 3 most important differences:<\/p>\n You can experiment faster with Docker \u2014 whether you\u2019re testing a new ML model or setting up a data pipeline. You can package everything in a container and run it immediately. And you don\u2019t have any \u201cIt works on my machine\u201d-problems. Your container runs the same everywhere \u2014 so you can simply share it.<\/p>\n As a data scientist, your main task is to analyze, process and model data to gain valuable insights and predictions, which in turn are important for management.<\/p>\n Of course, you don\u2019t need to have the same in-depth knowledge of containers, Docker or Kubernetes as a DevOps Engineer or a Site Reliability Engineer (SRE). Nevertheless, it is worth having container knowledge at a basic level\u200a\u2014\u200abecause these are 4 examples of where you will come into contact with it sooner or later:<\/p>\n You are training a model. You not only want to use it locally but also make it available to others. To do this, you can pack it into a container and make it available via a REST API.<\/p>\n Let\u2019s look at a concrete example: Your trained model runs in a Docker container with FastAPI or Flask. The server receives the requests, processes the data and returns ML predictions in real-time.<\/p>\n ML models and pipelines require specific libraries. For example, if you want to use a deep learning model like a Transformer, you need TensorFlow or PyTorch. If you want to train and evaluate classic machine learning models, you need Scikit-Learn, NumPy and Pandas. A Docker container now ensures that your code runs with exactly the same dependencies on every computer, server or in the cloud. You can also deploy a Jupyter Notebook environment as a container so that other people can access it and use exactly the same packages and settings.<\/p>\n Containers include all packages, dependencies and configurations that an application requires. They therefore run uniformly on local computers, servers or cloud environments. This means you don\u2019t have to reconfigure the environment.<\/p>\n For example, you write a data pipeline script. This works locally for you. As soon as you deploy it as a container, you can be sure that it will run in exactly the same way on AWS, Azure, GCP or the IBM Cloud.<\/p>\n Kubernetes helps you to orchestrate containers. But more on that below. If you now get a lot of requests for your ML model, you can scale it automatically with Kubernetes. This means that more instances of the container are started.<\/p>\n Let\u2019s take a look at an example that anyone can run through with minimal time \u2014 even if you haven\u2019t heard much about Docker and containers. It took me 30 minutes.<\/p>\n We\u2019ll set up a Jupyter Notebook inside a Docker container, creating a portable, reproducible Data Science environment. Once it\u2019s up and running, we can easily share it with others and ensure that everyone works with the exact same setup.<\/p>\n To be able to use containers, we need Docker Desktop. To do this, we\u00a0download Docker Desktop from the official website<\/a>.<\/p>\n Now we create a new folder for the project. You can do this directly in the desired folder. I do this via Terminal \u2014 on Windows with Windows + R and open CMD.<\/p>\n We use the following command:<\/p>\n Now we open VS Code or another editor and create a new file with the name \u2018Dockerfile\u2019. We save this file without an extension in the same directory.\u00a0Why doesn\u2019t it need an extension?<\/strong><\/p>\n We add the following code to this file:<\/p>\n We have thus defined a container environment for Jupyter Notebook that is based on the official Jupyter SciPy Notebook image.<\/p>\n First, we define with With With With Next, we will build the Docker image. Make sure you have the previously installed Docker desktop open. We now go back to the terminal and use the following command:<\/p>\n With The Docker image is the template for the container. This image contains everything needed for the application such as the operating system base (e.g. Ubuntu, Python, Jupyter), dependencies such as Pandas, Numpy, Jupyter Notebook, the application code and the startup commands. When we \u201cbuild\u201d a Docker image, this means that Docker reads the Dockerfile and executes the steps that we have defined there. The container can then be started from this template (Docker image).<\/p>\n We can now watch the Docker image being built in the terminal.<\/p>\n We use If yes, we see the data for the created Docker image:<\/p>\n Next, we want to start the container and use this command to do so:<\/p>\n We start a container with Alternatively, you can also perform this step in Docker desktop:<\/p>\n Now we open the URL [http:\/\/localhost:8888](http:\/\/localhost:8888\/)\u00a0in the browser. You should now see the Jupyter Notebook interface.<\/p>\n Here we will now create a Python 3 notebook and insert the following Python code into it.<\/p>\n Running the code will display the sine curve:<\/p>\n At the end, we end the container either with \u2018CTRL + C\u2019 in the terminal or in Docker Desktop.<\/p>\n With If you now want to upload your Docker image to a registry, you can do this with the following command. This will upload your image to Docker Hub (you need a Docker Hub account for this). You can also upload it to a private registry of AWS Elastic Container, Google Container, Azure Container or IBM Cloud Container.<\/p>\n If you then open Docker Hub and go to your repositories in your profile, the image should be visible.<\/p>\n This was a very simple example to get started with Docker. If you want to dive a little deeper, you can deploy a trained ML model with FastAPI via a container.<\/p>\n You can actually think of a container like a shipping container. Regardless of whether you load it onto a ship (local computer), a truck (cloud server) or a train (data center) \u2014 the content always remains the same.<\/p>\n And then there\u2019s Kubernetes.<\/p>\n A term that comes up a lot in this context: An orchestration tool that automates container management, ensuring scalability, load balancing and fault recovery. This is particularly useful for microservices and cloud applications.<\/p>\n Before Docker, VMs were the go-to solution (see more in \u2018Virtualization & Containers for Data Science Newbiews<\/a>\u2019.) VMs offer strong isolation, but require more resources and start slower.<\/p>\n So, Docker was developed in 2013 by Solomon Hykes to solve this problem. Instead of virtualizing entire operating systems, containers run independently of the environment \u2014 whether on your laptop, a server or in the cloud. They contain all the necessary dependencies so that they work consistently everywhere.<\/p>\n I simplify tech for curious minds The post Why Data Scientists Should Care about Containers \u2014 and Stand Out with This Knowledge<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n\n
1 \u2014 Containers vs. Virtual Machines: Why containers are more flexible than VMs<\/a>
2 \u2014 Containers & Data Science: Do I really need Containers? And 4 reasons why the answer is yes.<\/a>
3 \u2014 First Practice, then Theory: Container creation even without much prior knowledge<\/a>
4 \u2014 Your 101 Cheatsheet: The most important Docker commands & concepts at a glance<\/a>
Final Thoughts: Key takeaways as a data scientist<\/a>
Where Can You Continue Learning?<\/a><\/p>\n<\/blockquote>\n1 \u2014 Containers vs. Virtual Machines: Why containers are more flexible than VMs<\/h2>\n
\n
![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_V3oynmoJpAZNgw7lIViyrg-1024x673.webp?resize=1024%2C673&ssl=1\")
2 \u2014 Containers & Data Science: Do I really need Containers? And 4 reasons why the answer is yes.<\/h2>\n
Model deployment<\/h3>\n
Reproducibility and easier collaboration<\/h3>\n
Cloud integration<\/h3>\n
Scaling with Kubernetes<\/h3>\n
3 \u2014 First Practice, then Theory: Container creation even without much prior knowledge<\/h2>\n
0 \u2014 Install Docker Dekstop and create a project directory<\/h3>\n
![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_V1A9LcqYOANw6E0dARlFdA.webp?resize=653%2C171&ssl=1\")
1. Create a Dockerfile<\/h3>\n
# Use the official Jupyter notebook image with SciPy\nFROM jupyter\/scipy-notebook:latest \n\n# Set the working directory inside the container\nWORKDIR \/home\/jovyan\/work \n\n# Copy all local files into the container\nCOPY . .\n\n# Start Jupyter Notebook without token\nCMD [\"start-notebook.sh\", \"--NotebookApp.token=''\"]<\/code><\/pre>\nFROM<\/code> on which base image the container is built. jupyter\/scipy-notebook:latest<\/code> is a preconfigured Jupyter notebook image and contains libraries such as NumPy, SiPy, Matplotlib or Pandas. Alternatively, we could also use a different image here.<\/p>\nWORKDIR<\/code> we set the working directory within the container. \/home\/jovyan\/work<\/code> is the default path used by Jupyter. User jovyan<\/code> is the default user in Jupyter Docker images. Another directory could also be selected \u2014 but this directory is best practice for Jupyter containers.<\/p>\nCOPY . .<\/code> we copy all files from the local directory \u2014 in this case the Dockerfile, which is located in the jupyter-docker<\/code> directory \u2014 to the working directory \/home\/jovyan\/work<\/code> in the container.<\/p>\nCMD [\u201cstart-notebook.sh\u201d, \u201c \u2014 NotebookApp.token=\u2018\u2019\u2019\u201d]<\/code> we specify the default start command for the container, specify the start script for Jupyter Notebook and define that the notebook is started without a token \u2014 this allows us to access it directly via the browser.<\/p>\n2. Create the Docker image<\/h3>\n
cd jupyter-docker\ndocker build -t my-jupyter .<\/code><\/pre>\ncd jupyter-docker<\/code> we navigate to the folder we created earlier. With docker build<\/code> we create a Docker image from the Dockerfile. With -t my-jupyter<\/code> we give the image a name. The dot means that the image will be built based on the current directory. What does that mean? Note the space between the image name and the dot.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_eL3OSNKJTwRRI8pFkHyggw-1024x535.webp?resize=1024%2C535&ssl=1\")
docker images<\/code> to check whether the image exists. If the output my-jupyter<\/code> appears, the creation was successful.<\/p>\ndocker images<\/code><\/pre>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_kMTvvysX74TI108ccqevnQ.webp?resize=761%2C83&ssl=1\")
3. Start Jupyter container<\/h3>\n
docker run -p 8888:8888 my-jupyter<\/code><\/pre>\ndocker run<\/code>. First, we enter the specific name of the container that we want to start. And with -p 8888:8888<\/code> we connect the local port (8888) with the port in the container (8888). Jupyter runs on this port. I do not understand.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_kryRnbK_xYGev5b2zghazw-1024x367.webp?resize=1024%2C367&ssl=1\")
4. Open Jupyter Notebook & create a test notebook<\/h2>\n
import numpy as np\nimport matplotlib.pyplot as plt\n\nx = np.linspace(0, 10, 100)\ny = np.sin(x)\n\nplt.plot(x, y)\nplt.title(\"Sine Wave\")\nplt.show()<\/code><\/pre>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1__sle74116Vo2ltBrIE24Yg-1024x512.webp?resize=1024%2C512&ssl=1\")
5. Terminate the container<\/h3>\n
docker ps<\/code> we can check in the terminal whether containers are still running and with docker ps -a<\/code> we can display the container that has just been terminated:<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_Oe8l1asejIIaPuQF1qGrLQ-1024x111.webp?resize=1024%2C111&ssl=1\")
6. Share your Docker image<\/h3>\n
docker login\n\ndocker tag my-jupyter your-dockerhub-name\/my-jupyter:latest\n\ndocker push dein-dockerhub-name\/mein-jupyter:latest<\/code><\/pre>\n4 \u2014 Your 101 Cheatsheet: The most important Docker commands & concepts at a glance<\/h2>\n
The most important Docker terms<\/h3>\n
\n
The basic concepts behind containers<\/h3>\n
\n
The most basic Docker commands<\/strong><\/h3>\n
docker --version # Check if Docker is installed\ndocker ps # Show running containers\ndocker ps -a # Show all containers (including stopped ones)\ndocker images # List of all available images\ndocker info # Show system information about the Docker installation\n\ndocker run hello-world # Start a test container\ndocker run -d -p 8080:80 nginx # Start Nginx in the background (-d) with port forwarding\ndocker run -it ubuntu bash # Start interactive Ubuntu container with bash\n\ndocker pull ubuntu # Load an image from Docker Hub\ndocker build -t my-app . # Build an image from a Dockerfile\n<\/code><\/pre>\nFinal Thoughts: Key takeaways as a data scientist<\/h2>\n
![\"\ud83d\udc49\"](\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f449.png?ssl=1\")
With Containers you can solve the \u201cIt works on my machine\u201d problem. Containers ensure that ML models, data pipelines, and environments run identically everywhere, independent of OS or dependencies.<\/p>\n Containers are more lightweight and flexible than virtual machines. While VMs come with their own operating system and consume more resources, containers share the host operating system and start faster.<\/p>\n There are three key steps when working with containers: Create a Dockerfile to define the environment, use docker build to create an image, and run it with docker run \u2014 optionally pushing it to a registry with docker push.<\/p>\n![\"\ud83d\ude80\"](\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f680.png?ssl=1\")
If you enjoy my tech insights on Python, data science, Data Engineering<\/a>, machine learning and AI, consider subscribing to my\u00a0substack<\/a>.<\/p>\nWhere Can You Continue Learning?<\/h2>\n
\n