Welcome back, and thank you for joining for for another Docker tutorial. As we alluded to in a previous post, the skills and capabilities we’ve begun to acquire are starting to become quite powerful. From the very beginning, we’ve installed Docker on Windows, learned about the Docker Desktop interface, learned about Docker containers, built a Docker Compose file to build our containers, and connected two containers on a network and via shared volume. Together, these capabilities give us the ability to work with a dataset in real time. But we’re still missing one thing: the ability to automatically run an iterative script to process the data.
The main idea behind the setup we’ll be building is this:
- Build a single Docker container; the container’s image should have Python installed.
- The container will have a
/datadirectory which will house two files: a Bash file and a Python script.
- The container will be built using Docker Compose; during the build process, we’ll pass in a command to run the Bash script.
- The Bash script contains a command to iteratively run the Python script.
- Each time the Python script is run, it prints a message to the screen.
Pretty simple! In the interest of transparency, our folder hierarchy will look like something like this. We have a folder (“docker_intro”) housing a Docker file (
dockerfile), a Compose file (
docker-compose.yml), and a subdirectory titled “smee”. The
smee folder contains a Bash file (
iterfile.sh) and a Python script (
repeat.py). Now we need to describe the Docker file, the Compose file, the Bash file, and the Python script:
Unlike our previous Docker files, we’ll need something with a bit more “oomph” so we can run Python within our container. Instead of using Alpine:3.14, we’ll now just directly use a Python image taken from the official Python images in Docker Hub. For our current demonstration, there’s nothing particularly special about the
FROM python:3.10-slim-bullseye AS build
Our Compose file is pretty similar to our previous tutorials. We’re running Compose version 3.9, and we only have a single service,
container. The container service builds a container called
first_container using our Docker file titled
dockerfile. Furthermore, we’re going to go ahead and bind our external folder,
smee, to a directory (
/data) within our container,
Finally, we hit pay dirt — the new component of this tutorial! After binding our host directory to the container directory, we tell Compose to run the Bash script using the command:
sh /data/iterfile.sh . In compose, we break this command up into two portions: the program (or command) we want to call (“
sh“, which will call Bash), and an argument. In this case, the argument is the directory location and name of the Bash file.
version: '3.9' # Define services services: container: container_name: first_container build: context: . dockerfile: dockerfile volumes: - type: bind source: "/C/Users/apung/Desktop/docker_intro/smee" target: /data command: ["sh", "/data/iterfile.sh"]
Although the Bash file can become as complex as you’d like, I’m only going to show a simple loop. The loop will run indefinitely due to the while true condition. So while true is equal to true (which is always), we’re asking Bash to run a Python script (
repeat.py) located in the
/data directory using…well…Python. After it runs the Python script, Bash will wait (“
sleep“) 10 seconds, then it will call
#!/bin/bash while true do python /data/repeat.py sleep 10 done
Similar to the Bash script, the Python script is also very simple. Although inefficient, we import the
datetime package each time the script is run. Based on the current date and time, we define a variable
now, and format the string into a readable format. The new string is saved as a second variable,
current_time, which is then printed to the console.
from datetime import datetime now = datetime.now() current_time = now.strftime("%H:%M:%S") print("Current Time =", current_time)
Putting it all together
Now that we’ve defined our directories and necessary files, let’s go ahead and see what happens. Ideally, we’d expect Compose to build a single container in the background. Within that container, a Bash script will keep calling a Python script indefinitely. Hopefully, we’ll then be able to attach to the running container and demonstrate that the Python script is being called repeatedly. If successful, we’ll see a series of strings printed to the Terminal, each with a 10 second offset from the previous.
# Run the Compose file; rebuild containers, and run in Detached mode vscode ➜ /com.docker $ docker-compose up --build -d Creating network "comdocker_default" with the default driver Building container [+] Building 49.1s (5/5) FINISHED ... => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:08c7ea39270228f0ff12a6b467ec000b25db6384411c3877c6631bafc0f69596 0.0s => => naming to docker.io/library/comdocker_container 0.0s Creating first_container ... done # Look at running containers vscode ➜ /com.docker $ docker-compose ps Name Command State Ports ------------------------------------------------------ first_container sh /data/iterfile.sh Up # Attach to `container` service, and look at logs vscode ➜ /com.docker $ docker-compose logs --follow container Attaching to first_container first_container | Current Time = 15:52:58 first_container | Current Time = 15:53:08 first_container | Current Time = 15:53:18 first_container | Current Time = 15:53:28 first_container | Current Time = 15:53:38
Perfect! From the first block of text, we see that our container (“
first_container“) builds successfully and written to an image. We further show the existence of the container using
docker-compose ps, which shows that the container is running and it is running our Bash file! Lastly, we can check the output of the Python script called by our Bash file by look at the logs. This is done using the command:
docker-compose logs --follow container, which allows us to
follow the output of our
container service in real time.
So what does this mean? It means that we can now merge our capability of mounting a data volume within a container with our new ability to automatically run a processing script on that dataset! Every step we take, this process is becoming more and more automated. Imagine what we can do from here!
In this tutorial, we’ve built a Docker container using a Compose file. After Compose builds our container, we asked Compose to run a Bash file within the container. In turn, the Bash script called a Python script, and the Python script printed the current date and time to the terminal every 10 seconds. We then demonstrated our newfound capability by following the container logs and showing that our process was indeed running!
Join me next time as we learn how to incorporate unit tests into our Docker builds! As usual, thank you for dropping by — your support and attention means a great deal to me! If you’ve found the content, please feel free to Like, comment, or subscribe!
Get new content delivered directly to your inbox.
(Header image: Vortex vector by liuzishan)