Introduction
Welcome back, and thank you for joining for for another Docker tutorial. As we alluded to in a previous post, the skills and capabilities we’ve begun to acquire are starting to become quite powerful. From the very beginning, we’ve installed Docker on Windows, learned about the Docker Desktop interface, learned about Docker containers, built a Docker Compose file to build our containers, and connected two containers on a network and via shared volume. Together, these capabilities give us the ability to work with a dataset in real time. But we’re still missing one thing: the ability to automatically run an iterative script to process the data.
The Setup
The main idea behind the setup we’ll be building is this:
- Build a single Docker container; the container’s image should have Python installed.
- The container will have a
/datadirectory which will house two files: a Bash file and a Python script. - The container will be built using Docker Compose; during the build process, we’ll pass in a command to run the Bash script.
- The Bash script contains a command to iteratively run the Python script.
- Each time the Python script is run, it prints a message to the screen.
Pretty simple! In the interest of transparency, our folder hierarchy will look like something like this. We have a folder (“docker_intro”) housing a Docker file (dockerfile), a Compose file (docker-compose.yml), and a subdirectory titled “smee”. The smee folder contains a Bash file (iterfile.sh) and a Python script (repeat.py). Now we need to describe the Docker file, the Compose file, the Bash file, and the Python script:
Docker file
Unlike our previous Docker files, we’ll need something with a bit more “oomph” so we can run Python within our container. Instead of using Alpine:3.14, we’ll now just directly use a Python image taken from the official Python images in Docker Hub. For our current demonstration, there’s nothing particularly special about the 3.10-slim-bullseye build.
FROM python:3.10-slim-bullseye AS build
Compose file
Our Compose file is pretty similar to our previous tutorials. We’re running Compose version 3.9, and we only have a single service, container. The container service builds a container called first_container using our Docker file titled dockerfile. Furthermore, we’re going to go ahead and bind our external folder, smee, to a directory (/data) within our container, first_container.
Finally, we hit pay dirt — the new component of this tutorial! After binding our host directory to the container directory, we tell Compose to run the Bash script using the command: sh /data/iterfile.sh . In compose, we break this command up into two portions: the program (or command) we want to call (“sh“, which will call Bash), and an argument. In this case, the argument is the directory location and name of the Bash file.
version: '3.9'
# Define services
services:
container:
container_name: first_container
build:
context: .
dockerfile: dockerfile
volumes:
- type: bind
source: "/C/Users/apung/Desktop/docker_intro/smee"
target: /data
command: ["sh", "/data/iterfile.sh"]
Bash file
Although the Bash file can become as complex as you’d like, I’m only going to show a simple loop. The loop will run indefinitely due to the while true condition. So while true is equal to true (which is always), we’re asking Bash to run a Python script (repeat.py) located in the /data directory using…well…Python. After it runs the Python script, Bash will wait (“sleep“) 10 seconds, then it will call repeat.py again.
#!/bin/bash
while true
do
python /data/repeat.py
sleep 10
done
Python script
Similar to the Bash script, the Python script is also very simple. Although inefficient, we import the datetime package each time the script is run. Based on the current date and time, we define a variable now, and format the string into a readable format. The new string is saved as a second variable, current_time, which is then printed to the console.
from datetime import datetime
now = datetime.now()
current_time = now.strftime("%H:%M:%S")
print("Current Time =", current_time)
Putting it all together
Now that we’ve defined our directories and necessary files, let’s go ahead and see what happens. Ideally, we’d expect Compose to build a single container in the background. Within that container, a Bash script will keep calling a Python script indefinitely. Hopefully, we’ll then be able to attach to the running container and demonstrate that the Python script is being called repeatedly. If successful, we’ll see a series of strings printed to the Terminal, each with a 10 second offset from the previous.
# Run the Compose file; rebuild containers, and run in Detached mode
vscode ➜ /com.docker $ docker-compose up --build -d
Creating network "comdocker_default" with the default driver
Building container
[+] Building 49.1s (5/5) FINISHED
...
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:08c7ea39270228f0ff12a6b467ec000b25db6384411c3877c6631bafc0f69596 0.0s
=> => naming to docker.io/library/comdocker_container 0.0s
Creating first_container ... done
# Look at running containers
vscode ➜ /com.docker $ docker-compose ps
Name Command State Ports
------------------------------------------------------
first_container sh /data/iterfile.sh Up
# Attach to `container` service, and look at logs
vscode ➜ /com.docker $ docker-compose logs --follow container
Attaching to first_container
first_container | Current Time = 15:52:58
first_container | Current Time = 15:53:08
first_container | Current Time = 15:53:18
first_container | Current Time = 15:53:28
first_container | Current Time = 15:53:38
Perfect! From the first block of text, we see that our container (“first_container“) builds successfully and written to an image. We further show the existence of the container using docker-compose ps, which shows that the container is running and it is running our Bash file! Lastly, we can check the output of the Python script called by our Bash file by look at the logs. This is done using the command: docker-compose logs --follow container, which allows us to follow the output of our container service in real time.
So what does this mean? It means that we can now merge our capability of mounting a data volume within a container with our new ability to automatically run a processing script on that dataset! Every step we take, this process is becoming more and more automated. Imagine what we can do from here!
Conclusion
In this tutorial, we’ve built a Docker container using a Compose file. After Compose builds our container, we asked Compose to run a Bash file within the container. In turn, the Bash script called a Python script, and the Python script printed the current date and time to the terminal every 10 seconds. We then demonstrated our newfound capability by following the container logs and showing that our process was indeed running!
Join me next time as we learn how to incorporate unit tests into our Docker builds! As usual, thank you for dropping by — your support and attention means a great deal to me! If you’ve found the content, please feel free to Like, comment, or subscribe!
Get new content delivered directly to your inbox.
(Header image: Vortex vector by liuzishan)
Aaron J. Pung, Ph.D. 