Progress bars are valuable tools for estimating and displaying the amount of time that the task will take.
These can also be added to your Python scripts or code blocks to indicate the amount of time that the code execution will require.
There are various libraries available in Python such as
tqdm that can be used for this purpose.
What is tqdm?
tqdm is a Python library that provides functions that wrap around the specified iterable to give a smart progress bar as an output.
Python is a widely-used language to perform computationally intensive tasks that run over longer periods.
tqdm progress bar gives an indicator of the progress of these tasks.
The name “tqdm” is derived from the Arabic word ‘taqadum’ meaning progress in Arabic.
The library does allow for customizable progress bars, however at its base,
tqdm(iterable) is enough to get you started with a smart progress meter displaying the progress of the iterable.
Install tqdm progress
Before you use
tqdm, you need to install it using pip. You can run:
pip install tqdm
pip3 install tqdm
For conda environments, you can run:
conda install tqdm
Once the installation is done, you can wrap any iterable (for instance range, list, tuple, etc.) inside the function
to see the progress of the iteration of the iterable when using, for example, a for loop to traverse it.
from tqdm import tqdm for i in tqdm(range(0,100)): pass # do nothing
In this code, we first import the tqdm library.
Then we write a normal for loop to iterate over a
range object, except that we pass the
range object inside the method
tqdm of the
As the for loop cannot be empty, and since we don’t want to do anything in the loop, we use the
NOTE: Ensure that you do not name the file in which you are using the tqdm library as
This may result in an undesired output as your file may end up overriding the
tqdm library for all the Python code that runs from the same directory.
Print a message with the progress bar
A good developer practice involves not only making your code self-explanatory with descriptive comments,
but also making the output of your code as descriptive and self-explanatory as possible.
As a step in that direction, we can print text along with the progress bar in the terminal to indicate the purpose of the code being executed, or to add any other desired comments.
For this, you need to set the desc parameter in the
tqdm function call with your desired comment.
This text will then be printed beside the progress bar in the terminal.
from tqdm import tqdm for i in tqdm(range(0,100),desc="This loop does nothing"): pass
Nested Progress bars
Tqdm bars can also be used to indicate the progress of nested loops.
Multiple bars indicating the progress of each of the loops will be displayed.
In the code below, we will be using the labels ‘Outer loop’ and ‘inner loop’ to display the respective progress bars.
from tqdm import tqdm from tqdm import trange for i in tqdm(range(5), desc='Progress of Outer loop'): for j in trange((100), desc='Progress of inner loop'): pass
After some time, we see multiple bars, each corresponding to one inner loop,
while there is only one progress bar showing the progress of the outer loop
Note that in the code above, we have used the
trange function from the tqdm library.
This function acts as a replacement for using the range object parameter with the tqdm function.
trange function call indicates that a tqdm bar is to be displayed for the given range and all the additional features of tqdm such as desc can be used the same way.
tqdm with async task
To run concurrent tasks in Python, we use the
It has a single-thread, single process design and uses ‘cooperative multi-tasking’ to give a feel of concurrency.
While executing tasks with the
asyncio library, we can use the
tqdm progress bar to track its progress.
The following code example runs multiple sleep threads simultaneously, and the tqdm bar is wrapped around the tasks to display a progress bar.
import asyncio import time import tqdm import tqdm.asyncio async def sleep_duration(value = 1): start_time = time.time() await asyncio.sleep(delay=value) ending_time = time.time() intermittent_time = ending_time - starting_time return intermittent_time async def run_async(values): taskslist =  for value in range(len(values)): i_task = asyncio.create_task(sleep_duration(value=value)) taskslist.append(i_task) actual_async = [ await x for x in tqdm.tqdm(asyncio.as_completed(taskslist), total=len(taskslist)) ] return actual_async n = 10 values = [3.12,3.44,4.66,2.33,1.87,4.77,5.68] actual_async = asyncio.run( run_async(values=values))
The above code example may be a bit convoluted to grasp, which is normal for Python codes using async-await patterns!
run_async is used to run multiple sleep threads using the coroutine obtained with the help of the
sleep_duration function which uses time module to measure the intermittent time.
tqdm.tqdm is wrapped around the asynchronous process to generate the progress bar.
total parameter is optional and can be removed while
asyncio.as_completed wrapper is used.
Additional tqdm parameters
tqdm progress bar has some additional features that can be invoked with the help of a few additional parameters.
Here’s a list of some of them:
1. total – If it is not already implicitly specified (for eg. by the length of the iterable), the total number of iterations can be explicitly specified by setting the
from tqdm import tqdm from time import sleep for i in tqdm(range(0, 50), total = 10, desc ="total demo"): sleep(1)
Here, since we specify the total number of iterations to be 10, the progress bar will be displayed only for the first 10 iterations and will disappear after that.
This is how it looks after 10 iterations are over:
2. ncols – The width of the description text (specified by the
desc parameter) along with the progress bar, is determined dynamically based on the size of the window.
However, we can fix this width with the help of the
In the following example, the text along with the progress bar will be displayed using just 40 columns of the output screen, even though more are available.
from tqdm import tqdm from time import sleep for i in tqdm(range(0, 50), ncols = 40, desc ="ncols demo"): sleep(1)
3. initial – The
initial parameter can be set if you desire to start the progress from an arbitrary value such as 50 instead of the standard 0.
from tqdm import tqdm from time import sleep for i in tqdm(range(0, 20), initial = 10, desc ="initial demo"): sleep(1)
The above output is a snapshot of the first iteration.
The progress bar appears and shows iterations ’11-20′ for the first 10 iterations and then it disappears for the remaining iterations
4. disable – The progress bar can be disabled by setting the
disable parameter to
from tqdm import tqdm from time import sleep for i in tqdm(range(0, 10), disable = True, desc ="disable demo"): sleep(1)
As can be seen from the output, nothing appears for the entire duration of the iteration.
Using this parameter is as good as not using the tqdm library at all!
5. mininterval – The progress of the bar is updated every 0.1 seconds by default. This behavior can be changed with the use of the
from tqdm import tqdm from time import sleep for i in tqdm(range(0, 50), mininterval = 5, desc ="min interval demo"): sleep(1)
Output for the first 5 seconds:
Output for the next 5 seconds(5th to 10th sec):
Integration with pandas
Pandas is a popular library that is mainly used to manipulate numerical data.
It is one of the most commonly used libraries today for handling tabular data.
Pandas uses two different data types for data representation, ‘Series’ and ‘DataFrame‘.
While Series is a one-dimensional data structure analogous to a column in an excel sheet,
DataFrame is a two-dimensional data structure with rows and columns and can also represent heterogeneous data.
As pandas is built on top of the NumPy library, it is extensively used in the field of Data Science.
In pandas, you can use the
apply method to apply a function to every value of a series or every row/column of a dataframe.
We can use the tqdm progress bar with this method.
To use pandas, first install it using pip as:
pip install pandas
(For Python3, replace
pip3, and for conda environment, replace it with
import pandas as pd import numpy as np from tqdm import tqdm df = pd.DataFrame(np.random.randint(0, 100, (100, 100))) print(df.head(10).iloc[:,:5]) #print first 10 rows and first 5 columns tqdm.pandas(desc='pandas integration demo') df=df.progress_apply(lambda number:number +5) #add 5 to each number print(df.head(10).iloc[:,:5])
After importing the
tqdm library, we initialize a dataframe of size 100×100 with random integers between 0 and 100.
Now we use the
tqdm.pandas method to register the
pandas.progress_apply function with tqdm.
Now, instead of the usual
apply function, we use the
progress_apply function to display a tqdm bar to denote the percentage of progress.
progress_apply methods accept a function as a parameter.
In our case, we pass a lambda function that takes a number and adds 5 to it, and returns it.
Jupyter notebooks are open-source notebooks that can run multiple programming languages such as Python, R although it is most commonly used by Python users.
It provides a live and interactive Python runtime environment.
It is a popular choice, especially among ML practitioners and data engineers, to inspect data interactively or to test step-by-step the components of an ML pipeline.
We can display tqdm loaders inside a jupyter notebook.
After setting up your jupyter notebook, while the usage of
tqdm remains somewhat similar, the tqdm.notebook module is used to display the progress bar in notebooks.
pip3 install ipywidgets to ensure that the progress bar is displayed and dynamically updated in the notebook.
The tqdm notebook bar can contain one of the following three colors:
Green for successfully completed process.
Blue for an ongoing process.
Red for a process that has been terminated midway.
from tqdm.notebook import tqdm_notebook import time for i in tqdm_notebook(range(15), desc = 'Tqdm notebook’): time.sleep(1.5)
Output for an ongoing process:
Output for a terminated process:
Output for a successful process:
Similar to tqdm,
tqdm_notebook should be wrapped around an iterable.
The notebook function can also be used with all the features of the tqdm library by using keywords such as using
desc keyword to print a description of the progress bar,
total to specify the total iterations, etc.
Alternatives to tqdm
tqdm is a powerful tool to track the progress of an operation in Python.
Along with tqdm, several other libraries can be also used for achieving the same or similar goal.
Some of these libraries are
progressbar is a simple library and is fairly intuitive.
The default progress bar is printed with the # symbol. We can also print the progress bar as a spinner like the loading circle shown in web browsers.
Similar to tqdm, it can be wrapped around an iterable or can be used along with a context manager.
If you want to use the print function along with the progress bar, an extremely messy output will be displayed as a new progress bar is displayed on the output screen after every print function.
To avoid this, the
progressbar2 library can be used. It allows standard redirect to output, allowing for a clean progress bar along with the print function.
The last alternative mentioned here is the
alive-progress library which offers some of the coolest looking progress bar options.
If animated progress bars or an interactive progress bar is a requirement for your project then
alive-progress will be best suited for you.
Advantages of tqdm
One of the major advantages that the tqdm progress bar has over its alternatives is its speed.
The tqdm bar has an overhead of 60ns per iteration as compared to
progressbar which has an overhead of 800ns per iteration.
The tqdm bar also has other advantages such as it can not only be used with jupyter notebook but also with popular Python frameworks such as Keras, Dask, etc.
Options for custom integrations with Discord, Slack, and Telegram based on project requirements are also available.
Progress bars offer multiple advantages such as :
- Visual estimation of the progress of your task
- An estimate of the time required to complete your task
- An indication of an issue or error in your program.
The tqdm bar is a low overhead, smart progress bar that you can leverage in your project with the multiple customizations it offers.
Some of them were mentioned in this article and you can refer to the documentation at https://github.com/tqdm/tqdm for more detailed documentation.
Mokhtar is the founder of LikeGeeks.com. He is a seasoned technologist and accomplished author, with expertise in Linux system administration and Python development. Since 2010, Mokhtar has built an impressive career, transitioning from system administration to Python development in 2015. His work spans large corporations to freelance clients around the globe. Alongside his technical work, Mokhtar has authored some insightful books in his field. Known for his innovative solutions, meticulous attention to detail, and high-quality work, Mokhtar continually seeks new challenges within the dynamic field of technology.