One of the significant advantages when using Python is the large amount of the available modules, we can say that there are python modules for almost everything you need.
You need to work with robots, or some hardware in a spaceship, you will find a python module for that.
In this post, we will talk about python modules and how to create, install, upgrade, reload and use them.
Table of Contents
- 1 What is the python module?
- 2 Standard Library Modules
- 3 How import statement works
- 4 Python Search for a Module
- 5 Creating a Python Module
- 6 Installing Python Modules from Source
- 7 Installing Python Modules Using pip
- 8 Importing Modules
- 9 The Danger of Using from Statement
- 10 Using Aliases
- 11 Import Module Scope
- 12 Reloading Modules
What is the python module?
The python module allows you to organize your Python code, so that makes your code easier to understand and easier to maintain later.
Simply, a module is a file consisting of Python programming. Inside a module, you can define functions, classes, and variables.
You can send your module to your friends and share it with others so they can use it.
Standard Library Modules
Python shipped with a large collection of modules known as the standard library. This collection contains more than 200 modules for common programming tasks.
None of these modules are part of the Python language itself, but you can use them by importing the needed modules on any of your files. Because they are standard library modules, you can be sure that they will be available on most platforms on which you will run Python.
These modules can be used by importing them using the import statement as we’ve seen in the previous post python programming basics.
How import statement works
Some newcomers might think that the import statement works like the include directive in C language or require in PHP, but that far from the truth because the other languages all it does is just textual insertions of one file into another.
In python, the situation is little different. There are three steps performed the first time a program imports a given file.
- Find the module’s file.
- Compile it to bytecode if needed.
- Run the module’s code to build the objects.
First, Python must find the module file referenced by the import statement.
You type the imported file without the extension and without its full directory path.
Then Python compiles it to bytecode. If the bytecode file (.pyc file) is older than the source file or was created by a different Python version, Python automatically regenerates the bytecode.
If Python finds only a bytecode file on the search path and no source file for it, it simply loads the bytecode directly; this means you can ship a program as just bytecode files and avoid sending source.
The final step of the import operation is to execute the generated bytecode.
Python Search for a Module
As we said earlier, you just type the name of the module without extension or complete path and python will take care of the rest.
So where python does the search for the modules?
- The home directory of your program
- PATH environment variable directories
- Standard library directories
- The contents of any .pth files if found
- The site-packages home of your third-party extensions
The combination of these items becomes sys.path list.
Actually, sys.path is the path where Python searches for modules. Python configures it at program startup.
You can get your path like this:
The result is the list of directories that Python searches on each import of a new file.
You can modify the search path for all future imports. This modification can be only for the duration of the script.
If you use sys.path.append or sys.path.insert will append your path for a single program run only.
When you type import mymodule, python will load any of the followings:
- Source code file named mymodule.py
- Bytecode file named mymodule.pyc
- Optimized bytecode file named mymodule.pyo
- Directory named mymodule, for package imports
- Compiled extension module like mymodule.so on Linux, or mymodule.dll on Windows
- Compiled module coded in C and statically linked into Python
- ZIP file that is automatically extracted when imported
- Java class, in the Jython version of Python
- .NET component, written in the IronPython version of Python
If you have two different files like mymodule.py and mymodule.dll, python will load the first file found in the search path during the left-to- right search of sys.path. But what if python found both files in the same directory?
Well, in this case, python follows a standard picking order, which is not guaranteed to stay the same over time. So stay away from doing that.
Creating a Python Module
To create a python module, nothing special is required, just open your editor and type your code and save your file with .py extension.
All the names defined at the top level of the module become its attributes.
For example, if you define a function like the following inside your module
You can call the function myfunc from that imported module inside your code.
Keep in mind that when creating a module, the module name becomes a variable (without the extension) inside your python program, so you can’t name a module if.py. Always avoid python reserved keywords.
Installing Python Modules from Source
Many modules are written to use a system called distutils, which is included with Python. Distutils is a very basic system that is included with Python.
The module includes a special file named setup.py that handles the details.
You can install python modules from the source like this
$ python setup.py install
This command will install the module within the site-packages folder which could be like this C:\Python27\lib\site-packages
Or like this
Installing Python Modules Using pip
You can use pip utility to easily install modules and packages directly from the PyPI repository.
Most developers prefer installing modules using this method. The best thing about pip is that it handles dependency checking.
You can install python modules using pip like this:
$ pip install numpy
Also, you can upgrade your installed modules using pip like this:
$ pip install --upgrade numpy
This command will update all the dependencies of the modules.
After writing your awesome modules or installing them, now you need to reuse it anywhere in your code, or maybe send it to your friends so they can use it, or share it with others on the web. How to import modules in your code?
There are two ways to achieve that, the first way is to use import statement, and the second way is to use from statement. Both of them will find, compile and run.
The major difference is that import statement fetches the whole module, while from statement fetches the specified attributes from the modules.
from mymodule import myfunc
The import Statement
Because import statement loads all attributes from the module so you must go through the module name to fetch its attributes.
The from Statement
As we said before, the from statement loads only the specified attributes.
Using the from statement makes it less typing because we don’t have to specify the whole module name when calling it.
from mymodule import myfunc
Actually, behind the scenes the from statement is an extension of the import statement, the entire file is also loaded, but you will be provided with all attributes directly from your code.
You can write the from statement like this:
from mymodule import *
The asterisk here means import everything at the top level from the module.
The from statement copies all names in the module into the importing space.
The Danger of Using from Statement
Some python developers recommend using the import statement over using the from statement.
But is this true? Does the from statement not safe?
Well, when using the from statement, if you use it to import variables that happen to have the same names as existing variables in your scope, your variables will be silently overwritten.
This problem, of course, doesn’t happen with the import statement. Since you type the module name to get its attributes, so no collision happen.
You should take care when using the from statement. Consider the following situation
print("Hello from module 1")
print("Hello from module 2")
Now, let’s import the above modules and try to use them
from module1 import myfunc
from module2 import myfunc
The output here will be from module 2 since the function will be overwritten.
Actually, this is not a common thing to encounter, but anyway, there is a solution for that.
To avoid the name collision happened in the above example, you can use aliases like this:
from module1 import myfunc as myfunc1
from module2 import myfunc as myfunc2
It’s like a renaming the imports. Sometimes developers use aliases to shorten the module names they are importing when the module name is bigger.
After using aliases, do you think that using the from statements is dangerous? For me, it’s not.
Import Module Scope
You know that we can’t access the module attributes unless we import it.
Consider the following situation
M = 10
M = 20
M = 50
When you run module2, the module1.myfunc() changes the M variable in module1 not the M in module2.
The result will be 50 20.
That means the global scope for module1.myfunc is the file that enclosing it, not the file it runs from.
The import statements don’t promote the visibility of the attributes, and the imported file can’t see the attributes in the importing file.
Python modules are loaded and run on the first import and the first only.
If you try to import the module again, python will fetch the already loaded module again.
To reload a module, you can use the reload function that can achieve that.
Why do we need to reload the imported module?
The answer is so simple. This allows some parts of your program to be modified without restarting your program. You can imagine any situation that needs dynamic customization.
Python can only reload modules written in python only. So compiled extensions (like the discussed previously) can’t be reloaded.
You MUST import the module first before you reload it. You can reload modules like this:
from importlib import reload
You can change the code of module1 in your text editor and it will be reloaded.
Reloading modules is a very powerful feature in python if it is used carefully.
I hope you find the post useful and interesting. Keep coming back.