Python Functions Basics

Table of Contents

What is python module ?

Modules are the way to structure large programs and create reusable libraries.
Each file in python with a .py, which contains statements and definitions is a module.
Generally the python modules are designed to be used by other python program files, but not to run directly.
Modules can be reloaded and rerun as many times as needed.

Built in python modules

When you install python there are many modules (called as standard library modules) have been already installed/copied to your system
For example sys, math, os.
More modules can be found here https://docs.python.org/3/library/index.html

Create a python module

Let us create a module (which is nothing but a text file with .py extension) and name it mod1.py and write few lines of code in it.
Module name mod1.py can be any name as per the rules of identifier.

P = 10
def add(x, y):
    print(x + y)
class Example(object):
    pass

The functions, classes and variables defined (written) inside the modules are called as module attributes.
The module can be imported to other files (modules/scripts/programs) and these attributes can be accessed and reused

Python import statement

A module can be imported to another program file/module using import keyword.
Example
Let us create a python file main.py and imports a standard library modules

import sys
import os

OR

import sys,  os # Same line - comma separated

Let us try to import the module we have created earlier mod1.py into main.py
The main.py need to be created in the same location/directory where mod1.py exists
main.py

import mod1
mod1.add(5, 6) # prints 11
print(mod1.P) # prints 10
print(mod1.Example) # prints <class 'mod1.Example'>

Here module mod1.py is imported or loaded using import keyword
Note that the name mod1 is same as mod1.py without a .py extension.
So using that name mod1 , the definitions/attributes or names of mod1.py can be accessed, like we have done in this script.(mod1.add(5, 6))

When python finds import statement , it executes the statements (only top-level) inside a module and creates objects for each definition (like P, add only top-level ) and makes those available for use.
Example
mod2.py

P = 10
print('Executing mod2 module')

main.py

import mod2

Running main,py gives follwing result

>> python main.py
Executing mod2 module

Here the print statement print('Executing mod2 module') is executed only when import it finds and makes the name P available to use.

main.py

import mod2
print(mod2.P)

Running main,py gives follwing result

>> python main.py
Executing mod2 module
10

Before running module, python does few more steps when import it finds in a statement, which are discussed in the next section

Only the top level attributes can be accessed not nested ones
Example

P = 10
def add(x, y):
    def subtract(a, b):
        print(a - b)
    print(x + y)
    Y = 10
class Example(object):
    Pass

in main.py

import mod1
mod1.subtract(10, 5)

AttributeError: module 'mod1' has no attribute 'subtract'

Import mod1
print(mod1.Y)

AttributeError: module 'mod1' has no attribute 'Y'
So the nested or inner attributes can not be accessed by module name

What happens when import statement executed ?

While importing python does the following steps

  1. Find the module’s file (module search path)
  2. Compile it to byte code (if needed). (.pyc)
  3. Run the module’s code to build the objects it defines like variables, functions and classes

Python checks for the module search path to find the module.
A module is imported once during the programs execution. If you try to import same module many times python imports it only once and does the above 3 steps
After compiling the modules source code, python stores the bytecodes in a subdirectory named __pycache__ in the same directory where module is present.
The byte codes helps in loading the module faster, When we run a program with a import of a module for which byte codes (.pyc) is already available, then python loads the .pyc file directly into the program. So the compilation time is saved. This step only happens if there is no change in source file of the module .

Python from .. import statement

We can load module attributes or names directly into our program without using from and import keyword together
mod1.py


P = 10

def add(x, y):
    print("In add function - ", x + y)

class Example(object):
    pass

main.py

from mod1 import add
from mod1 import Example, P # in a single line, separating by comma
add(5, 6)
print(P)
print(Example)

In add function - 11
10
<class 'mod1.Example'>

Here the attributes P, add, Example can directly be used without using module name mod1 ,
The from...import makes these names available in the current scope

Renaming a module while importing using "as"

While importing a module we can rename the module name using keyword as
Example

import mod1 as m
m.add(5, 6)

Here we have renamed mod1 to m and we have used m name to access the module attributes (like m.add() )
So once we rename the module, we can not use the old name to access the attributes like mod1.add()

Similarly the attributes/names imported directly from the module also can be renamed like below

from mod1 import add as a
a(5, 6)

Here we have renamed add to a, so we can directly use a to invoke the same function
Renaming helps to minimize typing effort if module or attribute name are long.
It also helps to avoid conflicts if you wish to use multiple attributes with same name from different modules.
Example
If we have 2 modules mod1 and mod2 have 2 functions with same name add, we can import both these functions by renaming them to different names

from mod1 import add as add1
from mod2 import add as add2
# now use add1 and add2
add1(1, 3)
add2(“welcome”, “here”)

Module search path

Let us check this example
If we want to include a module mod5.py present in path D://sourcepath in program file main.py present in path C://dir1

import mod5

When we run main.py we get this error

ModuleNotFoundError: No module named 'mod5'
Python could not locate the module you have imported.

While importing a module (import mod1), python looks several places to find the module.
Python searches the following paths in order to find the module
1. The home directory of the program
2. PYTHONPATH directories (if set by user)
3. Standard library directories
4. The site-packages home
All these paths gets stored in sys.path list

Home directory:

Python first checks if the module is present in the current directory where the top level script is available, then it picks the module to run. Or else move to the next.

PYTHONPATH

This is an environment variable can be set by the user if user wants to tell python where to find user defined modules
This is configurable as follows
For example :
on Windows:
If you have copied all your modules in a directory called "D:/mymodules", then
C:> set PYTHONPATH=D:/mymodules
On Linux:
If modules are in "/path/to/mymodules"
$ export PYTHONPATH=/path/to/mymodules
Same as on Mac
$ export PYTHONPATH=/path/to/mymodules

Standard library directories:

If python does not find the module you have imported in the above 2 places , it searches it in the standard python library locations
Path examples on windows: C:\Python36\lib, C:\Python36\DLLs, C:\Python36\python36.zip
These paths are automatically searched

The site-packages:

This is the path where the third party extensions are getting saved when you install any external module using pip
Python automatically adds this path to search list and tries to find your module you imported
Path example on windows: C:\Python36\lib\site-packages, C:\Python36\lib\site-packages\win32

All these search paths are eventually getting stored as a string of paths in a list sys.path of sys module.
Python searches each of these directories listed in sys.path from left to right and tries to find out the imported module

Example:
Let us create a program file main.py in a directory "C\mymodules" as follows

import sys
print(sys.path) # print the module search path

Set the PYTHONPATH to D:/usermodules and run the program main.py as follows

C:>set PYTHONPATH=D:/usermodules
C:>python main.py
['C:\mymodule',
'D:\usermodules',
'C:\Python36\python36.zip',
'C:\Python36\DLLs',
'C:\Python36\lib',
'C:\Python36',
'C:\Python36\lib\site-packages',
'C:\Python36\lib\site-packages\win32',
'C:\Python36\lib\site-packages\win32\lib',
'C:\Python36\lib\site-packages\Pythonwin']

So it prints all the path than python searches.
'C:\mymodule' is current programs home path , 'D:\usermodules' is PYTHONPATH that we set before run
We can add or modify sys.path and tell python to search module in a different path

Change Module search path dynamically

Say we want to include a module mod5.py present in path D://sourcepath in program file main.py present in path C://dir1

import mod5
print(mod5)

When we run main.py we get this error

ModuleNotFoundError: No module named 'mod5'
Python could not to locate the module you have imported.
We can change the code as follows to find our module
import sys
sys.path.append('D://sourcepath')
import mod5
print(mod5)

<module 'mod5' from 'D://sourcepath\mod5.py'>

Importing a python module by name string

Module names in python are hardcoded variable names like math,string,sys,os
Sometimes in your program you need to import module which values are in string at runtime.

For example:

import 'string; # Not allowed , SyntaxError: invalid syntax
module_string = 'sys'
import module_string # Not allowed , ModuleNotFoundError
import 'os' # Not allowed , SyntaxError: invalid syntax

You can not do that directly.
You need to use special tools/methods

Using exec() built-in function

module_string = 'sys'
exec('import '+ module_string)
print(sys) # prints - <module 'sys' (built-in)>

Using __import__() built-in function

module_string = 'sys'
sys = __import__(module_string)
print(sys) # prints - <module 'sys' (built-in)>

Using importlib python module function

import importlib
module_string = 'sys'
sys = importlib.import_module(module_string)
print(sys) # prints - <module 'sys' (built-in)>
Click any Link
to navigate to certain page easily
Write a line to us
Your Email
Title
Description