ModuleNotFoundError when trying to unpickle a model in Flask app

It seems a bit strange that you get that error when executing predict_edu.py, as it is in the same directory as predictors.py, and thus, using absolute import such as from predictors import predictor_transformer (without the dot . operator) should normally work as expected. However, below are a few options that you could try out, if the error persists.

Option 1

You could add the parent directory of the predictors file to the system PATH variable, before attempting to import the module, as described here. This should work fine for smaller projects.

import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from predictors import predictor_transformer

Option 2

Use relative imports, e.g., from .predictors import..., and make sure you run the script from the parent directory of your package, as shown below. The -m option “searches the sys.path for the named module and execute its contents as the __main__ module”, and not as the top-level script. Read more about the -m option in the following references: [1], [2], [3], [4], [5], [6]. Read more about “relative imports” here: [1], [2], [3], [4].

python -m video_discovery.data_science.predict_edu

However, the PEP 8 style guide recommends using absolute imports in general.

Absolute imports are recommended, as they are usually more readable
and tend to be better behaved (or at least give better error messages)
if the import system is incorrectly configured (such as when a
directory inside a package ends up on sys.path)

In certain cases, however, absolute imports can get quite verbose, depending on the complexity of the directory structure, as shown below. On the other hand, “relative imports can be messy, particularly for shared projects where directory structure is likely to change”. They are also “not as readable as absolute ones, and it is hard to tell the location of the imported resources”. Read more about Python Import and Absolute vs Relative Imports.

from package1.subpackage2.subpackage3.subpackage4.module5 import function6

Option 3

Include the directory containing your package directory in PYTHONPATH and use absolute imports instead. PYTHONPATH is used to set the path for user-defined modules, so that they can be directly imported into a Python script. The PYTHONPATH variable is a string with a list of directories that need to be added to the sys.path directory list by Python. The primary use of this variable is to allow users to import modules that have not yet made into an installable Python package. Read more about it here and here.

For instance, let’s say you have a package named video_discovery (under /Users/my_user/code/video_discovery) and wanted to add the directory /Users/my_user/code to the PYTHONPATH:

On Mac

1. Open Terminal.app
2. Open the file ~/.bash_profile in your text editor – e.g. atom ~/.bash_profile
3. Add the following line to the end: export PYTHONPATH="/Users/my_user/code"
4. Save the file.
5. Close Terminal.app
6. Start Terminal.app again, to read in the new settings, and type
   echo $PYTHONPATH. It should show something like /Users/my_user/code.

On Linux

1. Open your favorite terminal program

2. Open the file ~/.bashrc in your text editor – e.g. atom ~/.bashrc

3. Add the following line to the end: export PYTHONPATH=/home/my_user/code

4. Save the file.

5. Close your terminal application.

6. Start your terminal application again, to read in the new settings,
   and type echo $PYTHONPATH. It should show something like /home/my_user/code.

On Windows

1. Open This PC (or Computer), right-click inside and select
   Properties.
2. From the computer properties dialog, select Advanced system settings on the left.
3. From the advanced system settings dialog, choose the Environment variables button.
4. In the Environment variables dialog, click the New button in the
   top half of the dialog, to make a new user variable:
5. Give the variable name as PYTHONPATH and in value add the path to
   your module directory. Choose OK and OK again to save this variable.
6. Now open a cmd window and type echo %PYTHONPATH% to confirm the environment variable is correctly set. Remember to open a new cmd window to run your Python program, so that it picks up the new settings in PYTHONPATH.

Option 4

Another solution would be to install the package in an editable state (all edits made to the .py files will be automatically included in the installed package), as described here and here. However, the amount of work required to get this to work might make Option 3 a better choice for you.

The contents for the setup.py should be as shown below, and the command for installing the package should be pip install -e . (-e flag stands for “editable” and . stands for “current directory”).

from setuptools import setup, find_packages
setup(name="myproject", version='1.0', packages=find_packages())