virtualenv & virtualenvwrapper

A Virtual Environment enables us to keep the dependencies required by different projects in separate places, by creating virtual Python environments.

In other words, virtualenv is a tool to create isolated Python environments. The virtualenv creates a folder which contains all the necessary executables to use the packages that a Python project would need.

So, each project can have its own dependencies, regardless of what dependencies every other project has.

The site packages (3rd party libraries) installed using easy_install or pip are typically placed in one of the directories pointed to by site.getsitepackages:

>>> import site
>>> site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
>>> 

$ python3
Python 3.5.2 (default, Jul  5 2016, 12:43:10) 
>>> import site
>>> site.getsitepackages()
['/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.5/dist-packages']

Pip is a tool that fetched Python packages from the Python package Index and its mirrors. We use it to manage and install Python packages. It is similar to easy_install but pip was originally written to improve on easy_install. So, it has more features, the key feature being support for virtualenv.

To install it with pip:

$ sudo pip install virtualenv
Password:
Downloading/unpacking virtualenv
  Downloading virtualenv-13.0.1-py2.py3-none-any.whl (1.7MB): 1.7MB downloaded
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...

When we install a package from PyPI using the copy of pip that's created by the virtualenv tool, it will install the package into the site-packages directory inside the virtualenv directory. We can then use it in our program just as before.

We only need the virtualenv tool itself when we want to create a new environment. This is really simple. Start by changing directory into the root of our project directory, and then use the virtualenv command-line tool to create a new environment:

$ mkdir myproject
$ cd myproject

$ virtualenv env
New python executable in env/bin/python
Installing setuptools, pip, wheel...done.

Here, env is just the name of the directory we want to create our virtual environment inside. It's a common convention to call this directory env, and to put it inside our project directory (so, say we keep our code at ~/myproject/, the environment will be at ~/myproject/env/ - each project gets its own env). But we can put it whatever we like.

If we look inside the env directory we just created, we'll see a few subdirectories:

The one we care about the most is bin. This is where the local copy of the python binary and the pip installer exists. Let's start by using the copy of pip to install requests into the virtualenv (rather than globally):

$ env/bin/pip install requests
/home/k/myproject/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: 
Collecting requests
/home/k/myproject/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: 
  Downloading requests-2.7.0-py2.py3-none-any.whl (470kB)
    100% |------------------| 471kB 783kB/s
Installing collected packages: requests
Successfully installed requests-2.7.0

Notice that we didn't need to use sudo this time, because we're not installing requests globally, we're just installing it inside our home directory.

Now, instead of typing python to get a Python shell, we type env/bin/python:

$ env/bin/python
Python 2.7.5 (default, Mar  9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('http://bogotobogo.com')
<response [200]>
>>>
</response>

Instead of typing env/bin/python and env/bin/pip every time, we can run a script to activate the environment. This script, which can be executed with source env/bin/activate, simply adjusts a few variables in our shell (temporarily) so that when we type python, we actually get the Python binary inside the virtualenv instead of the global one:

$ which python
/usr/bin/python

$ source env/bin/activate

(env)$ which python
/home/k/myproject/env/bin/python
(env)$

So, now we can just run pip install requests (instead of env/bin/pip install requests) and pip will install the library into the environment, instead of globally. The adjustments to our shell only last for as long as the terminal is open, so we'll need to remember to rerun source env/bin/activate each time you close and open our terminal window. If we switch to work on a different project (with its own environment), we can run deactivate to stop using one environment, and then source env/bin/activate to activate the other.

virtualenvwrapper is a set of extensions to virtualenv tool. The extensions include wrappers for creating and deleting virtual environments and otherwise managing our development workflow, making it easier to work on more than one project at a time without introducing conflicts in their dependencies.

To install it:

$ sudo pip install virtualenvwrapper
...
Successfully installed stevedore-1.18.0 virtualenv-15.1.0 virtualenv-clone-0.2.6 virtualenvwrapper-4.7.2

Using virtualenv without virtualenvwrapper is a little bit painful because everytime we want to activate a virtual environment, so we have to type long command like this:

$ source ~/myproject/env/bin/activate

But with virtualenvwrapper, we only need to type:

$ workon myproject

Note that we haven't installed virtualenvwrapper.sh, this may not work yet.

As initialization steps, we will want to add the command to source /usr/local/bin/virtualenvwrapper.sh to our shell startup file, changing the path to virtualenvwrapper.sh depending on where it was installed by pip:

$ vi ~/.bashrc
export WORKON_HOME=~/Envs

Run the script:

$ source ~/.bashrc

Then, execute the following:

$ mkdir -p $WORKON_HOME

$ echo $WORKON_HOME
/home/k/Envs

Run virtualenvwrapper.sh:

$ source /usr/local/bin/virtualenvwrapper.sh
virtualenvwrapper.user_scripts creating /home/k/Envs/premkproject
virtualenvwrapper.user_scripts creating /home/k/Envs/postmkproject
virtualenvwrapper.user_scripts creating /home/k/Envs/initialize
virtualenvwrapper.user_scripts creating /home/k/Envs/premkvirtualenv
virtualenvwrapper.user_scripts creating /home/k/Envs/postmkvirtualenv
virtualenvwrapper.user_scripts creating /home/k/Envs/prermvirtualenv
virtualenvwrapper.user_scripts creating /home/k/Envs/postrmvirtualenv
virtualenvwrapper.user_scripts creating /home/k/Envs/predeactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/postdeactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/preactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/postactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/get_env_details

We may want to run the script from ~/.bashrc:

$ echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc

Let's create our first virtualenv:

$ mkvirtualenv env1
New python executable in /home/k/Envs/env1/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/k/Envs/env1/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/env1/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/env1/bin/preactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/env1/bin/postactivate
virtualenvwrapper.user_scripts creating /home/k/Envs/env1/bin/get_env_details
(env1) k@laptop:~$

We are not limited to a single virtualenv, and we can add another env:

(env1) k@laptop:~$ mkvirtualenv env2

List the envs:

(env1) k@laptop:~$ ls $WORKON_HOME
env1	env2  ...

Now, we can switch between envs with workon command:

(env2) k@laptop:~$ workon env1

(env1) k@laptop:~$ echo $VIRTUAL_ENV
/home/k/Envs/env1

(env1) k@laptop:~$

Now we can install some software into the environment:

(env1) k@laptop:~$ pip install django
Collecting django
  Using cached Django-1.10.3-py2.py3-none-any.whl
Installing collected packages: django
Successfully installed django-1.10.3
(env1) k@laptop:~$ 

We can check the new package with lssitepackages:

(env1) k@laptop:~$ lssitepackages
django                   pip                  setuptools-29.0.1.dist-info
Django-1.10.3.dist-info  pip-9.0.1.dist-info  wheel
easy_install.py          pkg_resources        wheel-0.29.0.dist-info
easy_install.pyc         setuptools
(env1) k@laptop:~$

How to leave/exit/deactivate a python virtualenv?

(env1) k@laptop:~$ deactivate
k@laptop:~$ workon env1

In this section, we have a sample of using virtualenv for Django application. The source is available from sfvue2.

We get the following from the source:

The following is from install.mb, and we can see how it works:

#Installation Instructions *Make sure you have Python2.7.3, virtualenv, pip and sqlite3 installed*
1. Download or clone this repo.
2. Go to project home folder and run these commands:
   

   $ cp sfvue/example_local.py sfvue/local_settings.py 
   $ virtualenv venv 
   $ source venv/bin/activate 
   

3. This will create a virtual environment and activate it. (note) We may want to install mariadb-devel to avoid vertualenv EnvironmentError: mysql_config not found Now use pip to install dependencies with:
   

   $ pip install -r dev-requirements.txt
   

   (note) --allow-external PIL --allow-unverified PIL PIL==1.1.7
4. Now we have to prepare a database:
   

   $ python manage.py syncdb
   

5. It will ask you to provide username, email and password. Give them and run following migrations:
   

   $ python manage.py migrate guardian 
   $ python manage.py migrate resources 
   $ python manage.py migrate profiles 
   

6. Run django server

   $ python manage.py runserver 
   

7. Go to [http://127.0.0.1:8000/admin/](http://127.0.0.1:8000/admin/)
8. Create Resource Types named Book, Ebook, Tutorial, Online Course, Other.
9. Go to home page.