Python - Install from Source - Local

This tutorial is about installing the latest versions of python from source into the users home folder as opposed to a server wide install. Normally /opt/python/python-3.6.1 would be the best choice for the installation. In this case installing locally is fine as well.

References

Dependencies

In order to build from source you will need the build essentials:

$ sudo apt-get install build-essential

You should also install the following dependencies:

$ sudo apt-get install libsqlite3-dev
$ sudo apt-get install sqlite3
$ sudo apt-get install bzip2 libbz2-dev
$ sudo apt-get install libncursesw5-dev

$ sudo apt-get install lzma-dev liblz-dev liblzma-dev
$ sudo apt-get install tk8.6-dev  # <-----<<<< This is for ubuntu 14.04
$ sudo apt-get install libreadline6 libreadline6-dev
$ sudo apt-get install libssl-dev
$ sudo apt-get install libgdbm-dev
$ sudo apt-get install libc6-dev
$ sudo apt-get install tk-dev

$ sudo apt-get install libdb-dev
$ sudo apt-get install tcl-dev
$ sudo apt-get install libreadline6-dev

Note: You may need to install other dependencies later on in order to properly build python 3.x.

Download Source

Dowload and compile Python:

$ mkdir ~/tmp/compile
$ cd ~/tmp/compile
$ wget https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz
$ tar xf ./Python-3.6.1.tar.xz
$ cd ./Python-3.6.1

Upgrading Python

With each new release of Python that you want to install, whether it be a completely new version or a minor upgrade. Simply follow the process and install to a new folder. The virtualenv will take care of the rest.

Adjusting Setup

http://stackoverflow.com/a/15013895

If you have problems building and it is related to zlib:

Edit ./Modules/Setup.dist and uncomment the line:

zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz

Install Python

$ ./configure --enable-optimizations --prefix=/home/troy/opt/python/python-3.6.1

Note: The configure line with the --prefix option set. This will install python to that particular folder. Virtualenv could be used, but in my case is a bit of overkill. I just need a separate python 3.6 installation
that doesn't interfere with the version that the system uses.

$ make

Note: After running make, check to see if it complains about any dependencies that are missing. Once the dependencies are installed, run make again to confirm. The dependencies can be installed from the repos.

$ make test
$ make install <- shouldn't need to invoke sudo as we are installing it in a local folder
$ make clean

Testing:

$ ./~/opt/python/python-3.6.1/bin/python3

Python 3.6.1 (default, May  7 2017, 11:49:10)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Creating the Virtual Environments

https://pymotw.com/3/venv/

In the past I would have a file in the bin folder of my home directory that linked to the python executable. This time we will be configuring virtual environments for our scripts to run.

Create the folder where will will store the virtual environments

~$ mkdir .envs

Create a test environment:

~$ ./ opt/python/python-3.6.1/bin/python3 -m venv ~/.envs/test

Activate the environment:

~$ source .envs/test/bin/activate

Test python out:

(test):~$ python
Python 3.6.1 (default, May  7 2017, 11:49:10)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Deactivate the virtual environment:

(test):~$ deactivate

~$

Scipy Stack

This is the general purpose scientific computing stack that I use.

Make the environment:

~$ ./opt/python/python-3.6.1/bin/python3 -m venv ~/.envs/jupyter5

Activate the environment:

~$ source .envs/jupyter5/bin/activate

Install the bits that I need:

(jupyter5):~$ pip install numpy scipy matplotlib ipython jupyter pandas sympy nose seaborn
(jupyter5):~$ pip install notebook jupyter_nbextensions_configurator yapf widgetsnbextension

After the items are installed, generate a requirements file. This should be done when anything else is added. Generate the requirements file:

jupyter5):~$ cd .envs/jupyter5

(jupyter5):~$ pip freeze > requirements.txt

NOTE: The requirements file should be recreated anytime something is installed

Upgrade pip packages

$ pip install -r requirements.txt --upgrade

Troy Williams

My name is Troy Williams and I am a professional mining engineer and programmer writing software for the mining industry.