Build TensorFlow from Source Code using Docker
To contribute to TensorFlow, we need to build TensorFlow from source code. The official guide is great. However, it interleaves the native building process and that using Docker and makes it confusing because packages needed by the former are not by the latter. Also, we found some useful tricks to start Docker containers in practices that are not in the official guide. Hence this document.
Build the Pip Package in TensorFlow Development Container
On either Mac or Linux, or any other OS, we don’t have to install and configure the building tools; instead, we can use a Docker image where all tools have been installed and properly configured.
Get the Docker image containing all the building tools:
docker pull tensorflow/tensorflow:latest-devel
Then, let’s get the source code. On any OS, please install git using the native package manager. For example, on Ubuntu, please
sudo apt-get install git
or, on Mac,
brew install git
Then, use the git just installed, let’s clone tensorflow source code:
git clone https://github.com/tensorflow/tensorflow cd tensorflow
By default, we will be on the
masterbranch. Feel free to do you change in your feature branches, or switch to a release branch, for example:
git checkout v1.11.0 git checkout -b v1.11.0
Then, let us start a Docker container running the
docker run --rm -it \ -w /tensorflow \ -v $PWD:/tensorflow \ -v $HOME/.cache:/root/.cache \ -e "HOST_PERMS=$(id -u):$(id -g)" \ tensorflow/tensorflow:latest-devel \ /bin/bash
-w /tensorflowbrings us to the
/tensorflowdirectory in the container once after we start it.
-v $PWD:/tensorflowmaps the current directory, which is the just cloned TensorFlow source directory on the host, to
/tensorflowin the container.
-v $HOME/.cache:/root/.cachemaps the Bazel temporary directory on the host into the container, so the intermediate files generated by Bazel running in the container are actually saved on the host. This allows us to interrupt the container during the build and restart it later to resume the building.
e "HOST_PERMS=$(id -u):$(id -g)"passes the user identification on the host into the container as an environment variable. We can reset the mode of files generated in the container to this user identity.
From now on, we will be working in the container. Let us first configure the building:
Usually, I would simply choose all the default options by hitting enter all the way down.
Build the pip package builder:
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
Build the pip package and save it into
Change the generated wheel file into the usual mode as on the host:
chown $HOST_PERMS /mnt/tensorflow-*.whl
Install the Pip Pacakge in TensorFlow Development Container
Let’s now try the new pip package. First, we need to uninstall the current tensorflow pip package, then we install the newly built one:
pip uninstall tensorflow pip install /tensorflow/tensorflow-*.whl
Now, we can verify if the new package works. First, we need to switch to a directory out from
/tensorflow, so we don’t import from the source directory:
cd /tmp # other directories also work
then, we can try import the newly installed tensorflow package and verify it:
python >>> import tensorflow >>> print(tensorflow.__version__)
Now, let us quit from Python and from the Docker container. We should see the
tensorflow-*.whlfile on the host in the current directory.
Install the Pip Package in a Clean Python Package
After we quit from the development container, we should see the wheel file in the TensorFlow source directory on the host. Now, we can start a Python container and install the Pip package in it.
Start the Python container
docker run --rm -it -v $PWD:/tensorflow python:2.7 bash
Install the pip pacakge in the container
pip install /tensorflow/tensorflow*.whl
Try TensorFlow by starting Python in the container
python >>> import tensorflow as tf >>> print(tf.__version__)