Recently, I have been heavily working with Python and Jupyer Notebook. When I tried to run Tensorflow 1.0 code in Tensorflow 2.0 environment, so many errors occured.
That’s hard to solve at the begining, after I changed the
tf.compact.v1.abc(), it works a little bit normal. But I just want run the old code in its original way, this way also does not change the code to the pattern and paradigm of Tensorflow 2.0. It just let your code can run(no crash) in Tensorflow 2.0 environment.
The best way and far way is the understand 1.0 and 2.0, and totally refactor 1.0 code to 2.0.
Then I created two Python environments, one has tf 1.0 and one has 2.0. I thought if I change the env to tf 1.0, the Notebook will use tf 1.0. But, it’s not worked.
Then I began to run between
pip install tensorflow==1.13.1 and
pip install tensorflow==2.2.0. It’s hard.
I recalled in a platform, we are allowed to change the kernel of a Notebook. I guess that should be the right way. Indeed, that’s it, it works.
tf1 has Tensorflow 1.0 and env
tf2 has Tensorflow 2.0. But, there is another problem, if you want to use pandas, pandas not changed much, you need also install the version of pandas in both tf1 and tf2, the pandas’ code will duplicate in your system.
So, you want use the system’s packages at the same time, you need to add
--system-site-packages parameter when you are creating your env.
virtualenv venv --system-site-packages
Now, to switch different versions of package, the only thing you need to do is to change kernel in Notebook, the following will show you how to achieve this setup.
Create Virtual Environment with Virtualenv/venv
You can also model venv to create the virtual env.
Install virtualenv package.
pip install virtualenv
Create virtual env of tf1
virtualenv tf1 --system-site-packages
Install Tensorflow 1 in tf1
source ~/tf1/bin/activate pip install tensorflow==1.13.1 deactivate
Create virtual env of tf2
virtualenv tf2 --system-site-packages
Install Tensorflow 2 in tf2
source ~/tf2/bin/activate pip install tensorflow==2.2.0 deactivate
Add Virtual Environment to Jupyter Notebook
source ~/tf1/bin/activate pip install ipykernel python3 -m ipykernel install --name=tf1 deactivate
source ~/tf2/bin/activate pip install ipykernel python3 -m ipykernel install --name=tf2
Then you should see the tf1 and tf2 kernel in Notebook.
Activate a virtual env
Deactivate a env
Specify a Python version when creating env.
virtualenv -p /usr/python/python3.7 py37_env
Save all packages in an env into a txt file
pip freeze --local > requirements.txt
List all package you have installed
Upgrade a package
pip install tensorflow --upgrade
Create a env using venv module
python3 -m venv project_env # see the env which python # see packages installed pip list
delete the env, delete the directory
rm -rf project_env
create a venv in project dir
python3 -m venv my_project/venv pip install -r requirements.txt
create a venv and using the system packages
python3 -m venv venv --system-site-packages # see local packages(only in env, system packages is not included) pip list --local
virtualenv -p /usr/local/bin/python3.7m tf-nightly --system-site-packages source tf-nightly/bin/activate python3 -m ipykernel install --name=tf-nightly
Why I use
tf-nightly has been installed in python3.7m environment, I don’t know install it under another python version.
Remove Virtual Environment from Jupyter Notebook
list kernels in Jupyter notebook
➜ ~ jupyter kernelspec list Available kernels: python3 /Users/yq/Library/Python/3.7/lib/python/site-packages/ipykernel/resources tf1 /Users/yq/Library/Jupyter/kernels/tf1 tf2 /Users/yq/Library/Jupyter/kernels/tf2 tf-nightly /usr/local/share/jupyter/kernels/tf-nightly
delete a kernel
jupyter kernelspec uninstall myenv
Checking a package belongs to which version of Python
A package is tied to a version of Python. Suppose you installed a package
Tenserflow=2.3.0-dev20200615 on python3.7, if you enter a wrong version of Python, say Python2.7, you will not be able use it.
check how may Pythons are in you system.
➜ ~ which python /usr/bin/python ➜ ~ which python3 /usr/local/bin/python3 ➜ ~ ls /usr/local/bin/python* /usr/local/bin/python3 /usr/local/bin/python3.7-config /usr/local/bin/python3.8 /usr/local/bin/python3-config /usr/local/bin/python3.7m /usr/local/bin/python3.8-config /usr/local/bin/python3.7 /usr/local/bin/python3.7m-config ➜ ~ ls /usr/bin/python* /usr/bin/python /usr/bin/python2 /usr/bin/python2.7-config /usr/bin/pythonw /usr/bin/python-config /usr/bin/python2.7 /usr/bin/python3 /usr/bin/pythonw2.7
Where the pythons are pointing to? We can see that python, python2 and python2.7 are all the same python.
➜ ~ ls -l /usr/bin/python* lrwxr-xr-x 1 root wheel 75 Feb 17 21:44 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 lrwxr-xr-x 1 root wheel 82 Feb 17 21:44 /usr/bin/python-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config lrwxr-xr-x 1 root wheel 75 Feb 17 21:44 /usr/bin/python2 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 lrwxr-xr-x 1 root wheel 75 Feb 17 21:44 /usr/bin/python2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 lrwxr-xr-x 1 root wheel 82 Feb 17 21:44 /usr/bin/python2.7-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config -rwxr-xr-x 1 root wheel 31488 Apr 7 04:38 /usr/bin/python3 lrwxr-xr-x 1 root wheel 76 Feb 17 21:44 /usr/bin/pythonw -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7 lrwxr-xr-x 1 root wheel 76 Feb 17 21:44 /usr/bin/pythonw2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7
Python2.7 is the system version, it is in
➜ Versions cd /System/Library/Frameworks/Python.framework/Versions/ ➜ Versions ls 2.3 2.5 2.6 2.7 Current
Where the python3s are pointing to?
➜ ~ ls -l /usr/local/bin/python* lrwxr-xr-x 1 root wheel 69 Jun 22 11:41 /usr/local/bin/python3 -> ../../../Library/Frameworks/Python.framework/Versions/3.8/bin/python3 lrwxr-xr-x 1 root wheel 76 Jun 22 11:41 /usr/local/bin/python3-config -> ../../../Library/Frameworks/Python.framework/Versions/3.8/bin/python3-config lrwxr-xr-x 1 yq wheel 38 May 26 10:37 /usr/local/bin/python3.7 -> ../Cellar/python/3.7.6_1/bin/python3.7 lrwxr-xr-x 1 yq wheel 45 May 26 10:37 /usr/local/bin/python3.7-config -> ../Cellar/python/3.7.6_1/bin/python3.7-config lrwxr-xr-x 1 yq wheel 39 May 26 10:37 /usr/local/bin/python3.7m -> ../Cellar/python/3.7.6_1/bin/python3.7m lrwxr-xr-x 1 yq wheel 46 May 26 10:37 /usr/local/bin/python3.7m-config -> ../Cellar/python/3.7.6_1/bin/python3.7m-config lrwxr-xr-x 1 root wheel 71 Jun 22 11:41 /usr/local/bin/python3.8 -> ../../../Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 lrwxr-xr-x 1 root wheel 78 Jun 22 11:41 /usr/local/bin/python3.8-config -> ../../../Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8-config
Python3.8 is installed via pkg file from python.org.
➜ Versions cd /Library/Frameworks/Python.framework/Versions ➜ Versions ls 3.8
To explore 3.8 directory.
➜ 3.8 ll total 6848 lrwxr-xr-x 1 root admin 17B Jun 22 11:41 Headers -> include/python3.8 -rwxrwxr-x 1 root admin 3.3M May 14 04:33 Python drwxrwxr-x 5 root admin 160B May 14 04:30 Resources drwxrwxr-x 3 root admin 96B May 14 04:33 _CodeSignature drwxrwxr-x 28 root admin 896B Jun 26 11:51 bin drwxrwxr-x 3 root admin 96B May 14 04:30 etc drwxrwxr-x 3 root admin 96B May 14 04:30 include drwxrwxr-x 36 root admin 1.1K Jun 22 11:41 lib drwxrwxr-x 4 root admin 128B May 14 04:30 share ➜ 3.8 cd bin ➜ bin ls 2to3 google-oauthlib-tool pydoc3.8 python3 2to3-3.8 idle3 pyrsa-decrypt python3-config chardetect idle3.8 pyrsa-encrypt python3.8 easy_install-3.8 markdown_py pyrsa-keygen python3.8-config f2py pip3 pyrsa-priv2pub wheel f2py3 pip3.8 pyrsa-sign f2py3.8 pydoc3 pyrsa-verify ➜ bin ./python3 Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> ^D ➜ bin ./python3-config Usage: /usr/local/bin/python3-config [--prefix|--exec-prefix|--includes|--libs|--cflags|--ldflags|--extension-suffix|--help|--abiflags|--configdir|--embed]
If we want to use
Tenserflow=2.3.0-dev20200615, we must enter
➜ ~ python3.7 Python 3.7.6 (default, Dec 30 2019, 19:38:26) [Clang 11.0.0 (clang-122.214.171.124)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow as tf >>> tf.__version__ '2.3.0-dev20200615'
If you type
python3, you may enter
Tenserflow=2.3.0-dev20200615 can not be used.
➜ ~ python3 Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow as tf >>> tf.__version__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'tensorflow' has no attribute '__version__' >>>
If you type
python, you will enter
➜ ~ python WARNING: Python 2.7 is not recommended. This version is included in macOS for compatibility with legacy software. Future versions of macOS will not include Python 2.7. Instead, it is recommended that you transition to using 'python3' from within Terminal. Python 2.7.16 (default, Feb 29 2020, 01:55:37) [GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1126.96.36.199) (-macos10.15-objc- on darwin Type "help", "copyright", "credits" or "license" for more information.