Wiki source code of Conda installation

Last modified by Jan Rhebergen on 2022/01/24 15:56
Show last authors
1 = Creating a basic functional Jupyter data-science/machine learning environment =
2
3
4 Anaconda or conda offers a well known and standardised environment for data-science and other activities. Anaconda is the graphically and visually appealing shell whereas ##miniconda## is a much smaller and lighter command line tool that achieves the same purpose.
5
6 Download the basic version from [[https:~~/~~/docs.conda.io/en/latest/miniconda.html>>https://docs.conda.io/en/latest/miniconda.html]] and make it executable##(chmod +x)##. We place the downloaded version in ##/home/4all/Downloads## This way it is available for others to install as well. When installing accept all the default options except the one where it is asked: //"Do you wish the installer to initialize Miniconda3"// which we do want so we answer //"y"//
7
8 After installation you can add another channel to expand the available packages and versions see: [[https:~~/~~/conda-forge.org/>>https://conda-forge.org/]]
9
10 (% class="box" %)
11 {{{conda config --add channels conda-forge
12 conda config --set channel_priority strict
13 conda update -n base -c defaults conda
14 }}}
15
16 Attached to this page one can find the handy [[conda-cheat-sheet.pdf>>attach:conda-cheat-sheet.pdf]] which contains some examples of regularly used commands. For each user we will install a simple **##data-science##** environment with default tools that are considered essential. The default **##base##** environment we will leave as it is.
17
18 * Jupyter lab
19 * spyder (optional console based python IDE)
20 * pandas
21 * numpy (will be install due to panadas dependency)
22
23 Some are dependent on one another an may be automatically installed when selecting a package.
24
25 (% class="box" %)
26 {{{conda create -n data-science
27 conda activate data-science
28 conda install jupyterlab
29 conda install pandas
30 conda install jupyterlab-git
31 }}}
32
33 After executing the above we have a basic environment suitable for data science.
34
35 //"The littelest Jupyter Hub"// a.k.a. ##TLJH## we usually execute the following. When installing this in our own environment we can (% class="mark" %)ommit(%%) the ##sudo -E## part.
36
37 (% class="box" %)
38 {{{sudo -E conda install pandas
39 sudo -E conda install numpy
40 sudo -E conda install scikit-learn
41 sudo -E conda install nltk
42 sudo -E conda install nodejs -c conda-forge --repodata-fn=repodata.json
43 #JBRv if applicable this one too
44 sudo -E jupyter labextension uninstall @jupyterlab/plotly-extension
45 #JBRv this one can take while
46 sudo -E jupyter labextension install jupyterlab-plotly plotlywidget @jupyter-widgets/jupyterlab-manager
47 #JBRv next one causes a lot of conflicts and need to be looked at
48 sudo -E conda install scikit-learn-intelex
49 sudo -E jupyter lab build
50
51 sudo -E conda install geopandas folium plotly matplotlib geopy
52 sudo -E conda install r-base r-essentials r-irkernel
53 }}}
54
55 For remote access later, you need to do a basic configuration and set a password.
56
57 (% class="box" %)
58 {{{Jupyter server --generate-config
59 Jupyter server password}}}
60
61 In case we also want to be able to use the GPU we need to install either of the following as well (please also refer to (% class="mark" %)[[this page>>doc:UOG.Technical Documentation.Tensorflow.WebHome]](%%) for more info).
62
63 (% class="box" %)
64 {{{conda install keras-gpu}}}
65
66 or this:
67
68 (% class="box" %)
69 {{{conda install tensorflow-gpu
70 }}}
71
72 = Subsequent steps =
73
74 Depending on the complete system setup, the above described set up may not work as you expect. This particularly is the case when you are using a docker environment for ##tensorflow##. (% class="mark" %)System76(%%) has created a great tool called ##tensorman## which eases the manipulation of ##tensorflow## docker containers. The reason for this is that lining up the correct versions of all the tools and libraries can be tricky and prone to fail especially after upgrades/updates.
75
76 When using docker libraries employing ##tensorman## the tools and ##conda## environment don't know about each other. The tools (and libraries) are in the container while the the ##conda## modules are in its own environment on the host. One solution to this problem is to not use ##conda## at all but use ##pip## inside the ##tensorflow## docker container. Basically you treat the container the same way as you would treat python environments in ##pip## or ##conda##. This is perfectly legitimate but can be cumbersome and hard for novices. Hence we choose the ##conda## approach partly because we also prefer this approach when we're not using ##tensorflow## docker containers.
77
78 We're assuming you have a ##tensorflow## container running with ##jupyter-lab## installed as described [[here>>doc:UOG.Technical Documentation.Creating a custom Jupyter GPU enabled tensorflow image.WebHome]]. As you are running this in subdirectory ##/project## on your ##$HOME## it is not aware of your (possibly) other environment(s) (be it from ##conda## or ##pip## origines). From a bash command prompt inside your ##tensorflow## container you can install ##conda## in the ##/project/miniconda3## subdirectory. It will create the usual ##(base)## enviroment which we will use per default (NB: it also creates a ##.bashrc## file!). If you started Jupyter it will not know about this enviroment because it uses the ##ipython## kernel from the ##tensorflow## docker container. This will result in ##"unknown module"## error messages if not remedied.
79
80 To remedy this situation execute the following steps:
81
82
83 (% class="box" %)
84 {{{conda install ipykernel
85 python -m ipykernel install --user --name conda-base --display-name "Conda (base)"
86 jupyter kernelspec list
87 }}}
88
89 You can now (re)start Jupyter from your ##tensorflow## command shell (replace port number with yours from [[here>>doc:UOG.Technical Documentation.Nginxproxymanager.WebHome]]):
90
91 (% class="box" %)
92 {{{jupyter lab --ip=0.0.0.0 --port=8889 --no-browser
93 }}}
94
95 //(NB: Jupyter needs to be restarted once for this to take effect and have the newly defined kernel available)//
96
97 You can now select the kernel named "Conda (base)". This will make your conda enviroment (ie. packages/modules) available for use.