Wiki source code of Git & Gitea

Last modified by Jan Rhebergen on 2022/01/24 15:56
Show last authors
1 = Basic setup and use of git and gitea =
2
3 We use git together with jupyter-lab and gitea as separate webinterface for source code version control of jupyter notebooks. Here we describe the basic setup in use.
4
5 == git basic setup ==
6
7 Start by specifying common (fixed) variables like your name and email address. Normally this is a one-time setup.
8
9 (% class="box" %)
10 {{{git config --global user.name "Jan Rhebergen"
11 git config --global user.email "jan@rhebergen.net"
12 }}}
13
14 Unix. MacOS and DOS (Windows) handle line endings differently according to the table below:
15
16 (% border="2" cellpadding="1" cellspacing="1" style="margin-right:auto; width:503px" summary="line endings per operating system" %)
17 |=(% style="width: 173px;" %)Operating system|=(% style="width: 323px;" %)Character used for line ending
18 |(% style="width:173px" %)Unix|(% style="width:323px" %)LF (line feed) = '\n'
19 |(% style="width:173px" %)MacOS|(% style="width:323px" %)CR (carriage return) = '\r'
20 |(% style="width:173px" %)DOS/Windows|(% style="width:323px" %)CR LF (carriage return & line feed) = `\r\n`
21
22 To ensure correct handling in a mixed environment we set the following:
23
24 (% class="box" %)
25 {{{git config --global core.autocrlf input
26 git config --global core.safecrlf warn
27 }}}
28
29 And just in case you're using Windows:
30
31 (% class="box" %)
32 {{{git config --global core.autocrlf true
33 git config --global core.safecrlf warn
34 }}}
35
36 == git initialisation and setup ==
37
38 This step basically consists of creating a directory and initialising it as a ##git## repository. In case there already is a repository available elsewhere one can copy (clone) it here.
39
40 (% class="box" %)
41 {{{mkdir ~/Data-Science/OGR-II/
42 cd Data-Science/OGR-II
43 git init
44 }}}
45
46 The ##git## repository consists of the ##'working directory'## (holding the files) the ##'index'## or staging area and the ##'head'## pointing to the most recent commit made. These three trees are maintained for you by ##git##. In our example we will copy the files (jupyter notebooks) from our nextcloud instance using ##rclone##. We assume ##rclone## [[has already been configured>>doc:UOG.Technical Documentation.Nextcloud access.WebHome]] and has a known ##remote## called ##beethoven##.
47
48 Normally one would ##clone## the files from a git repository. In this case however it does not exist yet. The jupyter notebooks are stored in the [[##nextcloud## instance>>https://nextcloud.bladzij.de/apps/files/?dir=/UOG/OGR-II/Notebooks&fileid=1119]]. using the joint UOG group directory.
49
50 (% class="box" %)
51 {{{cd ~/Data-Science
52 rclone copy beethoven:UOG/OGR-II/Notebooks/ ./OGR-II/
53 }}}
54
55 Once the files are located in the working directory they can be added to to the git repository by first adding them to the index (staging) and subsequently committing them to the head of the repository as illustrated below.
56
57 (% class="box" %)
58 {{{git add *
59 git commit -m "my initial/first commit, yeah!"
60 }}}
61
62 Perhaps this is a good occasion to install a git tool like ##lazygit##. Strictly speaking this is not needed but it can come in handy when interacting with git repositories. ##lazygit## is a powerful text based tool which runs straight from the commandline. It can be install using ##conda##. In case conda cannot be found (for whatever reason) also execute the first line below.
63
64 **optional!**
65
66 (% class="box" %)
67 {{{source $HOME/miniconda3/etc/profile.d/conda.sh
68 conda activate data-science
69 conda install lazygit
70 }}}
71
72
73 == interact with remote git repository (gitea instance) ==
74
75 Next we want to create a remote repository on ##beethoven## where we have a ##gitea## instance running. It is assumed that ##beethoven## appears in the ##/etc/hosts## file (see below). Otherwise the local/private IP should be used.
76
77 (% class="box" %)
78 {{{127.0.0.1 localhost
79 ::1 localhost
80 127.0.1.1 pop-os.localdomain pop-os
81 192.168.166.130 liszt
82 192.168.166.222 beethoven
83 192.168.166.103 wagner
84 }}}
85
86 This specifies the git/gitea repository on ##beethoven## as our remote repository:
87
88 (% class="box" %)
89 {{{git remote add origin http://beethoven:3000/UOG/OGR-II.git
90 }}}
91
92 **NB:** the UOG organisation and data-science team have been created beforehand using the [[##gitea## webinterface.>>https://gitea.bladzij.de/UOG]]
93
94 For a private repository this would be:
95
96 (% class="box" %)
97 {{{git remote add origin http://beethoven:3000/jan_rhebergen/my-first-private-repo.git
98 }}}
99
100 If you happen to make a mistake, just delete the remote entry like so:
101
102 (% class="box" %)
103 {{{git remote remove origin
104 }}}
105
106 For the above to work you ##gitea## needs to be have the variables below configured in ##/var/lib/docker/volumes/gitea_gitea/_data/gitea/conf/app.ini## (don't forget to restart ##gitea##). If this is not the case the push (and create) will fail. The alternative is to use the web interface to create an empty repository and then push the local one to the gitea server. We also want to **//disable//** the self registration possibility for security reasons.
107
108 (% class="box" %)
109 {{{[repository]
110 ENABLE_PUSH_CREATE_USER = true
111 ENABLE_PUSH_CREATE_ORG = true
112
113 [service]
114 DISABLE_REGISTRATION = true
115 }}}
116
117 Finally the ##OGR-II## repository need to be pushed to ##gitea##
118
119 (% class="box" %)
120 {{{git push -u origin master
121 }}}
122
123 == using git with jupyter-lab ==
124
125 The while idea of using git is to make sharing and maintaining code (e.g. jupyter notebooks) easy and well organised. Jupyter-lab as an excellent git plugin which takes care of a lot of the usual hassles. The sections above are assume a (bash) login shell on ##liszt##. This section assumes a running jupyter-lab instance with the ##jupyterlab-git## extention installed by using: ##conda install jupyterlab-git##. This will make available a handy  and powerful ##git## interface.
126
127 ##git## has several authentication mechanisms. Currently it is up to the user to make sure that ##jupter-lab## can authenticate to the ##git## server. There currently is no dialogue in the ##jupter-lab## interface that facilitates this. Hence to make authentication as painless as possible we need to modify the local ##git## config file ##$HOME/Data-Science/OGR-II/.git/config## according to the snippet below bye adding the ##[credential]## section:
128
129 (% class="box" %)
130 (((
131 **FROM:**
132
133 (% class="box" %)
134 {{{[core]
135 repositoryformatversion = 0
136 filemode = true
137 bare = false
138 logallrefupdates = true
139 [remote "origin"]
140 url = http:~/~/beethoven:3000/UOG/OGR-II.git
141 fetch = +refs/heads/*:refs/remotes/origin/*
142 [branch "master"]
143 remote = origin
144 merge = refs/heads/master
145 }}}
146
147 **TO:**
148
149 (% class="box" %)
150 {{{[core]
151 repositoryformatversion = 0
152 filemode = true
153 bare = false
154 logallrefupdates = true
155 [remote "origin"]
156 url = http:~/~/beethoven:3000/UOG/OGR-II.git
157 fetch = +refs/heads/*:refs/remotes/origin/*
158 [credential]
159 helper = store
160 [branch "master"]
161 remote = origin
162 merge = refs/heads/master
163 }}}
164
165 **USING:**
166
167 {{{git config credential.helper store}}}
168 )))
169
170 This will cause credentials that are supplied to be stored in the local git setup. To get this working one only needs to authenticate once from the bash command line (e.g. ##git pull##). Then subsequent requests for authetications that originate from ##jupyter-lab## will use these stored credentials automatically.
171
172 An alternate way to grant/get access is by means of tokens that can be generated using the gitea webinterface. We'll employ this method below.
173
174 == using git with TLJH ==
175
176 We'll be using ##nbgitpuller## to get the jupyter notebooks from the git repository. First [[##nbgitpuller##>>https://github.com/jupyterhub/nbgitpuller#constructing-the-nbgitpuller-url]] needs to be installed. This can be acompllished by logingin as user ##ganymede## (the admin user) and execute from a terminal (bash shell):
177
178 (% class="box" %)
179 {{{sudo -E conda install nbgitpuller
180 }}}
181
182 ##nbgitpuller## is a jupyter-hub (and also TLJH) plugin which shields the user from the complexities of ##git## and makes it easy to obtain code from the git repository. It works by supplying the user with a ##URL## which contains all the information needed. This ##URL## can be crafted by hand or by using a generator [[like this one>>https://jupyterhub.github.io/nbgitpuller/link]]. (see also [[this documentation>>https://jupyterhub.github.io/nbgitpuller/]]).
183
184 (% class="box" %)
185 [[[[image:nbgitpuller-linkgenerator.png||height="377" style="float:left" width="800"]]>>https://jupyterhub.github.io/nbgitpuller/link]]
186
187 Using nbgitpuller has this [[desireable behaviour>>https://jupyterhub.github.io/nbgitpuller/topic/automatic-merging.html#case-1-the-instructor-changed-a-file-that-the-student-has-not-changed]] and properly handles the following //merge// cases:
188
189 (% style="margin-right:auto" summary="merg cases" %)
190 |Case 1: |The instructor changed a file that the student has not changed
191 |Case 2: |Student & instructor changed different lines in same file
192 |Case 3: |Student & instructor change same lines in same file
193 |Case 4: |Student deletes file locally, but instructor doesn’t
194 |Case 5: |Student creates file manually, but instructor adds file with same name
195
196 === gitea preparations ===
197
198 For this to work we need to make some preparations. These consist of:
199
200 * [[create a dedicated TLJH user on gitea (i.e. (% class="wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink" %)##tljh_user##) and limit this users privileges.>>attach:gitea-create-tljh-user.png]]
201 * [[add this user as a collaborator to the UOG organisation / OGR-II repository>>attach:gitea-add-collaborator.png]].
202 * [[login as user (% class="wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink wikiattachmentlink" %)##tljh_user## and generate a token (with a desired tag).>>attach:gitea-manage-accesss-tokens.png]]
203
204 All these actions take place using the gitea webinterface (mostly as admin). **NB:** the token generated allows one to login using the API and without the need for a password. This token is generated ad displayed only once and is **//not//** stored by the system itself (on purpose). This means you should save it yourself. When executing the above set the rights/permissions appropriately (minimised).

Need help?

If you need help with XWiki you can contact: