aboutsummaryrefslogtreecommitdiff
path: root/INSTALL
blob: 0116833073b646540f86ba9818bfb050d32e7ff6 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
RhodeCode Installation and Setup
================================

RhodeCode is an open soruce git hosting solution written in Python. Linaro
uses its own RhodeCode instance to server git repositories.

Installation
============

TL;DR
-----

In the 'scripts/' directory there is a Python script to automate the setup
process.

To use that script, these are the steps:

 * Edit the 'config/production.ini' file and change the following lines:

    sqlalchemy.db1.url = postgresql://rhodecode:XXXX@localhost/rhodecode
    broker.password = XXXX
    beaker.session.sa.url = postgresql://rhodecode:XXXX@localhost/rhodecode

   Change the 'XXXX' with appropriate passwords, that need to be passed on the
   command line to run the script.

 * Modify the file 'config/rhodecode' for Apache, adjusting the default or
    empty values based on the installation environment.

 * Run the following command:

    python scripts/rhodecode-setup --rhodecode-config config/production.ini \
    --rhodecode-admin-email $EMAIL --rhodecode-admin-pwd $ADMIN_PWD \
    --assume-yes --rabbitmq-pwd $RABBIT_PWD --postgres-role-pwd $DB_PWD \
    --apache-conf config/rhodecode

   Changing $RABBIT_PWD with the password written in 'broker.password', and
   $DB_PWD with the one set up on the database configuration line. Fill in also
   the other values: $EMAIL and $ADMIN_PWD for the RhodeCode administration.

At the end, the script will print all the installation details.

Dependencies
------------

 * apache2
 * build-essential
 * git
 * mercurial (v2.4.1, not in Ubuntu repositories, need to install via pip)
 * pgadmin3
 * postgresql-9.1
 * postgresql-server-dev-9.1
 * python
 * python-amqplib
 * python-anyjson
 * python-babel
 * python-bcrypt
 * python-celery
 * python-dateutil
 * python-dev
 * python-docutils
 * python-formencode
 * python-ldap
 * python-mailer
 * python-markdown
 * python-nose
 * python-pastescript
 * python-pip
 * python-psycopg2
 * python-pylons
 * python-webhelpers
 * python-webob
 * rabbitmq-server
 * waitress (v0.8.1, not in Ubuntu repositories, need to install via pip)

The package git-core can also be installed via the Ubuntu Git Maintainers PPA
that usually has more updated versions of git. The PPA is: ppa:git-core/ppa

The pip installed packages are handled automatically by the RhodeCode install
steps, and will be installed in the home directory of the user running the
installation, not in the whole system.

Users Configuration
-------------------

The RhodeCode installation will be done with a dedicated system user and group.

 * System user: rhodecode
 * Group: rhodecode
 * Home: /home/rhodecode

Necessary Directories
---------------------

The following directories need to be created and be writable at least by the
'rhodecode' user and/or group:

 * /var/log/celery
 * /var/log/rhodecode
 * /opt/rhodecode
 * /opt/rhodecode/git_repos

The '/opt/rhodecode' directory is where RhodeCode will store its local data and
caches.

The '/opt/rhodecode/git_repos' directory is where RhodeCode will store the git
repositories directories.

If some of these directories need to be changed, it is necessary to update also
the RhodeCode configuration files.

PostgreSQL Configuration
------------------------

For PostgreSQL it is necessary to create a role called 'rhodecode' with its
own password and the permit to create databases, and a database always called
'rhodecode'.

Once this is done, modify the file 'config/production.ini' at the line:

    sqlalchemy.db1.url = postgresql://rhodecode:XXXX@localhost/rhodecode

with the correct values.

The default config file assumes PostreSQL to be installed on the same host of
RhodeCode.

Celery and RabbitMQ
-------------------

For better performance with more than 10 repositories hosted on RhodeCode, it
is recommended to use Celery with RabbitMQ.

Once RabbitMQ has been installed, it is necessary to set up the correct user
and vhost. Commands have to be run as root.

 * Add a new user called 'rhodecode' to RabbitMQ, set up its own password:

    sudo rabbitmqctl add_user rhodecode $PASSWORD

 * Create a new RabbitMQ vhost called 'rhodecode-vhost':

    sudo rabbitmqctl add_vhost rhodecode-vhost

 * Set the correct permission to the new user for the created vhost:

    sudo rabbitmqctl set_permissions -p rhodecode-vhost rhodecode ".*" ".*" ".*"

Once RabbitMQ has been configured, modify the file 'config/production.ini' with
the correct values. The lines to modify are:

    broker.vhost = rhodecode-vhost
    broker.user = rhodecode
    broker.password = XXXX

The default config file assumes RabbitMQ to be installed on the same host of
RhodeCode.

For more information about Celery and RabbitMQ configurations, see:

 http://docs.celeryproject.org/en/master/getting-started/brokers/rabbitmq.html

In the 'config/' directory there is also a simple RabbitMQ configuration file
that tries to lower RabbitMQ memory consumption, and tweaks other parameters:

    cp config/rabbitmq.config /etc/rabbitmq/
    service rabbitmq-server restart

For more information, see:

 http://www.rabbitmq.com/configure.html#customise-general-unix-environment

RhodeCode Configuration File
----------------------------

In the 'config/' directory there is a default configuration file for a production
RhodeCode instance.

Copy the file 'config/production.ini' into the home directory of the 'rhodecode'
user and sets its permissions accordingly:

    sudo cp config/production.ini /home/rhodecode/production.ini
    sudo chown rhodecode:rhodecode /home/rhodecode/production.ini

Download, Setup and Install RhodeCode
-------------------------------------

We install RhodeCode from source, from a Linaro dedicated branch, wich might
contain changes applied by us. All these operation are assumed to be performed
under the 'rhodecode' home directoy, with the user 'rhodecode'.

 * Clone the 'rhodecode' repository from git.linaro.org:

    git clone http://git.linaro.org/git-ro/infrastructure/rhodecode.git

 * Move in the 'rhodecode' directory, checkout the decicated branch and update:

    cd rhodecode && git checkout linaro && git pull origin linaro

 * Always as the 'rhodecode' user, and in the 'rhodecode' git directory:

    paster setup-rhodecode $CONF_FILE --user=$USER --email=$EMAL \
    --password=$PWD --repos=$REPOS

   The variable means:
   * $CONF_FILE: path to the 'production.ini' file, that for these instructions
     should be '/home/rhodecode/production.ini'
   * $USER: the name for the administrator user on the website
   * $EMAIL: the email of the administrator
   * $PWD: the password for the administrator
   * $REPOS: the path of the directory where the git repositories will be saved,
     that for these instructions should be '/opt/rhodecode/git_repos'

 * Install RhodeCode, using the 'rhodecode' user home directory as the base:
    python setup.py install --user

Apache Configuration
--------------------

In the 'config/' directory there is a working Apache configuration to serve
RhodeCode on port 80 and on 443 for HTTPS. Values need to be adjusted based on
the installation environment.

To use that configuration it is necessary to enable two Apache modules:

 * proxy
 * proxy_http
 * ssl (if HTTPS is going to be used)

Disable also the 'default' Apache module.

If the directory where git repositories will be stored, is different than
'/opt/rhodecode/git_repos', modify the Apache RhodeCode virtual host
accordingly.

Copy the file 'config/rhodecode' into the appropriate Apache directory and
reload the configuration.

Start the Services
------------------

In order to be able to start the services when the server boots, there are two
upstart configuration files that can be used.

They are located in the 'scripts/' directory:

 * celeryd-upstart.conf
 * rhodecode-upstart.conf

Copy these files respectively in:

    /etc/init/celeryd.conf
    /etc/init/rhodecode.conf

To start Celery and RhodeCode:

    service celeryd start
    service rhodecode start

To manually start the services, as the 'rhodecode' user:

    paster celeryd $PATH/production.ini
    paster server $PATH/production.ini

LDAP Support
============

At the moment, LDAP support is not enabled in RhodeCode nor in the automated
install script.

To enable LDAP:

 * Install the python-ldap package:

    sudo apt-get install python-ldap

 * Restart RhodeCode

 * Login as admin to RhodeCode, and from the Admin menu choose LDAP

 * LDAP configuration has to be done in there. For more info see:

    http://packages.python.org/RhodeCode/setup.html#setting-up-ldap-support

Configuration
=============

All RhodeCode configuration, like creating groups and modifying some of its
setting, happens from the web interface when logged in as the admin user.

User Permissions
----------------

Under admin → permissions it is possible to:

 * Disable anonymous access: only user with user name and password will be able
    to browse the web site.
 * Disable user registration: restrict access only to Linaro account holders.

User Groups and Repository Groups
-----------------------------------

Repository groups are mapped to directories on the file system under the directory
storing git repositories. Keeping everything as it is now on git.linaro.org
we will have "automatic" repository groups.

Be default, users will only be able to read the repositories under a repository
gorup. In order to set write permissions for a repository group:

 * Under admin → user groups, create a new user group
 * Once created, click on its name to open the preferences
 * From 'Available members' choose the members to include in the group, and save
 * Move in admin → repositories groups
 * Select the goup that holds the repositories to give special permission
 * Click 'Add another member', type the name of the user group (it should 
    autocomplete), and set the permission.

Repositories Defaults
---------------------

Under admin → defaults it is possible to set global repositories default values.

 * Private repository: if repositories, by default are private
 * Enable statistics: useful to see commit statistics

Known Problems
--------------

 * With RhodeCode v1.5.1 there is a known problem with the dashboard not showing
    more than 5 groups. Issue: http://goo.gl/WJ7f7
    A workaround is to enable the 'Use lightweight dashboard' option in
    admin → settings


RhodeCode Staging Instance
==========================

In order to have staging instance as similar as possible to the production one,
we need to clone git.linaro.org repositories, mirror them and update them from
time to time.

In order to mirror all hosted repositories, we need a list of all the '.git'
directories in git.linaro.org, and clone those.

Once we have the mirrored clones, we need to set up a cron job to update them.

In the 'scripts/' directory there are two Python scripts to automate the clone
and update operations. They are respectively:

    mirror-repos
    update-repos

In the 'config' directory there is also a file with all the '.git' directories
from git.linaro.org. The file is:

    git_repos.txt

This file has to be passed on the command line when cloning the repositories.
To clone all git.linaro.org repositories:

    python scripts/mirror-repos --repos-list config/git_repos.txt \
    --checkout-dir /opt/rhodecode/git_repos --user rhodecode

The script will mirror the repositories maintaining their directory structures.
In order for RhodeCode to pick up the newly added respositories, it is necessary
to trigger RhodeCode for a directory re-scan. To do so, the command used before
will look like:

    python scripts/mirror-repos --repos-list config/git_repos.txt \
    --checkout-dir /opt/rhodecode/git_repos --user rhodecode --rescan-repos \
    --api-key $ADMIN_API_KEY

Where $ADMIN_API_KEY is found in the RhodeCode 'admin' user web profile. The
script will mirror the repositories maintaining their directory structures.

To update the repositories, as a cron job for the 'rhodecode' user, run:

    python scripts/update-repos --repos-dir /opt/rhodecode/git_repos

If set up as root, use:

    python scripts/update-repos --repos-dir /opt/rhodecode/git_repos \
    --user rhodecode

Fix Problem When Mirroring Repositories
---------------------------------------

In order to use RhodeCode to serve git repositories via HTTP, after repositories
have been mirrored, it is necessary to run a small script that will run a git
command on each of them in order to update the server information.

The command to run is:

    python scripts/update-server-info

Default values are used as described in these installation instructions.


Upgrading the Rhodecode instance
================================

The rhodecode-setup script also supports update of the Rhodecode to the
specific tag or branch, while doing the reinstallation of the Rhodecode
automatically. Downgrade of the database is not supported by the Rhodecode
tools, so bare this in mind when updating to an older version/tag/branch.

To update the system to a specific tag/branch run the following command
from the rhodecode-config base directory:

    python scripts/rhodecode-setup --rhodecode-config config/production.ini \
    --rhodecode-usr $USR --rhodecode-branch $TAG \
    --rhodecode-checkout-dir $CO_DIR --dbname $DB_NAME \
    --assume-yes --update

   Explaining variables:
   * $USR: Rhodecode system user (defaults is: rhodecode)
   * $TAG: Tag or branch in the Rhodecode cloned repository to which
     the code base will be updated.
   * $CO_DIR: path to the directory where Rhodecode was previously cloned.
   * $DB_NAME: Database name for Rhodecode (default is: rhodecode)