path: root/INSTALL

RhodeCode Installation and Setup
================================

RhodeCode is an open soruce git hosting solution written in Python. Linaro
uses its own RhodeCode instance to server git repositories.

Installation
============

TL;DR
-----

In the 'scripts/' directory there is a Python script to automate the setup
process.

To use that script, these are the steps:

 * Edit the 'config/production.ini' file and change the following lines:

    sqlalchemy.db1.url = postgresql://rhodecode:XXXX@localhost/rhodecode
    broker.password = XXXX
    beaker.session.sa.url = postgresql://rhodecode:XXXX@localhost/rhodecode

   Change the 'XXXX' with appropriate passwords, that need to be passed on the
   command line to run the script.

 * Modify the file 'config/rhodecode' for Apache, adjusting the default or
    empty values based on the installation environment.

 * Run the following command:

    python scripts/rhodecode-setup --rhodecode-config config/production.ini \
    --rhodecode-admin-email $EMAIL --rhodecode-admin-pwd $ADMIN_PWD \
    --assume-yes --rabbitmq-pwd $RABBIT_PWD --postgres-role-pwd $DB_PWD \
    --apache-conf config/rhodecode

   Changing $RABBIT_PWD with the password written in 'broker.password', and
   $DB_PWD with the one set up on the database configuration line. Fill in also
   the other values: $EMAIL and $ADMIN_PWD for the RhodeCode administration.

At the end, the script will print all the installation details.

Dependencies
------------

 * apache2
 * build-essential
 * git
 * mercurial (v2.4.1, not in Ubuntu repositories, need to install via pip)
 * pgadmin3
 * postgresql-9.1
 * postgresql-server-dev-9.1
 * python
 * python-amqplib
 * python-anyjson
 * python-babel
 * python-bcrypt
 * python-celery
 * python-dateutil
 * python-dev
 * python-docutils
 * python-formencode
 * python-ldap
 * python-mailer
 * python-markdown
 * python-nose
 * python-pastescript
 * python-pip
 * python-psycopg2
 * python-pylons
 * python-webhelpers
 * python-webob
 * rabbitmq-server
 * waitress (v0.8.1, not in Ubuntu repositories, need to install via pip)

The package git-core can also be installed via the Ubuntu Git Maintainers PPA
that usually has more updated versions of git. The PPA is: ppa:git-core/ppa

The pip installed packages are handled automatically by the RhodeCode install
steps, and will be installed in the home directory of the user running the
installation, not in the whole system.

Users Configuration
-------------------

The RhodeCode installation will be done with a dedicated system user and group.

 * System user: rhodecode
 * Group: rhodecode
 * Home: /home/rhodecode

Necessary Directories
---------------------

The following directories need to be created and be writable at least by the
'rhodecode' user and/or group:

 * /var/log/celery
 * /var/log/rhodecode
 * /opt/rhodecode
 * /opt/rhodecode/git_repos

The '/opt/rhodecode' directory is where RhodeCode will store its local data and
caches.

The '/opt/rhodecode/git_repos' directory is where RhodeCode will store the git
repositories directories.

If some of these directories need to be changed, it is necessary to update also
the RhodeCode configuration files.

PostgreSQL Configuration
------------------------

For PostgreSQL it is necessary to create a role called 'rhodecode' with its
own password and the permit to create databases, and a database always called
'rhodecode'.

Once this is done, modify the file 'config/production.ini' at the line:

    sqlalchemy.db1.url = postgresql://rhodecode:XXXX@localhost/rhodecode

with the correct values.

The default config file assumes PostreSQL to be installed on the same host of
RhodeCode.

Celery and RabbitMQ
-------------------

For better performance with more than 10 repositories hosted on RhodeCode, it
is recommended to use Celery with RabbitMQ.

Once RabbitMQ has been installed, it is necessary to set up the correct user
and vhost. Commands have to be run as root.

 * Add a new user called 'rhodecode' to RabbitMQ, set up its own password:

    sudo rabbitmqctl add_user rhodecode $PASSWORD

 * Create a new RabbitMQ vhost called 'rhodecode-vhost':

    sudo rabbitmqctl add_vhost rhodecode-vhost

 * Set the correct permission to the new user for the created vhost:

    sudo rabbitmqctl set_permissions -p rhodecode-vhost rhodecode ".*" ".*" ".*"

Once RabbitMQ has been configured, modify the file 'config/production.ini' with
the correct values. The lines to modify are:

    broker.vhost = rhodecode-vhost
    broker.user = rhodecode
    broker.password = XXXX

The default config file assumes RabbitMQ to be installed on the same host of
RhodeCode.

For more information about Celery and RabbitMQ configurations, see:

 http://docs.celeryproject.org/en/master/getting-started/brokers/rabbitmq.html

RhodeCode Configuration File
----------------------------

In the 'config/' directory there is a default configuration file for a production
RhodeCode instance.

Copy the file 'config/production.ini' into the home directory of the 'rhodecode'
user and sets its permissions accordingly:

    sudo cp config/production.ini /home/rhodecode/production.ini
    sudo chown rhodecode:rhodecode /home/rhodecode/production.ini

Download, Setup and Install RhodeCode
-------------------------------------

We install RhodeCode from source, from a Linaro dedicated branch, wich might
contain changes applied by us. All these operation are assumed to be performed
under the 'rhodecode' home directoy, with the user 'rhodecode'.

 * Clone the 'rhodecode' repository from git.linaro.org:

    git clone http://git.linaro.org/git-ro/infrastructure/rhodecode.git

 * Move in the 'rhodecode' directory, checkout the decicated branch and update:

    cd rhodecode && git checkout linaro && git pull origin linaro

 * Always as the 'rhodecode' user, and in the 'rhodecode' git directory:

    paster setup-rhodecode $CONF_FILE --user=$USER --email=$EMAL \
    --password=$PWD --repos=$REPOS

   The variable means:
   * $CONF_FILE: path to the 'production.ini' file, that for these instructions
     should be '/home/rhodecode/production.ini'
   * $USER: the name for the administrator user on the website
   * $EMAIL: the email of the administrator
   * $PWD: the password for the administrator
   * $REPOS: the path of the directory where the git repositories will be saved,
     that for these instructions should be '/opt/rhodecode/git_repos'

 * Install RhodeCode, using the 'rhodecode' user home directory as the base:
    python setup.py install --user

Apache Configuration
--------------------

In the 'config/' directory there is a working Apache configuration to serve
RhodeCode on port 80 and on 443 for HTTPS. Values need to be adjusted based on
the installation environment.

To use that configuration it is necessary to enable two Apache modules:

 * proxy
 * proxy_http
 * ssl (if HTTPS is going to be used)

Disable also the 'default' Apache module.

If the directory where git repositories will be stored, is different than
'/opt/rhodecode/git_repos', modify the Apache RhodeCode virtual host
accordingly.

Copy the file 'config/rhodecode' into the appropriate Apache directory and
reload the configuration.

Start the Services
------------------

In order to be able to start the services when the server boots, there are two
upstart configuration files that can be used.

They are located in the 'scripts/' directory:

 * celeryd-upstart.conf
 * rhodecode-upstart.conf

Copy these files respectively in:

    /etc/init/celeryd.conf
    /etc/init/rhodecode.conf

To start Celery and RhodeCode:

    service celeryd start
    service rhodecode start

To manually start the services, as the 'rhodecode' user:

    paster celeryd $PATH/production.ini
    paster server $PATH/production.ini

LDAP Support
============

At the moment, LDAP support is not enabled in RhodeCode nor in the automated
install script.

To enable LDAP:

 * Install the python-ldap package:

    sudo apt-get install python-ldap

 * Restart RhodeCode

 * Login as admin to RhodeCode, and from the Admin menu choose LDAP

 * LDAP configuration has to be done in there. For more info see:

    http://packages.python.org/RhodeCode/setup.html#setting-up-ldap-support

Configuration
=============

All RhodeCode configuration, like creating groups and modifying some of its
setting, happens from the web interface when logged in as the admin user.

User Permissions
----------------

Under admin → permissions it is possible to:

 * Disable anonymous access: only user with user name and password will be able
    to browse the web site.
 * Disable user registration: restrict access only to Linaro account holders.

User Groups and Repository Groups
-----------------------------------

Repository groups are mapped to directories on the file system under the directory
storing git repositories. Keeping everything as it is now on git.linaro.org
we will have "automatic" repository groups.

Be default, users will only be able to read the repositories under a repository
gorup. In order to set write permissions for a repository group:

 * Under admin → user groups, create a new user group
 * Once created, click on its name to open the preferences
 * From 'Available members' choose the members to include in the group, and save
 * Move in admin → repositories groups
 * Select the goup that holds the repositories to give special permission
 * Click 'Add another member', type the name of the user group (it should 
    autocomplete), and set the permission.

Repositories Defaults
---------------------

Under admin → defaults it is possible to set global repositories default values.

 * Private repository: if repositories, by default are private
 * Enable statistics: useful to see commit statistics

Known Problems
--------------

 * With RhodeCode v1.5.1 there is a known problem with the dashboard not showing
    more than 5 groups. Issue: http://goo.gl/WJ7f7
    A workaround is to enable the 'Use lightweight dashboard' option in
    admin → settings


RhodeCode Staging Instance
==========================

In order to have staging instance as similar as possible to the production one,
we need to clone git.linaro.org repositories, mirror them and update them from
time to time.

In order to mirror all hosted repositories, we need a list of all the '.git'
directories in git.linaro.org, and clone those.

Once we have the mirrored clones, we need to set up a cron job to update them.

In the 'scripts/' directory there are two Python scripts to automate the clone
and update operations. They are respectively:

    mirror-repos
    update-repos

In the 'config' directory there is also a file with all the '.git' directories
from git.linaro.org. The file is:

    git_repos.txt

This file has to be passed on the command line when cloning the repositories.
To clone all git.linaro.org repositories:

    python scripts/mirror-repos --repos-list config/git_repos.txt \
    --checkout-dir /opt/rhodecode/git_repos --user rhodecode

The script will mirror the repositories maintaining their directory structures.
In order for RhodeCode to pick up the newly added respositories, it is necessary
to trigger RhodeCode for a directory re-scan. To do so, the command used before
will look like:

    python scripts/mirror-repos --repos-list config/git_repos.txt \
    --checkout-dir /opt/rhodecode/git_repos --user rhodecode --rescan-repos \
    --api-key $ADMIN_API_KEY

Where $ADMIN_API_KEY is found in the RhodeCode 'admin' user web profile. The
script will mirror the repositories maintaining their directory structures.

To update the repositories, as a cron job for the 'rhodecode' user, run:

    python scripts/update-repos --repos-dir /opt/rhodecode/git_repos

If set up as root, use:

    python scripts/update-repos --repos-dir /opt/rhodecode/git_repos \
    --user rhodecode

Fix Problem When Mirroring Repositories
---------------------------------------

In order to use RhodeCode to serve git repositories via HTTP, after repositories
have been mirrored, it is necessary to run a small script that will run a git
command on each of them in order to update the server information.

The command to run is:

    python scripts/update-server-info

Default values are used as described in these installation instructions.


Upgrading the Rhodecode instance
================================

The rhodecode-setup script also supports update of the Rhodecode to the
specific tag or branch, while doing the reinstallation of the Rhodecode
automatically. Downgrade of the database is not supported by the Rhodecode
tools, so bare this in mind when updating to an older version/tag/branch.

To update the system to a specific tag/branch run the following command
from the rhodecode-config base directory:

    python scripts/rhodecode-setup --rhodecode-config config/production.ini \
    --rhodecode-usr $USR --rhodecode-branch $TAG \
    --rhodecode-checkout-dir $CO_DIR --dbname $DB_NAME \
    --assume-yes --update

   Explaining variables:
   * $USR: Rhodecode system user (defaults is: rhodecode)
   * $TAG: Tag or branch in the Rhodecode cloned repository to which
     the code base will be updated.
   * $CO_DIR: path to the directory where Rhodecode was previously cloned.
   * $DB_NAME: Database name for Rhodecode (default is: rhodecode)