RhodeCode Installation and Setup ================================ RhodeCode is an open soruce git hosting solution written in Python. Linaro uses its own RhodeCode instance to server git repositories. Installation ============ TL;DR ----- In the 'scripts/' directory there is a Python script to automate the setup process. To use that script, these are the steps: * Edit the 'config/production-0.ini' file and change the following lines: sqlalchemy.db1.url = postgresql://rhodecode:XXXX@localhost/rhodecode broker.password = XXXX beaker.session.sa.url = postgresql://rhodecode:XXXX@localhost/rhodecode Change the 'XXXX' with appropriate passwords, that need to be passed on the command line to run the script. * Modify the file 'config/rhodecode' for Apache, adjusting the default or empty values based on the installation environment. * Run the following command: python scripts/rhodecode-setup --rhodecode-config config/production-0.ini \ --rhodecode-admin-email $EMAIL --rhodecode-admin-pwd $ADMIN_PWD \ --assume-yes --rabbitmq-pwd $RABBIT_PWD --postgres-role-pwd $DB_PWD \ --apache-conf config/rhodecode Changing $RABBIT_PWD with the password written in 'broker.password', and $DB_PWD with the one set up on the database configuration line. Fill in also the other values: $EMAIL and $ADMIN_PWD for the RhodeCode administration. At the end, the script will print all the installation details. Dependencies ------------ * apache2 * build-essential * git * libapache2-mod-xsendfile * members * mercurial * pgadmin3 * postgresql-9.1 * postgresql-server-dev-9.1 * python * python-amqplib * python-anyjson * python-babel * python-bcrypt * python-celery * python-dateutil * python-dev * python-docutils * python-formencode * python-ldap * python-mailer * python-markdown * python-nose * python-objgraph * python-pastescript * python-pip * python-psycopg2 * python-pylons * python-webhelpers * python-webob * rabbitmq-server The package git-core can also be installed via the Ubuntu Git Maintainers PPA that usually has more updated versions of git. The PPA is: ppa:git-core/ppa The pip installed packages are handled automatically by the RhodeCode install steps, and will be installed in the home directory of the user running the installation, not in the whole system. Users Configuration ------------------- The RhodeCode installation will be done with a dedicated system user and group. * System user: rhodecode * Group: rhodecode * Home: /home/rhodecode Necessary Directories --------------------- The following directories need to be created and be writable at least by the 'rhodecode' user and/or group: * /var/log/celery * /var/log/rhodecode * /opt/rhodecode * /opt/rhodecode/git_repos The '/opt/rhodecode' directory is where RhodeCode will store its local data and caches. The '/opt/rhodecode/git_repos' directory is where RhodeCode will store the git repositories directories. If some of these directories need to be changed, it is necessary to update also the RhodeCode configuration files. PostgreSQL Configuration ------------------------ For PostgreSQL it is necessary to create a role called 'rhodecode' with its own password and the permit to create databases, and a database always called 'rhodecode'. Once this is done, modify the file 'config/production-0.ini' at the line: sqlalchemy.db1.url = postgresql://rhodecode:XXXX@localhost/rhodecode with the correct values. The default config file assumes PostreSQL to be installed on the same host of RhodeCode. Celery and RabbitMQ ------------------- For better performance with more than 10 repositories hosted on RhodeCode, it is recommended to use Celery with RabbitMQ. Once RabbitMQ has been installed, it is necessary to set up the correct user and vhost. Commands have to be run as root. * Add a new user called 'rhodecode' to RabbitMQ, set up its own password: sudo rabbitmqctl add_user rhodecode $PASSWORD * Create a new RabbitMQ vhost called 'rhodecode-vhost': sudo rabbitmqctl add_vhost rhodecode-vhost * Set the correct permission to the new user for the created vhost: sudo rabbitmqctl set_permissions -p rhodecode-vhost rhodecode ".*" ".*" ".*" Once RabbitMQ has been configured, modify the file 'config/production-0.ini' with the correct values. The lines to modify are: broker.vhost = rhodecode-vhost broker.user = rhodecode broker.password = XXXX The default config file assumes RabbitMQ to be installed on the same host of RhodeCode. For more information about Celery and RabbitMQ configurations, see: http://docs.celeryproject.org/en/master/getting-started/brokers/rabbitmq.html In the 'config/' directory there is also a simple RabbitMQ configuration file that tries to lower RabbitMQ memory consumption, and tweaks other parameters: cp config/rabbitmq.config /etc/rabbitmq/ service rabbitmq-server restart For more information, see: http://www.rabbitmq.com/configure.html#customise-general-unix-environment RhodeCode Configuration File ---------------------------- In the 'config/' directory there is a default configuration file for a production RhodeCode instance. Copy the file 'config/production-0.ini' into the home directory of the 'rhodecode' user and sets its permissions accordingly: sudo cp config/production-0.ini /home/rhodecode/production-0.ini sudo chown rhodecode:rhodecode /home/rhodecode/production-0.ini Download, Setup and Install RhodeCode ------------------------------------- We install RhodeCode from source, from a Linaro dedicated branch, wich might contain changes applied by us. All these operation are assumed to be performed under the 'rhodecode' home directoy, with the user 'rhodecode'. * Clone the 'rhodecode' repository from git.linaro.org: git clone http://git.linaro.org/git-ro/infrastructure/rhodecode.git * Move in the 'rhodecode' directory, checkout the decicated branch and update: cd rhodecode && git checkout linaro && git pull origin linaro * Always as the 'rhodecode' user, and in the 'rhodecode' git directory: paster setup-rhodecode $CONF_FILE --user=$USER --email=$EMAL \ --password=$PWD --repos=$REPOS The variable means: * $CONF_FILE: path to the 'production-0.ini' file, that for these instructions should be '/home/rhodecode/production-0.ini' * $USER: the name for the administrator user on the website * $EMAIL: the email of the administrator * $PWD: the password for the administrator * $REPOS: the path of the directory where the git repositories will be saved, that for these instructions should be '/opt/rhodecode/git_repos' * Install RhodeCode, using the 'rhodecode' user home directory as the base: python setup.py install --user Apache Configuration -------------------- In the 'config/' directory there is a working Apache configuration to serve RhodeCode on port 80 and on 443 for HTTPS. Values need to be adjusted based on the installation environment. To use that configuration it is necessary to enable two Apache modules: * proxy * proxy_http * ssl (if HTTPS is going to be used) Disable also the 'default' Apache module. If the directory where git repositories will be stored, is different than '/opt/rhodecode/git_repos', modify the Apache RhodeCode virtual host accordingly. Copy the file 'config/rhodecode' into the appropriate Apache directory and reload the configuration. Start the Services ------------------ In order to be able to start the services when the server boots, there are two upstart configuration files that can be used. They are located in the 'scripts/' directory: * celeryd.conf * rhodecode.conf * rhodecode-stop.conf * rhodecode-start.conf Copy these files in `/etc/init/'. To start Celery and RhodeCode: service rhodecode-start start To stop the services: service rhodecode-stop start The RhodeCode upstart configurations are made so that it is possible to automatically start and stop multiple RhodeCode instances if there are multiple `production-*.ini' file in the `/home/rhodecode' directory. LDAP Support ============ At the moment, LDAP support is not enabled in RhodeCode nor in the automated install script. To enable LDAP: * Install the python-ldap package: sudo apt-get install python-ldap * Restart RhodeCode * Login as admin to RhodeCode, and from the Admin menu choose LDAP * LDAP configuration has to be done in there. For more info see: http://packages.python.org/RhodeCode/setup.html#setting-up-ldap-support * To support ACL with SSH keys for public repositories you will need to enable rhodecode group to have system groupadd and groupmod privileges. In order to enable this type: sudo visudo And under the "# User alias specification" add the following line: %rhodecode ALL=(ALL) NOPASSWD: /usr/sbin/groupadd, /usr/sbin/groupmod, /usr/bin/gpasswd, /usr/bin/change-repo-ownership, /usr/bin/getent Configuration ============= All RhodeCode configuration, like creating groups and modifying some of its setting, happens from the web interface when logged in as the admin user. User Permissions ---------------- Under admin → permissions it is possible to: * Disable user registration: restrict access only to Linaro account holders. User Groups and Repository Groups ----------------------------------- Repository groups are mapped to directories on the file system under the directory storing git repositories. Keeping everything as it is now on git.linaro.org we will have "automatic" repository groups. Be default, users will only be able to read the repositories under a repository gorup. In order to set write permissions for a repository group: * Under admin → user groups, create a new user group * Once created, click on its name to open the preferences * From 'Available members' choose the members to include in the group, and save * Move in admin → repositories groups * Select the goup that holds the repositories to give special permission * Click 'Add another member', type the name of the user group (it should autocomplete), and set the permission. Repositories Defaults --------------------- Under admin → defaults it is possible to set global repositories default values. * Private repository: if repositories, by default should be private * Enable statistics: useful to see commit statistics * Enable downloads: to enable ZIP files download RhodeCode Staging Instance ========================== In order to have staging instance as similar as possible to the production one, we need to clone git.linaro.org repositories, mirror them and update them from time to time. In order to mirror all hosted repositories, we need a list of all the '.git' directories in git.linaro.org, and clone those. Once we have the mirrored clones, we need to set up a cron job to update them. In the 'scripts/' directory there are two Python scripts to automate the clone and update operations. They are respectively: mirror-repos update-repos In the 'config' directory there is also a file with all the '.git' directories from git.linaro.org. The file is: git_repos.txt This file has to be passed on the command line when cloning the repositories. To clone all git.linaro.org repositories: python scripts/mirror-repos --repos-list config/git_repos.txt \ --checkout-dir /opt/rhodecode/git_repos --user rhodecode The script will mirror the repositories maintaining their directory structures. In order for RhodeCode to pick up the newly added respositories, it is necessary to trigger RhodeCode for a directory re-scan. To do so, the command used before will look like: python scripts/mirror-repos --repos-list config/git_repos.txt \ --checkout-dir /opt/rhodecode/git_repos --user rhodecode --rescan-repos \ --api-key $ADMIN_API_KEY Where $ADMIN_API_KEY is found in the RhodeCode 'admin' user web profile. The script will mirror the repositories maintaining their directory structures. To update the repositories, as a cron job for the 'rhodecode' user, run: python scripts/update-repos --repos-dir /opt/rhodecode/git_repos If set up as root, use: python scripts/update-repos --repos-dir /opt/rhodecode/git_repos \ --user rhodecode Sync via Rsync -------------- In the `scripts/' directory there is also a file called `rsync-mirror'. With that script it is possible to sync the repositories via rsync. It needs a special user setup on the remote server in order to open the connection. Fix Problem When Mirroring Repositories --------------------------------------- In order to use RhodeCode to serve git repositories via HTTP, after repositories have been mirrored, it is necessary to run a small script that will run a git command on each of them in order to update the server information. The command to run is: python scripts/update-server-info Default values are used as described in these installation instructions. Upgrading the Rhodecode instance ================================ The rhodecode-setup script also supports update of the Rhodecode to the specific tag or branch, while doing the reinstallation of the Rhodecode automatically. IMPORTANT: Database downgrade is not supported by the Rhodecode tools, so bare this in mind when updating to an older version/tag/branch. To update the system to a specific tag/branch run the following command from the rhodecode-config base directory: python scripts/rhodecode-setup --rhodecode-config config/production-0.ini \ --rhodecode-usr $USR --rhodecode-branch $TAG \ --rhodecode-checkout-dir $CO_DIR --dbname $DB_NAME \ --assume-yes --update Explaining variables: * $USR: Rhodecode system user (defaults is: rhodecode) * $TAG: Tag or branch in the Rhodecode cloned repository to which the code base will be updated. * $CO_DIR: path to the directory where Rhodecode was previously cloned. * $DB_NAME: Database name for Rhodecode (default is: rhodecode) After upgrade, the Apache config should be copied manually to the /etc/apache2/sites-available/ directory.