1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
|
Linaro downloads license protection
===================================
Linaro builds sometimes contain "binary blobs"—pieces of binary-only code which enable extra features like accelerated graphics or multimedia. These pieces are distributed under a separate license, and downloading images or collections containing them requires some sort of license protection.
This code provides such license protection on the hosting web server: other parts of infrastructure need to properly integrate with it (see eg. android-build.linaro.org section). At the moment, it's implemented as a click-through license acceptance on the Apache side.
To ensure and prove this keeps working, we are also using automated tests with this code.
Setup
-----
This tree is the base for snapshots.linaro.org and releases.linaro.org www roots as served by Apache. It makes heavy use of mod_rewrite from within the .htaccess file.
Currently, all directories/files containing either 'origen' or 'snowball' in the URL path are protected with appropriate license (Samsung or ST-E) click-through.
Dependencies
............
libapache2-mod-php5
Testing: phpunit, testrepository, python-html2text
Technical details
-----------------
releases.linaro.org
...................
Root directory for releases.linaro.org includes .htaccess and licenses/*.
It has mod_rewrite enabled.
It allows a few hard-coded hosts to bypass the click-through protection, by their IPs:
* android-build.linaro.org (50.17.250.69)
* validation.linaro.org (213.123.120.124)
Currently hosted on mombin.canonical.com.
snapshots.linaro.org
....................
Same basic set-up as releases.linaro.org.
Further, to allow android-build.linaro.org to push to snapshots.linaro.org
www area directly, we set-up two different users on the system with SSH keys:
* android-build-linaro:
chrooted to /srv/snapshots.linaro.org/www/android/ and allows sftp
access to push files over; home directory
* android-build-linaro-trigger:
limited to executing only a single command through
/etc/ssh/user-authorized-keys/android-build-linaro-trigger
command="/home/android-build-linaro-trigger/scripts/trigger-android-build-linaro.sh ${SSH_ORIGINAL_COMMAND#* }"
(this passes the arguments received from the remote end as well)
This script does a sudo to 'android-build-linaro' and then runs
/home/android-build-linaro/scripts/jenkins-post-sftp.sh
script which moves files from /android/.tmp/ into appropriate
build directory. It expects "build_name/build_number" as command
line parameters.
android-build.linaro.org
........................
Runs Jenkins and uses SFTP plugin to access the above two users. Private keys live in
/home/ubuntu/snapshots-sync/snapshots-sync — android-build-linaro
/home/ubuntu/snapshots-sync/snapshots-filemove — android-build-linaro-trigger
To ensure serialization of steps, and allow LAVA submission, these happen as
build steps, and not as publishing steps.
Tests
-----
Testing infrastructure is based on 'testrepository' and requires at least
Python and Apache2:
$ testr init
$ testr run
To run the production test plans (NOTE: They will take a bit more time to run)
execute the following:
$ testr init
$ testr run testplans.test_suite
Tests for PHP license-matching logic
....................................
There's currently only one unit test file, LicenseHelperTest.php under testing directory. You first need to install the phpunit package from ubuntu repos:
$ sudo apt-get install php-unit
Recent (as of 2012-05-08) Ubuntu/Debian releases have a broken phpunit package and thus require installation in a different manner:
$ sudo apt-get install php-pear
$ sudo pear config-set auto_discover 1
$ sudo pear install pear.phpunit.de/PHPUnit
PHPUnit tests execution is already included in the python integration tests, but if you wish to run unit tests separately, do the following:
$ phpunit testing/LicenseHelperTest
|
