doc: measure the efficiency of cpufreq governors

DVFS adds a latency in the execution of task because of the time to
decide to move at max freq. We need to measure this latency and check
that the governor stays in an acceptable range.

When rt-app runs a json file, a log file is created for each thread.
This log file records the number of loop that has been executed and
the duration for executing these loops (per phase). We can use these
figures to evaluate to latency that is added by a cpufreq governor
and its "performance efficiency".

We use the run+sleep pattern to do the measurement, for the run time per
loop, the performance governor should run the expected duration as the
CPU stays a max freq. At the opposite, the powersave governor will give
use the longest duration (as it stays at lowest OPP). Other governor will
be somewhere between the 2 previous duration as they will use several OPP
and will go back to max frequency after a defined duration which depends
on its monitoring period.

The formula:

    duration of powersave gov - duration of the gov
--------------------------------------------------------  x 100%
 duration of powersave gov - duration of performance gov

will give the efficiency of the governor. 100% means as efficient as
the perf governor and 0% means as efficient as the powersave governor.

This patch offers json files and shell scripts to do the measurement.

Usage:
./calibration.sh <cpu>
cpu:            cpu number on which you want to run the test

./test.sh <governor> <cpu> <runtime> <sleeptime> [<loops>]
governor:       target CPUFreq governor you want to test
cpu:            cpu number on which you want to run the test. Be the same
                as the one passed to "calibration.sh".
runtime:        running time in ms per loop of the workload pattern
sleeptime:      sleeping time in ms per loop of the workload pattern
loops:          repeat times of the workload pattern. default: 10

Example:
"./calibration.sh 0" means to calculate the computing capacity of CPU0 which
will be used in the following test.

"./test.sh ondemand 0 100 100 20" means to
test "ondemand" on CPU0 with workload pattern "run 100ms + sleep 100ms"(20 loops).

NOTE:
- Make sure there are "sed", "cut", "grep", "rt-app", etc tools on
  your test machine, and run the scripts under root privilege.

- Run the test while the system is idle.

- You can change the target governor's parameters after running "calibration.sh",
  but before "test.sh".

Signed-off-by: Xunlei Pang <pang.xunlei@linaro.org>

6 files changed, 272 insertions, 0 deletions

diff --git a/doc/examples/cpufreq_governor_efficiency/README b/doc/examples/cpufreq_governor_efficiency/README
new file mode 100755
index 0000000..10482b8
--- /dev/null
+++ b/doc/examples/cpufreq_governor_efficiency/README
@@ -0,0 +1,60 @@
+Measure the efficiency of cpufreq governors using rt-app
+
+BACKGROUND:
+    DVFS adds a latency in the execution of task because of the time to
+    decide to move at max freq. We need to measure this latency and check
+    that the governor stays in an acceptable range.
+
+    When rt-app runs a json file, a log file is created for each thread.
+    This log file records the number of loop that has been executed and
+    the duration for executing these loops (per phase). We can use these
+    figures to evaluate to latency that is added by a cpufreq governor
+    and its "performance efficiency".
+
+    We use the run+sleep pattern to do the measurement, for the run time per
+    loop, the performance governor should run the expected duration as the
+    CPU stays a max freq. At the opposite, the powersave governor will give
+    use the longest duration (as it stays at lowest OPP). Other governor will
+    be somewhere between the 2 previous duration as they will use several OPP
+    and will go back to max frequency after a defined duration which depends
+    on its monitoring period.
+
+    The formula:
+
+        duration of powersave gov - duration of the gov
+    --------------------------------------------------------  x 100%
+     duration of powersave gov - duration of performance gov
+
+    will give the efficiency of the governor. 100% means as efficient as
+    the perf governor and 0% means as efficient as the powersave governor.
+
+    This test offers json files and shell scripts to do the measurement.
+
+Usage:
+    ./calibration.sh <cpu>
+    cpu:            cpu number on which you want to run the test
+
+    ./test.sh <governor> <cpu> <runtime> <sleeptime> [<loops>]
+    governor:       target CPUFreq governor you want to test
+    cpu:            cpu number on which you want to run the test. Be the same
+                    as the one passing to "calibration.sh".
+    runtime:        running time in ms per loop of the workload pattern
+    sleeptime:      sleeping time in ms per loop of the workload pattern
+    loops:          repeat times of the workload pattern. default: 10
+
+Example:
+    "./calibration.sh 0" means to calculate the computing capacity of CPU0 which
+    will be used in the following test.
+
+    "./test.sh ondemand 0 100 100 20" means to
+    test "ondemand" on CPU0 with workload pattern "run 100ms + sleep 100ms"(20 loops).
+
+NOTE:
+    - Make sure there are "sed", "cut", "grep", "rt-app", etc tools on
+      your test machine, and run the scripts under root privilege.
+
+    - Run the test while the system is idle.
+
+    - You can change the target governor's parameters after running "calibration.sh",
+      but before "test.sh".
+
diff --git a/doc/examples/cpufreq_governor_efficiency/calibration.json b/doc/examples/cpufreq_governor_efficiency/calibration.json
new file mode 100755
index 0000000..2d5870c
--- /dev/null
+++ b/doc/examples/cpufreq_governor_efficiency/calibration.json
@@ -0,0 +1,26 @@
+{
+	"tasks" : {
+		"thread" : {
+			"instance" : 1,
+			"loop" : 1,
+			"phases" : {
+				"run" : {
+					"loop" : 1,
+					"run" : 2000,
+				},
+				"sleep" : {
+					"loop" : 1,
+					"sleep" : 2000,
+				}
+			}
+		}
+	},
+	"global" : {
+		"default_policy" : "SCHED_FIFO",
+		"calibration" : "CPU0",
+		"lock_pages" : true,
+		"ftrace" : false,
+		"logdir" : "./",
+	}
+}
+
diff --git a/doc/examples/cpufreq_governor_efficiency/calibration.sh b/doc/examples/cpufreq_governor_efficiency/calibration.sh
new file mode 100755
index 0000000..a217487
--- /dev/null
+++ b/doc/examples/cpufreq_governor_efficiency/calibration.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+set -e
+
+if [ ! $1 ] ; then
+	echo "Please input one cpu"
+	exit
+fi
+
+echo performance > /sys/devices/system/cpu/cpu$1/cpufreq/scaling_governor
+
+sleep 1
+
+sed 's/"calibration" : "CPU.*",/"calibration" : "CPU'$1'",/' -i calibration.json
+pLoad=$(rt-app calibration.json 2>&1 |grep pLoad |sed 's/.*= \(.*\)ns.*/\1/')
+sed 's/"calibration" : .*,/"calibration" : '$pLoad',/' -i dvfs.json
+echo CPU$1\'s pLoad is $pLoad
diff --git a/doc/examples/cpufreq_governor_efficiency/dvfs.json b/doc/examples/cpufreq_governor_efficiency/dvfs.json
new file mode 100755
index 0000000..c8447df
--- /dev/null
+++ b/doc/examples/cpufreq_governor_efficiency/dvfs.json
@@ -0,0 +1,27 @@
+{
+	"tasks" : {
+		"thread" : {
+			"instance" : 1,
+			"cpus" : [0],
+"loop" : 21,
+			"phases" : {
+				"running" : {
+					"loop" : 1,
+					"run" : 100,
+				},
+				"sleeping" : {
+					"loop" : 1,
+					"sleep" : 100,
+				}
+			}
+		}
+	},
+	"global" : {
+		"default_policy" : "SCHED_OTHER",
+		"calibration" : 90,
+		"lock_pages" : true,
+		"ftrace" : false,
+		"logdir" : "./",
+	}
+}
+
diff --git a/doc/examples/cpufreq_governor_efficiency/dvfs.sh b/doc/examples/cpufreq_governor_efficiency/dvfs.sh
new file mode 100755
index 0000000..00ce81d
--- /dev/null
+++ b/doc/examples/cpufreq_governor_efficiency/dvfs.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+# $1 $2 $3 $4 $5: governor cpu run sleep loops
+set -e
+
+echo $1 > /sys/devices/system/cpu/cpu$2/cpufreq/scaling_governor
+#echo $1 > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
+sed 's/"cpus" : \[.*\],/"cpus" : \['$2'\],/' -i dvfs.json
+sleep 3
+
+if [ $3 ] ; then
+	sed 's/"run" : .*,/"run" : '$3',/' -i dvfs.json
+fi
+
+if [ $4 ] ; then
+	sed 's/"sleep" : .*,/"sleep" : '$4',/' -i dvfs.json
+fi
+
+if [ $5 ] ; then
+	sed 's/^"loop" : .*,/"loop" : '$5',/' -i dvfs.json
+fi
+
+rt-app dvfs.json 2> /dev/null
+
+if [ $1 ] ; then
+	mv -f rt-app-thread-0.log rt-app_$1_run$3us_sleep$4us.log
+
+	sum=0
+	loop=0
+	for i in $(cat rt-app_$1_run$3us_sleep$4us.log | sed 'n;d' | sed '1d' |cut -f 3); do
+		loop=$(expr $loop + 1)
+		sum=$(expr $sum + $i)
+	done
+	sum=$(expr $sum / $loop)
+	echo $sum
+	rm -f rt-app_$1_run$3us_sleep$4us.log
+fi
+
diff --git a/doc/examples/cpufreq_governor_efficiency/test.sh b/doc/examples/cpufreq_governor_efficiency/test.sh
new file mode 100755
index 0000000..0160952
--- /dev/null
+++ b/doc/examples/cpufreq_governor_efficiency/test.sh
@@ -0,0 +1,104 @@
+#!/bin/sh
+
+set -e
+
+test_efficiency() {
+
+	FILENAME="results_$RANDOM$$.txt"
+
+	if [ -e /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors ]; then
+		for i in $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors); do
+			if [ $i = $1 ] ; then
+				gov_target=$i
+			fi
+			export gov_$i=$(echo $i)
+		done
+	else
+		echo "cpufreq sysfs is not available!"
+		exit
+	fi
+
+	if [ ! $gov_target ] ; then
+		echo " Can't find $1 governor!"
+		exit
+	fi
+
+	if [ ! $gov_performance ] ; then
+		echo "Can't find performance governor!"
+		exit
+	fi
+
+	if [ ! $gov_powersave ] ; then
+		echo "Can't find powersave governor!"
+		exit
+	fi
+
+	if [ $gov_target = $gov_performance ] || [ $gov_target = $gov_powersave ] ; then
+		echo "Please input a governor other than \"performance\" or \"powersave\""
+		exit
+	fi
+
+	# Get target gov data first
+	dvfs.sh $1 $2 $3 $4 $5 > $FILENAME
+	target=$(cat $FILENAME |sed -n '1p')
+
+	# Get powersave data
+	dvfs.sh powersave $2 $3 $4 $5 > $FILENAME
+	powersave=$(cat $FILENAME |sed -n '1p')
+
+	# Get performance data
+	dvfs.sh performance $2 $3 $4 $5 > $FILENAME
+	performance=$(cat $FILENAME |sed -n '1p')
+
+	if [ $performance -ge $powersave ] ; then
+		echo "powersave: $powersave"
+		echo "performance: $performance"
+		echo "Error! performance spent more time than powersave!"
+		exit
+	fi
+
+	echo "\"powersave\" efficiency: 0%"
+	echo "\"performance\" efficiency: 100%"
+
+	denominator=$(expr $powersave - $performance)
+
+	if [ $powersave -le $target ]; then
+		target=0
+	else
+		numerator=$(expr $powersave - $target)
+		numerator=$(expr $numerator \* 100)
+		target=$(expr $numerator / $denominator)
+		if [ $target -gt 100 ]; then
+			target=100
+		fi
+	fi
+
+	echo "\"$gov_target\" efficiency: $target%"
+
+	rm -f $FILENAME
+}
+
+if [ $# -lt 4 ]; then
+	echo "Usage: ./test.sh <governor> <cpu> <runtime> <sleeptime> [<loops>]"
+	echo "governor:	target CPUFreq governor you want to test"
+	echo "cpu:		cpu number on which you want to run the test"
+	echo "runtime:	running time in ms per loop of the workload pattern"
+	echo "sleeptime:	sleeping time in ms per loop of the workload pattern"
+	echo "loops:		repeat times of the workload pattern. default: 10"
+	echo "\nExample:\n\"./test.sh ondemand 0 100 100 20\" means\nTest \"ondemand\" on CPU0 with workload pattern \"run 100ms + sleep 100ms\"(20 loops).\n"
+	exit
+fi
+
+if [ $# = 4 ]; then
+	loops=10
+else
+	loops=$5
+fi
+
+echo "Test \"$1\" on CPU$2 with workload pattern \"run $3ms + sleep $4ms\"($loops loops)."
+
+sleep 1
+PATH=$PATH:.
+
+test_efficiency $1 $2 $(expr $3 \* 1000) $(expr $4 \* 1000) $loops
+