author | Gustavo J. A. M. Carneiro <gjc@inescporto.pt> |
Sat, 24 Jan 2009 18:09:29 +0000 | |
changeset 4119 | 9ffcf2400aa5 |
parent 4114 | a6c5ccfa3451 |
child 4129 | 9b1f5d58dfb6 |
permissions | -rw-r--r-- |
4064
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
1 |
# python lib modules |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
2 |
import os |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
3 |
import sys |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
4 |
import shutil |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
5 |
import pproc as subprocess |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
6 |
import urllib |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
7 |
|
4064
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
8 |
# WAF modules |
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
9 |
import Build |
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
10 |
import Options |
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
11 |
import Utils |
4114
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
12 |
import Task |
4064
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
13 |
|
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
14 |
# local modules |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
15 |
import wutils |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
16 |
|
4064
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
17 |
|
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
18 |
def dev_null(): |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
19 |
if sys.platform == 'win32': |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
20 |
return open("NUL:", "w") |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
21 |
else: |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
22 |
return open("/dev/null", "w") |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
23 |
|
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
24 |
|
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
25 |
def _find_tests(testdir): |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
26 |
"""Return a list of test modules in the test directory |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
27 |
|
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
28 |
Arguments: |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
29 |
testdir -- the directory to look in for tests |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
30 |
""" |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
31 |
names = os.listdir(testdir) |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
32 |
tests = [] |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
33 |
for name in names: |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
34 |
if name[:5] == "test-" and name[-3:] == ".py": |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
35 |
testname = name[:-3] |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
36 |
tests.append(testname) |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
37 |
tests.sort() |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
38 |
return tests |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
39 |
|
4114
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
40 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
41 |
class regression_test_task(Task.TaskBase): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
42 |
after = 'cc cxx cc_link cxx_link' |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
43 |
color = 'BLUE' |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
44 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
45 |
def __init__(self, env, test_name, test_scripts_dir, build_traces_dir, reference_traces): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
46 |
super(regression_test_task, self).__init__() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
47 |
self.env = env |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
48 |
self.test_name = test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
49 |
self.test_scripts_dir = test_scripts_dir |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
50 |
self.build_traces_dir = build_traces_dir |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
51 |
self.reference_traces_dir = reference_traces |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
52 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
53 |
def __str__(self): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
54 |
return 'regression-test (%s)\n' % self.test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
55 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
56 |
def run(self): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
57 |
"""Run a single test""" |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
58 |
sys.path.insert(0, self.test_scripts_dir) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
59 |
try: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
60 |
mod = __import__(self.test_name, globals(), locals(), []) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
61 |
finally: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
62 |
sys.path.remove(self.test_scripts_dir) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
63 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
64 |
assert self.test_name.startswith('test-') |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
65 |
short_name = self.test_name[len('test-'):] |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
66 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
67 |
trace_dir_name = getattr(mod, "trace_dir_name", None) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
68 |
if trace_dir_name is None: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
69 |
trace_dir_name = "%s.ref" % short_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
70 |
trace_output_path = os.path.join(self.build_traces_dir, trace_dir_name) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
71 |
reference_traces_path = os.path.join(self.reference_traces_dir, trace_dir_name) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
72 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
73 |
if hasattr(mod, 'get_arguments'): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
74 |
arguments = mod.get_arguments(self.env, '..') |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
75 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
76 |
arguments = getattr(mod, "arguments", []) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
77 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
78 |
pyscript = getattr(mod, "pyscript", None) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
79 |
if pyscript: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
80 |
is_pyscript = True |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
81 |
program = pyscript |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
82 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
83 |
is_pyscript = False |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
84 |
program = getattr(mod, "program", short_name) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
85 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
86 |
if hasattr(mod, 'may_run'): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
87 |
reason_cannot_run = mod.may_run(self.env) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
88 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
89 |
reason_cannot_run = None |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
90 |
if reason_cannot_run: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
91 |
print "SKIP %s (%s)" % (self.test_name, reason_cannot_run) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
92 |
self.result = None |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
93 |
return 0 |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
94 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
95 |
if Options.options.regression_generate: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
96 |
# clean the target dir |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
97 |
shutil.rmtree(reference_traces_path, ignore_errors=True) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
98 |
os.makedirs(reference_traces_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
99 |
result = self.run_reference_generate(reference_traces_path, program, arguments, is_pyscript) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
100 |
if result == 0: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
101 |
print "GENERATE " + self.test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
102 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
103 |
print "GENERATE FAIL " + self.test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
104 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
105 |
# clean the target dir |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
106 |
shutil.rmtree(trace_output_path, ignore_errors=True) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
107 |
os.makedirs(trace_output_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
108 |
# run it |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
109 |
result = self.run_reference_test(reference_traces_path, trace_output_path, program, arguments, is_pyscript) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
110 |
if result == 0: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
111 |
print "PASS " + self.test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
112 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
113 |
print "FAIL " + self.test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
114 |
self.result = result |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
115 |
return 0 |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
116 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
117 |
def run_reference_test(self, reference_traces_path, trace_output_path, program, arguments, is_pyscript): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
118 |
if not os.path.exists(reference_traces_path): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
119 |
print "Cannot locate reference traces in " + reference_traces_path |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
120 |
return 1 |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
121 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
122 |
if is_pyscript: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
123 |
script = os.path.abspath(os.path.join('..', *os.path.split(program))) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
124 |
argv = [self.env['PYTHON'], script] + arguments |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
125 |
wutils.run_argv(argv, cwd=trace_output_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
126 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
127 |
wutils.run_program(program, |
4119
9ffcf2400aa5
Bug #323: waf --valgrind doesn't check for valgrind first
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4114
diff
changeset
|
128 |
command_template=wutils.get_command_template(self.env, arguments), |
4114
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
129 |
cwd=trace_output_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
130 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
131 |
if Options.options.verbose: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
132 |
#diffCmd = "diff traces " + refTestDirName + " | head" |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
133 |
diffCmd = subprocess.Popen([self.env['DIFF'], trace_output_path, reference_traces_path], |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
134 |
stdout=subprocess.PIPE) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
135 |
headCmd = subprocess.Popen("head", stdin=diffCmd.stdout) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
136 |
rc2 = headCmd.wait() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
137 |
diffCmd.stdout.close() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
138 |
rc1 = diffCmd.wait() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
139 |
rc = rc1 or rc2 |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
140 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
141 |
rc = subprocess.Popen([self.env['DIFF'], trace_output_path, reference_traces_path], stdout=dev_null()).wait() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
142 |
if rc: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
143 |
print "----------" |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
144 |
print "Traces differ in test: ", self.test_name |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
145 |
print "Reference traces in directory: " + reference_traces_path |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
146 |
print "Traces in directory: " + trace_output_path |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
147 |
print "Run the following command for details:" |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
148 |
print "\tdiff -u %s %s" % (reference_traces_path, trace_output_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
149 |
print "----------" |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
150 |
return rc |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
151 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
152 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
153 |
def run_reference_generate(self, trace_output_path, program, arguments, is_pyscript): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
154 |
if is_pyscript: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
155 |
script = os.path.abspath(os.path.join('..', *os.path.split(program))) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
156 |
argv = [self.env['PYTHON'], script] + arguments |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
157 |
retval = wutils.run_argv(argv, cwd=trace_output_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
158 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
159 |
retval = wutils.run_program(program, |
4119
9ffcf2400aa5
Bug #323: waf --valgrind doesn't check for valgrind first
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4114
diff
changeset
|
160 |
command_template=wutils.get_command_template(self.env, arguments), |
4114
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
161 |
cwd=trace_output_path) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
162 |
return retval |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
163 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
164 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
165 |
class regression_test_collector_task(Task.TaskBase): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
166 |
after = 'regression_test_task' |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
167 |
color = 'BLUE' |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
168 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
169 |
def __init__(self, test_tasks): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
170 |
super(regression_test_collector_task, self).__init__() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
171 |
self.test_tasks = test_tasks |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
172 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
173 |
def __str__(self): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
174 |
return 'regression-test-collector\n' |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
175 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
176 |
def run(self): |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
177 |
failed_tests = [test for test in self.test_tasks if test.result is not None and test.result != 0] |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
178 |
skipped_tests = [test for test in self.test_tasks if test.result is None] |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
179 |
print "Regression testing summary:" |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
180 |
if skipped_tests: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
181 |
print "SKIP: %i of %i tests have been skipped (%s)" % ( |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
182 |
len(skipped_tests), len(self.test_tasks), |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
183 |
', '.join([test.test_name for test in skipped_tests])) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
184 |
if failed_tests: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
185 |
print "FAIL: %i of %i tests have failed (%s)" % ( |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
186 |
len(failed_tests), len(self.test_tasks), |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
187 |
', '.join([test.test_name for test in failed_tests])) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
188 |
return 1 |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
189 |
else: |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
190 |
print "PASS: %i of %i tests passed" % (len(self.test_tasks) - len(skipped_tests), |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
191 |
len(self.test_tasks)) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
192 |
return 0 |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
193 |
|
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
194 |
def run_regression(bld, reference_traces): |
3872
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
195 |
"""Execute regression tests. Called with cwd set to the 'regression' subdir of ns-3. |
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
196 |
|
4109
1a251c8ad317
Cleanup: remove regression reference traces downloading code (moved to allinone)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4064
diff
changeset
|
197 |
@param reference_traces: reference traces directory. |
3872
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
198 |
|
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
199 |
""" |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
200 |
|
4110
04170734fa8b
Don't abuse os.chdir so much (not thread safe)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4109
diff
changeset
|
201 |
testdir = os.path.join("regression", "tests") |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
202 |
if not os.path.exists(testdir): |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
203 |
print "Tests directory does not exist" |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
204 |
sys.exit(3) |
3872
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
205 |
|
4064
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
206 |
if Options.options.regression_tests: |
10222f483860
Upgrade to new WAF, work in progress
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3872
diff
changeset
|
207 |
tests = Options.options.regression_tests.split(',') |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
208 |
else: |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
209 |
tests = _find_tests(testdir) |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
210 |
|
3872
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
211 |
if not os.path.exists(reference_traces): |
8e757a83fb36
Add a --with-regression-traces configure option, to allow using externally supplied regression traces instead of fetching them from the network.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
3866
diff
changeset
|
212 |
print "Reference traces directory (%s) does not exist" % reference_traces |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
213 |
return 3 |
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
214 |
|
4114
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
215 |
test_scripts_dir = bld.path.find_dir('regression/tests').abspath() |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
216 |
build_traces_dir = bld.path.find_or_declare('regression/traces').abspath(bld.env) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
217 |
tasks = [] |
3866
9e946fee902c
Refactor wscript code to move regression testing code to a separate python module (bug 325)
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
diff
changeset
|
218 |
for test in tests: |
4114
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
219 |
tasks.append(regression_test_task(bld.env, test, test_scripts_dir, build_traces_dir, reference_traces)) |
a6c5ccfa3451
Run regression tests as WAF tasks, plus simplify how unit tests are declared. Closes #480.
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents:
4110
diff
changeset
|
220 |
regression_test_collector_task(tasks) |