test.py
author Tom Henderson <tomh@tomh.org>
Fri, 20 Aug 2010 12:17:19 -0700
changeset 6589 9c325569fb01
parent 6374 787aef59db0b
permissions -rwxr-xr-x
Help waf to guess release versions of nsc and pybindgen
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     1
#! /usr/bin/env python
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     2
## -*- Mode: python; py-indent-offset: 4; indent-tabs-mode: nil; coding: utf-8; -*-
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     3
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     4
# Copyright (c) 2009 University of Washington
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     5
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     6
# This program is free software; you can redistribute it and/or modify
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     7
# it under the terms of the GNU General Public License version 2 as
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     8
# published by the Free Software Foundation;
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
     9
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    10
# This program is distributed in the hope that it will be useful,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    11
# but WITHOUT ANY WARRANTY; without even the implied warranty of
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    12
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    13
# GNU General Public License for more details.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    14
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    15
# You should have received a copy of the GNU General Public License
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    16
# along with this program; if not, write to the Free Software
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    17
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    18
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    19
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    20
import os
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    21
import sys
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
    22
import time
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    23
import optparse
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    24
import subprocess
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    25
import threading
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    26
import Queue
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    27
import signal
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    28
import xml.dom.minidom
5239
2e753de86174 Remove and replace attribute unit tests (bug 675)
Craig Dowell <craigdo@ee.washington.edu>
parents: 4772
diff changeset
    29
import shutil
5912
679c04aa43e7 Update test.py to detect still reachable memory blocks as errors, it will fail if there is some reachable memory blocks reported by valgrind
fmoatamr
parents: 5909
diff changeset
    30
import re
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    31
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    32
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    33
# XXX This should really be part of a waf command to list the configuration
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    34
# items relative to optional ns-3 pieces.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    35
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    36
# A list of interesting configuration items in the waf configuration 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    37
# cache which we may be interested in when deciding on which examples
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    38
# to run and how to run them.  These are set by waf during the 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    39
# configuration phase and the corresponding assignments are usually
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    40
# found in the associated subdirectory wscript files.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    41
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    42
interesting_config_items = [
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    43
    "NS3_BUILDDIR",
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    44
    "NS3_MODULE_PATH",
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    45
    "ENABLE_NSC",
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    46
    "ENABLE_REAL_TIME",
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    47
    "ENABLE_EXAMPLES",
6243
a597d6d2da85 Bug 882 - ./test.py reports 6 FAILs with --disable-python
Gustavo J. A. M. Carneiro <gjc@inescporto.pt>
parents: 6227
diff changeset
    48
    "ENABLE_PYTHON_BINDINGS",
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    49
]
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    50
5295
c1bd4ffb5e47 fix test.py -v
Craig Dowell <craigdo@ee.washington.edu>
parents: 5279
diff changeset
    51
ENABLE_NSC = False
c1bd4ffb5e47 fix test.py -v
Craig Dowell <craigdo@ee.washington.edu>
parents: 5279
diff changeset
    52
ENABLE_REAL_TIME = False
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    53
ENABLE_EXAMPLES = True
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    54
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    55
#
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    56
# If the user has constrained us to run certain kinds of tests, we can tell waf
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    57
# to only build
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    58
#
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    59
core_kinds = ["bvt", "core", "system", "unit"]
5295
c1bd4ffb5e47 fix test.py -v
Craig Dowell <craigdo@ee.washington.edu>
parents: 5279
diff changeset
    60
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    61
#
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    62
# There are some special cases for test suites that kill valgrind.  This is
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    63
# because NSC causes illegal instruction crashes when run under valgrind.
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    64
#
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    65
core_valgrind_skip_tests = [
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    66
    "ns3-tcp-cwnd",
6198
cd1b101cae62 suppress valgrind testing of new nsc test suite
Tom Henderson <tomh@tomh.org>
parents: 6127
diff changeset
    67
    "nsc-tcp-loss",
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    68
    "ns3-tcp-interoperability",
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    69
]
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    70
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    71
#
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    72
# A list of examples to run as smoke tests just to ensure that they remain 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    73
# buildable and runnable over time.  Also a condition under which to run
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    74
# the example (from the waf configuration), and a condition under which to
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    75
# run the example under valgrind.  This is because NSC causes illegal 
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    76
# instruction crashes when run under valgrind.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    77
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    78
# XXX Should this not be read from a configuration file somewhere and not
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    79
# hardcoded.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    80
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
    81
example_tests = [
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    82
    ("csma/csma-bridge", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    83
    ("csma/csma-bridge-one-hop", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    84
    ("csma/csma-broadcast", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    85
    ("csma/csma-multicast", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    86
    ("csma/csma-one-subnet", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    87
    ("csma/csma-packet-socket", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    88
    ("csma/csma-ping", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    89
    ("csma/csma-raw-ip-socket", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    90
    ("csma/csma-star", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    91
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    92
    ("emulation/emu-ping", "False", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    93
    ("emulation/emu-udp-echo", "False", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    94
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    95
    ("error-model/simple-error-model", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
    96
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    97
    ("ipv6/icmpv6-redirect", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    98
    ("ipv6/ping6", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
    99
    ("ipv6/radvd", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   100
    ("ipv6/radvd-two-prefix", "True", "True"),    
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   101
    ("ipv6/test-ipv6", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   102
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   103
    ("mesh/mesh", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   104
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   105
    ("naming/object-names", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   106
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   107
    ("realtime/realtime-udp-echo", "ENABLE_REAL_TIME == True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   108
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   109
    ("routing/dynamic-global-routing", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   110
    ("routing/global-injection-slash32", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   111
    ("routing/global-routing-slash32", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   112
    ("routing/mixed-global-routing", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   113
    ("routing/nix-simple", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   114
    ("routing/nms-p2p-nix", "False", "True"), # Takes too long to run
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   115
    ("routing/simple-alternate-routing", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   116
    ("routing/simple-global-routing", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   117
    ("routing/simple-point-to-point-olsr", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   118
    ("routing/simple-routing-ping6", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   119
    ("routing/static-routing-slash32", "True", "True"),
5741
83919caa6208 [aodv] examples/routing/aodv added to tested examples
Pavel Boyko <boyko@iitp.ru>
parents: 5483
diff changeset
   120
    ("routing/aodv", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   121
6349
4bab6b10a034 merged spectrum framework
Nicola Baldo <nbaldo@cttc.es>
parents: 6337
diff changeset
   122
    ("spectrum/adhoc-aloha-ideal-phy", "True", "True"),
4bab6b10a034 merged spectrum framework
Nicola Baldo <nbaldo@cttc.es>
parents: 6337
diff changeset
   123
    ("spectrum/adhoc-aloha-ideal-phy-with-microwave-oven", "True", "True"),
4bab6b10a034 merged spectrum framework
Nicola Baldo <nbaldo@cttc.es>
parents: 6337
diff changeset
   124
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   125
    ("stats/wifi-example-sim", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   126
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   127
    ("tap/tap-wifi-dumbbell", "False", "True"), # Requires manual configuration
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   128
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   129
    ("tcp/star", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   130
    ("tcp/tcp-large-transfer", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   131
    ("tcp/tcp-nsc-lfn", "ENABLE_NSC == True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   132
    ("tcp/tcp-nsc-zoo", "ENABLE_NSC == True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   133
    ("tcp/tcp-star-server", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   134
6127
b5bc10de166d merge topology read system
Tommaso Pecorella <tpecorella@mac.com>
parents: 6111
diff changeset
   135
    ("topology-read/topology-read --input=../../examples/topology-read/Inet_small_toposample.txt", "True", "True"),
6374
787aef59db0b Bug 946 - Rocketfuel topology dataset support for topology reader
Hajime Tazaki <tazaki@sfc.wide.ad.jp>
parents: 6349
diff changeset
   136
    ("topology-read/topology-read --format=Rocketfuel --input=../../examples/topology-read/RocketFuel_toposample_1239_weights.txt", "True", "True"),
6127
b5bc10de166d merge topology read system
Tommaso Pecorella <tpecorella@mac.com>
parents: 6111
diff changeset
   137
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   138
    ("tunneling/virtual-net-device", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   139
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   140
    ("tutorial/first", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   141
    ("tutorial/hello-simulator", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   142
    ("tutorial/second", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   143
    ("tutorial/third", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   144
    ("tutorial/fourth", "True", "True"),
5483
ad83d869c08c Make tests pass on MinGW, add fifth tutorial example
Craig Dowell <craigdo@ee.washington.edu>
parents: 5481
diff changeset
   145
    ("tutorial/fifth", "True", "True"),
6046
46665d75667d output stream object inherits from simple ref count
Craig Dowell <craigdo@ee.washington.edu>
parents: 5917
diff changeset
   146
    ("tutorial/sixth", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   147
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   148
    ("udp/udp-echo", "True", "True"),
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   149
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   150
    ("wireless/mixed-wireless", "True", "True"),
6311
f8ca463163fd update multirate example for test.py
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6310
diff changeset
   151
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::AarfcdWifiManager", "True", "True"), 
f8ca463163fd update multirate example for test.py
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6310
diff changeset
   152
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::AmrrWifiManager", "True", "True"), 
f8ca463163fd update multirate example for test.py
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6310
diff changeset
   153
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::CaraWifiManager", "True", "True"), 
f8ca463163fd update multirate example for test.py
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6310
diff changeset
   154
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::IdealWifiManager", "True", "True"), 
6337
92c95748a915 Fixed Bug 802 and Bug 919
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6327
diff changeset
   155
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::MinstrelWifiManager", "True", "True"), 
6311
f8ca463163fd update multirate example for test.py
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6310
diff changeset
   156
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::OnoeWifiManager", "True", "True"), 
f8ca463163fd update multirate example for test.py
Duy Nguyen <duy@soe.ucsc.edu>
parents: 6310
diff changeset
   157
    ("wireless/multirate --totalTime=0.3s --rateManager=ns3::RraaWifiManager", "True", "True"), 
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   158
    ("wireless/simple-wifi-frame-aggregation", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   159
    ("wireless/wifi-adhoc", "False", "True"), # Takes too long to run
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   160
    ("wireless/wifi-ap --verbose=0", "True", "True"), # Don't let it spew to stdout
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   161
    ("wireless/wifi-clear-channel-cmu", "False", "True"), # Requires specific hardware
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   162
    ("wireless/wifi-simple-adhoc", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   163
    ("wireless/wifi-simple-adhoc-grid", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   164
    ("wireless/wifi-simple-infra", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   165
    ("wireless/wifi-simple-interference", "True", "True"),
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   166
    ("wireless/wifi-wired-bridging", "True", "True"),
6111
0b22ae082153 Merge WiMAX module
Mohamed Amine Ismail <amine.ismail@sophia.inria.fr>
parents: 6083
diff changeset
   167
0b22ae082153 Merge WiMAX module
Mohamed Amine Ismail <amine.ismail@sophia.inria.fr>
parents: 6083
diff changeset
   168
    ("wimax/wimax-simple", "True", "True"),
0b22ae082153 Merge WiMAX module
Mohamed Amine Ismail <amine.ismail@sophia.inria.fr>
parents: 6083
diff changeset
   169
    ("wimax/wimax-ipv4", "True", "True"),
0b22ae082153 Merge WiMAX module
Mohamed Amine Ismail <amine.ismail@sophia.inria.fr>
parents: 6083
diff changeset
   170
    ("wimax/wimax-multicast", "True", "True"),
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   171
]
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   172
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   173
#
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   174
# A list of python examples to run as smoke tests just to ensure that they 
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   175
# runnable over time.  Also a condition under which to run the example (from
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   176
# the waf configuration)
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   177
#
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   178
# XXX Should this not be read from a configuration file somewhere and not
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   179
# hardcoded.
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   180
#
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   181
python_tests = [
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   182
    ("csma/csma-bridge.py", "True"),
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   183
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   184
    ("flowmon/wifi-olsr-flowmon.py", "True"),
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   185
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   186
    ("routing/simple-routing-ping6.py", "True"),
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   187
6202
c6b85405f6fe some python examples require enable-sudo, skip
Craig Dowell <craigdo@ee.washington.edu>
parents: 6201
diff changeset
   188
    ("tap/tap-csma-virtual-machine.py", "False"), # requires enable-sudo
c6b85405f6fe some python examples require enable-sudo, skip
Craig Dowell <craigdo@ee.washington.edu>
parents: 6201
diff changeset
   189
    ("tap/tap-wifi-virtual-machine.py", "False"), # requires enable-sudo
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   190
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   191
    ("tutorial/first.py", "True"),
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   192
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   193
    ("wireless/wifi-ap.py", "True"),
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   194
    ("wireless/mixed-wireless.py", "True"),
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   195
]
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   196
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   197
#
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   198
# The test suites are going to want to output status.  They are running
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   199
# concurrently.  This means that unless we are careful, the output of
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   200
# the test suites will be interleaved.  Rather than introducing a lock
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   201
# file that could unintentionally start serializing execution, we ask
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   202
# the tests to write their output to a temporary directory and then 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   203
# put together the final output file when we "join" the test tasks back
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
   204
# to the main thread.  In addition to this issue, the example programs
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
   205
# often write lots and lots of trace files which we will just ignore.
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
   206
# We put all of them into the temp directory as well, so they can be
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
   207
# easily deleted.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   208
#
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
   209
TMP_OUTPUT_DIR = "testpy-output"
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   210
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   211
def get_node_text(node):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   212
    for child in node.childNodes:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   213
        if child.nodeType == child.TEXT_NODE:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   214
            return child.nodeValue
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   215
    return "None"
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   216
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   217
#
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   218
# A simple example of writing a text file with a test result summary.  It is 
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   219
# expected that this output will be fine for developers looking for problems.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   220
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   221
def translate_to_text(results_file, text_file):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   222
    f = open(text_file, 'w')
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   223
    dom = xml.dom.minidom.parse(results_file)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   224
    for suite in dom.getElementsByTagName("TestSuite"):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   225
        result = get_node_text(suite.getElementsByTagName("SuiteResult")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   226
        name = get_node_text(suite.getElementsByTagName("SuiteName")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   227
        time = get_node_text(suite.getElementsByTagName("SuiteTime")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   228
        output = "%s: Test Suite \"%s\" (%s)\n" % (result, name, time)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   229
        f.write(output)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   230
        if result != "CRASH":
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   231
            for case in suite.getElementsByTagName("TestCase"):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   232
                result = get_node_text(case.getElementsByTagName("CaseResult")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   233
                name = get_node_text(case.getElementsByTagName("CaseName")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   234
                time = get_node_text(case.getElementsByTagName("CaseTime")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   235
                output =   "  %s: Test Case \"%s\" (%s)\n" % (result, name, time)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   236
                f.write(output)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   237
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   238
                if result == "FAIL":
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   239
                    for details in case.getElementsByTagName("FailureDetails"):
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   240
                        f.write("    Details:\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   241
                        f.write("      Message:   %s\n" % get_node_text(details.getElementsByTagName("Message")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   242
                        f.write("      Condition: %s\n" % get_node_text(details.getElementsByTagName("Condition")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   243
                        f.write("      Actual:    %s\n" % get_node_text(details.getElementsByTagName("Actual")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   244
                        f.write("      Limit:     %s\n" % get_node_text(details.getElementsByTagName("Limit")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   245
                        f.write("      File:      %s\n" % get_node_text(details.getElementsByTagName("File")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   246
                        f.write("      Line:      %s\n" % get_node_text(details.getElementsByTagName("Line")[0]))
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   247
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   248
    for example in dom.getElementsByTagName("Example"):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   249
        result = get_node_text(example.getElementsByTagName("Result")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   250
        name = get_node_text(example.getElementsByTagName("Name")[0])
5460
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   251
        time = get_node_text(example.getElementsByTagName("ElapsedTime")[0])
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   252
        output = "%s: Example \"%s\" (%s)\n" % (result, name, time)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   253
        f.write(output)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   254
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   255
    f.close()
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   256
    
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   257
#
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   258
# A simple example of writing an HTML file with a test result summary.  It is 
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   259
# expected that this will eventually be made prettier as time progresses and
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   260
# we have time to tweak it.  This may end up being moved to a separate module
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   261
# since it will probably grow over time.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   262
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   263
def translate_to_html(results_file, html_file):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   264
    f = open(html_file, 'w')
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   265
    f.write("<html>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   266
    f.write("<body>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   267
    f.write("<center><h1>ns-3 Test Results</h1></center>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   268
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   269
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   270
    # Read and parse the whole results file.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   271
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   272
    dom = xml.dom.minidom.parse(results_file)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   273
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   274
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   275
    # Iterate through the test suites
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   276
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   277
    f.write("<h2>Test Suites</h2>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   278
    for suite in dom.getElementsByTagName("TestSuite"):
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   279
     
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   280
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   281
        # For each test suite, get its name, result and execution time info
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   282
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   283
        name = get_node_text(suite.getElementsByTagName("SuiteName")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   284
        result = get_node_text(suite.getElementsByTagName("SuiteResult")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   285
        time = get_node_text(suite.getElementsByTagName("SuiteTime")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   286
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   287
        # 
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   288
        # Print a level three header with the result, name and time.  If the 
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   289
        # test suite passed, the header is printed in green. If the suite was
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   290
        # skipped, print it in orange, otherwise assume something bad happened
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   291
        # and print in red.
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   292
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   293
        if result == "PASS":
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   294
            f.write("<h3 style=\"color:green\">%s: %s (%s)</h3>\n" % (result, name, time))
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   295
        elif result == "SKIP":
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   296
            f.write("<h3 style=\"color:#ff6600\">%s: %s (%s)</h3>\n" % (result, name, time))
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   297
        else:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   298
            f.write("<h3 style=\"color:red\">%s: %s (%s)</h3>\n" % (result, name, time))
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   299
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   300
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   301
        # The test case information goes in a table.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   302
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   303
        f.write("<table border=\"1\">\n")
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   304
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   305
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   306
        # The first column of the table has the heading Result
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   307
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   308
        f.write("<th> Result </th>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   309
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   310
        #
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   311
        # If the suite crashed or is skipped, there is no further information, so just
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   312
        # delare a new table row with the result (CRASH or SKIP) in it.  Looks like:
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   313
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   314
        #   +--------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   315
        #   | Result |
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   316
        #   +--------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   317
        #   | CRASH  |
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   318
        #   +--------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   319
        #
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   320
        # Then go on to the next test suite.  Valgrind and skipped errors look the same.
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   321
        #
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   322
        if result in ["CRASH", "SKIP", "VALGR"]:
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   323
            f.write("<tr>\n")
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   324
            if result == "SKIP":
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   325
                f.write("<td style=\"color:#ff6600\">%s</td>\n" % result)
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   326
            else:
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   327
                f.write("<td style=\"color:red\">%s</td>\n" % result)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   328
            f.write("</tr>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   329
            f.write("</table>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   330
            continue
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   331
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   332
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   333
        # If the suite didn't crash, we expect more information, so fill out
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   334
        # the table heading row.  Like,
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   335
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   336
        #   +--------+----------------+------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   337
        #   | Result | Test Case Name | Time |
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   338
        #   +--------+----------------+------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   339
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   340
        f.write("<th>Test Case Name</th>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   341
        f.write("<th> Time </th>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   342
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   343
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   344
        # If the test case failed, we need to print out some failure details
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   345
        # so extend the heading row again.  Like,
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   346
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   347
        #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   348
        #   | Result | Test Case Name | Time | Failure Details |
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   349
        #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   350
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   351
        if result == "FAIL":
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   352
            f.write("<th>Failure Details</th>\n")
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   353
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   354
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   355
        # Now iterate through all of the test cases.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   356
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   357
        for case in suite.getElementsByTagName("TestCase"):
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   358
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   359
            #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   360
            # Get the name, result and timing information from xml to use in
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   361
            # printing table below.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   362
            #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   363
            name = get_node_text(case.getElementsByTagName("CaseName")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   364
            result = get_node_text(case.getElementsByTagName("CaseResult")[0])
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   365
            time = get_node_text(case.getElementsByTagName("CaseTime")[0])
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   366
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   367
            #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   368
            # If the test case failed, we iterate through possibly multiple
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   369
            # failure details
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   370
            #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   371
            if result == "FAIL":
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   372
                #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   373
                # There can be multiple failures for each test case.  The first
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   374
                # row always gets the result, name and timing information along
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   375
                # with the failure details.  Remaining failures don't duplicate
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   376
                # this information but just get blanks for readability.  Like,
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   377
                #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   378
                #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   379
                #   | Result | Test Case Name | Time | Failure Details |
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   380
                #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   381
                #   |  FAIL  | The name       | time | It's busted     |   
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   382
                #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   383
                #   |        |                |      | Really broken   |   
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   384
                #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   385
                #   |        |                |      | Busted bad      |   
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   386
                #   +--------+----------------+------+-----------------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   387
                #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   388
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   389
                first_row = True
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   390
                for details in case.getElementsByTagName("FailureDetails"):
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   391
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   392
                    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   393
                    # Start a new row in the table for each possible Failure Detail
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   394
                    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   395
                    f.write("<tr>\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   396
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   397
                    if first_row:
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   398
                        first_row = False
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   399
                        f.write("<td style=\"color:red\">%s</td>\n" % result)
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   400
                        f.write("<td>%s</td>\n" % name)
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   401
                        f.write("<td>%s</td>\n" % time)
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   402
                    else:
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   403
                        f.write("<td></td>\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   404
                        f.write("<td></td>\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   405
                        f.write("<td></td>\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   406
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   407
                    f.write("<td>")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   408
                    f.write("<b>Message: </b>%s, " % get_node_text(details.getElementsByTagName("Message")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   409
                    f.write("<b>Condition: </b>%s, " % get_node_text(details.getElementsByTagName("Condition")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   410
                    f.write("<b>Actual: </b>%s, " % get_node_text(details.getElementsByTagName("Actual")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   411
                    f.write("<b>Limit: </b>%s, " % get_node_text(details.getElementsByTagName("Limit")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   412
                    f.write("<b>File: </b>%s, " % get_node_text(details.getElementsByTagName("File")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   413
                    f.write("<b>Line: </b>%s" % get_node_text(details.getElementsByTagName("Line")[0]))
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   414
                    f.write("</td>\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   415
                    
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   416
                    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   417
                    # End the table row
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   418
                    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   419
                    f.write("</td>\n")
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   420
            else:
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   421
                #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   422
                # If this particular test case passed, then we just print the PASS
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   423
                # result in green, followed by the test case name and its execution
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   424
                # time information.  These go off in <td> ... </td> table data.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   425
                # The details table entry is left blank.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   426
                #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   427
                #   +--------+----------------+------+---------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   428
                #   | Result | Test Case Name | Time | Details |
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   429
                #   +--------+----------------+------+---------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   430
                #   |  PASS  | The name       | time |         |   
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   431
                #   +--------+----------------+------+---------+
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   432
                #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   433
                f.write("<tr>\n")
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   434
                f.write("<td style=\"color:green\">%s</td>\n" % result)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   435
                f.write("<td>%s</td>\n" % name)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   436
                f.write("<td>%s</td>\n" % time)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   437
                f.write("<td></td>\n")
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   438
                f.write("</tr>\n")
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   439
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   440
        # All of the rows are written, so we need to end the table.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   441
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   442
        f.write("</table>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   443
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   444
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   445
    # That's it for all of the test suites.  Now we have to do something about 
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   446
    # our examples.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   447
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   448
    f.write("<h2>Examples</h2>\n")
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   449
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   450
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   451
    # Example status is rendered in a table just like the suites.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   452
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   453
    f.write("<table border=\"1\">\n")
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   454
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   455
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   456
    # The table headings look like,
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   457
    #
5460
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   458
    #   +--------+--------------+--------------+
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   459
    #   | Result | Example Name | Elapsed Time |
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   460
    #   +--------+--------------+--------------+
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   461
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   462
    f.write("<th> Result </th>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   463
    f.write("<th>Example Name</th>\n")
5460
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   464
    f.write("<th>Elapsed Time</th>\n")
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   465
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   466
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   467
    # Now iterate through all of the examples
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   468
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   469
    for example in dom.getElementsByTagName("Example"):
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   470
        
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   471
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   472
        # Start a new row for each example
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   473
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   474
        f.write("<tr>\n")
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   475
        
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   476
        #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   477
        # Get the result and name of the example in question
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   478
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   479
        result = get_node_text(example.getElementsByTagName("Result")[0])
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   480
        name =   get_node_text(example.getElementsByTagName("Name")[0])
5460
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   481
        time =   get_node_text(example.getElementsByTagName("ElapsedTime")[0])
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   482
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   483
        #
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   484
        # If the example either failed or crashed, print its result status
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   485
        # in red; otherwise green.  This goes in a <td> ... </td> table data
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   486
        #
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   487
        if result == "PASS":
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   488
            f.write("<td style=\"color:green\">%s</td>\n" % result)
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   489
        elif result == "SKIP":
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   490
            f.write("<td style=\"color:#ff6600\">%s</fd>\n" % result)
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   491
        else:
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   492
            f.write("<td style=\"color:red\">%s</td>\n" % result)
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   493
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   494
        #
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   495
        # Write the example name as a new tag data.
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   496
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   497
        f.write("<td>%s</td>\n" % name)
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   498
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   499
        #
5460
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   500
        # Write the elapsed time as a new tag data.
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   501
        #
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   502
        f.write("<td>%s</td>\n" % time)
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   503
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   504
        #
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   505
        # That's it for the current example, so terminate the row.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   506
        #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   507
        f.write("</tr>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   508
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   509
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   510
    # That's it for the table of examples, so terminate the table.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   511
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   512
    f.write("</table>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   513
5324
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   514
    #
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   515
    # And that's it for the report, so finish up.
0ba73cdd2a43 Refalgamize test framework to allow multiple test failures
Craig Dowell <craigdo@ee.washington.edu>
parents: 5295
diff changeset
   516
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   517
    f.write("</body>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   518
    f.write("</html>\n")
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   519
    f.close()
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   520
    
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   521
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   522
# Python Control-C handling is broken in the presence of multiple threads.  
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   523
# Signals get delivered to the runnable/running thread by default and if 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   524
# it is blocked, the signal is simply ignored.  So we hook sigint and set 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   525
# a global variable telling the system to shut down gracefully.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   526
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   527
thread_exit = False
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   528
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   529
def sigint_hook(signal, frame):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   530
    global thread_exit
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   531
    thread_exit = True
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   532
    return 0
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   533
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   534
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   535
# Waf can be configured to compile in debug or optimized modes.  In each
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   536
# case, the resulting built goes into a different directory.  If we want
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   537
# test tests to run from the correct code-base, we have to figure out which
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   538
# mode waf is running in.  This is called its active variant.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   539
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   540
# XXX This function pokes around in the waf internal state file.  To be a
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   541
# little less hacky, we should add a commmand to waf to return this info
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   542
# and use that result.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   543
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   544
def read_waf_active_variant():
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   545
    for line in open("build/c4che/default.cache.py").readlines():
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   546
        if line.startswith("NS3_ACTIVE_VARIANT"):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   547
            exec(line, globals())
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   548
            break
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   549
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   550
    if options.verbose:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   551
        print "NS3_ACTIVE_VARIANT == %s" % NS3_ACTIVE_VARIANT
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   552
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   553
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   554
# In general, the build process itself naturally takes care of figuring out
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   555
# which tests are built into the test runner.  For example, if waf configure
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   556
# determines that ENABLE_EMU is false due to some missing dependency,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   557
# the tests for the emu net device simply will not be built and will 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   558
# therefore not be included in the built test runner.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   559
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   560
# Examples, however, are a different story.  In that case, we are just given
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   561
# a list of examples that could be run.  Instead of just failing, for example,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   562
# nsc-tcp-zoo if NSC is not present, we look into the waf saved configuration
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   563
# for relevant configuration items.  
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   564
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   565
# XXX This function pokes around in the waf internal state file.  To be a
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   566
# little less hacky, we should add a commmand to waf to return this info
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   567
# and use that result.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   568
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   569
def read_waf_config():
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   570
    for line in open("build/c4che/%s.cache.py" % NS3_ACTIVE_VARIANT).readlines():
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   571
        for item in interesting_config_items:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   572
            if line.startswith(item):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   573
                exec(line, globals())
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   574
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   575
    if options.verbose:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   576
        for item in interesting_config_items:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   577
            print "%s ==" % item, eval(item)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   578
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   579
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   580
# It seems pointless to fork a process to run waf to fork a process to run
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   581
# the test runner, so we just run the test runner directly.  The main thing 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   582
# that waf would do for us would be to sort out the shared library path but
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   583
# we can deal with that easily and do here.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   584
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   585
# There can be many different ns-3 repositories on a system, and each has 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   586
# its own shared libraries, so ns-3 doesn't hardcode a shared library search
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   587
# path -- it is cooked up dynamically, so we do that too.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   588
#
6227
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   589
def make_paths():
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   590
    have_DYLD_LIBRARY_PATH = False
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   591
    have_LD_LIBRARY_PATH = False
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   592
    have_PATH = False
6227
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   593
    have_PYTHONPATH = False
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   594
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   595
    keys = os.environ.keys()
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   596
    for key in keys:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   597
        if key == "DYLD_LIBRARY_PATH":
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   598
            have_DYLD_LIBRARY_PATH = True
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   599
        if key == "LD_LIBRARY_PATH":
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   600
            have_LD_LIBRARY_PATH = True
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   601
        if key == "PATH":
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   602
            have_PATH = True
6227
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   603
        if key == "PYTHONPATH":
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   604
            have_PYTHONPATH = True
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   605
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   606
    pypath = os.environ["PYTHONPATH"] = os.path.join (NS3_BUILDDIR, NS3_ACTIVE_VARIANT, "bindings", "python")
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   607
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   608
    if not have_PYTHONPATH:
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   609
        os.environ["PYTHONPATH"] = pypath
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   610
    else:
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   611
        os.environ["PYTHONPATH"] += ":" + pypath
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   612
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   613
    if options.verbose:
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
   614
        print "os.environ[\"PYTHONPATH\"] == %s" % os.environ["PYTHONPATH"]
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   615
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   616
    if sys.platform == "darwin":
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   617
        if not have_DYLD_LIBRARY_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   618
            os.environ["DYLD_LIBRARY_PATH"] = ""
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   619
        for path in NS3_MODULE_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   620
            os.environ["DYLD_LIBRARY_PATH"] += ":" + path
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   621
        if options.verbose:
5466
c206a8ce5644 Fix testing under Mac OS X
fmoatamr
parents: 5461
diff changeset
   622
            print "os.environ[\"DYLD_LIBRARY_PATH\"] == %s" % os.environ["DYLD_LIBRARY_PATH"]
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   623
    elif sys.platform == "win32":
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   624
        if not have_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   625
            os.environ["PATH"] = ""
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   626
        for path in NS3_MODULE_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   627
            os.environ["PATH"] += ';' + path
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   628
        if options.verbose:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   629
            print "os.environ[\"PATH\"] == %s" % os.environ["PATH"]
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   630
    elif sys.platform == "cygwin":
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   631
        if not have_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   632
            os.environ["PATH"] = ""
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   633
        for path in NS3_MODULE_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   634
            os.environ["PATH"] += ":" + path
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   635
        if options.verbose:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   636
            print "os.environ[\"PATH\"] == %s" % os.environ["PATH"]
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   637
    else:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   638
        if not have_LD_LIBRARY_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   639
            os.environ["LD_LIBRARY_PATH"] = ""
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   640
        for path in NS3_MODULE_PATH:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   641
            os.environ["LD_LIBRARY_PATH"] += ":" + path
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   642
        if options.verbose:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   643
            print "os.environ[\"LD_LIBRARY_PATH\"] == %s" % os.environ["LD_LIBRARY_PATH"]
5275
d4008f2981ba Windows needs entire path
Craig Dowell <craigdo@ee.washington.edu>
parents: 5274
diff changeset
   644
5909
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   645
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   646
# Short note on generating suppressions:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   647
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   648
# See the valgrind documentation for a description of suppressions.  The easiest
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   649
# way to generate a suppression expression is by using the valgrind 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   650
# --gen-suppressions option.  To do that you have to figure out how to run the 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   651
# test in question.
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   652
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   653
# If you do "test.py -v -g -s <suitename> then test.py will output most of what
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   654
# you need.  For example, if you are getting a valgrind error in the
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   655
# devices-mesh-dot11s-regression test suite, you can run:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   656
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   657
#   ./test.py -v -g -s devices-mesh-dot11s-regression 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   658
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   659
# You should see in the verbose output something that looks like:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   660
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   661
#   Synchronously execute valgrind --suppressions=/home/craigdo/repos/ns-3-allinone-dev/ns-3-dev/testpy.supp
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   662
#   --leak-check=full --error-exitcode=2 /home/craigdo/repos/ns-3-allinone-dev/ns-3-dev/build/debug/utils/test-runner 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   663
#   --suite=devices-mesh-dot11s-regression --basedir=/home/craigdo/repos/ns-3-allinone-dev/ns-3-dev 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   664
#   --tempdir=testpy-output/2010-01-12-22-47-50-CUT 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   665
#   --out=testpy-output/2010-01-12-22-47-50-CUT/devices-mesh-dot11s-regression.xml
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   666
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   667
# You need to pull out the useful pieces, and so could run the following to 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   668
# reproduce your error:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   669
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   670
#   valgrind --suppressions=/home/craigdo/repos/ns-3-allinone-dev/ns-3-dev/testpy.supp
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   671
#   --leak-check=full --error-exitcode=2 /home/craigdo/repos/ns-3-allinone-dev/ns-3-dev/build/debug/utils/test-runner 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   672
#   --suite=devices-mesh-dot11s-regression --basedir=/home/craigdo/repos/ns-3-allinone-dev/ns-3-dev 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   673
#   --tempdir=testpy-output 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   674
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   675
# Hint: Use the first part of the command as is, and point the "tempdir" to 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   676
# somewhere real.  You don't need to specify an "out" file.
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   677
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   678
# When you run the above command you should see your valgrind error.  The 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   679
# suppression expression(s) can be generated by adding the --gen-suppressions=yes
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   680
# option to valgrind.  Use something like:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   681
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   682
#   valgrind --gen-suppressions=yes --suppressions=/home/craigdo/repos/ns-3-allinone-dev/ns-3-dev/testpy.supp
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   683
#   --leak-check=full --error-exitcode=2 /home/craigdo/repos/ns-3-allinone-dev/ns-3-dev/build/debug/utils/test-runner 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   684
#   --suite=devices-mesh-dot11s-regression --basedir=/home/craigdo/repos/ns-3-allinone-dev/ns-3-dev 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   685
#   --tempdir=testpy-output 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   686
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   687
# Now when valgrind detects an error it will ask:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   688
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   689
#   ==27235== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ----
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   690
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   691
# to which you just enter 'y'<ret>.
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   692
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   693
# You will be provided with a suppression expression that looks something like
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   694
# the following:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   695
#   {
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   696
#     <insert_a_suppression_name_here>
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   697
#     Memcheck:Addr8
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   698
#     fun:_ZN3ns36dot11s15HwmpProtocolMac8SendPreqESt6vectorINS0_6IePreqESaIS3_EE
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   699
#     fun:_ZN3ns36dot11s15HwmpProtocolMac10SendMyPreqEv
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   700
#     fun:_ZN3ns36dot11s15HwmpProtocolMac18RequestDestinationENS_12Mac48AddressEjj
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   701
#     ...
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   702
#     the rest of the stack frame
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   703
#     ...
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   704
#   }
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   705
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   706
# You need to add a supression name which will only be printed out by valgrind in 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   707
# verbose mode (but it needs to be there in any case).  The entire stack frame is
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   708
# shown to completely characterize the error, but in most cases you won't need 
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   709
# all of that info.  For example, if you want to turn off all errors that happen
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   710
# when the function (fun:) is called, you can just delete the rest of the stack
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   711
# frame.  You can also use wildcards to make the mangled signatures more readable.
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   712
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   713
# I added the following to the testpy.supp file for this particular error:
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   714
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   715
#   {
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   716
#     Supress invalid read size errors in SendPreq() when using HwmpProtocolMac
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   717
#     Memcheck:Addr8
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   718
#     fun:*HwmpProtocolMac*SendPreq*
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   719
#   }
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   720
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   721
# Now, when you run valgrind the error will be suppressed.
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   722
#
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   723
VALGRIND_SUPPRESSIONS_FILE = "testpy.supp"
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   724
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   725
def run_job_synchronously(shell_command, directory, valgrind, is_python):
5909
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   726
    (base, build) = os.path.split (NS3_BUILDDIR)
766442c7f240 Bug 781: Suppress the valgrind error: Invalid read size of 8 in TestSuite devices-mesh-dot11s-regression
Faker Moatamri <faker.moatamri@sophia.inria.fr>
parents: 5741
diff changeset
   727
    suppressions_path = os.path.join (base, VALGRIND_SUPPRESSIONS_FILE)
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   728
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   729
    if is_python:
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   730
        path_cmd = "python " + os.path.join (base, shell_command)
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   731
    else:
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   732
        path_cmd = os.path.join (NS3_BUILDDIR, NS3_ACTIVE_VARIANT, shell_command)
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   733
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   734
    if valgrind:
5917
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   735
        cmd = "valgrind --suppressions=%s --leak-check=full --show-reachable=yes --error-exitcode=2 %s" % (suppressions_path, 
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   736
            path_cmd)
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   737
    else:
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   738
        cmd = path_cmd
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   739
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   740
    if options.verbose:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   741
        print "Synchronously execute %s" % cmd
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   742
5460
7c6f8ebf4a34 test.py prints example elapsed times
craigdo@ee.washington.edu
parents: 5459
diff changeset
   743
    start_time = time.time()
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   744
    proc = subprocess.Popen(cmd, shell = True, cwd = directory, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
5917
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   745
    stdout_results, stderr_results = proc.communicate()
5912
679c04aa43e7 Update test.py to detect still reachable memory blocks as errors, it will fail if there is some reachable memory blocks reported by valgrind
fmoatamr
parents: 5909
diff changeset
   746
    elapsed_time = time.time() - start_time
5917
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   747
5912
679c04aa43e7 Update test.py to detect still reachable memory blocks as errors, it will fail if there is some reachable memory blocks reported by valgrind
fmoatamr
parents: 5909
diff changeset
   748
    retval = proc.returncode
5917
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   749
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   750
    #
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   751
    # valgrind sometimes has its own idea about what kind of memory management
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   752
    # errors are important.  We want to detect *any* leaks, so the way to do 
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   753
    # that is to look for the presence of a valgrind leak summary section.
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   754
    #
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   755
    # If another error has occurred (like a test suite has failed), we don't 
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   756
    # want to trump that error, so only do the valgrind output scan if the 
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   757
    # test has otherwise passed (return code was zero).
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   758
    #
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   759
    if valgrind and retval == 0 and "== LEAK SUMMARY:" in stderr_results:
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   760
        retval = 2
aa8364846523 Bug 792: Neither test.py nor waf --valgrind catch all kinds of memory leaks
Craig Dowell <craigdo@ee.washington.edu>
parents: 5912
diff changeset
   761
    
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   762
    if options.verbose:
5912
679c04aa43e7 Update test.py to detect still reachable memory blocks as errors, it will fail if there is some reachable memory blocks reported by valgrind
fmoatamr
parents: 5909
diff changeset
   763
        print "Return code = ", retval
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   764
        print "stderr = ", stderr_results
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   765
5912
679c04aa43e7 Update test.py to detect still reachable memory blocks as errors, it will fail if there is some reachable memory blocks reported by valgrind
fmoatamr
parents: 5909
diff changeset
   766
    return (retval, stdout_results, stderr_results, elapsed_time)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   767
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   768
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   769
# This class defines a unit of testing work.  It will typically refer to
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   770
# a test suite to run using the test-runner, or an example to run directly.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   771
#
5415
5c2f96b507a9 Stray () in test.py (bug 718)
Craig Dowell <craigdo@ee.washington.edu>
parents: 5414
diff changeset
   772
class Job:
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   773
    def __init__(self):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   774
        self.is_break = False
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   775
        self.is_skip = False
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   776
        self.is_example = False
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   777
        self.is_pyexample = False
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   778
        self.shell_command = ""
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   779
        self.display_name = ""
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   780
        self.basedir = ""
5481
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   781
        self.tempdir = ""
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   782
        self.cwd = ""
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   783
        self.tmp_file_name = ""
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   784
        self.returncode = False
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   785
        self.elapsed_time = 0
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   786
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   787
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   788
    # A job is either a standard job or a special job indicating that a worker
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   789
    # thread should exist.  This special job is indicated by setting is_break 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   790
    # to true.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   791
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   792
    def set_is_break(self, is_break):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   793
        self.is_break = is_break
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   794
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   795
    #
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   796
    # If a job is to be skipped, we actually run it through the worker threads
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   797
    # to keep the PASS, FAIL, CRASH and SKIP processing all in one place.
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   798
    #
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   799
    def set_is_skip(self, is_skip):
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   800
        self.is_skip = is_skip
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   801
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   802
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   803
    # Examples are treated differently than standard test suites.  This is
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   804
    # mostly because they are completely unaware that they are being run as 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   805
    # tests.  So we have to do some special case processing to make them look
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   806
    # like tests.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   807
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   808
    def set_is_example(self, is_example):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   809
        self.is_example = is_example
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   810
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   811
    #
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   812
    # Examples are treated differently than standard test suites.  This is
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   813
    # mostly because they are completely unaware that they are being run as 
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   814
    # tests.  So we have to do some special case processing to make them look
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   815
    # like tests.
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   816
    #
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   817
    def set_is_pyexample(self, is_pyexample):
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   818
        self.is_pyexample = is_pyexample
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   819
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   820
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   821
    # This is the shell command that will be executed in the job.  For example,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   822
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   823
    #  "utils/test-runner --suite=some-test-suite"
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   824
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   825
    def set_shell_command(self, shell_command):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   826
        self.shell_command = shell_command
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   827
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   828
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   829
    # This is the dispaly name of the job, typically the test suite or example 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   830
    # name.  For example,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   831
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   832
    #  "some-test-suite" or "udp-echo"
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   833
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   834
    def set_display_name(self, display_name):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   835
        self.display_name = display_name
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   836
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   837
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   838
    # This is the base directory of the repository out of which the tests are
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   839
    # being run.  It will be used deep down in the testing framework to determine
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   840
    # where the source directory of the test was, and therefore where to find 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   841
    # provided test vectors.  For example,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   842
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   843
    #  "/home/user/repos/ns-3-dev"
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   844
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   845
    def set_basedir(self, basedir):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   846
        self.basedir = basedir
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   847
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   848
    #
5481
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   849
    # This is the directory to which a running test suite should write any 
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   850
    # temporary files.
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   851
    #
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   852
    def set_tempdir(self, tempdir):
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   853
        self.tempdir = tempdir
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   854
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   855
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   856
    # This is the current working directory that will be given to an executing
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   857
    # test as it is being run.  It will be used for examples to tell them where
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   858
    # to write all of the pcap files that we will be carefully ignoring.  For
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   859
    # example,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   860
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   861
    #  "/tmp/unchecked-traces"
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   862
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   863
    def set_cwd(self, cwd):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   864
        self.cwd = cwd
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   865
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   866
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   867
    # This is the temporary results file name that will be given to an executing 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   868
    # test as it is being run.  We will be running all of our tests in parallel
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   869
    # so there must be multiple temporary output files.  These will be collected
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
   870
    # into a single XML file at the end and then be deleted.  
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   871
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   872
    def set_tmp_file_name(self, tmp_file_name):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   873
        self.tmp_file_name = tmp_file_name
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   874
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   875
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   876
    # The return code received when the job process is executed.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   877
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   878
    def set_returncode(self, returncode):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   879
        self.returncode = returncode
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   880
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   881
    #
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   882
    # The elapsed real time for the job execution.
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   883
    #
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   884
    def set_elapsed_time(self, elapsed_time):
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   885
        self.elapsed_time = elapsed_time
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   886
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   887
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   888
# The worker thread class that handles the actual running of a given test.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   889
# Once spawned, it receives requests for work through its input_queue and
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   890
# ships the results back through the output_queue.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   891
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   892
class worker_thread(threading.Thread):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   893
    def __init__(self, input_queue, output_queue):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   894
        threading.Thread.__init__(self)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   895
        self.input_queue = input_queue
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   896
        self.output_queue = output_queue
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   897
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   898
    def run(self):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   899
        while True:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   900
            job = self.input_queue.get()
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   901
            #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   902
            # Worker threads continue running until explicitly told to stop with
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   903
            # a special job.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   904
            #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   905
            if job.is_break:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   906
                return
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   907
            #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   908
            # If the global interrupt handler sets the thread_exit variable,
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   909
            # we stop doing real work and just report back a "break" in the
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   910
            # normal command processing has happened.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   911
            #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   912
            if thread_exit == True:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   913
                job.set_is_break(True)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   914
                self.output_queue.put(job)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   915
                continue
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   916
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   917
            #
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   918
            # If we are actually supposed to skip this job, do so.  Note that
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   919
            # if is_skip is true, returncode is undefined.
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   920
            #
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   921
            if job.is_skip:
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   922
                if options.verbose:
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   923
                    print "Skip %s" % job.shell_command
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   924
                self.output_queue.put(job)
6253
3aa30fa758f5 bug 886
Craig Dowell <craigdo@ee.washington.edu>
parents: 6247
diff changeset
   925
                continue
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
   926
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   927
            #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   928
            # Otherwise go about the business of running tests as normal.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   929
            #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   930
            else:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   931
                if options.verbose:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   932
                    print "Launch %s" % job.shell_command
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   933
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   934
                if job.is_example or job.is_pyexample:
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   935
                    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   936
                    # If we have an example, the shell command is all we need to
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   937
                    # know.  It will be something like "examples/udp-echo" or 
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   938
                    # "examples/mixed-wireless.py"
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   939
                    #
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   940
                    (job.returncode, standard_out, standard_err, et) = run_job_synchronously(job.shell_command, 
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   941
                        job.cwd, options.valgrind, job.is_pyexample)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   942
                else:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   943
                    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   944
                    # If we're a test suite, we need to provide a little more info
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   945
                    # to the test runner, specifically the base directory and temp
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   946
                    # file name
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   947
                    #
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   948
                    (job.returncode, standard_out, standard_err, et) = run_job_synchronously(job.shell_command + 
5481
de01c67af975 pass explicit temp directory to test suites
Craig Dowell <craigdo@ee.washington.edu>
parents: 5470
diff changeset
   949
                        " --basedir=%s --tempdir=%s --out=%s" % (job.basedir, job.tempdir, job.tmp_file_name), 
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
   950
                        job.cwd, options.valgrind, False)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   951
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   952
                job.set_elapsed_time(et)
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
   953
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   954
                if options.verbose:
5370
a27163124701 add valgrind powers back to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5369
diff changeset
   955
                    print "returncode = %d" % job.returncode
6327
246934d7b912 Fixed typo in test.py
Lalith Suresh
parents: 6315
diff changeset
   956
                    print "---------- begin standard out ----------"
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   957
                    print standard_out
5351
6abced63cd7f keep wifi-ap from spewing all over std_out during tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5350
diff changeset
   958
                    print "---------- begin standard err ----------"
6abced63cd7f keep wifi-ap from spewing all over std_out during tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5350
diff changeset
   959
                    print standard_err
6abced63cd7f keep wifi-ap from spewing all over std_out during tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5350
diff changeset
   960
                    print "---------- end standard err ----------"
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   961
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   962
                self.output_queue.put(job)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   963
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   964
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   965
# This is the main function that does the work of interacting with the test-runner
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   966
# itself.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   967
#
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   968
def run_tests():
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   969
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   970
    # Run waf to make sure that everything is built, configured and ready to go
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   971
    # unless we are explicitly told not to.  We want to be careful about causing
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   972
    # our users pain while waiting for extraneous stuff to compile and link, so
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   973
    # we allow users that know what they''re doing to not invoke waf at all.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
   974
    #
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   975
    if not options.nowaf:
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   976
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   977
        #
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   978
        # If the user is running the "kinds" or "list" options, there is an 
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   979
        # implied dependency on the test-runner since we call that program
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   980
        # if those options are selected.  We will exit after processing those
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   981
        # options, so if we see them, we can safely only build the test-runner.
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   982
        #
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   983
        # If the user has constrained us to running only a particular type of
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   984
        # file, we can only ask waf to build what we know will be necessary.
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   985
        # For example, if the user only wants to run BVT tests, we only have
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   986
        # to build the test-runner and can ignore all of the examples.
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   987
        #
5470
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   988
        # If the user only wants to run a single example, then we can just build
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   989
        # that example.
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   990
        #
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   991
        # If there is no constraint, then we have to build everything since the
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   992
        # user wants to run everything.
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   993
        #
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
   994
        if options.kinds or options.list or (len(options.constrain) and options.constrain in core_kinds):
5470
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   995
            if sys.platform == "win32":
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   996
                waf_cmd = "waf --target=test-runner"
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   997
            else:
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   998
                waf_cmd = "./waf --target=test-runner"
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
   999
        elif len(options.example):
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1000
            if sys.platform == "win32":
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1001
                waf_cmd = "waf --target=%s" % os.path.basename(options.example)
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1002
            else:
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1003
                waf_cmd = "./waf --target=%s" % os.path.basename(options.example)
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1004
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
  1005
        else:
5470
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1006
            if sys.platform == "win32":
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1007
                waf_cmd = "waf"
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1008
            else:
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1009
                waf_cmd = "./waf"
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
  1010
5470
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1011
        if options.verbose:
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1012
            print "Building: %s" % waf_cmd
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1013
d3aabb63dd12 remind test.py how to build on unix and limit build scope when posible
Craig Dowell <craigdo@ee.washington.edu>
parents: 5466
diff changeset
  1014
        proc = subprocess.Popen(waf_cmd, shell = True)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1015
        proc.communicate()
6315
1f53e122254b fix bug 908
Andrey Mazo <mazo@iitp.ru>
parents: 6312
diff changeset
  1016
        if proc.returncode:
1f53e122254b fix bug 908
Andrey Mazo <mazo@iitp.ru>
parents: 6312
diff changeset
  1017
            print >> sys.stderr, "Waf died. Not running tests"
1f53e122254b fix bug 908
Andrey Mazo <mazo@iitp.ru>
parents: 6312
diff changeset
  1018
            return proc.returncode
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1019
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1020
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1021
    # Pull some interesting configuration information out of waf, primarily
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1022
    # so we can know where executables can be found, but also to tell us what
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1023
    # pieces of the system have been built.  This will tell us what examples 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1024
    # are runnable.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1025
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1026
    read_waf_active_variant()
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1027
    read_waf_config()
6227
38a3d54b5742 python path helps when running python programs
Craig Dowell <craigdo@ee.washington.edu>
parents: 6203
diff changeset
  1028
    make_paths()
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1029
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1030
    #
6083
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1031
    # If lots of logging is enabled, we can crash Python when it tries to 
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1032
    # save all of the text.  We just don't allow logging to be turned on when
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1033
    # test.py runs.  If you want to see logging output from your tests, you
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1034
    # have to run them using the test-runner directly.
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1035
    #
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1036
    os.environ["NS_LOG"] = ""
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1037
a4a7ded174dc test.py turns off NS_LOG locally
Craig Dowell <craigdo@ee.washington.edu>
parents: 6046
diff changeset
  1038
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1039
    # There are a couple of options that imply we can to exit before starting
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1040
    # up a bunch of threads and running tests.  Let's detect these cases and 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1041
    # handle them without doing all of the hard work.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1042
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1043
    if options.kinds:
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1044
        path_cmd = os.path.join("utils", "test-runner --kinds")
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1045
        (rc, standard_out, standard_err, et) = run_job_synchronously(path_cmd, os.getcwd(), False, False)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1046
        print standard_out
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1047
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1048
    if options.list:
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1049
        path_cmd = os.path.join("utils", "test-runner --list")
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1050
        (rc, standard_out, standard_err, et) = run_job_synchronously(path_cmd, os.getcwd(), False, False)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1051
        print standard_out
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1052
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1053
    if options.kinds or options.list:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1054
        return
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1055
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1056
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1057
    # We communicate results in two ways.  First, a simple message relating 
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1058
    # PASS, FAIL, CRASH or SKIP is always written to the standard output.  It 
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1059
    # is expected that this will be one of the main use cases.  A developer can
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1060
    # just run test.py with no options and see that all of the tests still 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1061
    # pass.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1062
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1063
    # The second main use case is when detailed status is requested (with the
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1064
    # --text or --html options).  Typicall this will be text if a developer
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1065
    # finds a problem, or HTML for nightly builds.  In these cases, an
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1066
    # XML file is written containing the status messages from the test suites.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1067
    # This file is then read and translated into text or HTML.  It is expected
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1068
    # that nobody will really be interested in the XML, so we write it somewhere
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1069
    # with a unique name (time) to avoid collisions.  In case an error happens, we
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1070
    # provide a runtime option to retain the temporary files.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1071
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1072
    # When we run examples as smoke tests, they are going to want to create
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1073
    # lots and lots of trace files.  We aren't really interested in the contents
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1074
    # of the trace files, so we also just stash them off in the temporary dir.
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1075
    # The retain option also causes these unchecked trace files to be kept.
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1076
    #
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1077
    date_and_time = time.strftime("%Y-%m-%d-%H-%M-%S-CUT", time.gmtime())
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1078
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1079
    if not os.path.exists(TMP_OUTPUT_DIR):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1080
        os.makedirs(TMP_OUTPUT_DIR)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1081
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1082
    testpy_output_dir = os.path.join(TMP_OUTPUT_DIR, date_and_time);
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1083
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1084
    if not os.path.exists(testpy_output_dir):
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1085
        os.makedirs(testpy_output_dir)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1086
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1087
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1088
    # Create the main output file and start filling it with XML.  We need to 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1089
    # do this since the tests will just append individual results to this file.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1090
    #
5412
00d7fe69d024 refalgamize temporary files in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5403
diff changeset
  1091
    xml_results_file = os.path.join(testpy_output_dir, "results.xml")
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1092
    f = open(xml_results_file, 'w')
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1093
    f.write('<?xml version="1.0"?>\n')
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1094
    f.write('<TestResults>\n')
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1095
    f.close()
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1096
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1097
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1098
    # We need to figure out what test suites to execute.  We are either given one 
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1099
    # suite or example explicitly via the --suite or --example/--pyexample option,
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1100
    # or we need to call into the test runner and ask it to list all of the available
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1101
    # test suites.  Further, we need to provide the constraint information if it
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1102
    # has been given to us.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1103
    # 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1104
    # This translates into allowing the following options with respect to the 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1105
    # suites
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1106
    #
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
  1107
    #  ./test,py:                                           run all of the suites and examples
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
  1108
    #  ./test.py --constrain=core:                          run all of the suites of all kinds
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1109
    #  ./test.py --constrain=unit:                          run all unit suites
5369
86beb5869f67 split examples, add examples, tweak test.py to minimize builds
Craig Dowell <craigdo@ee.washington.edu>
parents: 5351
diff changeset
  1110
    #  ./test,py --suite=some-test-suite:                   run a single suite
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1111
    #  ./test,py --example=udp/udp-echo:                    run no test suites
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1112
    #  ./test,py --pyexample=wireless/mixed-wireless.py:    run no test suites
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1113
    #  ./test,py --suite=some-suite --example=some-example: run the single suite
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1114
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1115
    # We can also use the --constrain option to provide an ordering of test 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1116
    # execution quite easily.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1117
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1118
    if len(options.suite):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1119
        suites = options.suite + "\n"
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1120
    elif len(options.example) == 0 and len(options.pyexample) == 0:
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1121
        if len(options.constrain):
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1122
            path_cmd = os.path.join("utils", "test-runner --list --constrain=%s" % options.constrain)
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1123
            (rc, suites, standard_err, et) = run_job_synchronously(path_cmd, os.getcwd(), False, False)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1124
        else:
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1125
            path_cmd = os.path.join("utils", "test-runner --list")
6200
2bf2700b6e67 Add some python examples to test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 6127
diff changeset
  1126
            (rc, suites, standard_err, et) = run_job_synchronously(path_cmd, os.getcwd(), False, False)
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1127
    else:
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1128
        suites = ""
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1129
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1130
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1131
    # suite_list will either a single test suite name that the user has 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1132
    # indicated she wants to run or a list of test suites provided by
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1133
    # the test-runner possibly according to user provided constraints.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1134
    # We go through the trouble of setting up the parallel execution 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1135
    # even in the case of a single suite to avoid having two process the
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1136
    # results in two different places.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1137
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1138
    suite_list = suites.split('\n')
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1139
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1140
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1141
    # We now have a possibly large number of test suites to run, so we want to
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1142
    # run them in parallel.  We're going to spin up a number of worker threads
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1143
    # that will run our test jobs for us.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1144
    #
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1145
    input_queue = Queue.Queue(0)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1146
    output_queue = Queue.Queue(0)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1147
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1148
    jobs = 0
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1149
    threads=[]
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1150
5273
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1151
    #
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1152
    # In Python 2.6 you can just use multiprocessing module, but we don't want
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1153
    # to introduce that dependency yet; so we jump through a few hoops.
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1154
    #
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1155
    processors = 1
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1156
5459
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1157
    if sys.platform != "win32":
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1158
        if 'SC_NPROCESSORS_ONLN'in os.sysconf_names:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1159
            processors = os.sysconf('SC_NPROCESSORS_ONLN')
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1160
        else:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1161
            proc = subprocess.Popen("sysctl -n hw.ncpu", shell = True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1162
            stdout_results, stderr_results = proc.communicate()
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1163
            if len(stderr_results) == 0:
3c240ed38d99 try and make test.py a bit more portable
Craig Dowell <craigdo@ee.washington.edu>
parents: 5415
diff changeset
  1164
                processors = int(stdout_results)
5273
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1165
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1166
    #
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1167
    # Now, spin up one thread per processor which will eventually mean one test
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1168
    # per processor running concurrently.
c95bd39cc980 Remove multiprocessing dependency in test.py
Craig Dowell <craigdo@ee.washington.edu>
parents: 5257
diff changeset
  1169
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1170
    for i in range(processors):
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1171
        thread = worker_thread(input_queue, output_queue)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1172
        threads.append(thread)
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1173
        thread.start()
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1174
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1175
    #
5402
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1176
    # Keep track of some summary statistics
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1177
    #
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1178
    total_tests = 0
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1179
    skipped_tests = 0
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1180
b73b6da8ead6 teach test.py to skip selected tests
Craig Dowell <craigdo@ee.washington.edu>
parents: 5390
diff changeset
  1181
    #
4772
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1182
    # We now have worker threads spun up, and a list of work to do.  So, run 
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1183
    # through the list of test suites and dispatch a job to run each one.
7b6ae6bf0055 add test and validation framework
Craig Dowell <craigdo@ee.washington.edu>
parents:
diff changeset
  1184
    #