diff options
author | Johnny Chen <johnny.chen@apple.com> | 2010-10-19 21:46:48 +0000 |
---|---|---|
committer | Johnny Chen <johnny.chen@apple.com> | 2010-10-19 21:46:48 +0000 |
commit | 843429b0e7de6381b668d0b6d85886368a1e4da7 (patch) | |
tree | 2b6c693ff420ed7d1c96b0310b9e664030eaa975 /docs | |
parent | 44820ec9b49bc08c0f4529091019b5bc95ec237a (diff) | |
download | lldb-843429b0e7de6381b668d0b6d85886368a1e4da7.tar.gz |
Finish the first draft of a detailed walkthrough of an lldb test method.
git-svn-id: https://llvm.org/svn/llvm-project/llvdb/trunk@116865 91177308-0d34-0410-b5e6-96231b3b80d8
Diffstat (limited to 'docs')
-rw-r--r-- | docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log | 43 | ||||
-rw-r--r-- | docs/testsuite/a-detailed-walkthrough.txt | 142 |
2 files changed, 184 insertions, 1 deletions
diff --git a/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log b/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log new file mode 100644 index 000000000..e18199537 --- /dev/null +++ b/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log @@ -0,0 +1,43 @@ + +os command: [['/bin/sh', '-c', 'make clean; make']] +stdout: rm -rf "a.out" "a.out.dSYM" main.o main.d +g++ -arch x86_64 -gdwarf-2 -O0 -c -o main.o main.cpp +g++ -arch x86_64 -gdwarf-2 -O0 main.o -o "a.out" +/usr/bin/dsymutil -o "a.out.dSYM" "a.out" + +stderr: None +retcode: 0 + + +runCmd: file /Volumes/data/lldb/svn/trunk/test/settings/a.out +output: Current executable set to '/Volumes/data/lldb/svn/trunk/test/settings/a.out' (x86_64). + + +runCmd: settings set target.process.output-path 'stdout.txt' +output: + +runCmd: settings show target.process.output-path +output: target.process.output-path (string) = 'stdout.txt' + + +Expecting start string: target.process.output-path (string) = 'stdout.txt' +Matched + +runCmd: run +output: Process 43533 launched: '/Volumes/data/lldb/svn/trunk/test/settings/a.out' (x86_64) + + +FAIL + +runCmd: process kill +check of return status not required +runCmd failed! +error: Process must be launched. + + +Traceback (most recent call last): + File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 125, in test_set_output_path + "'stdout.txt' exists due to target.process.output-path.") +AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path. + + diff --git a/docs/testsuite/a-detailed-walkthrough.txt b/docs/testsuite/a-detailed-walkthrough.txt index 12543baff..b29fcef28 100644 --- a/docs/testsuite/a-detailed-walkthrough.txt +++ b/docs/testsuite/a-detailed-walkthrough.txt @@ -161,4 +161,144 @@ the specified command to the lldb command interpreter. It's like you're typing the command within an interactive lldb session. The CURRENT_EXECUTABLE_SET is an assert message defined in the lldbtest module -so that it can reused from other test modules. +so that it can be reused from other test modules. + +By default, the runCmd() is going to check the return status of the command +execution and fails the test if it is not a success. The assert message, in our +case CURRENT_EXECUTABLE_SET, is used in the exception printout if this happens. + +There are cases when we don't care about the return status from the command +execution. This can be accomplished by passing the keyword argument pair +'check=False' to the method. + +After the current executable is set, we'll then execute two more commands: + + # Set the output-path and verify it is set. + self.runCmd("settings set target.process.output-path 'stdout.txt'") + self.expect("settings show target.process.output-path", + SETTING_MSG("target.process.output-path"), + startstr = "target.process.output-path (string) = 'stdout.txt'") + +The first uses the 'settings set' command to set the static setting +target.process.output-path to be 'stdout.txt', instead of the default +'/dev/stdout'. We then immediately issue a 'settings show' command to check +that, indeed, the setting did take place. Notice that we use a new method +expect() to accomplish the task, which in effect issues a runCmd() behind the +door and grabs the output from the command execution and expects to match the +start string of the output against what we pass in as the value of the keyword +argument pair: + + startstr = "target.process.output-path (string) = 'stdout.txt'" + +Take a look at TestBase.expect() within lldbtest.py for more details. Among +other things, it can also match against a list of regexp patterns as well as a +list of sub strings. And it can also perform negative matching, i.e., instead +of expecting something from the output of command execution, it can perform the +action of 'not expecting' something. + +This will launch/run the program: + + self.runCmd("run", RUN_SUCCEEDED) + +And this asserts that the file 'stdout.txt' should be present after running the +program. + + # The 'stdout.txt' file should now exist. + self.assertTrue(os.path.isfile("stdout.txt"), + "'stdout.txt' exists due to target.process.output-path.") + +Also take a look at main.cpp which emits some message to the stdout. Now, if we +pass this assertion, it's time to examine the contents of the file to make sure +it contains the same message as programmed in main.cpp: + + # Read the output file produced by running the program. + with open('stdout.txt', 'r') as f: + output = f.read() + + self.expect(output, exe=False, + startstr = "This message should go to standard out.") + +We open the file and read its contents into output, then issue an expect() +method. The 'exe=False' keyword argument pair tells expect() that don't try to +execute the first arg as a command at all. Instead, treat it as a string to +match against whatever is thrown in as keyword argument pairs! + +There are also other test methods present in the TestSettings.py mode: +test_set_prompt(), test_set_term_width(), test_set_auto_confirm(), +test_with_dsym(), and test_with_dwarf(). We are using the default test loader +from unittest framework, which uses the 'test' method name prefix to identify +test methods automatically. + +This finishes the walkthrough of the test method test_set_output_path(self). +Before we say goodbye, notice the little method definition at the top of the +file: + + @classmethod + def classCleanup(cls): + system(["/bin/sh", "-c", "rm -f output.txt"]) + system(["/bin/sh", "-c", "rm -f stdout.txt"]) + +This is a classmethod (as shown by the @classmethod decorator) which allows the +individual test class to perform cleanup actions after the test harness finishes +with the particular test class. This is part of the so-called test fixture in +the unittest framework. From http://docs.python.org/library/unittest.html: + +A test fixture represents the preparation needed to perform one or more tests, +and any associate cleanup actions. This may involve, for example, creating +temporary or proxy databases, directories, or starting a server process. + +The TestBase class uses such fixture with setUp(self), tearDown(self), +setUpClass(cls), and tearDownClass(cls). And within teraDownClass(cls), it +checks whether the current class has an attribute named 'classCleanup', and +executes as a method if present. In this particular case, the classCleanup() +calls a utility function system() defined in lldbtest.py in order to remove the +files created by running the program as the tests are executed. + +This system() function uses the Python subprocess module to spawn the process +and to retrieve its results. If the test instance passes the keyword argument +pair 'sender=self', the detailed command execution through the operating system +also gets recorded in a session object. If the test instance fails or errors, +the session info automatically gets dumped to a file grouped under a directory +named after the timestamp of the particular test suite run. + +For simple cases, look for the timestamp directory in the same directory of the +test driver program dotest.py. For example, if we comment out the +@expectedFailure decorator for TestSettings.py, and then run the test module: + +/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v settings +---------------------------------------------------------------------- +Collected 6 tests + +test_set_auto_confirm (TestSettings.SettingsCommandTestCase) +Test that after 'set auto-confirm true', manual confirmation should not kick in. ... ok +test_set_output_path (TestSettings.SettingsCommandTestCase) +Test that setting target.process.output-path for the launched process works. ... FAIL +test_set_prompt (TestSettings.SettingsCommandTestCase) +Test that 'set prompt' actually changes the prompt. ... ok +test_set_term_width (TestSettings.SettingsCommandTestCase) +Test that 'set term-width' actually changes the term-width. ... ok +test_with_dsym (TestSettings.SettingsCommandTestCase) +Test that run-args and env-vars are passed to the launched process. ... ok +test_with_dwarf (TestSettings.SettingsCommandTestCase) +Test that run-args and env-vars are passed to the launched process. ... ok + +====================================================================== +FAIL: test_set_output_path (TestSettings.SettingsCommandTestCase) +Test that setting target.process.output-path for the launched process works. +---------------------------------------------------------------------- +Traceback (most recent call last): + File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 125, in test_set_output_path + "'stdout.txt' exists due to target.process.output-path.") +AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path. + +---------------------------------------------------------------------- +Ran 6 tests in 8.219s + +FAILED (failures=1) +/Volumes/data/lldb/svn/trunk/test $ ls 2010-10-19-14:10:49.059609 +TestSettings.SettingsCommandTestCase.test_set_output_path.log +/Volumes/data/lldb/svn/trunk/test $ + +We get one failure and a timestamp directory 2010-10-19-14:10:49.059609. +For education purposes, the directory and its contents are reproduced here in +the same directory as the current file. |