Unit Tests
Software testing is important for a complex package such as PyFlag. Rigorous tests ensure a number of ends:
- Basic functionality of each component is tested in isolation as well as in combination with the whole of the package.
- The API of using each component is documented through a test cases.
PyFlags testing regime
PyFlag uses a system of tests based on python's unit test framework. Tests can be dispersed through the plugin code and will be run by the test harness in the utilities directory:
./tests/launch python2.4 utilities/Tester.py --------------------------------------- Running tests in <class 'pyflag.DB.DBOTest'> --------------------------------------- Test to make sure we can locate the pyflag default database ... ok Test to make sure DBO temporary tables get cleaned up after handle gc ... ok Test to ensure that dbhs reconnect after an aborted server side connection ... ok Test to make sure slow queries are aborted ... Warning: Killing query in thread 26 because it took too long ok Test the mass insert mechanism ... ok Test that query caching works properly ... ok ---------------------------------------------------------------------- Ran 6 tests in 1.340s
Writing Unit tests
Unit tests are classes defined in any file within the plugins directory which is derived from unittest.TestCase. There are a number of derived classes which PyFlag uses to write specific kinds of tests.
For example:
1 ## Unit Tests
2 import unittest
3
4 class DBOTest(unittest.TestCase):
5 order = 1
6 def test01validinstall(self):
7 """ Test to make sure we can locate the pyflag default database """
8 dbh = DBO(None)
9 dbh.execute("show tables")
10 tables = [ row.values()[0] for row in dbh ]
11 self.assert_( 'meta' in tables)
12
13 def test02TemporaryTables(self):
14 """ Test to make sure DBO temporary tables get cleaned up after handle gc """
15 dbh = DBO(None)
16 tablename = dbh.get_temp()
17 dbh.execute("create table %s(field1 text)", tablename)
18 dbh.execute("select * from %s", tablename)
19 result = [ row['field1'] for row in dbh ]
20 self.assertEqual(result, [])
21
22 dbh2 = DBO(None)
23 tablename2 = dbh2.get_temp()
24 self.assert_(tablename2 != tablename)
25
26 del dbh
27 def ExceptionTest():
28 dbh = DBO(None)
29 dbh.execute("select * from %s", tablename)
30
31 self.assertRaises(DBError, ExceptionTest)
The following observations can be made:
- Unit tests are run in the order specified by the order attributes. Your test should be set contained (i.e. it should be possible to run your test on a new installation with out any effects - so you should not rely on the order that tests are run).
- Tests are methods starting with "test". They are ordered alphabetically. (For more information see Python's unit test doco).
- Tests must be designed with well defined outcomes which produce a pass/fail outcome. This can then be tested using self.assert_, self.failIf etc (see unittest documentation for all the methods).
A number of helpful classes are already written in the module pyflag.tests:
- The FDTest class is designed to test a VFS File driver - it checks to ensure that the driver present the required file like behaviour.
ScannerTest class is a convenience test which loads a filesystem automatically. You should then write a test method which actually scans it using the scanner under test, and checks for the expected results.
Pre-requisites for tests
Most of PyFlag's tests are designed to operate on test cases designed to exercise as much functionality as possible. These test images are quite large and sometimes do get updated. To make management of the test set easier we have a script "utilities/test_repo.py".
The script downloads an inventory file, which contains a set of md5 hashes of the test images. It then checks the test directory for the presence of the images, and their md5 sums. If the sums are different, the script downloads a new version of the image. The script will transparently take care of compressed images, and can decompress them, or convert to a number of image formats. The script is also able to download other images (which are not hosted on www.pyflag.net).
Its quite safe to run the script multiple times, existing files will not be downloaded again.
/testimages$ python utilities/test_repo.py 2003(Debug): Fetching inventory 2003(Debug): Done 2003(Debug): Fetching pyflag_stdimage_0.4.e01 2003(Debug): Done 2003(Debug): EWF Decompressing pyflag_stdimage_0.4.e01 2003(Debug): Done 2003(Debug): Fetching stdcapture_0.4.pcap.e01 2003(Debug): Done 2003(Debug): Fetching pyflag_apache_standard_log.gz 2003(Debug): Done 2003(Debug): Fetching pyflag_iis_standard_log.gz 2003(Debug): Done 2003(Debug): Fetching ntfs_image.e01 ....
The test files path can be set using the --target command line option
