This file is part of IVRE.

Copyright 2011 - 2014 [[mailto:pierre.lalet@cea.fr|Pierre LALET]]

====== Why should I test IVRE? ======

The ''%%tests%%'' directory is mainly intended for people who want to contribute to IVRE and want to make sure their changes do not break IVRE.

====== The tests ======

===== Dependencies =====

To run IVRE tests you will need [[http://nedbatchelder.com/code/coverage/|coverage.py]].

===== Test case =====

The first thing is to find samples. You need both (recent) Nmap XML scan result files and PCAP files (dump while you browse, and browse a lot, or sniff a busy open Wi-Fi network, if that's legal in your country).

A good test case should have a lot of various data (sniff from different places, scan different hosts with different Nmap options).

It is mandatory to have at least, for the Nmap test:

  * Two scanned (and up) hosts with different IP addresses
  * One host scanned with the script ''%%http-robots.txt%%'' reporting ''%%/cgi-bin%%'' in its output.
  * One host scanned with an anonymous FTP server.
  * One scan result with traceroute and at least one hop with a hostname.
  * One host scanned with a hostname ending with ".com".

For the passive test:

  * Two records with different IP addresses.
  * One SSL certificate.

===== First run =====

From the ''%%tests%%'' directory, create the ''%%samples%%'' subdirectory and place your samples there (the PCAP files must have the extension ''%%.pcap%%'', and the Nmap XML result files must have the extension ''%%.xml%%'').

Then, run ''%%./run%%''. The first run will create a ''%%samples/results%%'' file with the expected values for some results. The runs after will use those values to check if something has been broken.

For this reason, it is important to:

  * Run the tests for the first time with a "known-working" version.
  * Remove the file ''%%samples/results%%'' whenever a sample file is added, modified or removed.

===== Improving the test case =====

If you want to make sure to have enough samples, you can:

  * Check the ''%%samples/results%%'' file for ''%%*_count%%'' entries with low values (particularly 0, of course) and find or create new samples that will improve those values.
  * Check the report generated by coverage.py under the ''%%htmlcov%%'' directory, and check whether your current test case covers at least the code you want to change.

===== Failures =====

Tests failure are not always an issue. Apart from a new bug, of course, here are some reasons that can explain test failures:

  * You have added new samples and have not removed the ''%%samples/results%%'' file.
  * Your samples do not match the minimum requirements detailed above.
  * A new feature has been added to IVRE and the new results are actually better than the stored ones.

