From f5c4408ccde387c00881abe0dc2ceef41df5ca5b Mon Sep 17 00:00:00 2001 From: Alberto Bertogli Date: Sat, 12 Sep 2009 00:57:54 -0300 Subject: [PATCH 10/14] Document how the library is tested Signed-off-by: Alberto Bertogli --- tests/README | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 50 insertions(+), 0 deletions(-) create mode 100644 tests/README diff --git a/tests/README b/tests/README new file mode 100644 index 0000000..2e11857 --- /dev/null +++ b/tests/README @@ -0,0 +1,50 @@ + +The library is normally tested using several tools, which are described below. + +Each commit must pass at least the behaviour and stress tests successfully, +invasive commits must be tested with the ones that are most likely affected by +it, and releases must be checked with all the tests. + +In any case, remember that testing is not a replacement for careful code +inspection. + + +Behaviour tests: + Check how the library behaves in different situations, from basic ones to + weird crash scenarios, using fault injection. They can be found in the + behaviour/ directory. + +Stress test: + It's a randomized stress test that performs different transactions and + checks the result was the one expected. Can also run with randomized fault + injection both in libjio's predetermined points and in POSIX functions. It + can be found in the stress/ directory. + +Performance tests: + Check the performance of simple operations like streaming and randomized + writes. Not the most interesting tests, but can be useful for profiling. + +Valgrind: + Run the other tests under valgrind and see there are no libjio-related + issues. Performance tests are the easier ones, sadly behaviour and stress + tests are more painful because Python makes valgrind emit lots of warnings, + but it's definitely worth the effort and should be done frequently. + +Code coverage: + Use gcov and lcov to check that the behaviour and stress tests cover most of + the code. Currently, it's over 90%, and it shouldn't go down. However, this + test shouldn't be given more relevance that the one it deserves, since it's + mostly useful to see if we're missing some real scenario in the other tests. + +Profiling: + Use kcachegrind and prof (the Linux Kernel tool) to check for suspicious + bottlenecks. + +Static checkers: + Build using sparse (make CC=cgcc) and clang (make CC=ccc), and run cppcheck. + +Portability: + Build in some different architectures. At least Linux and some BSD must + be checked before releases. + + -- 1.6.2.2.646.gb214