Testing of wgrib2             9/2015


I have had a script for testing wgrib2 for many years.
The script was never released because it depends on comparing
a good version of wgrib2 with a new, untested version.  Most
users are compiling wgrib2 for the first time and can't run
this script.

This is version 2 of the verify script.  It saves the results 
into a file so it can be used to test ports of wgrib2.  The code
has be rewritten from scratch so it looks like the code has an
intelligent design rather than it just evolved. 

The script will grow in time as more tests are added.  Someone,
not me, has the idea of adding a test for every option in wgrib2.
I am comfortable with adding new tests to as the need arises.

                              Usage

./verify2.sh [SAVE] [-wgrib2 filename] [-wgrib2new filename] [<RESULTS]
     if SAVE, create baseline using wgrib2, save results in RESULTS
     If !SAVE, test wgrib2new, compare results to RESULTS

   ex. to test ~/grib2/wgrib2/wgrib2

   cd (wherever)/verify
   ./verify2.sh -wgrib2new -wgrib2new ~/grib2/wgrib2/wgrib2 <RESULTS

   ex. To create a new RESULTS file using ~/bin/wgrib2

   cd (wherever)/verify
   ./verify2.sh -wgrib2 ~/bin/wgrib2 SAVE



                      Principle of Operations

The tests are run by running a command line and doing a check-sum
of stdout.  This allows to you check inventories and output files.

  ex. To check an inventory

        wgrib2 file | cksum

  ex. To check a binary output file
  
        wgrib2 file -bin - -inv /dev/null | cksum

  ex. To check a match command

        wgrib2 file -match "(UGRD|VGRD)" -grib - -inv /dev/null | cksum

The tests can be be run in three different ways, THREADING, EQV, VERF

                          THREADING Tests

The test is run with one and 3 threads (OpenMP).  The results
have to be identical.

                             EQV Tests

Often there is more than one way to get the same results.  For example

      example 1.  wgrib2 -match "(UGRD|VGRD)"
                  wgrib2 -match_fs "(UGRD|VGRD)"
                  wgrib2 -match "UGRD" -match "VGRD"
                  wgrib2 -match_fs "UGRD" -match_fs "VGRD"

      example 2.  wgrib2 -for 1:5:2
                  wgrib2 -match "^(1|3|5):"

The EQV test takes two command lines and makes sure that they get the
same results.

                        VERF (regression) Tests

When in save mode, the VERF (regression) tests runs the command line 
with the good version of wgrib2 (wgrib2) and writes the check sum to 
the file RESULTS.  When in testing mode, the command line is run 
with the wgrib2new and the check sum is compared the saved check sums.
The regression tests can fail but that may be fine.  For example, if
the grib table is changed, then the inventories may change.  It's
not a fatal error but needs to be examined.  Another example is
a conversion of a grib file to ieee.  Depending on the rounding,
the ieee files could be different.  Even a changing grib standard
can make a regression test fail.  For example, a recent change
allows the forecast hour to be negative.  Older versions of wgrib2
will return a different results when encountering a negative forecast
hour.

                            Test Files

There are three sets of test files.

    1. General grib files.
    2. Grib files supported by -new_grid
    3. Grib files with 3 or more grib messages