What is a *.idx file?

In the NODD archive, there are *.idx files.  These index
files are used by the "partial httd downloading" technique
to speed up downloads by only downloading the desired fields.

The *.idx files by are index files created by wgrib2.

    $ wgrib2 NAME.grb > NAME.idx

The *.idx are text files which list the contents of the
grib file.  A typical *.idx would look like

1:0:d=1950011000:DLWRF:surface:anl:ens mean
2:163798:d=1950011000:ULWRF:surface:anl:ens mean
3:307585:d=1950011000:DSWRF:surface:anl:ens mean
4:385591:d=1950011000:USWRF:surface:anl:ens mean
5:454943:d=1950011000:UGRD:10 m above ground:anl:ens mean
6:627805:d=1950011000:VGRD:10 m above ground:anl:ens mean
7:783805:d=1950011000:WIND:10 m above ground:anl:ens mean

Column

1    grib message number starting from 1
2    starting byte location starting from zero
3    d=YYYYMMDDHH, date code for the reference time
4    field name, ex TMP for temperature
5    level, ex surface
6    ftime, the timing information relative to the reference time
         ex. anl                = analysis, valic at the reference time
             0-3 hour ave fcst  = the 0-3 hour average of a forecast starting at the reference time
7+   extra information

To download the 4th grib message (USWRF:surface), you look at the location of the 4th
field (38559) and the 5th field (454943). So to ddownload the 4th field, you need to
download bytes 385590-454942.  The last number is one less than the start of the 5th
field.  Some servers allow you to download multiple byte ranges in one http request.  
Some servers only allow one byte range per http request. A some many not support partial
http downloading.  If the server supports partial http downloading, you can read the
*.idx file and determine the byte ranges to download.  This can save a considerable
amount of time in downloading.

                                 Software

This technique of using the wgrib/wgrib2 inventories has been in use since 2005.
So many software packages have their own implementation of this procedure.  For
NODD, the old perl scripts have been converted to python and customized for the
CORe archive on NODD.

   get_core.py       get core files from NODD
                     works on linux and windows with python3 installed