imagery Satellite Humidity And Instability product

Goal of iSHAI product

The name of version 2016 Clear Air Product Processor is GEO-iSHAI (imagery Satellite Humidity And Instability). It has been developed within the NWC SAF context, aiming to support nowcasting applications on clear air pixels.

GEO-iSHAI version 3.0 product aims to provide information about water vapour and instability distributions on clear air pixels using as main inputs MSG SEVIRI infrared channels and NWP fields. Products are generated at high spatial and temporal resolution to support real time meteorological applications.

The main outputs of GEO-iSHAI product are written in the iSHAI netCDF output file. The main outputs are:

1. Total Precipitable Water (TPW): Precipitable water in layer from surface pressure to top of atmosphere.
2. Layer Precipitable Water (LPW). Corresponding to the precipitable water in three layers:
2.a BL: Precipitable water in low layer [Psurface to 850 hPa]
2.b ML: Precipitable water in middle layer [850 to 500 hPa ]
2.c HL: Precipitable water in high layer [500 hPa to top of atmosphere]
3. Stability indices: they are calculated from the retrieved profiles of temperature and humidity. The calculated indices are:
3.a Lifted Index (LI)
3.b Showalter Index (SHW)
3.c K-index (KI)
4. Skin Temperature (SKT)
5. Total Ozone (TOZ). Note: This parameter is optional and must be activated by the users and the NWP GRIB files must contains ozone fields.
6. Besides the main outputs, the differences between the above parameters calculated from the retrieved profiles of temperature and humidity (and ozone profile if activated) with the parameters calculated from the spatial, temporal and vertical interpolated profiles from background NWP are written as other outputs (following 2007 Madrid Workshop recommendation). Thus the parameters diffTPW, diffBL, diffML, diffHL, diffLI, diffSHW, diffKI, diffSKT, diffTOZ are also written in the netCDF output file.

Besides the above main output parameters, GEO-iSHAI can be optionally configured to save input profiles, intermediate results and retrieved profiles in iSHAI optional binary files; these files can be useful for debugging purposes or to feed new user applications.

GEO-iSHAI products are useful in nowcasting applications, used in synergy with other data available to the forecaster. Since the physical basis of temperature and humidity retrieval is based in the minimization of the error between the SEVIRI BTs and the synthetic BTs calculated from the profiles and taking into account the limited number of channels and the spectral information of the SEVIRI instrument, GEO-iSHAI mainly improve the humidity profiles of the background NWP in middle and high levels. Despite of this, the retrieved fields have a higher spatial and tempoal resolution compared with the forecast. This fact must be taken into account by the users when the GEO-iSHAI outputs are used in nowcasting applications.

These web pages contain practical information on the use and configuration of GEO-iSHAI. The content has been extracted from the User Manual of GEO-iSHAI version v3.0 included in the NWC/GEO software package release 2016. Current documentation can be found at the NWC SAF Helpdesk web: http://www.nwcsaf.org.

It describes the characteristics, how to configure and how to execute the GEO-iSHAI processor; including the needed input files and the resulting outputs. A summary of definitions, acronyms and abbreviations can be seen here.

Summary of iSHAI algorithm

The iSHAI algorithm is a combination of one statistical and one optimal estimation algorithms and it has two main steps.

In the first step, the iSHAI First Guess profile is built using a set of non-linear regressions from collocated background NWP temperature and humidity profiles and bias corrected SEVIRI BTs.

In the second step, a physical retrieval algorithm (optimal estimation) is applied with some improvements over the classical approach. The improvements over the classical approach are: a) the use of EOFs to reduce the dimension of matrix and reduce the computation time (2 EOFs for T, 3 EOFs for q and 1 EOF for Tskin) and b) a regularization parameter.

The second step is only executed if the distance (BT_RMS) between BTSEVIRI and synthetic BTRTTOV (BTs calculated using RTTOV from FG profile) on the non-window channels (WV6.2, WV7.3 and IR13.4)) is greater than a configurable threshold (BT_RMS_THRESHOLD).

The Cloud Mask output is used to identify clear air pixels. Only if the pixel is labelled as clear air and the satellite zenith angle of this pixel lies within the configurable threshold (default 70º ), the processing is made.
A more detailed description of iSHAI algorithm is made here.
Full details of the algorithm can be found in "Algorithm Theoretical Basis Document for iSHAI Product” and in the “User Manual for the Clear Air Product Processor of the NWC/GEO: Science Part” document.

GEO-iSHAI algorithm remains based in Jun Li’s algorithm for legacy GOES Sounder physical retrieval algorithm (physical iterative approach with non-linear regression as first guess) but adapted for RTTOV and SEVIRI. Jun Li (CIMSS-Wisconsin) algorithm is also the proposed as Day-1 algorithm for the GOES-R processing by NOAA (Li, 2010).

GEO-iSHAI release 2016 has some major updates with respect to former PGE13 SPhR processor in version 2013. The summary of changes with respect to former version can be read here.

Description of GEO-iSHAI outputs

There are two kind of outputs:

The main outputs provided are parameters informing about the water vapour content in the vertical column (total and in selected layers) and about the atmospheric thermal instability. These parameters are computed using the final retrieved temperature and humidity profiles.

Since NWC/GEO release 2016, also the Skin Temperature (SKT) and Total Ozone (TOZ) are additional outputs.

As additional outputs, GEO-iSHAI also provides the differences between the above parameters, computed from the final retrieved profiles, and similar ones computed from the forecasted background profiles. These deviations are important for

The fields are now data fields only on clear pixels written as integer scaled fields. The NWCGEO netCDF has been designed in order to be standard. Thus, iSHAI netCDF files can be used with free and standard meteorological tools. As an example, all the iSHAI images in this web pages have been generated with the free and interactive McIDAS-V tool. Then, any user could use its own tools or use these free and standard meteorological tools to exploit iSHAI and NWCGEO netCDF files.

Optional iSHAI binary files outputs and 3D use.

Optionally, GEO-iSHAI can be configured to keep the final retrieved profiles and the intermediate profiles. This allows users:

Since GEO-iSHAI is executed locally by the users, the collocated temperature, specific humidity and ozone profiles with satellite data can be written in optional binary files at different steps of the algorithm. The main advantage to get spatial, temporal and vertically collocated (T, q, ozone) profiles with satellite data is to allow debugging activities or generating new parameters or stability indices or generation of 3D displays as vertical cross sections or 3D visualizations or generation of validation datasets. These applications are only a few among other applications that could be used.

Examples of the use of optional iSHAI binary files outputs with 3D tools can be seen below in visualization example chapter. More examples on the use of the GEO-iSHAI and former PGE13 SPhR binary file in nowcasting can be found in (Martinez, 2016), (Martinez, 2015), (Martinez, 2013), (Martinez, 2010) or (Martinez, 2010).

Examples on how to use the GEO-iSHAI binary files to generate validation dataset can be found in the Validation Report.

Description of inputs, coefficients and configurable parameters files

In this section a description of the inputs, coefficients files content and user configurable parameters is made.

The inputs and outputs scheme is shown in the Figure below. A more detatailed explanation of the implementation and execution steps can be found here.

Figure 1: GEO-iSHAI inputs and outputs scheme

GEO-iSHAI configurable parameters

A region configuration file and a GEO-iSHAI model configuration file (with the diverse parameters and options that are indicated by the user for calculation, are needed as mandatory parameters in order to execute the GEO-iSHAI command.

GEO-iSHAI has been designed in a highly modular manner and allows a user selectable configuration.

The user configurable parameters are determined through the GEO-iSHAI model configuration file. It is the third argument required by the GEO-iSHAI program. Thus, to change the configuration used to execute the GEO-iSHAI code is as easy as changing the third argument when the GEO-iSHAI is run. A more detailed description of the iSHAI configurable parameters can be found here.

Thus, the user can have several GEO-iSHAI model configurations files in the $SAFNWC/config and GEO-iSHAI can be executed on real time with different configurations depending on the region to process, the hour of the image, etc. As an example, users can execute GEO-iSHAI on the full disk with a large FOR width and with only one iteration step or, on the other side, to execute GEO-iSHAI over a small region centred in the area of interest with small FOR width and with up to three iterations.

Also users could choose between P or HYB mode using the keyword NWP_EXEC_MODE

Input files

The main inputs are: satellite data (in this case HRIT SEVIRI files), NWP data, a Cloud Mask for the region to process and some ancillary data. More detailed description is available here.

Coefficient files

Concerning the coefficients files a list and a description is available here; more specific scientific information about theoretical aspects can be found in the GEO-iSHAI Algorithm Theoretical Basis Document.

A list of GEO-iSHAI Product Assumptions and Limitations can be seen here.

Example of iSHAI visualisation

Graphic displays of GEO-iSHAI product parameters generated at the AEMET NWCSAF/MSG Reference System area are available on real-time in the web site of the NWC SAF Help Desk (http://www.nwcsaf.org).

For the display of the clear air outputs, a set of colour enhancement tables similar to the ones used by CIMSS for the visualisation of the GOES derived TPW and LI products has been selected.

In release 2016, the format of the file is netCDF and GEO-iSHAI outputs are not image like fields and they are data fields; this has the advantage to allow use in calculations of derived fields as temporal trends and spatial gradients.

Because of this, it is the responsibility of the user to use adequate tools to generate images similar to the ones in previous versions.

Among others, there are some free available tools as McIDAS-V or IDV or HDF-View that can do this using just the GEO-iSHAI netCDF files.

As commented in the iSHAI outputs description, to complete the image on cloudy pixels and avoid black holes in the iSHAI images the BT from IR10.8 channel has been added to GEO-iSHAI netCDF file. BTs from IR10.8 channel have been scaled in order to store it in 7 pixels, after scaling, and the values are in the range [0-127] only in cloudy pixels determined by the cloud mask (GEO-CMA product).

As one example, the process with a generic tool to generate an image as the ones shown in iSHAI images could be the following:

In the case of McIDAS-V, the mechanism to represent in a color palette the clear air pixels, and as grey, the cloudy pixels is based in to using a color palette for the parameters shown on clear pixels and to use a gray color palette on cloudy pixels and all pixels where the value is equal to missing code are considered transparent.

In other tools a mechanism to generate separately the clear and cloudy images with different value ranges and fixing to a reserved low value, the missing data value and combine it using a combined color palette with color for clear air range value and grey for cloudy range value could be needed.

Also it is possible to generate separated images for iSHAI fields and for the IR band (using a grey color palette) and later use programs like ImageMagick to declare as transparent the black or white pixels in the iSHAI images and make a composition of the transparent iSHAI images over the grey IR band image.

In the Figures below, the images corresponding to the case study of 10th August 2016 at 12UTC are shown as an example. The images have been generated with GEO-iSHAI v3.0 algorithm netCDF files and McIDAS-V. The images have been reprocessed using:

BL: Precipitable water in low layer [Psurface to 850 hPa]

diffBL: difference between iSHAI and ECMWF in BL

ML: Precipitable water in low layer [850 to 500 hPa]

diffML: difference between iSHAI and ECMWF in ML

HL: Precipitable water in low layer [500 to 0 hPa]

diffHL: difference between iSHAI and ECMWF in HL

TPW: Total Precipitable water

diffTPW: difference between iSHAI and ECMWF in TPW

KI: K-Index

diffKI: difference between iSHAI and ECMWF in KI

LI: Lifted-Index

diffLI: difference between iSHAI and ECMWF in LI

SHW: Showalter-Index

diffSHW: difference between iSHAI and ECMWF in SHW

TOZ: Total Ozone

diffTOZ: difference between iSHAI and ECMWF in TOZ

SKT: Skin Temperature

diffSKT: difference between iSHAI and ECMWF in SKT

Images of iSHAI main fields from 12 UTZ 10th of August 2016.

One of the best ways to exploit the water vapour and the stability indices generated by GEO-iSHAI software is to take advantage of the high spatial and temporal resolution of SEVIRI and to use loops of images to monitor their evolution. See a summary of the images and loops for the case study of 10th August 2016. More case studies will be incorporated when the new NWCSAF web server be operational.


A list of references is provided here.