Why to use?¶

Why and when should you use pyCMBS?

Geospatial data analysis ...

• you work with geospatial data
• you work a lot with netCDF files
• you analyze timeseries
• you do geostatistical analysis
• you want to generate nice looking maps
• you are bored in re-coding you own scripts, but want to increase your scientific throughput using a powerfull tool just using a few lines of code?

Model benchmarking ... the easy way!

• you are widely using models or are developing models?
• you want to know how good your model performs
• you search for a flexible tool for automated model performance metrics, comparing e.g. different models or model versions against control simulations or observational datasets

Some of the features pyCMBS provides for model benchmarking are

• automated and efficient pre-processing and regridding of model data and observations in a standardized way
• standard, modular and flexible libraries for plots and diagnostics
• automatic report generation with different levels of detail
• comparison against up to four different observational datasets at the same time
• user friendly configuration of plots and processing
• state-of-the-art model skill scores

Coding principles¶

pyCMBS is written purely in Python (2.7). The coding is based largely on the concept of the test driven development. The code has the following characteristics

• thorough testing using unittests
• readable python code, following PEP8 convention
• object oriented programming, flexibility and modularity
• pyCMBS is an open source project and is hosted on github

Operating systems¶

pyCMBS is developed purely in python and is expected to be in general independent from a special operating system. The current version was however only developed on Linux and no tests at all were made on other operating systems.

Dependencies¶

pyCMBS was built to have only a small number of external dependencies. However, there is a minimum number of dependencies existing which are a required for using pyCMBS sucessfully.

Core python packages [obligatory]

• python 2.7.x
• matplotlib >v1.3.x
• numpy
• scipy >0.11

Note that you should really ensure that your matplotlib installation is greater than version 1.3, as otherwise some features will not work. Standard packages like e.g. shipped as standard with Ubuntu have smaller version numbers (e.g. v1.1.1).

Compatability with python 3.x is not supported yet

Obligatory additional dependencies

For file I/O of netCDF files, a single library is supported at the moment:

• netCDF4 library is used as default for file I/O [recommended]

For an efficient data pre-processing the climate data operators are used. The core CDO’s and the corresponding python wrapper is required.

• climate data operators (cdo) [obligatory]
• cdo python interface [obligatory, can be easily installed using pip]

Recommended dependencies

For plotting projected data (map plots), pyCMBS currently supports two different plotting backends. These are

• cartopy [recommended]
• matplotlib basemap [optional]
• yaml [required if you want to use plugins or do benchmarking]

Installation of dependencies¶

For convenience, we have put together an installation procedure to install all required dependencies (except the ones needed for cartopy). Please find it here for your convenience.

An Installation checklist summarizing the required packages is available as well.

Quick installation from scratch for experts¶

If you have not yet any of the above dependencies installed and are working on a Debian like operating system (e.g. Ubunutu), the easiest way to install pyCMBS is by executing the installation sequence which can be found in the file .travis.yml in the root directory of the source code.

The developers are using automatic code checking and builds and the installation sequence in the file .travis.yml is used to setup for each build a working installation.

Detailed installation instructions for pyCMBS¶

In the following, we will summarize the different approaches to install pyCMBS.

pip install numpy
pip install pycmbs

mkdir temp_dir
cd temp_dir
tar -xvf pycmbs-vx.x.x.tar.gz

cd pycmbs-vx.x.x
python setup.py install

git clone https://github.com/your_user_name/pycmbs.git

# compile cython code
sh compile_extensions.py

# do installation
python setup.py install

# or as an alternative for developers, just set the PYTHONPATH
# environment variable to the pycmbs root directory and also adapt
# you systempath (PATH) such that includes the pycmbs rootdirectory
export PYTHONPATH=/my/path/to/pycmbs:$PYTHONPATH

Installation of dependencies¶

Please find here a working installation procedure, which was tested under Ubuntu 32-bit. It installs all pyCMBS dependencies, except the ones needed for cartopy and installs pyCMBS itself.:

#!/usr/bin/env bash

#
# This file provides an installing procedure for pyCMBS WITHOUT Cartopy support
#    it was tested for ubuntu32
#

# update package database
apt-get update

#####################################################################
# DEPENDENCIES
#####################################################################

# the -qq option installs silent using defaults
apt-get -qq install texlive-latex-base texlive-latex-extra texlive-latex-recommended
apt-get -qq install python-pip python-dev
apt-get -qq install cdo libhdf5-openmpi-dev libnetcdf-dev libopenmpi-dev
apt-get -qq install python-numpy
apt-get -qq install cython
C_INCLUDE_PATH=/usr/include/mpi pip install netCDF4

# apt-get -qq install python-matplotlib  # this gives the system default package, which is currently v1.1.1 therefore it is not used here
# it is highly recommended to use matplotlib > 1.3
apt-get -qq install libfreetype6-dev libpng-dev  # required for matplotlib
sudo easy_install -U distribute
sudo pip install https://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.3.1/matplotlib-1.3.1.tar.gz
apt-get -qq install python-mpltoolkits.basemap
apt-get -qq install python-mpltoolkits.basemap-data

apt-get -qq install python-scipy
pip install pyshp

#####################################################################
# pycmbs
#####################################################################
pip install --upgrade pycmbs

#####################################################################
# test environment
#####################################################################
pip install nose

echo "Now you can run the unittests as follows:"
echo "    cd /usr/local/lib/python2.7/dist-packages/pycmbs/tests"
echo "    nosetests"

Final check of installation¶

Check that installation worked properly by going through the following checklist. In case of problems, please refer to the troublesolver .

Is the pyCMBS python module loaded properly?:

python -c "from pycmbs import *; print('Welcome to pyCMBS')"

This should give you a short welcome message, but no error messages.

Is the benchmarking script working properly?:

pycmbs-benchmarking.py

This will you give a short message like:

*******************************************
* WELCOME to pycmbs.py                    *
* Happy benchmarking ...                  *
*******************************************

and will end with an error message that the configuration file is not found (this is o.k.)

If you see the above, the installation has worked! Congratulations!

3. Check also the proper installation of the cdo’s and the cdo.py interface, as this is a prerequesite of beeing able to properly work with pyCMBS:

python -c "from cdo import *; cdo=Cdo(); print 'If you see this, everything went right ... have fun with pyCMBS and CDOs'"

Again, this should give you a short welcome message. Any error message is a bad sign. In that case, please check your installation again. Have a look at the troublesolver.

pycmbs init

Running tests¶

pyCMBS code comes with a rich suite of test routines. We follow the concept of unittests using the nosetests tools. Tests should be always executed in the following cases:

• after installation
• before comitting code to the repository
• before merging a branch in the master branch

Tests can be simply executed using the Makefile in the main installation directory as:

make tests

As an alternative you can also check the coverage of tests in the code using:

make coverage

which gives you a report on test coverage in /coverage/index.html.

Further information and trouble solving¶

pyCMBS makes use of a standard directory to look for observations. This directory is the Standard Evaluation Pool (SEP). The path to the SEP directory needs to be specified in the $SEP environment variable. In you .bashrc write:

export SEP=/path/to/directory

For users at MPI-M, the SEP variable needs to point to /pool/SEP. It is however possible to specify also for each observation an individual path where the observation is located. Then the SEP evnironment variable is not required. To check whether SEP is set, type:

echo $SEP

python -c "print 'Hello world'"

echo $PYTHONPATH

pycmbs-benchmarking.py

echo $PATH

Ipython notebook examples¶

The demo folder contains a set of ipython notebooks with code examples. If you have not installed ipython notebooks yet, the following examples can be also directly viewed:

• pyCMBS basics
• Analysis of the global temperature record
• Comparison of two dataset
• Example of plotting a portraet diagram

Gallery¶

The pyCMBS gallery provides some additional simple examples

Source code examples and documentation¶

The source code contains comprehensive documentation and also examples as docstrings. This can be read using the standard python doc() and help() functionalities.

Basic plotting¶

Read some sample data and just plot it:

from pycmbs.mapping import map_plot
import matplotlib.pyplot as plt
from pycmbs.examples import download
import numpy as np

# Read some sample data ...
air = download.get_sample_file(name='air')
air.label = 'air temperature'

# a quick plot as well as a projection plot
f1 = map_plot(air)  # unprojected
f2 = map_plot(air, use_basemap=True)  # projected
plt.show()

For plotting you have a rich suite of keyword parameters. Please use the python help() system to get full documentation from docstrings. Some more illustrations of options are provided:

map_plot(air,show_zonal=True, use_basemap=True,title='show_zonal=True')

And a few more details on customizing your map ...:

map_plot(air, use_basemap=True, title='vmin=-30.,vmax=30.,cmap_data=RdBu_r', vmin=-30., vmax=30., cmap_data='RdBu_r', ax=ax1)
map_plot(air, show_stat=True, use_basemap=True,title='show_stat=True',ax=ax3)
map_plot(air, show_stat=True, stat_type='median', use_basemap=True, title='show_stat=True,stat_type="median"', ax=ax4)

Your first benchmarking¶

The main input for model benchmarking is model data and observational datasets as well as a user specified configuration. After running the benchmarking, you get a report (PDF format) as well as a lot of figures and statistics which contain usefull information. The benchmarking is based ona modular approach and allows the user to customize the results by activating and deactivating specific components.

The next steps will guide you through a benchmarking session to get you started.

1. set up a working directory and change to it:

   # set up a working directory somewhere on your machine
   mkdir -p ~/temp/my_first_benchmarking
   cd ~/temp/my_first_benchmarking
   

2. set up an initial configuration:

   # simply run the pycmbs-benchmarking.py script
   # if properly installed, you should be able to just run it from the console
   pycmbs-benchmarking.py init
   

This gives you:

$ ls
configuration  template.cfg

If you do this for the first time, then it is recommended that you make yourself familiar with the content of the configuration directory. This contains

• INI files for each variable which specify the plot and processing configuration
• .json files which specify interface routines for data conversion

3. adapt the configuration file (xxx.cfg) to your needs. The configuration file is the center part where you specify

   1. which variables shall be diagnosed
   2. which models shall be analysed

   Details about the configuration file are specified below.

4. Do it! Run the benchmarking now by executing:

   pycmbs.py your_config_script.cfg
   

Benchmarking components¶

The benchmarking part of pyCMBS is based on a modular system of diagnostics. Currently the focus is on the comparison of climate mean states between observations and models. Below are examples of the currently implemented components which the user can combine to generate a specific report for a certain variable.

Plot configuration details¶

For each variable, a configuration file with the extension .ini is used to specify. The INI files are expected to be located in a directory which is specified in the main configuration file (.cfg). The plot configuration file specifies for each variable

• which diagnostics should be applied to a certain variable
• how plots for a particular diagnostic should look like (e.g. colorbars, limits ...)
• which observational datasets should be used

An INI file has two major parts:

1. Global plot options
2. Observation specific plot options (for each used observational dataset)

[OPTIONS]
map_difference =  True
map_seasons    =  True
map_season_difference = False
reichler_plot  =  True
gleckler_plot   =  True
hovmoeller_plot   =  False
regional_analysis = True
global_mean    = True
vmin           =  0.
vmax           =  8.
dmin           =  -1.
dmax           =  1.
units          =  $mm/day$
label          =  Daily evaporation
cticks         = [0.,2.,4.,6.,8.,10.]
nclasses       = 8
preprocess     = True
interpolation  = conservative
targetgrid     = t63grid
projection     = robin
region_file    = /home/m300028/shared/data/CMIP5/evap/evspsbl/merged/dummy_mask2.nc
region_file_varname = regmask

[CLARASAL]
obs_file =  #get_data_pool_directory() + 'data_sources/CMSAF/CLARA-SAL/DATA/SAL_all_t63.nc'#
obs_var  =  sal
scale_data = 0.01
gleckler_position = 2
add_to_report = True
valid_mask = land

How to add new variables for an already existing model output format?¶

A particular model output format is represented in pyCMBS by its own class. For an already existing model class, one needs to implement a reader for each variable. In principle, this is just a small subroutine that has the logic implemented how to properly read the data. Typically this requires:

• generation of filename dependent on e.g. experiment ID
• reading data into a Data object

To implement a new variable reader, there are several ways. You can either implement one routine per variable or make use of generic I/O routines (see routine get_model_data_generic in modely.py)

How to add new observational datasets?¶

The integration of new observational datasets is very simple as long as the datasets you use follow some standard conventions:

• datasets are in netCDF format
• optional: datasets have metadata attributes in the netCDF file. pyCMBS is making automatically use of CF conventions like netCDF attributes like scale_factor, add_offset, _FillValue, units. In case these attributes are provided, they are automatically used
• lat/lon coordinates are provided either as vectors (for simple lat/lon projections) or as 2D fields (a (lat, lon) tuple for each grid cell).
• observations are stored in a single file (all timesteps included)

Steps to integrate a new observational dataset into pyCMBS are as follows:

• decide for the variable the observational dataset belongs to –> variable name; you can look in the configuration file (.cfg) to get the currently supported variable names
• modify the corresponding INI file

• let’s say, that you have chosen sis (surface solar irradiance) as the variable and you have a new surface radiation dataset. Then the corresponding INI file would be sis.ini. The INI files can be found in the configuration folder.

• You can however also generate an own, new configuration folder, by simply typing pycmbs.py init in a fresh directory

• The content of the INI file is self explanatory. You have a global section which specifies how the analysis for this particular variable shall be made (e.g. which diagnostics and plots shall be generated). Below, you have for each observational dataset a section which specifies the details for each observation. Such a section looks e.g. like the following:

  [CERES]
  obs_file = #get_data_pool_directory() + 'data_sources/CERES/EBAF/ED_26r_SFC/DATA/CERES_EBAF-Surface__Ed2.6r__sfc_sw_down_all_mon__1x1__200003-201002.nc'#
  obs_var  = sfc_sw_down_all_mon
  scale_data = 1.
  gleckler_position = 2
  add_to_report = True
  valid_mask = global
  

The different entries have the following meaning

[id] : str

id of observational dataset. Will be used e.g also as labels for plots

obs_file : str

path to the netCDF file with the observations here you can either specify an absolute path or you can out a python code snipped between two hashes ‘#’. The latter approach is usefull, when you have some function that directs to a particular directory, like in the example given. Otherwise, the easiest way is to just put the absolute path to the netCDF file.

obs_var : str

specifies the name of the variable in the netCDF file

scale_data : float

a multiplicative scaling factor that is applied to the data when it is read

gleckler_position : int

[1...4]; This number specifies, where the observational dataset will be placed in the portraet diagram (???)

add_to_report : bool

[True,False], specifies if the observational dataset should be included in the report or not. This allows to have a lot of configurations in the INI file, but use only a few of them.

valid_mask : str

[land,ocean,global], specifies which area(s) are supposed to contain valid data. Other regions are automatically masked. Thus if you specify e.g. land, then the ocean will be masked. It is important, that you use a

Adding a new observation is as simple as copy/paste an already existing section and modify the entries like you need it. That’s it ... well at least on a technicla point. If everything is working properly and if the diagnostics you want to apply for this observational dataset are usefull is a different question.

How to integrate new variables in the analysis?¶

To add new variables in pyCMBS implies the following steps:

1. Define I/O routine: Implement for each model class that shall support the new variable a routine that allows to read the data. Let’s say you have a variable sis, then you would need e.g. to implement a routine get_sis() for the CMIP5RAW model class. Note that there is already a routine which can be used for generic I/O.

2. Register I/O routine: After you have implemented the routine to read the data, you need to let the program know about it. All data is read using a routine called get_data(). This routine gets the information which subroutines to call from details provided in a configuration file. The configuration file is found in:

./configuration/model_data_routines.json

The file is a simple JSON dictionary. Make yourself a bit familar with the structure and it should not be a problem to implement your new routine there.

3. Analysis script: Now you have the analysis script that can be used to read the data. However, you still need to tell pycmbs how to make use of this new information. This you do by implementing an analysis routine in analysis.py. For most variables supported so far, this analysis routine is just a wrapper which is calling a very generic analysis routine that basically does everything you tell it to do. What to do is specified in the INI files for each variable. Note however, that you are free to do what you want and you can implement a new analysis routine which is doing right the thing you want it to do.

4. Last step is to tell pycmbs that the analysis script you implemented is existing. This is again done, by simply registering it in the following file:

./configuration/analysis_scripts.json

5. [optional] Test environment As a best practice, you should then also integrate a unittest for this new variable. Details are specified in the section Testing the benchmarking framework (experts only).

How to add a new model format?¶

Each model is represented by its own class which is herited from the Model class. Each model object/class needs to support reading of the variables that should be used for the analysis. Please look at the current code in models.py to see how the actual implementation can look like. Important is a coherent support of the routines which are importing data from files for a model specific file format.

Project website¶

• pyCMBS on github (public development project)

Publications where pyCMBS has been used¶

• Brovkin, V. et al., 2013. Evaluation of vegetation cover and land-surface albedo in MPI-ESM CMIP5 simulations. Journal of Advances in Modeling Earth Systems, 5(1), pp.48-57. doi:10.1029/2012MS000169
• Hagemann, S., Loew, A. & Andersson, A., 2013. Combined evaluation of MPI-ESM land surface water and energy fluxes. Journal of Advances in Modeling Earth Systems, 5(2), pp.259-286. doi:10.1029/2012MS000173
• Loew, A. et al., 2013. Potential and limitations of multidecadal satellite soil moisture observations for selected climate model evaluation studies. Hydrology and Earth System Sciences, 17(9), pp.3523-3542. doi:10.5194/hess-17-3523-2013
• Loew, A., 2013. Terrestrial satellite records for climate studies: how long is long enough? A test case for the Sahel. Theoretical and Applied Climatology. doi:10.1007/s00704-013-0880-6

Projects where pyCMBS has been used¶

• ESA GlobAlbedo
• EvaCliMod (German Weather Service)
• ESA Climate Change Initiative (European Space Agency)