Reader Cache Library
Introduction and Overview
This Technical Note covers the concepts and the implementation details of the current (SNAP 13 and higher) reader caching library implementation used in the SNAP core product model.
Overview and Goal
The multiple ways of accessing product data stored on a hard-drive or remotely on cloud infrastructures are challenging to handle. The standard approach of the SNAP engine/desktop/GPT is to access and cache data in tiles of appropriate dimension with respect to the task being executed. These are configurable entities, using
snap.dataio.reader.tileHeight=512
snap.dataio.reader.tileWidth=512
snap.jai.defaultTileSize=512
This definition of tile sizes may not always be appropriate for the storage layout of the raw data, although a ProductReader can state the preferred tile sizes for the product, which will be considered.
This possible mismatch of tile sizes and storage layouts gets even worse in performance when the original data is stored as compressed data-blocks. It has been observed that performance issues arise from disk-blocks being loaded and decoded many times when SNAP-tiles overlap multiple disk-blocks.
The ReaderCachingLibrary has been developed to act as a bridging component between these two realms. It implements reading access to the raw data that is optimal for the storage layout and caches these data. Mapping the data to the tiles requested by the SNAP framework is handled in-memory and thus much faster.
Functionality
The cache library is composed of a central component that keeps track of the data allocated and ensures that data is released when a configurable threshold is reached. This is the CacheManager class. Each product reader using cache library functionality registers a ProductCache class at the CacheManager and de-registers on product close operation. This ProductCache is a container for variable specific caches which are created and removed on demand, i.e. when starting to read data from a variable that is not cached already.
The variable specific caches, which are objects of VariableCache, implement the caching- and mapping functionality that increases the reading performance.
On creation of a VariableCache, the class requests information about the variable from the ProductReader: data type, dimensions and tiling dimensions. This information is used to set up an empty cache structure for the specific variable.
When requested to deliver a certain area of data to a buffer the VariableCache does the following:
Identify the cache buffers affected by the request
Check if these are already kept in memory
If not -> load the missing data by calling the ProductReader
Copy the intersections between the cache buffer data and the data region requested to the appropriate target buffer location
Return the target buffer
Each allocation of a cache block triggered by the VariableCache is recorded at the CacheManager; if a threshold is hit, the CacheManager starts disposing data.
Data is always read as entire chunk following the dimensions stated by the ProductReader during the initialization phase. Thus, the access to raw data is always optimal for the storage medium of the data and compressed data is only de-compressed once.
Classes involved
CacheManager
This class implements the central cache coordination unit. It is a lazy loaded singleton which is instantiated by the first request to access it:
CacheManager.getInstance()
and it should be disposed when the process ends, e.g in a shutdown hook.
CacheManager.dispose()
Its functionality is controlled by two parameters:
· memoryLimit – the upper limit of data in bytes that is allowed to be allocated by all products in current process. Allocations beyond this limit trigger release operations on the cached data.
· disposeThreshold – this value defines the minimal size in bytes that the allocation tasks can overshoot before the release process is triggered.
Both parameters can be adapted either using the API:
public void setMemoryLimit(long memoryLimit)
public void setDisposeThreshold(long disposeThreshold)
or using the SNAP configuration files (e.g. snap.properties)
snap.dataio.cache.memoryLimit
snap.dataio.cache.disposeThreshold
Disposing data when the upper limit is reached is executed by sorting all ProductCaches registered by access time in an ascending order. This is list is iterated from oldest to newest, asking each object to release data. Each ProductCache responds with the amount of memory released; the iteration stops when the requested amount of memory is freed.
ProductCache
Is responsible for managing the cached data for a single product. It interfaces with a product reader using a method similar to the readBandRasterDataImpl() defined in the AbstractProductReader:
public ProductData read(String bandName, int[] offsets, int[] shapes, DataBuffer targetBuffer)
It creates and maintains VariableCache instances that implement access to the raw data. It also controls the data release procedures when called by the CacheManager to free data.
VariableCache
This class implements the caching functionality. It creates and keeps track of a variable-specific caching structure that consists of cache blocks of dimensions that are ideal for the underlying data. It exists in two variations, one for two-dimensional and one for three-dimensional variable storage.
It is responsible for loading data from the ProductReader if required, copying the cached data to the appropriate location in the target buffer and releasing data if requested. It keeps track of the access times for each cache block and can export its access times for cache management.
CacheDataProvider
This interface defines the contract between the cache library and the ProductReader. It consists of two methods:
VariableDescriptor getVariableDescriptor(String variableName)
To operate successfully, the cache requires information about the storage layout for the variables to be cached. This method provides the required information which is used to initiate the cache for each variable being read.
DataBuffer readCacheBlock(String variableName, int[] offsets, int[] shapes,
ProductData targetData)
The cache uses this method to actually read data into memory. The offset and shape parameters define the region of the raster to be read. These always match the boundaries that have been reported to be useful for the variable in the associated VariableDescriptor.
Integration into ProductReader
The ProductReader keeps the cache as instance variable. Ideally, it is created and registered with the CacheManager during the creation of the Product, i.e. in readProductNodesImpl():
productCache = new ProductCache(this);
CacheManager.getInstance().register(productCache);
Consequently, the cache is removed in the close() method of the ProductReader:
if (productCache != null) {
CacheManager.getInstance().remove(productCache);
productCache = null;
}
The CacheManager executes all cleanup chores and ensures that resources are release,
A ProductReader using the caching functionality must implement the CacheDataProvider interface as access means for the cache library. As sketched above, this interface consists of two methods, one for extracting storage pattern information to set up the caching structures and the other one to actually read the data.
VariableDescriptor getVariableDescriptor(String variableName)
The purpose of this method is to extract information about the storage pattern of the data. The variableName parameter denotes the physical variable name as stored in the original product format. When called, the product reader returns a VariableDescriptor object containing
· the variable name
· the data type as ProductData data type
· the overall dimension of the variable: width, height, number of layers (if applicable)
· the storage tile dimensions: width, height, number of layers (if applicable)
DataBuffer readCacheBlock(String variableName, int[] offsets, int[] shapes, ProductData targetData)
This method is called to read a complete cache tile into memory. Calls into this method are always aligned to the optimal access pattern defined by the associated VariableDescriptor.
If - as is normal in SNAP – three-dimensional variables are split into a list of two-dimensional layer-variables calls into readBandRasterDataImpl(…) will contain the band name, i.e. the two-dimensional layered name. Before dispatching this call to the ProductCache, the reader has to decode from this
· the variable name as stored in the product-file
· the associated layer index
Calls to readCacheBlock must use these decoded parameters. Offsets and shapes are passed as integer vectors. These follow the NetCDF notation using the fastest varying coordinate last, the vector length is defined by the variable dimensionality. E.g. for a three-dimensional variable:
· offsets[0]: z-axis (e.g. spectral dimension)
· offsets[1]: y-axis (along track)
· offsets[2]: x-axis (across track)
For a two-dimensional variable the vector length is of course two:
· shapes[0]: y-axis (i.e. height of the cache block)
· offsets[1]: x-axis (i.e. width of the cache block)
So, the implementation of readBandRasterDataImpl(…) in a product reader using the cache is the simplified to
detect if variable is 2d or 3d
if variable is 3d – detect the layer index and the original variable name
set up offset and shape arrays from the calling parameters and layer index if in 3d case
create a targetBuffer object that wraps the destBuffer parameter
call the cache: productCache.read(variableName, offsets, shapes, targetBuffer);
An example implementation can be found in the EnMAP product reader: