Multi-size Products Specification

Goals

• Other than BEAM, SNAP's product data model should be able to deal with products that have bands of different image sizes and geo-codings
• There shall be one model coordinate reference system for the whole product. 
• Users of the API shall be able to use all SNAP functionality for both single-size and multi-size products.
  
  

Findings of meeting from October 1st, 2015

This meeting took place after we ran into several problems resulting from the implemented changes described below. We found that the approach using SceneRasterTransforms was misguided, as it existed alongside the already present transformations between reference systems within SNAP.

We identified a need to exactly define the meanings of the various coordinate reference systems in SNAP. This becomes the more important as this concept needs to be adapted to consider bands of different sizes within the same product. At the core, it stands that there needs to be a single model crs for a product. Images provide imageToModelTransforms which transform from the respective image crs' onto the model crs'. Also, all vector data is saved within model coordinates. What is the model crs depends on the used geocoding. If a CrsGeocoding is used, there is a map crs which will be the model crs. In other Geocodings, there is no map crs (or, strictly, the geo crs will fill that role) and the model crs should be the image crs. In multi-size products, as there are multiple images it is not clear what is the image crs which shall constitute as the model crs.

Also, as there are multiple images, it makes less sense for a product to provide such things as a scenerastersize or a geocoding, as these are not necessarily valid for the whole product anymore. However, for single-size products, these values can still be provided as they still make sense. 

There are a lot of places within the SNAP code where the scenerastersize or the product's geocoding are requested. In these cases, the product can return the common rastersizes. If these are null, either the operation shall not be performed or the band- / rasterdatanode-specific sizes or geocodings should be used.

A major obstacle here is that there are numerous places where a careful code revision is required. Also, unexpected problems might still occur.

The big remaining question is whether the adaption of the existing image crs/geo crs/model crs - concept to multi-size images along with band-oriented raster sizes and geocodings is feasible.

Background and strategic fit

Raster data (“bands”) may be of different pixel resolution and/or are related to different coordinate systems. BEAM’s strategy is to upscale all bands with lower resolution to the highest one. This causes several problems in the toolbox and it’s APIs:

• Data explosion on disk when writing products
• Redundant computation of duplicated pixels (band maths, geometric operations)

Keeping all bands in their native resolution would cause the following problems:

• GPF tile processing is not defined anymore: what’s a tile then?
• Image display at native resolutions complicates visual comparison
• Band math cannot deal with that at all, as all pixels must be collocated

BEAM has some basic support for multi-resolutions / different geo-codings per band:

• Product.getSceneRasterWidth () and RasterDataNode.getSceneRasterWidth() vs. RasterDataNode.getRasterWidth()
• Product.getGeoCoding() vs. RasterDataNode.getGeoCoding()

Requirements

User interaction and design

General Approach

There are basically two approaches to deal with multi-size bands in the BEAM data model. The first one would be to let ProductReaders return multiple Product instances. For example, the S-2 MSI reader could return three products, one for each of the 10, 20, and 60 m bands. This way no spatial rescaling would be needed at ProductReader level, and the constraint to have a single image resolution within one Product instance would be retained. The second approach is to let the BEAM data model deal with multiple resolutions directly.

The disadvantage of the first approach is that it is very unhandy for implementors of GPF Operators. They have to deal with multiple source products at different resolutions. For a given target tile, how are source tiles computed? Also many visual tools in BEAM (e.g. spectrum toolwindow, spectral unmixing, magic wand) would need to be rewritten in order to account for the fact, that a spectrum is now distributed over a number of open products.

Therefore we propose to implement the second option – extending the BEAM data model to correctly handle multi-size bands.

Transformations Approach

This approach is based on the assumption that SNAP already provides everything required to support multi-size products, as there is already a system of image, model, map,and geographic coordinate reference systems (crs's). In single-size products, there is at least an image crs span by the product's scenerastersize. In most cases, there is also a geographic crs, which relates the pixels to a geographic spatial extent and location. Projected products in addition have a map crs which describes how the geographic coordinates are projected onto a surface. The model crs, finally, can be understood as the common product crs. Vector data is stored and exported in model coordinates and there exists an affine transformation between the image crs and the model crs. If a map crs is provided, it also poses as model crs (so there is an affine transformation between the image crs and the map crs). If there is no dedicated map crs, the image crs serves as the model crs, as there is not necessarily an affine transformation between the image crs and the geo crs.

In multi-size products, there are by definition multiple image crs's. Still, there should be only one model crs. When a map crs is provided, it shall serve as the model crs <- Here we make the assumption that a map crs is universal across the product. If there is no map crs, there must be an image crs which is assigned as the model crs. There is not a need that this image crs is actually employed by one of the actual bands.

Former Approaches

SceneImageTransform Approach

The basic idea is to equip the Product class with something we call a scene image geometry, which refers to the spatial collocation of all the native image geometries of the Product’s Bands where

Image Geometry = Pixel Resolution + Tiling + GeoCoding .

Product class

Existing properties for the scene image geometry of the Product class:

• geoCoding: GeoCoding (will be deprecated, use  a sceneGeoCoding instead)
• sceneRasterWidth: int
• sceneRasterHeight: int

New properties for the scene image geometry of the Product class:

• sceneRasterTileWidth: int
• sceneRasterTileHeight: int
• sceneRasterSize: Dimension
• sceneRasterTileSize: Dimension

RasterDataNode class

Existing properties for the native image geometry of the RasterDataNode class:

• geocoding: GeoCoding
• rasterWidth: int
• rasterHeight: int
• sceneRasterWidth: int (= Product. sceneRasterWidth, deprecate?)
• sceneRasterHeight:int (= Product. sceneRasterHeight, deprecate?)

New properties for the scene image geometry of the RasterDataNode class:

• rasterSize: Dimension
• rasterTileWidth: int
• rasterTileHeight: int
• rasterTileSize: Dimension
• sceneImageTransform: SceneImageTransform 

The value of RasterDataNode.sceneImageTransform allows a Band’s native raster data to be spatially transformed into the product’s scene image geometry and vice versa. It comprises a forward and inverse pixel coordinate transformation:

public interface SceneImageTransform {
    SceneImageTransform IDENT = ...;
    org.opengis.referencing.operation.MathTransform2D getForward();
    org.opengis.referencing.operation.MathTransform2D getInverse();
}

Note that it is legal to have SceneImageTransform instances whose either forward or inverse (but not both) pixel coordinate transformations are not available. ImageTransform instances may be explictely set (e.g. by ProductReader/Operator instances) on a RasterDataNode, or computed using both the Product's and the RasterDataNode's GeoCodings. The computation might not be possible at all or fail, in which case RasterDataNode.imageTransform does not exists. In (Python) pseudo code: 

def getSceneImageTransform():
	if self.sceneImageTransform: 
	    return self.sceneImageTransform
    else:
        return self.computeSceneImageTransform()
 
def computeSceneImageTransform():
	r = self
	p = self.product
    if r.geoCoding and not p.geoCoding:
        throw IllegalStateException('raster geo-coding found, but product geo-coding is undefined')
    if r.geoCoding == p.geoCoding and r.rasterSize == p.sceneRasterSize:
        return SceneImageTransform.IDENT
    ...
    return None

The SceneImageTransform shall allow to convert any pixel coordinate of a RasterDataNode's image CS into a pixel coordinate in the Product's image CS and vice versa. It must be used whenever vector data is displayed or interacted with in a ProductSceneView, because vector data uses the Product's image CS coordinates. For example, we can only add new vector data features (e.g. pins) to a displayed band if it is possible to convert the displayed band's pixel coordinates into pixel coordinates of the Product image CS. With other words, the band's SceneImageTransform must exists and have a valid inverse transform.

In accordance to methods getSceneRasterWidth and getSceneRasterHeight, the class RasterDataNode can provide further methods which operate on either a pixel in product or in band coordinates.

The current class hierarchy of RasterDataNodes can stay as it is right now.

ImageGeometry approach

Note: The following approach has been deprecated in favor of the SceneRasterTransform approach outlined above.

Currently, there is already a class named 'ImageGeometry'. It's main purpose is to store and compute the target image geometry (GeoCoding and resolution) for reprojection. To be able to use the interface name 'ImageGeometry' the existing class will be renamed to 'ReprojectionImageGeometry'. It should be possible to replace this class by the new implementation of the 'ImageGeometry' interface:

public interface ImageGeometry {
    int getWidth();
    int getHeight();
    int getTileWidth();
    int getTileHeight();
    Dimension getSize();
    Dimension getTileSize();
    GeoCoding getGeoCoding();
}
 
public interface ImageGeometryTransform {
    RenderedImage transform(RenderedImage source,
                            GeoCoding sourceGeoCoding,
                            ImageGeometry targetGeometry);
}

Possible operations used during spatial up-/down sampling: scale, crop, border-extend. By default, if no special transform is given per band, the scene image geometry is computed as follows:

TBD

GPF API Changes

At this point in time (SNAP 2.0, May 2015), we don’t want to change the GPF API at all. This means, GPF operators will always be asked to compute tiles defined in the scene image geometry of the target product. Required spatial transformations will be performed transparently. It is also important to retain all raster data accessors that can be used to retrieve the native raster data using the native raster data pixel coordinate system. This way, a GPF operator can operate on native source raster while providing target data in scene image geometry.

Issues

• Layers need to be placed on bands of all sizes. When the band size differs from the product size, the layer must be transformed.
• For multilevel-images, it must be considered that images from different bands might use different scalings

Use Cases

This table lists use cases which might prove problematic when dealing with bands of different sizes.

Questions

Below is a list of questions to be addressed as a result of this requirements document:

Not Doing

• We wont change the product reader interface so that it can read multiple products (as indicated in section "General Approach" above)

Decision

• We go for the 'Option 1: Very restrictive' in the above Use Cases table.
• For products which contain raster data in multiple resolutions, there should be one plugin for each resolution
  • All can return intended as the decode qualification
  • The ReadOp will be extended with a format name parameter, when doing this we can also change the file parameter to a string or to an URI.
  • When a product is opened in SNAP the user will be asked which of the intended product readers shall be used (also suitable reader are shown ans selectable). The user decision can be stored.
• In the Band Maths when the expression is parsed a parse exception is thrown if the used rasters are not compatible.