How to Use Tile Cache Operator to Improve Memory Consumption
This guide is directed to advanced/expert SNAP users. Some steps need insight into the implementation of an operator. The code can be obtained from https://github.com/senbox-org. If you struggle anyway, you are welcome to ask your questions in the STEP forum and the developer will give you some hints.
Introduction
SNAP is doing a lot of things very well. But it has also some flaws. One of the issues SNAP has is memory management during data processing. Especially when using more complex processing graphs the user system often runs out of memory and fails with an error. In such cases, users see error messages like “Cannot construct DataBuffer” or “java.lang.OutOfMemoryError : java heap space”. Actually, we use JAI in the background for memory management. And JAI should release data from the cache if not used anymore. Unfortunately, this doesn’t work out well in all cases. More data is kept in the cache as actually necessary and then we see the aforementioned error messages.
We want to improve this situation in the future. In the end, the memory should be handled in a better way automatically. In order to be able to implement a better algorithm for memory management we need some experience when data needs to be cached and when not. To gain this experience we have developed an operator that caches the data of products. When the global cache is disabled this operator allows caching only the data at certain points in a processing graph. The difficult part is now to find out at which points in the graph the caching operator makes the most sense. The following guide shall give you a first introduction to the usage of the operator.
However, we need your feedback regarding the experience you made with the operator. Please visit the STEP forum to report your finding or ask questions if you experience problems.
Outlook
To tease what is possible to achieve by using this operator we show first an example of a complex processing graph.
The graph on the left was used to atmospherically correct process OLCI data. 23 operators are used in the graph. For each of them, memory is used to cache the processed data. You can imagine, the memory consumption sums up to quite a huge amount.
In order to process a usual OLCI PDU, more than 12 GB of memory was consumed and the whole processing took 5 hours. After adding the TileCacheOp at selected positions in the graph we are able to process an OLCI product with 6 GB of memory and in less than 30 minutes.
Prerequisite
If you want to use the TileCacheOp you need to disable the global cache. Otherwise, data will be cached twice and things will get worse. The simplest way is to modify the snap.properties
file you find in the etc
folder of the installation directory. Add the following line to it: snap.gpf.disableTileCache = true
This way no data is put in the tile cache anymore. This property affects the processing in SNAP Desktop and the command line with gpt. Alternatively, if you only want to change this property for a single run with gpt, you can specify it along with the command line call, e.g.
>gpt -Dsnap.gpf.disableTileCache=true <GRAPH_XML_FILE> …
Tile
A tile is a rectangular chunk of data. SNAP does not load all data at once in memory but uses a tiling mechanism. The data of a product is split into multiple tiles. The size of the tiles of a product is often defined by the data format, if not an appropriate size is automatically determined. The tiles are placed in a cache to prevent reading or processing them again if the data is accessed again.
Rules where to add the TileCacheOp
During the first experiments, with the TileCacheOp some rules have emerged where the operator is placed best within a complex graph. For some of the rules, it is not easy to find out if they apply to your graph/operator. Sometimes insight of the implementation is necessary. That’s why in the header note it is mentioned that this guide is directed to the advanced/expert users.
In general 3 rules have been identified
Add the TileCacheOp before operators if they require not a single tile but also adjacent tiles
Add the TileCacheOp after operators if they implement computeTileStack()
Add the TileCacheOp after operators if their output is used by at least two subsequent operators
In the following, the rules are explained in more detail.
There are also operators which use other operators which use other operators internally. They are a graph, so to speak. In such cases, you can’t simply use the TileCacheOp without modifying the source code. If you have the suspicion that an operator might slow down the overall performance of your graph try to add the caching operator afterwards. See if the performance of your graph improves.
Rule 1: Adjacent Tiles
Usually, operators process one tile after the other. But sometimes surrounding data is needed to process the data for the target tile. This is the case for example when reprojecting data or if the algorithm averages several pixels or computing the location of the shadow of a cloud. In such cases data from adjacent tiles need to be loaded and therefor tiles are accessed multiple times. It is good to have the data cached to prevent multiple computations of the input data.
Rule 2: Tile Stack
If an operator computes the full tile stack of the output bands at once then it is good to cache this output. The computation of this stack is usually expensive. And therefore we should cache this data too.
A list of operators that implement the computeTileStack method can be found in the attached TileStackOperators.txt file.
Rule 3: Multiple Subsequent Operators
If the output is used by multiple subsequent operators it is also a good idea to cache its output. This is probably the most obvious rule.
Feedback
Especially regarding those rules, we would like to get your feedback. Maybe you have found more rules, found specific improvements to the existing rules or found cases where the rules don’t work. Whatever you find out, please report in the STEP forum.
Configure Tile Cache Operator
The operator has only one parameter. The cacheSize
parameter defines the cache size in MB of this operator instance. If the limit is reached old cached data is removed. If this removed data is required later the data must be recomputed and the request cannot be served by the cache anymore. In order to know how big the cache should be, you can consider three characteristics of the product which is cached.
The number of Bands
The data type of Bands
The image size of Bands
Let's assume the product has 13 bands, all with a size of 4000x4000 pixels. 10 of the band have the data type float32 (4 byte), 2 are short (2 byte) and one is a byte (1 byte, ) band.
numBytesPixel = 10*4 + 2*2 + 1 = 45 byte
numPixels = 4000 * 4000 = 16000000 pixels
oneMegabyte = 1024 * 1024
dataAmount = numBytesPixel * numPixels = 720000000 byte = 720000000 / oneMegabyte = 686 MB
The amount of data is roughly 700 MB. You should add a bit of a margin. For example, in this calculation, we haven’t considered masks and tie-point grids yet. But as a first guess it usable. You can calculate more accurately if you like.
Instead of letting the cache hold the complete data of the product you can try if maybe 4 tiles are sufficient. The tile size is shown in the information window of a product in SNAP Desktop. Assuming, the tile size is 256x256, we calculate a cache size for 4 tiles of 11,25 MB.