Introduced at the source level the concept of a RasterSource and RasterRef, enabling lazy/delayed read of sub-scene tiles.
Deprecation: Tile column functions (in RasterFunctions) and SQL registered names have all been renamed to follow snake_case conventions, matching SQL and Python. A temporary compatibility shim is included so that code built against 0.7.1 and earlier still work. These will be marked as deprecated.
Added withKryoSerialization extension methods on SparkSession.Builder and SparkConf.
Breaking: In Scala and SQL, ..._scalar functions (e.g. local_add_scalar) have been removed. Non-scalar forms now dynamically detect type of right hand side.
Breaking: tileToArray has been replaced with tile_to_array_double and tile_to_array_int.
Added render_matrix debugging function.
Breaking: renamed agg_histogram to agg_approx_histogram, local_agg_stats to agg_local_stats, local_agg_max to agg_local_max, local_agg_min to agg_local_min, local_agg_mean to agg_local_mean, local_agg_data_cells to agg_local_data_cells, local_agg_no_data_cells to agg_local_no_data_cells.
Breaking: CellHistogram no longer carries along approximate statistics, due to confusing behavior. Use agg_stats instead.
Introduced LocalCellStatistics class to wrap together results from LocalStatsAggregate.
Breaking: TileDimensions moved from astraea.spark.rasterframes to astraea.spark.rasterframes.model.
Moved Giter8 template to separate repository s22s/raster-frames.g8 due to sbt limitations
Updated Getting Started to reference new Giter8 repo
Changed SQL function name rf_stats and rf_histogram to rf_aggStats and rf_aggHistogram for consistency with DataFrames API
Created faster implementation of aggregate statistics.
Fixed bug in deserialization of TileUDTs originating from ConstantTiles
Fixed bug in serialization of NoDataFilter within SparkML pipeline
Refactoring of UDF organization
Various documentation tweaks and updates
Added Giter8 template
TileUDFs are encoded using directly into Catalyst–without Kryo–resulting in an insane decrease in serialization time for small tiles (int8, <= 128²), and pretty awesome speedup for all other cell types other than float32 (marginal slowing). While not measured, memory footprint is expected to have gone down.
aggStats and tileMean functions rewritten to compute simple statistics directly rather than using StreamingHistogram
tileHistogramDouble and tileStatsDouble were replaced by tileHistogram and tileStats
Added tileSum, tileMin and tileMax functions
Added aggMean, aggDataCells and aggNoDataCells aggregate functions.
Added localAggDataCells and localAggNoDataCells cell-local (tile generating) fuctions