Now an incubating project under Eclipse Foundation LocationTech! GitHub repo moved to
locationtech/rasterframes. PySpark support! See
to get started.
Added RasterFrames-enabled Jupyter Notebook Docker Container package. See
Updated to GeoMesa version 2.0.1.
Added for for writing GeoTIFFs from RasterFrames via
spark.read.geotrellis.withNumPartitions(Int) for setting the initial number of partitions to use when reading a layer.
spark.read.geotrellis.withTileSubdivisions(Int) for evenly subdividing tiles before they become rows in a RasterFrame.
experimental package for sandboxing new feature ideas.
Added experimental GeoJSON DataSource with schema inferfence on feature properties.
experimental feature for exporting the contents of a RasterFrame as a
SlippyMap tile image directory structure and Leaflet/OpenMaps-enabled HTML file. Added
experimental DataSource implementations for MODIS and Landsat 8 catalogs on AWS PDS.
Change: Default interpoation for
toMultibandRaster has been changed from
HistogramAggregateFunction now generates the new type
Added support for reading striped GeoTiffs (#64).
Moved extension methods associated with querying tagged columns to
DataFrameMethods for supporting temporal and spatial columns on non-RasterFrame DataFrames.
GeoTIFF and GeoTrellis DataSources automatically initialize RasterFrames.
Added utility for rendering multiband tile as RGB composite PNG.
RasterFrame.withRFColumnRenamed to lessen boilerplate in maintaining
RasterFrame type tag.
Upgraded to Spark 2.2.1. Added
VersionShims to allow for Spark 2.1.x backwards compatibility.
rasterframes-datasource library for hosting sources from which to read RasterFrames.
Implemented basic (but sufficient) temporal and spatial filter predicate push-down feature for the GeoTrellis layer datasource.
Added Catalyst expressions specifically for spatial relations, allowing for some polymorphism over JTS types.
Added a GeoTrellis Catalog
DataSource for inspecting available layers and associated metadata at a URI
Added GeoTrellis Layer DataSource for reading GeoTrellis layers from any SPI-registered GeoTrellis backend (which includes HDFS, S3, Accumulo, HBase, Cassandra, etc.).
Ability to save a RasterFrame as a GeoTrellis layer to any SPI-registered GeoTrellis backends. Multi-column RasterFrames are written as Multiband tiles.
Addd a GeoTiff DataSource for directly loading a (preferably Cloud Optimized) GeoTiff as a RasterFrame, each row containing tiles as they are internally organized.
Fleshed out support for
TileFeature support in datasource.
Added typeclass for specifying merge operations on
TileFeature data payload.
withTemporalComponent convenince method for creating appending a temporal key column with constant value.
withBounds, and now returns a JTS
EnvelopeEncoder for encoding JTS
Refactored build into separate
docs, paving way for
pyrasterframes polyglot module.
Added utility extension method
Writing multi-column RasterFrames to GeoTrellis layers requires all tiles to be of the same cell type.
withSpatialIndex to introduce a column assigning a z-curve index value based on the tile’s centroid in EPSG:4326.
Added column-appending convenience methods:
Documented example of creating a GeoTrellis layer from a RasterFrame.
Added Spark 2.2.0 forward-compatibility.
Upgraded to GeoTrellis 1.2.0-RC2.
Significant performance improvement in
explodeTiles (1-2 orders of magnitude). See
#38 Fixed bugs in
NoData handling when converting to
Upgraded to shapeless 2.3.2
Ported to sbt 1.0.3
Fixed bug in computing
aggMean when one or more tiles are
rfIinit in favor of
SQLContext.withRasterFrames extension methods
Upgraded to GeoTrellis 1.2.0-RC1
tour of RasterFrames
Moved Giter8 template to separate repository
s22s/raster-frames.g8 due to sbt limitations
Getting Started to reference new Giter8 repo Changed SQL function name
rf_aggHistogram for consistency with DataFrames API
Created faster implementation of aggregate statistics.
Fixed bug in deserialization of
TileUDTs originating from
Fixed bug in serialization of
NoDataFilter within SparkML pipeline
Refactoring of UDF organization
Various documentation tweaks and updates
Added Giter8 template
TileUDFs are encoded using directly into Catalyst–without Kryo–resulting in an insane decrease in serialization time for small tiles (
int8, <= 128²), and pretty awesome speedup for all other cell types other than
float32 (marginal slowing). While not measured, memory footprint is expected to have gone down.
tileMean functions rewritten to compute simple statistics directly rather than using
tileStatsDouble were replaced by
aggNoDataCells aggregate functions.
localAggNoDataCells cell-local (tile generating) fuctions
Overflow fix in