Core algorithm review

Previous implementation

Given shapefile, outputs a new shapefile and raster

For each vector feature class:

• reproject to 3857 if needed
• if point data, buffer -> output_data feature class
• if polygon data, CopyFeatures(reprojected_data) -> output_data feature class
• add sap field to output_data
• for each feature in output_data
  • calculate sap and update feature
    • get area of feature - optionally cap maximum
    • sap = weight / area - optionally cap maximum
  • for each row in output_data
    • create fishing_lyr in-memory FeatureLayer
    • RepairGeometry(fishing_lyr) - (does this actually change the shape?)
    • create raster with sap cell value
    • add to list of rasters
  • merge results into outCellStats - CellStatistics - calculate sum per-cell across entire list of raster rasters
  • Clip outCellStats to user-defined clipping bounds

Questions

• Why reproject to 3857 Web Mercator up-front? Is the varied area of each cell a problem?
  • Units are meters, allowing user to specify cell size in meters.
  • Web Mercator is ultimately used to display the rasters
  • As long as the study area is relatively small (how small?), the difference in area per cell across the extent of that study area will be minimal (how minimal?).
• Why is the user-defined clipping bounds needed?
  • It is to remove shape portions outside of the eez or overlapping with land.
  • Long-term this should not be needed if survey tool doesn't allow this in

Performance improvement opportunities

• don't generate all the intermediate rasters, one per feature, use burn-in algorithm
• separate core module that kicks off the actual heatmap jobs, one per processor (local) or one per Lambda
• use MemoryFile if within memory limits
• bucket features and use windowed process to parallelize (or create separate rasters and merge)

New Implementation

Define run parameters

• Output folder
• Input vector datasets
• Extent - used as clipping bounds
• Optional
  • Run name (defaults to 'heatmap')
  • Buffer distance - if point or line features (default to 500)
  • Output cell size (default to 100)
  • Calculate spatial access priority (SAP)? (Weight/Km2)
  • max SAP value
  • max area value

Generate heatmap for each input vector dataset

• Check if resulting raster, based on extent and cell size, is more than max, if it is too big, recommend the max cell size possible.
• Create output raster in memory with extent of clip region
• For each feature
  • Repair geometry
  • Reproject geometry to 3857
  • If point or line feature, buffer to create polygon
  • Clip features to clip region
  • Calculate SAP value
  • Burn polygon with SAP value into output raster

Write results to output folder

• Heatmap raster with name of input dataset
• Log with name of input dataset with run parameters, stats on number of features, any repairs needed?, timestamps