Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,4 @@
.DS_Store
.docusaurus
node_modules/
docs/_examples/
docs/_schema/
docs/schema/
docs/schema
static/_schema/
build/
17 changes: 17 additions & 0 deletions .markdownlint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"default": true,
"MD001": false,
"MD003": false,
"MD010": false,
"MD013": false,
"MD024": {
"siblings_only": true
},
"MD025": false,
"MD029": false,
"MD033": false,
"MD036": false,
"MD040": false,
"MD041": false,
"MD046": false
}
6 changes: 6 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
build/
.docusaurus/
node_modules/
package-lock.json
*.md
*.mdx
10 changes: 10 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"semi": true,
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "es5",
"printWidth": 100,
"bracketSpacing": true,
"arrowParens": "always",
"proseWrap": "preserve"
}
7 changes: 3 additions & 4 deletions blog/2024-04-21-welcome.mdx
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
---
title: Welcome to the Overture Maps engineering blog!
authors:
name: Dana Bauer
title: Technical Product Manager, Overture Maps
email: dana@overturemaps.org
authors: dana
---

Welcome to the Overture Maps engineering blog!
We're excited to tell the technical stories behind this big, exciting, ambitious, intensely collaborative project we’ve been [working on for more than a year](https://overturemaps.org/looking-forward-to-2024/).
The engineers building Overture Maps are eager to connect with our developer community.
In the posts we publish in the coming months, you’ll hear from team members who work at Amazon, Meta, Microsoft, TomTom, Esri, Development Seed, Precisely, and more.

<!-- truncate -->

And we’d like to hear from you: [feedback and contributions](https://github.com/OvertureMaps/data/discussions) from folks working with our data and schema are crucial to our success.
Thanks for joining us on this journey.
We’re glad you’re here.
51 changes: 28 additions & 23 deletions blog/2024-04-22-beta-release.mdx
Original file line number Diff line number Diff line change
@@ -1,79 +1,84 @@
---
title: Exploring our beta release
description: Highlights from Overture Maps beta release
authors:
name: Dana Bauer
title: Technical Product Manager, Overture Maps
email: dana@overturemaps.org
authors: dana
tags:
- tools
- geoparquet
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

Last week Overture Maps [announced the beta release](https://overturemaps.org/overture-maps-foundation-releases-beta-of-its-first-open-map-dataset/) of our schema and data. After months of hard work and steady improvements, we are nearing production-level stability. In a series of posts over the next few weeks -- starting with this one -- we’ll unpack the highlights and improvements you'll see in this release and beyond.
Last week Overture Maps [announced the beta release](https://overturemaps.org/overture-maps-foundation-releases-beta-of-its-first-open-map-dataset/) of our schema and data. After months of hard work and steady improvements, we are nearing production-level stability. In a series of posts over the next few weeks -- starting with this one -- we’ll unpack the highlights and improvements you'll see in this release and beyond.

<!-- truncate -->

## Making geospatial fast

The first thing you'll notice is how much faster you can [pull Overture Maps data out of the cloud](https://docs.overturemaps.org/getting-data/locally/). We made this possible by working closely with open-source collaborators on the repartitioning and [spatial optimization](https://cloudnativegeo.org/blog/2024/03/coming-soon-geoparquet-1.1/) of our GeoParquet files. Meanwhile, our friends at DuckDB, one of our favorite SQL tools, [added a feature](https://github.com/duckdb/duckdb/pull/10314) that improves the performance of queries with bounding boxes.

<details>

<summary>Query speeds have improved significantly from our February release to the April beta release. This example compares DuckDB queries for buildings in Philadelphia. [See here for more information about our performance testing](https://github.com/opengeospatial/geoparquet/discussions/188#discussioncomment-8909348).</summary>
<summary>
Query speeds have improved significantly from our February release to the April beta release. This
example compares DuckDB queries for buildings in Philadelphia. [See here for more information
about our performance
testing](https://github.com/opengeospatial/geoparquet/discussions/188#discussioncomment-8909348).
</summary>

<Tabs>
<TabItem value="february" label="2024-02-15-alpha.0 release" default>
```
SELECT
SELECT
*
FROM
FROM
read_parquet('s3://overturemaps-us-west-2/release/2024-02-15-alpha.0/theme=buildings/type=building/*', filename=true, hive_partitioning=1)
WHERE
WHERE
bbox.minx > -75.60154
AND bbox.maxx < -74.955763
AND bbox.miny > 39.867004
AND bbox.maxy < 40.137992;
```
820,915 buildings
820,915 buildings
~120s
</TabItem>


<TabItem value="april" label="2024-04-16-beta.0 release" default>
```
SELECT
SELECT
*
FROM
FROM
read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=buildings/type=building/*', filename=true, hive_partitioning=1)
WHERE
WHERE
bbox.xmin > -75.60154
AND bbox.xmax < -74.955763
AND bbox.ymin > 39.867004
AND bbox.ymax < 40.137992
```
852,487 buildings
852,487 buildings
~25s
</TabItem>
</Tabs>

</details>

We're continuing to make things faster and easier for users. Along with the folks at Development Seed, an [Overture Maps Foundation member](https://overturemaps.org/about/members/), we're [building special tools for Overture Maps data on top of lonboard](https://developmentseed.org/lonboard/latest/examples/overture-maps/), their Python library for visualizing large geospatial datasets in Jupyter. And recently our friends at Wherobots [took a comprehensive look](https://wherobots.com/overture-maps-data-cloud-native-geoparquet-apache-sedona/) at how our use of GeoParquet makes querying and analyzing our data with Apache Sedona very efficient.
We're continuing to make things faster and easier for users. Along with the folks at Development Seed, an [Overture Maps Foundation member](https://overturemaps.org/about/members/), we're [building special tools for Overture Maps data on top of lonboard](https://developmentseed.org/lonboard/latest/examples/overture-maps/), their Python library for visualizing large geospatial datasets in Jupyter. And recently our friends at Wherobots [took a comprehensive look](https://wherobots.com/overture-maps-data-cloud-native-geoparquet-apache-sedona/) at how our use of GeoParquet makes querying and analyzing our data with Apache Sedona very efficient.

As you can see, we're moving forward with the community to iterate on data, software, and specifications with the shared goal of making geospatial **_fast_**.

## Easier-to-use schema

Another highlight of the beta release is the transition to an easier-to-use schema for our administrative boundary data. We first [explored this idea](https://github.com/OvertureMaps/schema/discussions/117) with the Overture Maps community in February, and after two short months of work, the [new divisions schema and data](https://docs.overturemaps.org/guides/divisions/) are ready to go. Here's a query to `divisions` that grabs geometries for all the countries in the world:

```
SELECT *
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=divisions/type=division_area/*', filename=true, hive_partitioning=1)
WHERE subtype = 'country';
```

You can see that the `divisions` query above is much simpler than a comparable query to `admins`:

You can see that the `divisions` query above is much simpler than a comparable query to `admins`:

```
WITH admins AS (
Expand All @@ -87,14 +92,14 @@ WITH admins AS (
ON country_area.locality_id = country_locality.id
WHERE country_locality.locality_type = 'country';

```
```

We plan to [deprecate admins](https://docs.overturemaps.org/release-notes/) by the July release. In the meantime, both `admins` and `divisions` will be available to users.
We plan to [deprecate admins](https://docs.overturemaps.org/release-notes/) by the July release. In the meantime, both `admins` and `divisions` will be available to users.

## Bridges, islands, waterfalls, and more!

We added more rich detail to our `base` layer in this release, including an `infrastructure` type with familiar features from [Facebook’s Daylight map distribution](https://daylightmap.org/). We also added new subtypes and classes for the `land`, `land_use`, and `water` feature types. You'll find a comprehensive listing of the subtypes and classes for each feature type in our [schema reference docs](https://docs.overturemaps.org/schema/reference/base/infrastructure). Ready to make your own map? We have a [tutorial to help you get started](https://docs.overturemaps.org/examples/build-a-map/#13/47.6/-122.33/0/45).

## Stay tuned for more highlights
We'll be back soon with more posts that explore our path from the beta release to production. In the meantime, we invite you to [get started with our data](https://docs.overturemaps.org/getting-data/) and share with us your [comments and feedback](https://github.com/OvertureMaps/data/discussions).

## Stay tuned for more highlights

We'll be back soon with more posts that explore our path from the beta release to production. In the meantime, we invite you to [get started with our data](https://docs.overturemaps.org/getting-data/) and share with us your [comments and feedback](https://github.com/OvertureMaps/data/discussions).
42 changes: 19 additions & 23 deletions blog/2024-05-16-land-cover.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
---
title: Land cover is live
authors:
name: Dana Bauer
title: Technical Product Manager, Overture Maps
email: dana@overturemaps.org
authors: dana
tags:
- base
---
Expand All @@ -16,7 +13,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';

## Mapmakers rejoice!

The [May release](https://docs.overturemaps.org/release/latest) of Overture Maps includes new high-resolution land cover data *and* new cartographic schema properties.
The [May release](https://docs.overturemaps.org/release/latest) of Overture Maps includes new high-resolution land cover data _and_ new cartographic schema properties.

<!-- truncate -->

Expand All @@ -26,20 +23,19 @@ Our May release also includes [schema properties](https://docs.overturemaps.org/

## Exploring land cover

In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using [lonboard](https://developmentseed.org/lonboard/latest/) and the [Overture Maps Python command-line tool](https://github.com/OvertureMaps/overturemaps-py). We recommend that you consult [this example](https://developmentseed.org/lonboard/latest/examples/overture-maps/ ) in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on [Notebook Sharing Space](https://notebooksharing.space/view/b63f6b3dda1da99c45caf53284fbc508aaae0a43480b43bb0316db636c5e6677).

In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using [lonboard](https://developmentseed.org/lonboard/latest/) and the [Overture Maps Python command-line tool](https://github.com/OvertureMaps/overturemaps-py). We recommend that you consult [this example](https://developmentseed.org/lonboard/latest/examples/overture-maps/) in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on [Notebook Sharing Space](https://notebooksharing.space/view/b63f6b3dda1da99c45caf53284fbc508aaae0a43480b43bb0316db636c5e6677).

![land cover](/img/blog/notebook-overture-lc.png)

To follow along, you'll need to have [JupyterLab or Jupyter Notebook](https://jupyter.org/) running and the following dependencies installed:
- [lonboard](https://developmentseed.org/lonboard/latest/#install)
- [overturemaps-py](https://github.com/OvertureMaps/overturemaps-py)
- [pandas](https://pandas.pydata.org/)
- [GeoPandas](https://geopandas.org/en/stable/index.html)
- [Shapely](https://shapely.readthedocs.io/en/stable/index.html)
To follow along, you'll need to have [JupyterLab or Jupyter Notebook](https://jupyter.org/) running and the following dependencies installed:

- [lonboard](https://developmentseed.org/lonboard/latest/#install)
- [overturemaps-py](https://github.com/OvertureMaps/overturemaps-py)
- [pandas](https://pandas.pydata.org/)
- [GeoPandas](https://geopandas.org/en/stable/index.html)
- [Shapely](https://shapely.readthedocs.io/en/stable/index.html)

``` python
```python
import pandas as pd
import geopandas as gpd
import overturemaps
Expand All @@ -48,28 +44,28 @@ from lonboard import Map, PolygonLayer
from lonboard.colormap import apply_categorical_cmap
```

``` python
```python
# specify bounding box
bbox = -78.6429, 39.463, -73.7806, 41.6242
```

``` python
```python
# read in Overture Maps land_cover data type
table = overturemaps.record_batch_reader("land_cover", bbox).read_all()
table = table.combine_chunks()
```

``` python
```python
# convert to dataframe
df = table.to_pandas()
```

``` python
```python
# filter for higher resolution land_cover features
df_h = df[df.cartography.apply(lambda x: x['min_zoom'] == 8)]
```

``` python
```python
# create color map for land_cover subtypes, loosely based on natural-color palette: https://www.shadedrelief.com/shelton/c.html
color_map = {
"urban": [167, 162, 186],
Expand All @@ -85,12 +81,12 @@ color_map = {
}
```

``` python
```python
# apply color map to land_cover subtypes
colors = apply_categorical_cmap(df_h.subtype, color_map)
```

``` python
```python
# dataframe to geodataframe, set crs
gdf = gpd.GeoDataFrame(
df_h,
Expand All @@ -99,7 +95,7 @@ gdf = gpd.GeoDataFrame(
)
```

``` python
```python
# create map layer
layer = PolygonLayer.from_geopandas(
gdf= gdf[['id','subtype', 'cartography', 'geometry']].reset_index(drop=True),
Expand All @@ -108,7 +104,7 @@ layer = PolygonLayer.from_geopandas(
)
```

``` python
```python
#render map
view_state = {
"longitude": -76.2,
Expand Down
13 changes: 8 additions & 5 deletions blog/2024-07-24-explore-site.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
---
title: Exploring Overture data, no SQL required
authors:
name: Dana Bauer
title: Technical Product Manager, Overture Maps
email: dana@overturemaps.org
authors: dana
tags:
- tools
---
Expand All @@ -17,7 +14,13 @@ Overture has launched a new browser-based tool to give users a no-code option to
<source src={useBaseUrl('/img/blog/explore-philadelphia2.mp4')} type="video/mp4" />
</video>
<figcaption style={{ textAlign: 'center' }}>
<em>Checking out <a href="https://www.associationforpublicart.org/artwork/lion-crushing-a-serpent/">"Lion Crushing a Serpent"</a> in Philadelphia's Rittenhouse Park</em>
<em>
Checking out{' '}
<a href="https://www.associationforpublicart.org/artwork/lion-crushing-a-serpent/">
"Lion Crushing a Serpent"
</a>{' '}
in Philadelphia's Rittenhouse Park
</em>
</figcaption>
</figure>

Expand Down
7 changes: 2 additions & 5 deletions blog/2024-08-15-preview-august-release.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
---
title: What to expect in Overture’s August release
authors:
name: Dana Bauer
title: Technical Product Manager, Overture Maps
email: dana@overturemaps.org
title: What to expect in Overture's August release
authors: dana
tags:
- addresses
- base
Expand Down
Loading