Skip to content

New data catalog #58

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
silviasdfg merged 7 commits into from
Jun 25, 2025
Merged

New data catalog #58

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
22e98af
Update data catalog for issue #56
github-actions[bot] Jun 20, 2025
a7de559
minor format changes
sermars Jun 23, 2025
4ad35db
Automated installation of virtualenv
sermars Jun 23, 2025
a80614a
changed dataset_details path
sermars Jun 23, 2025
592f0f5
Merge branch 'develop' of github.com:CitComAI-Hub/CitComAI-Hub.github…
sermars Jun 23, 2025
a021d4e
Merge pull request #57 from CitComAI-Hub/request_new_dataset/branch-i…
silviasdfg Jun 23, 2025
f39ac35
Merge branch 'main' into develop
sermars Jun 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ PYTHON := python3
.PHONY: virtenv_create
## Create virtualenv
virtenv_create:
@python3 -c "import virtualenv" >/dev/null 2>&1 || pip install virtualenv
python3 -m virtualenv $(VENV_NAME)
source $(VENV_NAME)/bin/activate && pip install -r requirements.txt

Expand Down
4 changes: 4 additions & 0 deletions docs/data_catalog/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,7 @@ The data catalog is a centralized hub to keep track of available datasets. It is
| [Citybike locations in Aarhus](./metadata_datasets/nordic_citcom_gtm.md) | Nordic | Denmark | GTM | *no specific value* | *no specific value* | No | Aarhus Municipality | [Aarhus Bycykel - Dataset](https://www.opendata.dk/city-of-aarhus/aarhus-bycykel) |
| [Fast track bikeroutes in Aarhus](./metadata_datasets/nordic_citcom_gtm.md) | Nordic | Denmark | GTM | *no specific value* | Ongoing | No | Aarhus Municipality | [Supercykelsti i Aarhus Kommune - Dataset](https://www.opendata.dk/city-of-aarhus/supercykelsti) |
| [recreative bikeroutes in Aarhus](./metadata_datasets/nordic_citcom_gtm.md) | Nordic | Denmark | GTM | *no specific value* | Ongoing | No | Aarhus Municipality | [Rekreative cykelruter - Dataset](https://www.opendata.dk/city-of-aarhus/rekreative-cykelruter) |
| [AirQuality](./metadata_datasets/south_italy_uptown.md) | South | Italy | UpTown | *no specific value* | 30s | Yes | Politecnico di Milano | [neslab.it](https://www.neslab.it) |
| [Biodiversity](./metadata_datasets/south_italy_uptown.md) | South | Italy | UpTown | *no specific value* | 15m | Yes | Politecnico di Milano | [neslab.it](https://www.neslab.it) |
| [Energy distribution](./metadata_datasets/south_italy_uptown.md) | South | Italy | UpTown | *no specific value* | Hourly | Yes | A2A | [neslab.it](https://www.neslab.it) |
| [Archaeological Site](./metadata_datasets/south_italy_mithraeum-of-circus-maximus.md) | South | Italy | Mithraeum of Circus Maximus | *no specific value* | Real-time | Yes | Politecnico di Milano | [neslab.it](https://www.neslab.it) |
15 changes: 15 additions & 0 deletions docs/data_catalog/metadata_datasets/south_italy_mithraeum-of-circus-maximus.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Super Node: South | TEF Node: Italy

## Site: Mithraeum of Circus Maximus | Archaeological Site

| ATTRIBUTE | TYPE | UNITS (SI) | DESCRIPTION/COMMENTS |
|:--------------|:---------|:-------------|:------------------------------------------|
| id | Text | | Sensor identifier |
| temperature | Number | K | Ambient temperature |
| humidity | Number | % | Relative humidity |
| pm10 | Number | µg/m³ | Particulate matter < 10 µm concentration |
| pm25 | Number | µg/m³ | Particulate matter < 2.5 µm concentration |
| spectralPower | Number | mW/Hz | Spectral power density of vibrations |
| timestamp | Number | | Unix timestamp of the measurement |
| location | geo:json | | Geographical coordinates of the sensor |

62 changes: 62 additions & 0 deletions docs/data_catalog/metadata_datasets/south_italy_uptown.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Super Node: South | TEF Node: Italy

## Site: UpTown | AirQuality

| ATTRIBUTE | TYPE | UNITS (SI) | DESCRIPTION/COMMENTS |
|:-------------|:---------|:-------------|:------------------------------------------|
| id | Text | | Sensor identifier |
| temperature | Number | K | Ambient temperature |
| humidity | Number | % | Relative humidity |
| co | Number | mg/m³ | Carbon monoxide concentration |
| nh3 | Number | µg/m³ | Ammonia concentration |
| no | Number | µg/m³ | Nitric oxide concentration |
| no2 | Number | µg/m³ | Nitrogen dioxide concentration |
| o3 | Number | µg/m³ | Ozone concentration |
| pm25 | Number | µg/m³ | Particulate matter < 2.5 µm concentration |
| pm10 | Number | µg/m³ | Particulate matter < 10 µm concentration |
| so2 | Number | µg/m³ | Sulfur dioxide concentration |
| indoor | Boolean | | True if the sensor is located indoors |
| manufacturer | Text | | Sensor manufacturer name |
| sensorName | Text | | Human-readable sensor name |
| location | geo:json | | Geographical coordinates of the sensor |
| timestamp | Number | | Unix timestamp of the measurement |

## Site: UpTown | Biodiversity

| ATTRIBUTE | TYPE | UNITS (SI) | DESCRIPTION/COMMENTS |
|:-------------------------------|:-------|:-------------|:--------------------------------------------|
| id | Text | | Sensor identifier |
| timestamp | Number | | Unix timestamp of the measurement |
| device_hostname | Text | | Device hostname |
| firmware_version | Text | | Current firmware version |
| state | Text | | Reported high-level state of the device |
| activity_state | Text | | Current activity state (e.g., active, idle) |
| device_temperature | Number | °C | Internal device temperature |
| battery_voltage | Number | V | Battery voltage |
| battery_level | Number | % | Battery charge level |
| battery_power_input | Text | | Raw battery input power data |
| last_ping | Number | | Timestamp of last device ping |
| last_audio_record | Number | | Timestamp of last audio recording |
| last_detection_ts | Number | | Timestamp of last detection event |
| last_detection_scientific_name | Text | | Scientific name of last detected species |
| last_detection_common_name_en | Text | | Common name (EN) of last detected species |
| last_detection_common_name_it | Text | | Common name (IT) of last detected species |
| last_detection_confidence | Number | | Confidence score of last detection |

## Site: UpTown | Energy distribution

| ATTRIBUTE | TYPE | UNITS (SI) | DESCRIPTION/COMMENTS |
|:-----------------------------|:---------|:-------------|:-------------------------------------------------------------|
| id | Text | | Sensor identifier |
| timestamp | Number | | Unix timestamp of the measurement |
| user_identifier | Text | | Anonymized unique identifier for the user |
| pod_identifier | Number | | Anonymized identifier of the heating POD unit |
| pod_coordinates | geo:json | | Geographical coordinates of the POD |
| user_type | Text | | Category or role of the user (e.g., residential, commercial) |
| external_temperature | Number | °C | Outdoor air temperature |
| heating_temperature_delivery | Number | °C | Temperature of the fluid delivered to the heating system |
| heating_temperature_return | Number | °C | Temperature of the fluid returning from the heating system |
| heating_volume | Number | m³/h | Volume flow rate of heating fluid |
| heating_power | Number | kW | Thermal power delivered for heating |
| energy_consumption | Number | kWh | Cumulative heating energy consumed |

2 changes: 1 addition & 1 deletion scripts/csv_to_markdown/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ dataset:

markdowns:
index: "docs/data_catalog/index.md"
dataset_details: "docs/data_catalog/metadata_datasets"
dataset_details: "./metadata_datasets"

github_actions:
csv_path: "/tmp/csv_files"