Skip to content

Commit d991b15

Browse files
egeakmanegeakman
authored
All changes (#19)
* Convert SubmissionState to Enum * update * Update pretalx models * Add EuroPython models * socials * update config * update download * Add utils class * Add parse class * add timing relationships class * Add transform class * fix tests * add main function * pip-compile * add documentation * move to examples * Pr13 Small changes (#31) * Small changes * empty line * Split the main file into separate small categorized files (#32)
1 parent 1065791 commit d991b15

29 files changed

+1121
-310
lines changed

‎.pre-commit-config.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,6 @@ repos:
2323
hooks:
2424
- id: ruff
2525
- id: ruff-format
26-
args: ["--check"]
2726

2827
- repo: local
2928
hooks:

‎Makefile‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,11 @@ download:
1414
python -m src.download
1515

1616
transform:
17+
ifeq ($(WARN_DUPES), true)
18+
python -m src.transform --warn-dupes
19+
else
1720
python -m src.transform
18-
21+
endif
1922

2023
all: download transform
2124

‎README.md‎

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,41 @@
11
# programapi
2-
Program API
2+
3+
This project downloads, processes, saves, and serves the static JSON files containing details of accepted speakers and submissions via an API.
4+
5+
Used by the EuroPython 2024 website and the Discord bot.
6+
7+
**What this project does step-by-step:**
8+
9+
1. Downloads the Pretalx speaker and submission data, and saves it as JSON files.
10+
2. Transforms the JSON files into a format that is easier to work with and OK to serve publicly. This includes removing unnecessary/private fields, and adding new fields.
11+
3. Serves the JSON files via an API.
12+
13+
## Installation
14+
15+
1. Clone the repository.
16+
2. Install the dependency management tool: ``make deps/pre``
17+
3. Install the dependencies: ``make deps/install``
18+
4. Set up ``pre-commit``: ``make pre-commit``
19+
20+
## Configuration
21+
22+
You can change the event in the [``config.py``](src/config.py) file. It is set to ``europython-2024`` right now.
23+
24+
## Usage
25+
26+
- Run the whole process: ``make all``
27+
- Run only the download process: ``make download``
28+
- Run only the transformation process: ``make transform``
29+
30+
**Note:** Don't forget to set ``PRETALX_TOKEN`` in your ``.env`` file at the root of the project. And please don't make too many requests to the Pretalx API, it might get angry 🤪
31+
32+
## API
33+
34+
The API is served at ``https://programapi24.europython.eu/2024``. It has two endpoints (for now):
35+
36+
- ``/speakers.json``: Returns the list of confirmed speakers.
37+
- ``/sessions.json``: Returns the list of confirmed sessions.
38+
39+
## Schema
40+
41+
See [this page](data/examples/README.md) for the explanations of the fields in the returned JSON files.

‎data/.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# JSON files except the ones in examples/
2+
*.json
3+
!examples/**

‎data/examples/README.md‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Explaining the output data
2+
3+
**Note:** Some of the fields may be `null` or empty (`""`).
4+
5+
## `sessions.json`
6+
7+
<details>
8+
<summary>Example session data JSON</summary>
9+
10+
```json
11+
{
12+
"A1B2C3": {
13+
"code": "A1B2C3",
14+
"title": "Example talk",
15+
"speakers": [
16+
"B4D5E6",
17+
...
18+
],
19+
"session_type": "Talk",
20+
"slug": "example-talk",
21+
"track": "Some Track",
22+
"state": "confirmed",
23+
"abstract": "This is an example talk. It is a great talk.",
24+
"tweet": "This is an example talk.",
25+
"duration": "60",
26+
"level": "intermediate",
27+
"delivery": "in-person",
28+
"resources": [
29+
{
30+
"resource": "https://example.com/notebook.ipynb",
31+
"description": "Notebook used in the talk"
32+
},
33+
{
34+
"resource": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
35+
"description": "Video of the robot in action"
36+
}
37+
...
38+
],
39+
"room": "South Hall 2A",
40+
"start": "2024-07-10T14:00:00+02:00",
41+
"end": "2024-07-10T15:00:00+02:00",
42+
"website_url": "https://ep2024.europython.eu/session/example-talk/",
43+
"sessions_in_parallel": [
44+
"F7G8H9",
45+
...
46+
],
47+
"sessions_after": [
48+
"I0J1K2",
49+
...
50+
],
51+
"sessions_before": [
52+
"L3M4N5",
53+
...
54+
],
55+
"next_session": "O6P7Q8",
56+
"prev_session": "R9S0T1"
57+
},
58+
}
59+
```
60+
</details>
61+
62+
&nbsp;
63+
64+
The fields are as follows:
65+
66+
| Key | Type | Notes |
67+
|------------------------|-------------------------------------------|---------------------------------------------------------------|
68+
| `code` | `string` | Unique identifier for the session |
69+
| `title` | `string` | Title of the session |
70+
| `speakers` | `array[string]` | List of codes of the speakers |
71+
| `session_type` | `string` | Type of the session (e.g. Talk, Workshop, Poster, etc.) |
72+
| `slug` | `string` | URL-friendly version of the title |
73+
| `track` | `string` \| `null` | Track of the session (e.g. PyData, Web, etc.) |
74+
| `abstract` | `string` | Abstract of the session |
75+
| `tweet` | `string` | Tweet-length description of the session |
76+
| `duration` | `string` | Duration of the session in minutes |
77+
| `level` | `string` | Level of the session (e.g. beginner, intermediate, advanced) |
78+
| `delivery` | `string` | Delivery mode of the session (e.g. in-person, remote) |
79+
| `resources` | `array[object[string, string]]` \| `null` | List of resources for the session: `{"resource": <url>, "description": <description>}` |
80+
| `room` | `string` \| `null` | Room where the session will be held |
81+
| `start` | `string (datetime ISO format)` \| `null` | Start time of the session |
82+
| `end` | `string (datetime ISO format)` \| `null` | End time of the session |
83+
| `website_url` | `string` | URL of the session on the conference website |
84+
| `sessions_in_parallel` | `array[string]` \| `null` | List of codes of sessions happening in parallel |
85+
| `sessions_after` | `array[string]` \| `null` | List of codes of sessions happening after this session |
86+
| `sessions_before` | `array[string]` \| `null` | List of codes of sessions happening before this session |
87+
| `next_session` | `string` \| `null` | Code of the next session in the same room |
88+
| `prev_session` | `string` \| `null` | Code of the previous session in the same room |
89+
90+
&nbsp;
91+
92+
## `speakers.json`
93+
94+
<details>
95+
<summary>Example speaker data JSON</summary>
96+
97+
```json
98+
{
99+
"B4D5E6": {
100+
"code": "B4D5E6",
101+
"name": "A Speaker",
102+
"biography": "Some bio",
103+
"avatar": "https://pretalx.com/media/avatars/picture.jpg",
104+
"slug": "a-speaker",
105+
"submissions": [
106+
"A1B2C3",
107+
...
108+
],
109+
"affiliation": "A Company",
110+
"homepage": "https://example.com",
111+
"gitx": "https://github.com/B4D5E6",
112+
"linkedin_url": "https://www.linkedin.com/in/B4D5E6",
113+
"mastodon_url": "https://mastodon.social/@B4D5E6",
114+
"twitter_url": "https://x.com/B4D5E6"
115+
},
116+
...
117+
}
118+
```
119+
</details>
120+
121+
&nbsp;
122+
123+
The fields are as follows:
124+
125+
| Key | Type | Notes |
126+
|----------------|--------------------|-----------------------------------------------------------------------|
127+
| `code` | `string` | Unique identifier for the speaker |
128+
| `name` | `string` | Name of the speaker |
129+
| `biography` | `string` \| `null` | Biography of the speaker |
130+
| `avatar` | `string` | URL of the speaker's avatar |
131+
| `slug` | `string` | URL-friendly version of the name |
132+
| `submissions` | `array[string]` | List of codes of the sessions the speaker is speaking at |
133+
| `affiliation` | `string` \| `null` | Affiliation of the speaker |
134+
| `homepage` | `string` \| `null` | URL/text of the speaker's homepage |
135+
| `gitx` | `string` \| `null` | URL/text of the speaker's GitHub/GitLab/etc. profile |
136+
| `linkedin_url` | `string` \| `null` | URL of the speaker's LinkedIn profile |
137+
| `twitter_url` | `string` \| `null` | URL of the speaker's Twitter profile |
138+
| `mastodon_url` | `string` \| `null` | URL of the speaker's Mastodon profile |
139+
| `website_url` | `string` | URL of the speaker's profile on the conference website |

data/examples/output/sessions.json renamed to data/examples/europython/sessions.json

Lines changed: 24 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,22 +5,32 @@
55
"speakers": [
66
"F3DC8A", "ZXCVBN"
77
],
8-
"submission_type": "Talk (long session)",
8+
"session_type": "Talk (long session)",
99
"slug": "this-is-a-test-talk-from-a-test-speaker-about-a-test-topic",
1010
"track": "Software Engineering & Architecture",
11-
"state": "confirmed",
1211
"abstract": "This is the abstract of the talk, it should be about Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec condimentum viverra ante in dignissim. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec molestie lorem enim, id dignissim mi faucibus a. Suspendisse mollis lobortis mollis. Praesent eu lorem id velit maximus blandit eget at nisl. Quisque fringilla pharetra euismod. Morbi id ante vitae tortor volutpat interdum fermentum id tortor. Vivamus ligula nisl, mattis molestie purus vel, interdum venenatis nulla. Nam suscipit scelerisque ornare. Ut consequat sem vel sapien porta pretium. Nullam non lacinia nulla, a tincidunt dui. Sed consequat nibh in nibh ornare, rhoncus sollicitudin sem lobortis. Etiam molestie est et felis sollicitudin, commodo facilisis mi vehicula. Quisque pharetra consectetur ligula, sit amet tincidunt nibh consectetur fringilla. Suspendisse eu libero sed magna malesuada bibendum sed et enim. Phasellus convallis tortor nec lectus venenatis, id tristique quam finibus.",
1312
"tweet": "This is a short version of this talk, as a tweet.",
1413
"duration": "45",
1514
"level": "intermediate",
1615
"delivery": "in-person",
16+
"resources": [
17+
{
18+
"resource": "https://example.com/notebook.ipynb",
19+
"description": "Notebook used in the talk"
20+
},
21+
{
22+
"resource": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
23+
"description": "Video of the robot in action"
24+
}
25+
],
1726
"room": null,
1827
"start": null,
1928
"end": null,
20-
"talks_in_parallel": null,
21-
"talks_after": null,
22-
"next_talk_code": null,
23-
"prev_talk_code": null,
29+
"sessions_in_parallel": null,
30+
"sessions_after": null,
31+
"sessions_before": null,
32+
"next_session": null,
33+
"prev_session": null,
2434
"website_url": "https://ep2024.europython.eu/session/this-is-a-test-talk-from-a-test-speaker-about-a-test-topic"
2535
},
2636
"B8CD4F": {
@@ -29,22 +39,23 @@
2939
"speakers": [
3040
"G3DC8A"
3141
],
32-
"submission_type": "Talk",
42+
"session_type": "Talk",
3343
"slug": "a-talk-with-shorter-title",
3444
"track": "PyData: LLMs",
35-
"state": "confirmed",
36-
"abstract": "This is the abstract of the shoerter talk, it should be about Lorem ipsum dolor sit amet",
45+
"abstract": "This is the abstract of the shorter talk, it should be about Lorem ipsum dolor sit amet",
3746
"tweet": "Hey, short tweet",
3847
"duration": "30",
3948
"level": "beginner",
4049
"delivery": "in-person",
50+
"resources": null,
4151
"room": null,
4252
"start": null,
4353
"end": null,
44-
"talks_in_parallel": null,
45-
"talks_after": null,
46-
"next_talk_code": null,
47-
"prev_talk_code": null,
54+
"sessions_in_parallel": null,
55+
"sessions_after": null,
56+
"sessions_before": null,
57+
"next_session": null,
58+
"prev_session": null,
4859
"website_url": "https://ep2024.europython.eu/session/a-talk-with-shorter-title"
4960
}
5061
}

data/examples/output/speakers.json renamed to data/examples/europython/speakers.json

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,10 @@
88
"submissions": ["A8CD3F"],
99
"affiliation": "A Company",
1010
"homepage": null,
11-
"twitter": null,
12-
"mastodon": null
11+
"gitx": "https://github.com/F3DC8A",
12+
"linkedin_url": "https://www.linkedin.com/in/F3DC8A",
13+
"mastodon_url": null,
14+
"twitter_url": null,
15+
"website_url": "https://ep2024.europython.eu/speaker/a-speaker"
1316
}
1417
}

‎data/examples/pretalx/speakers.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@
8383
"en": "Social (LinkedIn)"
8484
}
8585
},
86-
"answer": "https://www.linkedin.com/in/F3DC8A/",
86+
"answer": "https://www.linkedin.com/in/F3DC8A",
8787
"answer_file": null,
8888
"submission": null,
8989
"review": null,

‎data/examples/pretalx/submissions.json‎

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,22 @@
2828
"abstract": "This is the abstract of the talk, it should be about Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec condimentum viverra ante in dignissim. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec molestie lorem enim, id dignissim mi faucibus a. Suspendisse mollis lobortis mollis. Praesent eu lorem id velit maximus blandit eget at nisl. Quisque fringilla pharetra euismod. Morbi id ante vitae tortor volutpat interdum fermentum id tortor. Vivamus ligula nisl, mattis molestie purus vel, interdum venenatis nulla. Nam suscipit scelerisque ornare. Ut consequat sem vel sapien porta pretium. Nullam non lacinia nulla, a tincidunt dui. Sed consequat nibh in nibh ornare, rhoncus sollicitudin sem lobortis. Etiam molestie est et felis sollicitudin, commodo facilisis mi vehicula. Quisque pharetra consectetur ligula, sit amet tincidunt nibh consectetur fringilla. Suspendisse eu libero sed magna malesuada bibendum sed et enim. Phasellus convallis tortor nec lectus venenatis, id tristique quam finibus.",
2929
"description": null,
3030
"duration": 45,
31+
"resources": [
32+
{
33+
"resource": "https://example.com/notebook.ipynb",
34+
"description": "Notebook used in the talk"
35+
},
36+
{
37+
"resource": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
38+
"description": "Video of the robot in action"
39+
}
40+
],
3141
"slot_count": 1,
3242
"do_not_record": false,
3343
"is_featured": false,
3444
"content_locale": "en",
3545
"slot": null,
3646
"image": null,
37-
"resources": [],
3847
"answers": [
3948
{
4049
"question": {
@@ -132,7 +141,7 @@
132141
},
133142
"track_id": 4493,
134143
"state": "confirmed",
135-
"abstract": "This is the abstract of the talk, it should be about Lorem ipsum dolor sit amet",
144+
"abstract": "This is the abstract of the shorter talk, it should be about Lorem ipsum dolor sit amet",
136145
"description": null,
137146
"duration": 30,
138147
"slot_count": 1,
@@ -157,6 +166,19 @@
157166
"person": null,
158167
"options": []
159168
},
169+
{
170+
"question": {
171+
"id": 3412,
172+
"question": {
173+
"en": "Abstract as a tweet / toot"
174+
}
175+
},
176+
"answer": "Hey, short tweet",
177+
"answer_file": null,
178+
"submission": "B8CD4F",
179+
"review": null,
180+
"person": null
181+
},
160182
{
161183
"question": {
162184
"id": 3412,

‎data/public/europython-2024/.gitignore‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

0 commit comments

Comments
 (0)