Commit 9e39a06
committed
Handle SQLite cache errors gracefully instead of aborting upload
Summary:
- Fix sqlite3.OperationalError: disk I/O error in the file handle cache from crashing the entire upload process
- PersistentCache.get() now catches sqlite3.DatabaseError and returns None (cache miss) instead of propagating the exception
- Cache failures are logged as warnings so they remain visible for debugging
Context
The upload file handle cache is a SQLite database that stores server-returned handles for already-uploaded images. It is purely an optimization to skip re-uploading — if the cache is
unavailable, the server's fetch_offset endpoint confirms prior uploads at the cost of one extra HTTP round-trip per image.
Previously, any SQLite error during cache lookup (disk I/O error, corrupt database, etc.) propagated up through _continue_or_fail which treated it as a fatal error, aborting the upload of all
remaining images. This was observed on Windows with 4 concurrent upload workers rapidly opening/closing SQLite connections to the same cache file.1 parent abc0056 commit 9e39a06
File tree
2 files changed
+24
-1
lines changed- mapillary_tools
- tests/unit
2 files changed
+24
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
122 | 128 | | |
123 | 129 | | |
124 | 130 | | |
125 | 131 | | |
126 | 132 | | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
127 | 136 | | |
128 | 137 | | |
129 | 138 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
372 | 372 | | |
373 | 373 | | |
374 | 374 | | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
375 | 389 | | |
376 | 390 | | |
377 | 391 | | |
| |||
0 commit comments