Enhance session handling and filename sanitation#48
Enhance session handling and filename sanitation#48VictorHarbo wants to merge 10 commits intobitdruid:mainfrom
Conversation
Added option to limit path depth Added option to limit snapshots pr. url Both are optional
…ters Add additional '#=!~' sanitizing to filenames and subdir foldernames
Fix non closed transaction
|
thank you i will check this out when ive enough time. meanwhile what are your thoughts about changes to the db into a single file (one table per request OR optional define a /etc/pywaybackup/config.yml for useing an existing sql / psql db)? id like to make this project more integratable into persistent infrastructures. currently im a bit playing around with a very simple flask ui. thats why i started implementing module functionality... |
…update related logic
…ds and added doc strings
Add more limiter functions to python-wayback-machine-downloader
There was a problem hiding this comment.
after i reviewed your code i like the additional verbose output but i think it should be moved behind a "debug" flag. either if --debug is already set or --verbose debug (loglevels...) so maybe a good time to implement loglevels at all. also i would prefer a smaller naming for the unique snapshots-limit. something like --unique or other. just to keep it simple and clean. thank you for the good work
|
i added the necessary tier-system in 964c519 |
3 participants
This pull request introduces some minor improvements to database session handling and filename sanitation.
The changes focus on ensuring resources are properly closed to prevent leaks. Also, filename sanitation is improved to handle more special characters as we have encountered errors with question marks for instance. The work has been done by @jorntx and I.
Database session management and error handling:
Database.close()indb.py, ensuring all pending transactions are handled and sessions are properly closedSnapshot.pywhen fetching and modifying snapshot recordsclose()method toWorker.pyto properly close database and HTTP connections with logging and error suppressionFilename and path sanitization: