Handle bitbucket deletion and redirection

[pombredanne]

Many repos were deleted when bitbucket dropped Mercurial support in 2020. See https://wiki.archiveteam.org/index.php/Bitbucket

https://bitbucket.org/pypa/distlib/ may be one of them. If this displayed consistently, we could follow the URL, and create a PURL "alias" in the DB?

This repository has been deleted

Our apologies, but the repository "pypa/distlib" has been deleted.

It now lives at https://github.com/pypa/distlib.

Also FWIW, some old versions were collected in SoftwareHeritage like for here https://bitbucket-archive.softwareheritage.org/new-static/e1/e1d6b725-3b7d-4c7a-8d3c-3f1ee73c96fd/attachments/
Or saved at https://archive.org/details/archiveteam_bitbucket

[Mahaboobunnisa123]

Hi @pombredanne I would like to work on this. Could you please assign this issue to me?

[ismaeeelxd]

I think we could add a new relationship to the PackageRelation model that corresponds to the alias you mentioned. This way, when Bitbucket returns a 404, we can search the metadata for other versions of the same package name. For example, if version 0.1 points to Bitbucket but version 0.2 points to GitHub, we can log the Bitbucket alias to the new VCS on GitHub. This seems about right to me or I could be missing something?

Edit: If we scale this up a little bit this should be generalized, if a package changed its VCS in a newer version we should create a new alias where the old VCS resolves to the new VCS.

Another option I would love to mention aswell, for simplicity why not just create a VCSAlias Model where old vcs maps to newer vcs in that model. That would be much simpler to implement

[ismaeeelxd]

Hello @pombredanne , just wanted to bump this up whether it's valid or not I've attached a draft PR demonstrating the last solution I've mentioned.

I think we could add a new relationship to the PackageRelation model that corresponds to the alias you mentioned. This way, when Bitbucket returns a 404, we can search the metadata for other versions of the same package name. For example, if version 0.1 points to Bitbucket but version 0.2 points to GitHub, we can log the Bitbucket alias to the new VCS on GitHub. This seems about right to me or I could be missing something?

Edit: If we scale this up a little bit this should be generalized, if a package changed its VCS in a newer version we should create a new alias where the old VCS resolves to the new VCS.

Another option I would love to mention aswell, for simplicity why not just create a VCSAlias Model where old vcs maps to newer vcs in that model. That would be much simpler to implement