Crates.io differences

AGENTS.md

@@ -0,0 +1 @@
+CLAUDE.md

CLAUDE.md

@@ -0,0 +1,68 @@
+# CLAUDE.md
+
+The role of this file is to describe common mistakes and confusion points that agents might encounter as they work in this project.
+If you ever encounter something in the project that surprises you, please alert the developer working with you and indicate that this is the case in the CLAUDE.md file to help prevent future agents from having the same issue.
+
+## Interacting with the developer environment
+
+Use the `pulp-cli` to interact with the Pulp API. Fallback on `httpie/curl` when the CLI doesn't support the endpoint/options needed. See the [user guides](docs/user/guides/) for workflow examples.
+
+Use the `oci-env` cli to interact with the developer's Pulp instance. It has commands for managing state, running tests, and executing commands against a running Pulp.
+
+```bash
+oci-env --help
+oci-env compose ps  # check status of the Pulp dev container
+oci-env compose up/down/restart  # start/stop/restart the Pulp dev container
+oci-env poll --attempts 10 --wait 10  # wait till Pulp container finishes booting up
+oci-env pstart/pstop/prestart  # start/stop/restart the services inside the Pulp container
+oci-env generate-client --help  # create the client bindings needed for the functional tests!
+oci-env test --help # run the functional/unit tests
+oci-env pulpcore-manager  # run any pulpcore or Django commands
+```
+
+## Running/Writing tests
+
+Prefer writing functional tests for new changes/bugfixes and only fallback on unit tests when the change is not easily testable through the API.
+
+pulpcore & pulp_rust functional tests require both client bindings to be installed. The bindings must be regenerated for any changes to the API spec.
+
+**Always** use the `oci-env` to run the functional and unit tests.
+
+## Modifying template_config.yml
+
+Use the `plugin-template` tool after any changes made to `template_config.yml`.
+
+```bash
+# typically located in the parent directory of pulpcore/plugin
+../plugin_template/plugin-template --github
+```
+
+## Fixing failed backports
+
+When patchback fails to cherry-pick a PR into an older branch, you need to manually apply the equivalent change. Key things to know:
+
+- When creating a PR include `[<version>]` in the PR title (e.g. `[0.1] Fix crate upload handling`).
+- Use `git cherry-pick -x`.
+
+## Contributing
+
+When preparing to commit and create a PR you **must** follow our [PR checklist](https://pulpproject.org/pulpcore/docs/dev/guides/pull-request-walkthrough/) Important to note is the AI attribution requirement in our commit messages. Also, note that our changelog entries are markdown.
+
+## Project status
+
+This project is in **Tech Preview**. APIs, behaviors, and data models are subject to breaking changes without prior notice. See the README for current limitations.
+
+## Cargo protocol & domain knowledge
+
+For understanding the Cargo registry protocol, refer to the upstream documentation:
+
+- [Cargo Book](https://doc.rust-lang.org/stable/cargo/)
+- [Registry Web API](https://doc.rust-lang.org/cargo/reference/registry-web-api.html)
+- [Registry Index](https://doc.rust-lang.org/cargo/reference/registry-index.html)
+
+## Common pitfalls
+
+- **Cargo.toml is authoritative during publish**: When a crate is published, dependencies are extracted from the `Cargo.toml` inside the `.crate` tarball, NOT from the JSON metadata submitted alongside it. This is an intentional security measure (see rust-lang/cargo#14492).
+- **RustDependency is NOT a Content type**: Unlike `RustContent` and `RustPackageYank`, `RustDependency` is a regular Django model with an FK to `RustContent`. Do not treat it as a Pulpcore Content subclass.
+- **Django app label is `rust`, not `pulp_rust`**: When running Django management commands (e.g. `makemigrations`), use the app label `rust`. The Python package is `pulp_rust` but the Django app label is set to `rust` in `PulpRustPluginAppConfig`.
+

docs/user/guides/private-registry.md

@@ -131,10 +131,16 @@ private registry. Any crate not present in the registry will fail to resolve.
 
 ## Combining with Pull-Through Caching
 
-If you need both private crates and public crates.io dependencies, we recommend keeping them as
-**separate registries** rather than mixing them into one. This avoids
-[dependency confusion](https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610) attacks,
-where a malicious package on a public registry could impersonate a private dependency.
+If you need both private crates and public crates.io dependencies, use separate distributions and
+separate repositories -- one for publishing and one for pull-through caching. Pulp enforces that
+a distribution cannot have both `allow_uploads` and a `remote` set at the same time.
+
+!!! warning
+    The two distributions should also use **separate repositories**. Mixing public and private
+    content in a single repository creates a risk of
+    [dependency confusion](https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610)
+    attacks, where a crate from the public registry conflicts with a private crate of the same
+    name and version.
 
 ```bash
 # Set up a separate pull-through cache for crates.io
@@ -167,11 +173,18 @@ serde = "1.0"                                              # resolved from crate
 my-internal-lib = { version = "1.0", registry = "my-crates" }  # resolved from private registry
 ```
 
-!!! warning
-    Avoid adding a public remote (such as crates.io) to a private registry's distribution. Mixing
-    public and private packages in a single registry index creates a risk of dependency confusion
-    attacks, where an attacker publishes a crate on the public registry with the same name as one
-    of your private crates.
+## Crate Name Handling
+
+Crate names in the Cargo spec are case-insensitive, and hyphens and underscores are treated as
+equivalent (e.g. `my-crate` and `my_crate` refer to the same package). Pulp enforces this:
+publishing `my-crate` when `my_crate` already exists in the same repository is rejected as a
+duplicate. Yank and unyank operations use the same matching.
+
+!!! tip "Separate Registries"
+    Keep private registries and public pull-through caches as separate distributions (and
+    preferably separate repositories). This makes it easy to audit which registries have
+    upstream access and reduces the risk of accidental misconfiguration. For additional
+    isolation or access control, they could be kept on entirely separate domains.
 
 ## Further Reading
 

pulp_rust/app/migrations/0001_initial.py

@@ -19,6 +19,7 @@ class Migration(migrations.Migration):
             fields=[
                 ('content_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='core.content')),
                 ('name', models.CharField(db_index=True, max_length=255)),
+                ('canonical_name', models.CharField(db_index=True, max_length=255)),
                 ('vers', models.CharField(db_index=True, max_length=64)),
                 ('cksum', models.CharField(db_index=True, max_length=64)),
                 ('features', models.JSONField(blank=True, default=dict)),
@@ -30,15 +31,15 @@ class Migration(migrations.Migration):
             ],
             options={
                 'default_related_name': '%(app_label)s_%(model_name)s',
-                'unique_together': {('name', 'vers', '_pulp_domain')},
+                'unique_together': {('name', 'vers', 'cksum', '_pulp_domain')},
             },
             bases=('core.content',),
         ),
         migrations.CreateModel(
             name='RustDistribution',
             fields=[
                 ('distribution_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='core.distribution')),
-                ('allow_uploads', models.BooleanField(default=True)),
+                ('allow_uploads', models.BooleanField(default=False)),
             ],
             options={
                 'default_related_name': '%(app_label)s_%(model_name)s',

pulp_rust/app/models.py

@@ -5,7 +5,11 @@
 from django.db import models
 from django_lifecycle import hook, AFTER_CREATE
 
-from pulp_rust.app.utils import extract_cargo_toml, extract_dependencies
+from pulp_rust.app.utils import (
+    canonicalize_crate_name,
+    extract_cargo_toml,
+    extract_dependencies,
+)
 
 from pulpcore.plugin.models import (
     Content,
@@ -55,8 +59,25 @@ class RustContent(Content):
     Cargo registry index specification. Each instance corresponds to one line
     in a package's index file.
 
+    The `name` field preserves the original crate name as it appears in the
+    package's `Cargo.toml` (e.g. `cfg-if`, `Serde-JSON`).  This matches
+    crates.io behavior and ensures that download paths and index entries use
+    the author's intended name form.
+
+    The `canonical_name` field stores the canonical form (lowercased, hyphens
+    replaced with underscores) for use in lookups where the Cargo spec's
+    case-insensitive, hyphen/underscore-equivalent matching is needed - for
+    example, duplicate detection during publish and yank operations.
+
+    Content uniqueness is enforced on `(name, vers, cksum, _pulp_domain)`.
+    Including `cksum` allows different crates with the same name and version
+    (e.g. a private crate and a public crate) to coexist as separate content
+    objects within a domain, while `repo_key_fields` prevents both from
+    appearing in the same repository version.
+
     Fields:
-        name: The package name (crate name)
+        name: The package name as it appears in Cargo.toml
+        canonical_name: Canonical form (lowercased, hyphens -> underscores)
         vers: The semantic version string (SemVer 2.0.0)
         cksum: SHA256 checksum of the .crate file (tarball)
         features: JSON object mapping feature names to their dependencies
@@ -69,9 +90,15 @@ class RustContent(Content):
     TYPE = "rust"
     repo_key_fields = ("name", "vers")
 
-    # Package name - alphanumeric characters, hyphens, and underscores allowed
+    # Package name as it appears in the crate's Cargo.toml.
     name = models.CharField(max_length=255, blank=False, null=False, db_index=True)
 
+    # Canonical form of the name: lowercased with hyphens replaced by underscores.
+    # Used for lookups where the Cargo spec's case-insensitive,
+    # hyphen/underscore-equivalent matching is needed (e.g. duplicate detection,
+    # yank operations).
+    canonical_name = models.CharField(max_length=255, blank=False, null=False, db_index=True)
+
     # Semantic version string following SemVer 2.0.0 specification
     vers = models.CharField(max_length=64, blank=False, null=False, db_index=True)
 
@@ -111,10 +138,12 @@ def init_from_artifact_and_relative_path(artifact, relative_path):
         Cargo.toml inside the .crate tarball.
         """
         crate_name, version = _parse_crate_relative_path(relative_path)
-        cargo_toml = extract_cargo_toml(artifact.file.path, crate_name, version)
+        with artifact.file.open("rb") as f:
+            cargo_toml = extract_cargo_toml(f, crate_name, version)
 
         content = RustContent(
             name=crate_name,
+            canonical_name=canonicalize_crate_name(crate_name),
             vers=version,
             cksum=artifact.sha256,
             features=cargo_toml.get("features", {}),
@@ -136,7 +165,7 @@ def _create_dependencies_from_parsed_data(self):
 
     class Meta:
         default_related_name = "%(app_label)s_%(model_name)s"
-        unique_together = (("name", "vers", "_pulp_domain"),)
+        unique_together = (("name", "vers", "cksum", "_pulp_domain"),)
 
 
 class RustDependency(models.Model):
@@ -277,6 +306,7 @@ class RustPackageYank(Content):
 
     name = models.CharField(max_length=255, db_index=True)
     vers = models.CharField(max_length=64, db_index=True)
+
     _pulp_domain = models.ForeignKey("core.Domain", default=get_domain_pk, on_delete=models.PROTECT)
 
     class Meta:
@@ -308,7 +338,7 @@ class RustDistribution(Distribution):
 
     TYPE = "rust"
 
-    allow_uploads = models.BooleanField(default=True)
+    allow_uploads = models.BooleanField(default=False)
 
     class Meta:
         default_related_name = "%(app_label)s_%(model_name)s"

pulp_rust/app/serializers.py

@@ -7,6 +7,7 @@
 from pulpcore.plugin import serializers as core_serializers
 
 from . import models
+from .utils import canonicalize_crate_name
 
 log = logging.getLogger(__name__)
 
@@ -153,6 +154,7 @@ class RustContentSerializer(core_serializers.SingleArtifactContentSerializer):
     def create(self, validated_data):
         """Create RustContent and related dependencies."""
         dependencies_data = validated_data.pop("dependencies", [])
+        validated_data["canonical_name"] = canonicalize_crate_name(validated_data["name"])
         content = super().create(validated_data)
 
         # Create dependency records
@@ -243,7 +245,7 @@ class Meta:
     """
 
     allow_uploads = serializers.BooleanField(
-        default=True, help_text=_("Allow packages to be uploaded to this index.")
+        default=False, help_text=_("Allow packages to be uploaded to this index.")
     )
     remote = core_serializers.DetailRelatedField(
         required=False,
@@ -253,6 +255,21 @@ class Meta:
         allow_null=True,
     )
 
+    def validate(self, data):
+        data = super().validate(data)
+        remote = data.get("remote", self.instance.remote if self.instance else None)
+        allow_uploads = data.get(
+            "allow_uploads", self.instance.allow_uploads if self.instance else False
+        )
+        if remote and allow_uploads:
+            raise serializers.ValidationError(
+                _(
+                    "A distribution cannot have both a remote and allow_uploads enabled. "
+                    "Use separate distributions for pull-through caching and publishing."
+                )
+            )
+        return data
+
     class Meta:
         fields = core_serializers.DistributionSerializer.Meta.fields + ("allow_uploads", "remote")
         model = models.RustDistribution