Skip to content

Conversation

@sarvekshayr
Copy link
Contributor

What changes were proposed in this pull request?

Added a flag --in-progress to include open files/keys and multipart uploads in ContainerToKeyMapping tool.
When --onlyFileNames is set, then add FSO and OBS tags to differentiate the keys.

What is the link to the Apache JIRA

HDDS-14518

How was this patch tested?

Only shows files and keys from fileTable and keyTable respectively.

ozone debug om container-key-mapping --db /data/metadata/om.db --containers 1,2,3,4
{
  "containers" : {
    "1" : {
      "keys" : [ "/vol1/fso-bucket/dir1/file1" ],
      "totalKeys" : 1
    },
    "2" : {
      "keys" : [ "/vol1/obs-bucket/key1" ],
      "totalKeys" : 1
    },
    "3" : {
      "keys" : [ "[unreferenced] unreferencedFile" ],
      "totalKeys" : 1,
      "unreferencedKeys" : 1
    },
    "4" : {
      "keys" : [ ],
      "totalKeys" : 0
    }
  }
}

Shows open files and keys from openFileTable and openKeyTable respectively. Also includes multipart uploads info from multipartInfoTable

ozone debug om container-key-mapping --db /data/metadata/om.db --containers 1,2,3,4 --in-progress
{
  "containers" : {
    "1" : {
      "keys" : [ "/vol1/fso-bucket/dir1/file1" ],
      "openKeys" : [ "/vol1/fso-bucket/dir1/openFile" ],
      "totalKeys" : 2
    },
    "2" : {
      "keys" : [ "/vol1/obs-bucket/key1" ],
      "openKeys" : [ "/vol1/obs-bucket/openKey" ],
      "totalKeys" : 2
    },
    "3" : {
      "keys" : [ "[unreferenced] unreferencedFile" ],
      "openKeys" : [ ],
      "totalKeys" : 1,
      "unreferencedKeys" : 1
    },
    "4" : {
      "keys" : [ ],
      "openKeys" : [ "[mpu-part-1] /vol1/obs-bucket/mpuKey/test-upload-id/part-1", "[mpu-part-2] /vol1/obs-bucket/mpuKey/test-upload-id/part-2" ],
      "totalKeys" : 2
    }
  }
}

Show only file names

ozone debug om container-key-mapping --db /data/metadata/om.db --containers 1,2,3,4 --in-progress --onlyFileNames
{
  "containers" : {
    "1" : {
      "keys" : [ "[fso] file1" ],
      "openKeys" : [ "[fso] openFile" ],
      "totalKeys" : 2
    },
    "2" : {
      "keys" : [ "[obs] key1" ],
      "openKeys" : [ "[obs] openKey" ],
      "totalKeys" : 2
    },
    "3" : {
      "keys" : [ "[fso] unreferencedFile" ],
      "openKeys" : [ ],
      "totalKeys" : 1
    },
    "4" : {
      "keys" : [ ],
      "openKeys" : [ "[mpu-part-1-obs] mpuKey/test-upload-id/part-1", "[mpu-part-2-obs] mpuKey/test-upload-id/part-2" ],
      "totalKeys" : 2
    }
  }
}

Copy link
Contributor

@sreejasahithi sreejasahithi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sarvekshayr for working on this.
Could you please reduce the duplication between processOpenFiles() and processOpenKeys().

@sarvekshayr sarvekshayr added the tools Tools that helps with debugging label Feb 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tools Tools that helps with debugging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants