feat: migrate tokenization to agentplatform#6729

copybara-service[bot] wants to merge 1 commit intomaingoogleapis/python-aiplatform:mainfrom

copybara_909678264googleapis/python-aiplatform:copybara_909678264Copy head branch name to clipboard