Bump `pyyaml` and `orjson` by mamercad · Pull Request #6029 · StackStorm/st2

mamercad · 2023-10-05T11:21:32Z

This is what I'm currently seeing under Python 3.8.10 with PyYAML 5.4.1:

❯ pip3 install st2client
Collecting st2client
  Using cached st2client-3.8.0-py3-none-any.whl (184 kB)
Requirement already satisfied: argcomplete==1.12.2 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages (from st2client) (1.12.2)
Requirement already satisfied: cffi<1.15.0 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages (from st2client) (1.14.6)
Requirement already satisfied: chardet<3.1.0 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages (from st2client) (3.0.4)
Collecting cryptography==3.4.7 (from st2client)
  Using cached cryptography-3.4.7-cp36-abi3-macosx_11_0_arm64.whl (1.9 MB)
Requirement already satisfied: importlib-metadata==3.10.1 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/importlib_metadata-3.10.1-py3.8.egg (from st2client) (3.10.1)
Requirement already satisfied: jsonpath-rw==1.4.0 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages (from st2client) (1.4.0)
Requirement already satisfied: jsonschema==2.6.0 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages (from st2client) (2.6.0)
Collecting orjson==3.5.2 (from st2client)
  Using cached orjson-3.5.2-cp38-cp38-macosx_10_9_universal2.whl (432 kB)
Requirement already satisfied: prettytable==2.1.0 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/prettytable-2.1.0-py3.8.egg (from st2client) (2.1.0)
Requirement already satisfied: prompt-toolkit==1.0.15 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/prompt_toolkit-1.0.15-py3.8.egg (from st2client) (1.0.15)
Requirement already satisfied: pyOpenSSL<=21.0.0 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/pyOpenSSL-21.0.0-py3.8.egg (from st2client) (21.0.0)
Requirement already satisfied: pysocks in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/PySocks-1.7.1-py3.8.egg (from st2client) (1.7.1)
Requirement already satisfied: python-dateutil==2.8.1 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/python_dateutil-2.8.1-py3.8.egg (from st2client) (2.8.1)
Requirement already satisfied: python-editor==1.0.4 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/python_editor-1.0.4-py3.8.egg (from st2client) (1.0.4)
Requirement already satisfied: pytz==2021.1 in /Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/pytz-2021.1-py3.8.egg (from st2client) (2021.1)
Collecting pyyaml==5.4.1 (from st2client)
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [62 lines of output]
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      /private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!
      
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
      
              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
      
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
      
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      Traceback (most recent call last):
        File "/Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/mmercado/.asdf/installs/python/3.8.10/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 318, in run
          self.find_sources()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 326, in find_sources
          mm.run()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 548, in run
          self.add_defaults()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 586, in add_defaults
          sdist.add_defaults(self)
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/command/sdist.py", line 113, in add_defaults
          super().add_defaults()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
        File "<string>", line 201, in get_source_files
        File "/private/var/folders/b3/p5g7v6vn4dxfw4wqdzt62k500000gq/T/pip-build-env-6ar1rfrn/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

arm4b · 2023-10-06T08:16:49Z

CHANGELOG.rst

+Changed
+~~~~~~~


There's already a Changed sub-section below under in development

arm4b · 2023-10-06T08:17:30Z

CHANGELOG.rst

+Changed
+~~~~~~~
+
+* Bumped `pyyaml` to `6.0.1` and `orjson` to `3.9.7`


Suggested change

* Bumped `pyyaml` to `6.0.1` and `orjson` to `3.9.7`

* Bumped `pyyaml` to `6.0.1` and `orjson` to `3.9.7`. #6029

arm4b · 2023-10-06T08:25:28Z

Thanks for the contribution!
Per https://github.com/StackStorm/st2/actions/runs/6418560182/job/17426603946?pr=6029#step:10:474
looks like the error:

ERROR: Could not find a version that satisfies the requirement orjson==3.9.7
ERROR: No matching distribution found for orjson==3.9.7

Additionally, there are more py lock files that need a regeneration.
Here is an example: #6015
@cognifloyd @amanda11 @nzlosh What were the instructions to update the pip dependencies and regenerate the lock files with the new pants builds? I couldn't find anything quick in the https://docs.stackstorm.com/development/index.html#contributing and https://github.com/StackStorm/st2/blob/master/CONTRIBUTING.rst, - probably looked in the wrong place.

mamercad · 2023-10-06T11:59:38Z

Thanks for the contribution! Per https://github.com/StackStorm/st2/actions/runs/6418560182/job/17426603946?pr=6029#step:10:474 looks like the error:
ERROR: Could not find a version that satisfies the requirement orjson==3.9.7
ERROR: No matching distribution found for orjson==3.9.7
Additionally, there are more py lock files that need a regeneration. Here is an example: #6015 @cognifloyd @amanda11 @nzlosh What were the instructions to update the pip dependencies and regenerate the lock files with the new pants builds? I couldn't find anything quick in the https://docs.stackstorm.com/development/index.html#contributing and https://github.com/StackStorm/st2/blob/master/CONTRIBUTING.rst, - probably looked in the wrong place.

Yeah, I'm not sure this'll ever work on Python 3.6, support was dropped in orjson==3.6.2.

mamercad · 2023-10-07T14:26:49Z

Thanks for the contribution! Per https://github.com/StackStorm/st2/actions/runs/6418560182/job/17426603946?pr=6029#step:10:474 looks like the error:
ERROR: Could not find a version that satisfies the requirement orjson==3.9.7
ERROR: No matching distribution found for orjson==3.9.7
Additionally, there are more py lock files that need a regeneration. Here is an example: #6015 @cognifloyd @amanda11 @nzlosh What were the instructions to update the pip dependencies and regenerate the lock files with the new pants builds? I couldn't find anything quick in the https://docs.stackstorm.com/development/index.html#contributing and https://github.com/StackStorm/st2/blob/master/CONTRIBUTING.rst, - probably looked in the wrong place.
Yeah, I'm not sure this'll ever work on Python 3.6, support was dropped in orjson==3.6.2.

A workaround might be, first, pip3 install "cython<3.0.0" pyyaml==5.4.1 --no-build-isolation -v.

amanda11 · 2023-10-09T06:54:12Z

If 3.9.7 is not available for python 3.6 then we can't bump to that version, as we are still using pip and therefore the same requirement files are used for python 3.6 and 3.8.
To re-generate the pants lock files then you can use:
./pants generate-lockfiles
If only need to update the st2 pants lock file then you can use:
./pants generate-lockfiles --resolve=st2
The pip requirements are just installed by using the pip requirements.txt files that are generated by the Makefile, so not sure how we could do the workaround suggested above.

There are some instructions in: #5789, and also in the st2/lockfiles/st2-constraints.txt and st2/pants-requirements.txt

Bump pyyaml and orjson

7d502dd

pull-request-size bot added the size/S PR that changes 10-29 lines. Very easy to review. label Oct 5, 2023

Update the changelog

24f6e31

mamercad changed the title ~~Bump pyyaml and orjson~~ Bump pyyaml and orjson Oct 5, 2023

arm4b reviewed Oct 6, 2023

View reviewed changes

CHANGELOG.rst

Comment on lines +7 to +8

Changed

~~~~~~~

Copy link

Member

arm4b Oct 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already a Changed sub-section below under in development

arm4b reviewed Oct 6, 2023

View reviewed changes

arm4b mentioned this pull request Dec 13, 2023

TSC: Promote Mark Mercado (@mamercad) from Contributors to Maintainers #6096

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Bump `pyyaml` and `orjson`#6029

Bump `pyyaml` and `orjson`#6029
mamercad wants to merge 2 commits intoStackStorm:masterfrom
mamercad:pyyaml

mamercad commented Oct 5, 2023 •

edited

Loading

Uh oh!

arm4b Oct 6, 2023

Uh oh!

arm4b Oct 6, 2023

Uh oh!

arm4b commented Oct 6, 2023

Uh oh!

mamercad commented Oct 6, 2023

Uh oh!

mamercad commented Oct 7, 2023

Uh oh!

amanda11 commented Oct 9, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	* Bumped `pyyaml` to `6.0.1` and `orjson` to `3.9.7`
	* Bumped `pyyaml` to `6.0.1` and `orjson` to `3.9.7`. #6029

Uh oh!

Conversation

mamercad commented Oct 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arm4b Oct 6, 2023

Choose a reason for hiding this comment

Uh oh!

arm4b Oct 6, 2023

Choose a reason for hiding this comment

Uh oh!

arm4b commented Oct 6, 2023

Uh oh!

mamercad commented Oct 6, 2023

Uh oh!

mamercad commented Oct 7, 2023

Uh oh!

amanda11 commented Oct 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mamercad commented Oct 5, 2023 •

edited

Loading

amanda11 commented Oct 9, 2023 •

edited

Loading