TSC Meeting (07 Jun 2022): Exchange "!bootstrap pack" demo, AWS disaster plan, v3.7.0 retrospective, v3.8.0 planning

[arm4b]

June 2022 @StackStorm/tsc 1 hour planned meeting: Tuesday, 07 Jun 2022, 09:30 AM US Pacific.

See #33 for more info on how to join.
(TLDR; Zoom link: https://us02web.zoom.us/j/81082101702?pwd=N1V4TWdYRVQ4SXBsaFh1TFYvVDA0UT09)

Meeting host: @armab

Meeting Agenda

Demo: StackStorm-Exchange !bootstrap pack DEMO (@cognifloyd)

• More info about the project: [RFC] Use Github Actions for StackStorm-Exchange CI and Maintenance #63

AWS account migration follow-up

• AWS credits expiration date 31-July-22 is getting closer
• AWS ownership transfer @punkrokk -> @bishopbm1 status
• AWS Cost Reduction project: https://github.com/orgs/StackStorm/projects/27
• Follow-up on the remaining distributed tasks (Slack link):
  • run st2-self-check in GH Actions, instead of st2cicd e2e tests (@winem)
  • Migrate api.stackstorm.com to GH static pages (@bishopbm1)
  • No Owner: Experiment with the AWS spot instances for running e2e test VMs: U16,U18,EL7,EL8 ()
  • Migrate docs.stackstorm.com to GH static pages #1096 (@guzzijones) New task owner?
• How we can further reduce infra/mgmt dependency risks?

AWS shutdown disaster plan

In case of AWS shutdown, what are the other options for shipping the st2 release?

• Just do a Github version release, and let the users build packages?
  • opportunity for the partners to provide customized st2 builds?
• Stop e2e VM OS tests, just deploy untested deb/rpm to the apt/yum repos?
• Drop OS support in favor of just core python version support (py3.8, etc)?
• Build Docker images only?

StackStorm v3.7 release feedback (@cognifloyd & @nzlosh)

• What went well?
• What could be improved?

StackStorm v3.8 release planning

• Project: https://github.com/orgs/StackStorm/projects/28
• Python 3.6 deprecation & OS support
• Release target date
• Release managers?
  • needs a volunteer from @StackStorm/maintainers

[arm4b]

Attendees

@cognifloyd @mickmcgrath13 @amanda11 @rush-skills @winem @armab @nzlosh @bishopbm1, Dale Smith @ Orchestral

Meeting Minutes

AWS account migration follow-up

• Good news: @bishopbm1 already had a chat with @punkrokk about AWS account migration. The ownership transfer is going by plan and hopefully will be completed before the next TSC meeting (July).

AWS cost reduction project

• AWS went from $500/mo to $350/mo with the optimizations from the https://github.com/orgs/StackStorm/projects/27
• Follow-up on the remaining distributed tasks (Slack link):
  • run st2-self-check in GH Actions, instead of st2cicd e2e tests - @winem will try to skip some CI tests and also try to use the ST2 CI containers (https://github.com/StackStorm/st2packaging-dockerfiles) that are shipped with ssh + systemd to run the ST2 tests in a reproducible/portable environment.
  • Migrate api.stackstorm.com to GH static pages - @bishopbm1 will start working on that as he's finished the Puppet metrics PR.
  • Migrate docs.stackstorm.com to GH static pages #1096 (@guzzijones) New task owner - @winem? Orchestral? @bishopbm1? In a meantime, @mickmcgrath13 drafted a PR that includes some first steps: GitHub action to publish docs to GH Pages st2docs#1129 so anyone could continue.
• More ideas by @amanda11 in https://github.com/orgs/StackStorm/projects/27:
  • Explore scheduling start/stop for st2cicd instance based on bi-weekly e2e runs to reduce idle cycles
  • Explore removing the AWS Load-Balancer in front of st2cicd (-$50/mo)
  • Rebuild the st2cicd with the cheaper instance type

AWS deprecation ideas and ST2 packaging & distribution

• AWS now holds the st2cicd machinery only
• st2cicd is there for e2e OS tests, deb/rpm packaging, releases
• OS support, OS migration work, versions, compatibility, and packaging is a burden for maintainers
  • See recent RabbitMQ/Erlang issues: EL8 install fails with issues reported in regard to Erlang 25 st2#5651 Use version of erlang < 1.25 and fix EL8 passlib ansible-st2#313
  • ideally, if we ship st2 and users have the ability to come with their own versions:
    • python3 version st2 core supports
    • MongoDB
    • RabbitMQ
    • OS they prefer
• Deprecating a st2cicd internal infra, doing just an st2 Github release with minimal artifacts, and supporting the core st2 with some CI py3 versions checks sounds promising to the maintainers.
• Docker/K8s is great, but still, need something for the raw OS installation.
• Find the balance between the platform portability, ease of installation for the community, and maintenance burden.
• Moving more services, tests, and release processes out of the AWS to public CI/GH Actions is a long-term strategy.
• Getting rid of AWS infra completely in the long-term future would be great eventually.
• @armab to start a written Github discussion to get more detailed/technical insights from the @StackStorm/tsc, partners, and community.

Demo: StackStorm-Exchange !bootstrap pack DEMO (@cognifloyd)

• Previously it was such a nightmare for @StackStorm/maintainers to onboard new st2 pack to Exchange, contributed by the community. Nobody wanted to touch it.
• Now it's a single !bootstrap Github command in the PR that bootstraps almost everything for the new pack.
• Jacob provided a really cool overview with the new pack bootstrap experience 🚀

StackStorm v3.7 release feedback (@cognifloyd & @nzlosh)

• moving some part of the release machinery to publicly visible GH actions would be nice.
• internal st2cicd and e2e tests is a burden, but can't replace everything with the GH Actions (windows, VMs).

Next TSC proposal: @bishopbm1 will show a demo for Metrics & Puppet (10min, start of the meeting)

StackStorm v3.8 release planning

• Release Manager: @nzlosh
• Release Assistant: @rush-skills

Partners Q/A

• Dale from Orchestral asked about the partners page mechanics & coordination.
• More info and follow-up in the StackStorm-Community #partners Slack: https://stackstorm-community.slack.com/archives/C020VUQH8P5/p1654628520257939