Skip to content

Improve debugging for hanging CI node. Show hanging spec files in the RSpec output and a command to reproduce the current batch of tests #287

Merged
ArturT merged 16 commits intomasterfrom
hung-node
Jan 23, 2025
Merged

Improve debugging for hanging CI node. Show hanging spec files in the RSpec output and a command to reproduce the current batch of tests #287
ArturT merged 16 commits intomasterfrom
hung-node

Conversation

@ArturT
Copy link
Member

@ArturT ArturT commented Jan 20, 2025

Story

https://trello.com/c/xvarNSRi

Related

Issue:

Related PR:

Description

Improve debugging for hanging CI node.

  • Show hanging spec files in the RSpec output.
  • Show a command to reproduce the current batch of tests. It includes seed if the RSpec tests were running in random order.

Checklist reminder

  • You added the changes to the UNRELEASED section of the CHANGELOG.md, including the needed bump (ie, patch, minor, major)
  • You follow the architecture outlined below for RSpec in Queue Mode, which is a work in progress (feel free to propose changes):
    • Pure: lib/knapsack_pro/pure/queue/rspec_pure.rb contains pure functions that are unit tested.
    • Extension: lib/knapsack_pro/extensions/rspec_extension.rb encapsulates calls to RSpec internals and is integration and e2e tested.
    • Runner: lib/knapsack_pro/runners/queue/rspec_runner.rb invokes the pure code and the extension to produce side effects, which are integration and e2e tested.

puts 'Use the following backtrace(s) to find the line of code that got stuck if the CI node hung and terminated your tests.'
puts 'How to read the backtrace: https://knapsackpro.com/perma/ruby/backtrace-debugging'

post_log_threads(threads)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plea

"Post" does not work because this happens in the middle of log_threads. What about log_current_tests?

Also, we are using both log and print, maybe we should converge into one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we are using both log and print, maybe we should converge into one?

Are you referring to the log_current_tests method implemented in the child class (lib/knapsack_pro/runners/queue/rspec_runner.rb)?

We use puts to print info to stdout there. You can't write to a log file from inside of a trap.

I think the general name log_current_tests is fine. We log stuff, under the hood they are logged to stdout.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an inconsistency: print_current_batch_rspec_command uses puts, log_current_tests uses puts.

So they both use puts but why is one called "print" and the other "log"?

ArturT and others added 8 commits January 23, 2025 14:28
Co-authored-by: Riccardo <riccardo.odone@gmail.com>
  Failure/Error: expect(actual.stdout).to match(/spec_integration\/b_spec\.rb:7:in .*kill/).twice

     NoMethodError:
       undefined method `twice' for an instance of RSpec::Matchers::BuiltIn::Match
@ArturT ArturT requested a review from 3v0k4 January 23, 2025 14:01
puts 'Use the following backtrace(s) to find the line of code that got stuck if the CI node hung and terminated your tests.'
puts 'How to read the backtrace: https://knapsackpro.com/perma/ruby/backtrace-debugging'

post_log_threads(threads)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an inconsistency: print_current_batch_rspec_command uses puts, log_current_tests uses puts.

So they both use puts but why is one called "print" and the other "log"?

Co-authored-by: Riccardo <riccardo.odone@gmail.com>
@ArturT
Copy link
Member Author

ArturT commented Jan 23, 2025

There's an inconsistency: print_current_batch_rspec_command uses puts, log_current_tests uses puts.

So they both use puts but why is one called "print" and the other "log"?

Ok. I'll do log_current_batch_rspec_command. log is a higher abstraction than print.

@ArturT ArturT merged commit 517b103 into master Jan 23, 2025
45 checks passed
@ArturT ArturT deleted the hung-node branch January 23, 2025 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants