Skip to content

Conversation

@aldbr
Copy link
Contributor

@aldbr aldbr commented Dec 8, 2025

While trying to debug the issue we have with job parameters, I noticed we didn't have any way (or at least any obvious way) to change the log level of the job wrapper (INFO by default from what I can see).

BEGINRELEASENOTES
*WorkloadManagement
NEW: ability to adjust job wrapper log level
ENDRELEASENOTES

self.pilotRef = gConfig.getValue("/LocalSite/PilotReference", "Unknown")
self.cpuNormalizationFactor = gConfig.getValue("/LocalSite/CPUNormalizationFactor", 0.0)
self.bufferLimit = gConfig.getValue(self.section + "/BufferLimit", 10485760)
logLevel = gConfig.getValue(self.section + "/LogLevel", "VERBOSE")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "VERBOSE" already the default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, it sounds like we can actually modify the LogLevel of the JobWrapper by either:

  • changing the log level in the JDL of a given job
  • or changing the log level of the JobAgent

I will give it a try

@aldbr aldbr force-pushed the feat-jobwrapper-set-log-level branch from b205740 to f662c62 Compare December 18, 2025 16:48
@aldbr aldbr changed the title [9.0] feat: ability to adjust job wrapper log level [9.0] fix: JobWrapper does not manage to send job parameters Dec 18, 2025
@aldbr
Copy link
Contributor Author

aldbr commented Dec 18, 2025

After installing the branch in certification, I managed to investigate the issue properly and set up a reproducer.
Here is the problem:

  • The CPUNormalizationFactor job parameter is submitted as a str in DIRAC.
  • In diracx-core, we have a pydantic model for the JobParameters. The model expects to get CPUNormalizationFactor as an int. A validator is expected to convert CPUNormalizationFactor from str to int.
  • Autorest is correctly generating the fields of the JobParameters but forget about generating the validators.

The JobReport.commit() was just silently failing. Even after adding some logs to it, the message is not crystal clear but it helped me debug the problem 🙌

@aldbr aldbr marked this pull request as draft December 18, 2025 16:56
@aldbr
Copy link
Contributor Author

aldbr commented Dec 18, 2025

I made a fix that I need to test properly in the certification machine again just to make sure it was the only problem.

@aldbr aldbr force-pushed the feat-jobwrapper-set-log-level branch 2 times, most recently from 7e93406 to 661efed Compare December 19, 2025 09:12
for key in ceOptions:
if key in INTEGER_PARAMETERS:
ceOptions[key] = int(ceOptions[key])
ceOptions[key] = int(float(ceOptions[key]))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In DIRACGrid/diracx#653 we said that CPUNormalizationFactor should be treated as a float.
I still process it as an int here, because I would also need to modify diracx and OS before adding it to FLOAT_PARAMETERS.
Once merged I will open an issue to keep track of it of course.

@aldbr aldbr force-pushed the feat-jobwrapper-set-log-level branch 2 times, most recently from c3a13b6 to 4e63611 Compare December 19, 2025 13:53
@aldbr aldbr force-pushed the feat-jobwrapper-set-log-level branch from 4e63611 to 635ab15 Compare December 19, 2025 15:27
@aldbr aldbr marked this pull request as ready for review December 19, 2025 15:28
@aldbr
Copy link
Contributor Author

aldbr commented Dec 19, 2025

Tested and fixed in certification: you can have a look to the job parameters of job 10179

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants