Some changes in handling the arrays that represent force fields, potentials and densities #347

mondracek · 2025-06-02T16:48:20Z

Partial step to address #198. Introduces the xyz order of numpy indices of arrays that represent fields in the CLI version. Unifies the functions that set the fields to be passed from a numpy array to the C++ code under one new function, setFF in ppafm/HighLevel.py.

so that vectors fields are stored as F[ix,iy,iz,i] and scalar fields as V[ix,iy,iz], both in C layout

…olate3DvecWrap, so as to prevent SegFault with the pyridineDensOverlap example.

…e index order redefinition

and memory layout in saveWSxM functions in io.py

by flipping the array indices of force fields

in ppafm/cli/relaxed_scan.py

…:, :, :, 1] in ppafm/cli/relaxed_scan.py

in ppafm/cli/plot_results.py

codecov · 2025-06-02T17:00:53Z

Codecov Report

Attention: Patch coverage is 45.91195% with 86 lines in your changes missing coverage. Please review.

Project coverage is 55.61%. Comparing base (c6e7e74) to head (248047e).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
ppafm/PPPlot.py	12.00%	44 Missing ⚠️
ppafm/HighLevel.py	66.66%	20 Missing ⚠️
ppafm/io.py	54.54%	5 Missing ⚠️
ppafm/cli/generateElFF.py	0.00%	4 Missing ⚠️
ppafm/common.py	50.00%	4 Missing ⚠️
ppafm/cli/relaxed_scan.py	57.14%	3 Missing ⚠️
ppafm/fieldFFT.py	80.00%	2 Missing ⚠️
ppafm/ocl/field.py	0.00%	2 Missing ⚠️
ppafm/cli/plot_results.py	0.00%	1 Missing ⚠️
ppafm/ocl/AFMulator.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #347   +/-   ##
=======================================
  Coverage   55.61%   55.61%           
=======================================
  Files          40       40           
  Lines        6252     6273   +21     
=======================================
+ Hits         3477     3489   +12     
- Misses       2775     2784    +9

Flag	Coverage Δ
python-3.11	`55.61% <45.91%> (?)`
python-3.12	`?`
python-3.13	`55.61% <45.91%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mondracek · 2025-06-03T08:43:53Z

I forgot to mention that this PR also resolves #346.

NikoOinonen

I didn't look at everything in detail, but I tried running all of the examples and this does seem to break the pyridineDensOverlap example. Some other examples were also broken but for other reasons: #348.

NikoOinonen · 2025-06-17T13:01:29Z

ppafm/cli/plot_results.py

-                # This is done as polynomial (2nd order) regression, the above equality need not be exact
+                # This is done as polynomial (2nd order) regression, the above equality needs not be exact


It was actually grammatically correct before.

mondracek · 2025-07-25T16:50:14Z

Hi @NikoOinonen, that is strange that the pyridineDensOverlap example should not work for you. I have tested this PR exactly on that example and it worked fine for me. Had you recompiled the C code before running the example? If it still does not work after you recompile, can you tell me more specifically how it fails? But not hurry, I have had no time for the coding recently anyway and I will not be able to work on it for at least two more weeks either...

NikoOinonen · 2025-07-28T08:08:29Z

@mondracek It specifically breaks the GPU version of the example. The code runs but the result is broken:

NikoOinonen · 2025-10-09T12:39:32Z

I tried running this again now, to see if I could figure out the problem, but now it's somehow broken the other way around. The GPU example works, but the CPU one is giving broken images. I'm just confused what is different now...

ProkopHapala · 2025-10-13T17:50:37Z

I tried running this again now, to see if I could figure out the problem, but now it's somehow broken the other way around. The GPU example works, but the CPU one is giving broken images. I'm just confused what is different now...

you can see the diff in branch history or not?

NikoOinonen · 2025-10-15T09:23:04Z

you can see the diff in branch history or not?

No, I mean I used the exact same commit on this branch, but got a different result. Maybe I had something different in my environment or some wrong file or something, I'm not sure.

mondracek · 2025-12-11T18:39:05Z

@NikoOinonen, does the pyridineDensOverlap example work out correctly for you at least with the main branch of the code? In other words, are you sure the problem is specific to the changes made in the cleanup branch?
In any case, I would very much like testing it myself, but I still cannot make the OCL version make running on any of the machines available to me. You mentioned the OCL version can run on CPU too: Can you explain me how exactly can I achieve that (in Linux)? Also, I attach here the output and error message from ppafm/examples/pyridineDensOverlap/run_gpu.py: output.txt. Any idea, what can be the problem here? I ran this with the code in the main branch, so nothing related to my changes.

NikoOinonen · 2025-12-12T09:49:24Z

You mentioned the OCL version can run on CPU too: Can you explain me how exactly can I achieve that (in Linux)?

The most general way is probably to use POCL. I wrote something about it in this issue recently: #332 (comment)

Also, I attach here the output and error message from ppafm/examples/pyridineDensOverlap/run_gpu.py: output.txt. Any idea, what can be the problem here? I ran this with the code in the main branch, so nothing related to my changes.
pyopencl._cl.MemoryError: clEnqueueNDRangeKernel failed: MEM_OBJECT_ALLOCATION_FAILURE

This is probably due to insufficient memory on the GPU. But I'm surprised it already fails at creating the Fourier transform for the tip density. How much VRAM do you have?

mondracek · 2025-12-12T15:48:39Z

@NikoOinonen, I have 2 GB of VRAM. As for using POCL: I have already installed the pocl-opencl-icd package, but the ppafm code still seems to use GPU preferentially as long as it is available. Or so I believe, as it says Initializing an OpenCL environment on NVIDIA CUDA when the calculation begins. Is there a way to tell ppafm (or Python in general) which opencl-icd it should prefer?

NikoOinonen · 2025-12-12T16:24:32Z

I have 2 GB of VRAM

This is not a lot, so that's probably the issue.

Is there a way to tell ppafm (or Python in general) which opencl-icd it should prefer?

Yes, on this line, change the i_platform:

ppafm/examples/pyridineDensOverlap/run_gpu.py

Line 14 in c0c8a9e

oclu.init_env(i_platform=0)

You can get the platform numbers easily by running ppafm-gui --list-devices.

mondracek · 2025-12-12T17:56:24Z

@NikoOinonen , thank you so much, it finally runs for me (in the main branch so far). Only, I need to be careful to close all other windows and apps I may be running at the same time (like a web browser, e-mail client etc.), otherwise the computer gets jammed and the whole XWin crashes after some time.
One suspicious message (warning) it gives during the run:

RepeatedKernelRetrieval: Kernel 'interp_tip_at' has been retrieved more than once. Each retrieval creates a new, independent kernel, at possibly considerable expense. To avoid the expense, reuse the retrieved kernel instance. To avoid this warning, use cl.Kernel(prg, name).

and somewhat later the same with interp_at' instead of interp_tip_at'. Does it indicate some inefficiency in the implementation of the GPU version? Should I file a separate issue about this? But perhaps it happens only when one runs the GPU version on CPU. If so, I think it is not worth worrying about it.

NikoOinonen · 2025-12-13T06:22:50Z

One suspicious message (warning) it gives during the run:

Yes, I saw this too, but haven't had the time to look into it. There is an issue already: #367

ProkopHapala · 2025-12-19T18:14:39Z

@NikoOinonen, I have 2 GB of VRAM. As for using POCL: I have already installed the pocl-opencl-icd package, but the ppafm code still seems to use GPU preferentially as long as it is available. Or so I believe, as it says Initializing an OpenCL environment on NVIDIA CUDA when the calculation begins. Is there a way to tell ppafm (or Python in general) which opencl-icd it should prefer?

I think it should be possible to choose specific OpenCL device by some command line argument but I do not remember which now, need to look in code or documentation

mondracek added 10 commits May 23, 2025 14:31

Rearange array indices and memory layout in CLI

326fb2a

so that vectors fields are stored as F[ix,iy,iz,i] and scalar fields as V[ix,iy,iz], both in C layout

Change the way indices are calculated in interpolate3DWrap and interp…

88a2591

…olate3DvecWrap, so as to prevent SegFault with the pyridineDensOverlap example.

Do forgotten switching of axes for np.roll inside getMGrid to complet…

583af51

…e index order redefinition

Correct index order for the capacitance term in cli/plot_results.py

18d9330

and memory layout in saveWSxM functions in io.py

Repair the bug in the --rotate option of ppafm-relaxed-scan

1125f5c

by flipping the array indices of force fields

Merge branch 'rotate' to repair the usage of rotate_ff

681b393

in ppafm/cli/relaxed_scan.py

Rename RG to RGB in ppafm/PPPlot.py

bf93f8c

Replace the erroneous expression ff_pauli[1, :, :, :, 1] by ff_pauli[…

8257dd9

…:, :, :, 1] in ppafm/cli/relaxed_scan.py

Use default "slices" argument to plot all slices with plotImages

24d488c

in ppafm/cli/plot_results.py

Merge functions prepareArrays, setFF_shape, and setFF into a new setFF

248047e

mondracek mentioned this pull request Jun 2, 2025

Suggest cleanup of geometry handling in the CPU version #198

Open

NikoOinonen reviewed Jun 17, 2025

View reviewed changes

ondrejkrejci mentioned this pull request Jul 29, 2025

CoPc-IETS example broken #348

Open

		# This is done as polynomial (2nd order) regression, the above equality need not be exact
		# This is done as polynomial (2nd order) regression, the above equality needs not be exact

Some changes in handling the arrays that represent force fields, potentials and densities #347

Are you sure you want to change the base?

Some changes in handling the arrays that represent force fields, potentials and densities #347

Uh oh!

Conversation

mondracek commented Jun 2, 2025

Uh oh!

codecov bot commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mondracek commented Jun 3, 2025

Uh oh!

NikoOinonen left a comment

Choose a reason for hiding this comment

Uh oh!

NikoOinonen Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

mondracek commented Jul 25, 2025

Uh oh!

NikoOinonen commented Jul 28, 2025

Uh oh!

NikoOinonen commented Oct 9, 2025

Uh oh!

ProkopHapala commented Oct 13, 2025

Uh oh!

NikoOinonen commented Oct 15, 2025

Uh oh!

mondracek commented Dec 11, 2025

Uh oh!

NikoOinonen commented Dec 12, 2025

Uh oh!

mondracek commented Dec 12, 2025

Uh oh!

NikoOinonen commented Dec 12, 2025

Uh oh!

mondracek commented Dec 12, 2025

Uh oh!

NikoOinonen commented Dec 13, 2025

Uh oh!

ProkopHapala commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Jun 2, 2025 •

edited

Loading