WIP native code ASE and LAMMPS export

[jameskermode]

This is an exploratory draft of compiling ACE potential models with juliac --trim=safe into a native executable/shared library for use with ASE and LAMMPS. Code so far is largely AI-generated so it will need careful testing.

TODO list:

• ACE basis in native code exeutable
• ACE potential energy, forces and virial in executable
• ACE potential energy, forces and virial from ASE-compatible Python using matscipy neighbourlist
  - [ ] ACE potential energy, forces and virial from ASE-compatible Python using Julia neighbourlist
  - [ ] ACE potential energy, forces and virial from LAMMPS plugin using Julia neighbourlist
• ACE potential energy, forces and virial from LAMMPS plugin using native LAMMPS neighbourlist

[tjjarvinen]

I am not sure are you aware that you can already use ACEpotentials with ASE and LAMMPS.

It is done with IPICalculator.jl and current used by e.g. DFTK for ASE. I have also used ACEpotentials with ASE extensively, so it is well tested.

Using the i-PI socket server protocol is the approach preferred by Ask Hjorth Larsen the ASE maintainer. This was discussed with him during Julia MolSSI Workshop last year and other occasions afterwards. The implementation at IPICalculator.jl is meant to serve all AtomsCalculators compatible calculators in JuliaMolSim ecosystem by providing i-PI protocol suport and general ASE interface.

LAMMPS supports Molssi Driver Interface (MDI) in server mode https://docs.lammps.org/mdi.html, which is compatible with i-PI protocol, with the exception of not having support for Unix pipe. You can use ACEpotentials with IPICalculators in LAMMPS this way. Although I have to admit that the LAMMPS documentation is not clear at all. I have not tested ACEpotentials in LAMMPS this way, but I expect it to work

Thus, I propose that instead of using resources to implement our own ASE and LAMMPS binary interfaces - this PR - we use the existing i-PI protocol interface to connect to ASE and LAMMPS, and make documentation entries for them. This the easiest way to get ASE and LAMMPS interface going and reduces maintenance burden for ACEpotentials.

[jameskermode]

Thanks for letting me know, I was aware of the IPICalculator.jl support but not that it could be used with LAMMPS. Nonetheless, I think native code export I am pursuing here is a good path to explore because it allows MPI-parallel domain decomposition. This should be much more scalable than socket communications for very large systems. ASE is then a fringe benefit that used the same native code export - and also benefits from OpenMP parallelisation that wouldn't be easy to do with IPICalculator. Initial tests give ca. 5x speed up on 8 threads even for small systems. Moreover, so far I only spent a day or so on it and it's nearly working.

[tjjarvinen]

MPI is a valid point. Although there is the executor keyword that allows you to perform distributed calculation already in Julia

# start Julia in parallel and then use DistributedEx() instead of the default ThreadedEx()
energy_forces_virial(system, ace_calc;  executor=DistributedEx(), ntasks=nworkers() )

I am not sure what the performance is with it, but you could compare it to the MPI implementation. (DistributedEx() might not be exported in ACEpotentials so you need to get it from Folds.jl or Transducers.jl)

Also, I am really curious how did you manage get 5x performance from OpenMP? The default Julia version is multi threaded and scales well with number of threads when neighbourlist calculation time is ignored.

[jameskermode]

It would indeed be interesting to compare LAMMPS MPI performance with Julia executor, but my experience in general on HPC machines is that MPI has been heavily vendor optimisated and in comparison's Julia distributed calculations are typically not competitive. This is likely to be particularly true for LAMMPS.

Standard Julia threads can't be used with --trim=safe native code, so I added OpenMP on the loop over atoms outside the Julia call. 5x on 8 cores isn't great but this was only 128 atom system and this is including the neighbourlist overhead. I was intending it for use either from Python, or within multilevel MPI/OpenMP parallelism from LAMMPS, just to give a bit of extra speedup and make full use of available hardware threads.

[jameskermode]

Another advantage of this approach: it looks like we can bundle the Julia runtime libraries, meaning no Julia installation required for end users.

[tjjarvinen]

MPI should be faster when communication limited, if computation limited then MPI and Julia should give about the same results. The point here would be get an idea of when the situation changes to communication limited case.

The point of including Julia runtime is also good. It should make ACEpotentials easier to use when only compute is needed.

• Standard Julia for small system where multi threading inside a single node - sockets with Unix pipe is same as using MPI on local node
• Large system with MPI using several nodes (might even be valid to use MPI in Julia too at some point)
• Easy to use binary package for compute only

I should also work on to finish the new neigbourlist package. It would be faster than the current one, is multithreaded and can be made to work with MPI.

[cortner]

@jameskermode -- this is entirely outside of what I want to have any control over and would be very happy to merge this so we can proceed with the other PRs.