Discussion: Remove rtapi_get_clocks()

[hdiethelm]

Hello

I was wondering why the .time value is so big in Linuxcnc. Probably this is documented somewhere but this value is not in ns but in arbitrary clocks returned by rtapi_get_clocks().

According to the documentation, rtapi_get_time() is really slow sometimes. So I created a component to test this:
https://github.com/hdiethelm/linuxcnc-fork/tree/timetest_component

It is not that slow according to my tests and the issues having with rtapi_get_clocks() returning random time units outweigh the 16ns rtapi_get_time() takes more in my opinion.

It might have been slow long time ago, the code is like 20 years old: ed4089d It is still an bit an issue on RTAI and it might still be an issue on certain systems. More tests would be good.

Results so far, Debian Trixie:

• VM, PREEMT_RT
  • Clock to time ratio: 4.3
  • rtapi_get_clocks(): 20ns
  • rtapi_get_time(): 46ns
• CNC PC, PREEMT_RT
  • Clock to time ratio: 3.2
  • rtapi_get_clocks(): 8ns
  • rtapi_get_time(): 20ns
• CNC PC, RTAI
  • Clock to time ratio: 3.2
  • rtapi_get_clocks(): 11ns
  • rtapi_get_time(): 112ns
• CNC PC, Xenomai3
  • Clock to time ratio: 3.2
  • rtapi_get_clocks(): 8ns
  • rtapi_get_time(): 24ns
• CNC PC, Xenomai4
  • Clock to time ratio: 3.2
  • rtapi_get_clocks(): 8ns
  • rtapi_get_time(): 22ns

Started using:
scripts/halrun -I -f timetest.hal

timetest.txt

I created a prototype removing this function all together: https://github.com/hdiethelm/linuxcnc-fork/tree/rtapi_get_clocks_removal

While doing so, I even discovered issues where the time to clock conversion is wrong. For example here:
https://github.com/LinuxCNC/linuxcnc/blob/master/src/hal/drivers/hal_gpio.c#L367
This assumes one clock is 1 ns. However, on my PC, one clock is 0.24 ns

Due to RTAI is still used and somewhat slow (+100ns), a hybrid solution might be needed like rtapi_get_clocks_ns() which returns rtapi_get_time() for uspace and something like 1000.0*rtapi_get_clocks()/cpu_khz for RTAI.

Any feedback on this?

[BsAtHome]

It actually makes me wonder why RTAI is so slow. Is this a kernel version issue?

Also interesting: machinekit#162 What would happen if you change rtapi to use rt_get_time_ns() instead?.

[hdiethelm]

Context:
https://en.wikipedia.org/wiki/Time_Stamp_Counter
TLDR: RDTSC returns a value randomly but constantly increasing, depending on CPU type and other factors.

https://stackoverflow.com/questions/79634297/how-does-clock-gettime-actually-work-in-the-kernel
TLDR: clock_gettime(CLOCK_MONOTONIC_RAW, &ts) is what should be used in uspace and works fine.
TBD: Change to CLOCK_MONOTONIC_RAW in some places? We don't need incremental adjustments. CLOCK_MONOTONIC_RAW is not faster for clock_gettime()

Cyclictest uses clock_gettime(CLOCK_MONOTONIC, ...) or clock_gettime(CLOCK_REALTIME, ...)
https://kernel.googlesource.com/pub/scm/utils/rt-tests/rt-tests/+/stable/devel/v1.0.1/src/cyclictest/cyclictest.c

[hdiethelm]

It actually makes me wonder why RTAI is so slow. Is this a kernel version issue?

Also interesting: machinekit#162 What would happen if you change rtapi to use rt_get_time_ns() instead?.

I can test this. At the moment, rt_get_cpu_time_ns() is used.

[hdiethelm]

I did some digging.

All 6.12 kernels RT and also standard have 20ns for clock_gettime()

The 5.4.302-rtai has always about 80...110ns for
clock_gettime() in userspace
rt_get_cpu_time_ns() / rt_get_time_ns() / ktime_get_ns() / ktime_get_raw_ns() / ktime_get_mono_fast_ns() in kernel

I guess there was an improvement somewhere between 5.4 and 6.12 kernels. I would need to test a standard 5.4 kernel but getting that on Deban Trixie is probably a bit cumbersome.

@NTULINUX Any insight? Can you test if you have the same result?

[NTULINUX]

What do you mean by a standard 5.4 kernel? One patched without RTAI, a vanilla 5.4 kernel? I can help test but it might be a bit, working on tying up my Gentoo image for LinuxCNC.

[hdiethelm]

It would be interesting vanilla / preemt / RTAI v5.4.
RTAI because the RTAI kernel behaves badly on my PC. I had already 3 full freezes today. So this 110us might just be a fluke on my PC.

No hurry, this is just an other side project I ran into while preparing for Xenomai RT network. I have now a ping component which works with PREEMT_RT and now I can add xenomai support in there to see what's needed.

[grandixximo]

Looking at this from the linuxcnc-ethercat perspective:

The codebase uses rtapi_get_time() exclusively (three sites in lcec_main.c), never rtapi_get_clocks(). The hot path is one call per cycle per master to feed ecrt_master_application_time(), which takes absolute ns, so a clocks-based API would still need conversion and cancel out the speed advantage.

EtherCAT only runs on uspace (PREEMPT_RT / Xenomai). At the 20-46 ns range you measured, rtapi_get_time() is well below the DC sync jitter budget at 1-8 kHz. Not a bottleneck in production.

No objection to removal. An rtapi_get_clocks_ns() hybrid that hides the unit sounds right. Current ambiguity (raw cycles vs ns depending on build) is a footgun, and the hal_gpio.c:367 bug shows how easy it is to repeat in timing-sensitive code.

[hdiethelm]

So that means removing it for ethercat would be fine.

rtapi_get_clocks() / rtapi_get_clocks_ns() have the issue that it gives a different value for each cpu the thread runs on according what I read. As long as the cpu is locked, which is normally the case for linuxcnc, this should be fine.

Also for uspace / not x86, linuxcnc falls back to rtapi_get_time().

It would be nice to have some more samples for rtapi_get_time() on different setups before removing or creating an optimized rtapi_get_clocks_ns() that measures the clock to ns ratio somehow.

I think even 100ns would be fine for most applications, you can call it 1250 times to for 8kHz and 10k times for 1kHz to use up all time.