Crashes occurring on multiple objects when changing fftsettings

[MattS6464]

Crashes occur in the help files of fluid.melbands~, fluid.sinefeature~ and fluid.mfcc~. To recreate the crash, enter the help file for any of these objects, turn on audio, then rapidly increase the third parameter of fftsettings. This is occurring on a debut/test build.

[tremblap]

I got it to crash with 1) audio on and 2) wiggle the FFT size. it is not all the time:

----------begin_max5_patcher----------
446.3ocuTEsaBCBE841uBBOtzYJXsV2S6+XYwfsnCSk1.TmNi9sO3Rq5VLpw
47k1bObgy8btWXSX.dR0JtFidA8FJHXSXP..4.BZiCvKXqxKYZHMrj+Y0j43
H+RF9JC.+zNTbGXMyj+gPNarhma7mMIMsWbDhFm490OFBn8hQu2tGYyhpFSI
2.rPZQ8Pl00b+wf0hYRVId+1DE.61J54g3CGkP1cRTG11vP2mn+lDyWmWx2g
HT54D5.pSaDRh6WqpuuBM4+VnSKaDE8zlolZal6PuNcpQaqNqR0nADJ5YBJi
Lhd49MIcDX.jK4CzK3CQH7I8BxI8BxM3ELiQ0HNijnIi7cVBnkrQ2Zq8jB4H
yTwVvMb0XtjMoD1Tb6ZEBccIa8hpB.teKrqz8MtC8oGuuj.9BYP7c0W5eM9x
dCvV4NOP7E+h5GHCWJj+94OPhN7eZJ5pFUdWo1VYnCZrfqs9NyHpjGkCwmyI
s8qkGxUvyv6.OIOH8PuId7sKVc8RtR2lLPgsmOuB59YQPnP5CG.gJ9RQW9dD
lxNAariuMJnrvqR8unhc2qTR6vteTwJNKkvMAoczSWy75.tvDtM7a7IBztK
-----------end_max5_patcher-----------

this is annoying. but it was the same as the OP which are 100% crashing - for instance, sines~ every time...

[tremblap]

this does it:

----------begin_max5_patcher----------
503.3ocuU1raiCBDG+r8SAhiqbiL3OpSO02ipUUDaRJUNXK.2MYqZd1WXHdS
xtNMNQt8hwL.ye9MCC7dX.dQyFtFid.8DJH38vf.vjyPv99A30rMk0LMLMro
Y0pZNNxOjrasPVyMvXj8FaYlxWDxUOq3kFuuuOeVbDhlk.Mo8eQ+7feZ5L+q
i7lLaa4dufERq+5WinB1PMKd8tB7ekVwVyMb0ybIaQMrtX2PeDF59DMRJk7e
Y8auWM7M.G3JV4tAQmddzIIdnmCMIYPG5YYO9+oKGOcD7icn3YjakAe9KM9B
Lb17mVrRxpGLElMgPVtsrluCQnzqFTxbGaj6Itl77u.PSmPPWV2IploERtdG
5Q6JVtznE+linwoEWaMZeRlBTSxKt.7IW.9ni+avPA4VBELiQ0I9T3pD51Z1
10MU7i2nCvLkbBrEyGYB2sI7Y.a.2Bu0qZ74hGCxNcpuxZDwkOI.LGB.jr3q
M.b3P20wexX4GVLt1cF+z2o.Zb1OMnna5Tk8Ru+wAzAbp3Za1hYDMxilSteN
CF1GqNYeS5PFgNYSfNzQnCYBzI8aRmjaRG+wOVa6abkd+jAIrm7esApAJhft
BouaFzUweSzOeuElxV7YrUdcJXag2jmh8K0dQkRZKdcEXgN3rRBU1Raogtk4
4.t.H7iv+3CLD0L
-----------end_max5_patcher-----------

[lewardo]

On MSVC Debug build both patches fail in same glamorous way, crash when spamming the third FFT parameter

FFT.hpp:86 fails assert on newSize <= mMaxSize

[tremblap]

I am sad to say that Windows has a better report - on Mac I cannot get it to print that assert string...

[tremblap]

@weefuzzy might cry, @AlexHarker might too. It seems all maxAttributes have a similar behaviour...

[lewardo]

managed to get it to crash, took some dedicated hwiggling

[weefuzzy]

This didn't used to happen, right? All that assiduous testing of help files on Windows would've caught it. So, is there a release version where this doesn't happen? If so, isolating the breaking change would be The Way (not least, for working out if this is a flucoma-max bug or a flucoma-core bug)

[AlexHarker]

@tremblap - can you repro on Mac?

[AlexHarker]

I'm going to bet this is threading issue. Look at this point in the code:

https://github.com/flucoma/flucoma-core/blob/2681d58fd4642c67a4135a28b7e19b0a5f9d5972/include/clients/common/ParameterTypes.hpp#L404

As far as I can see the member variable of the struct are set incorrectly at this point (and yet to be clipped) - if the audio thread reads at this point it will read incorrectly. @weefuzzy - am I wrong?

I would note also that because these settings should be read atomically, but are not written atomically there is the possibility of reading them in a non-related state.

Welcome to the horror of multi-threading - I will be happy to collect my prize now (BTW if I am correct then running in SIAI and changing the values in the high-priority thread only should never crash).

[AlexHarker]

Actually I think that would only be correct if v were the object state, but I think v should be a thread-local variable - the last bit (about the three values not writing atomically) is probably still true, but each element should still be atomic so it's hard to see how slicing could occur on the FFT size in this context...

The copy into the client parameters should be happening here if I understand the code correctly:

flucoma-max/source/include/FluidMaxWrapper.hpp

Line 1033 in 6de4328

x->params().template set<N>(std::move(a),

This might be a good place to check for spurious values (or just above it). If a can ever have spurious values then it would be a case of working back from there.

[weefuzzy]

Yeah, if it were that foundational, I feel like it would have bitten already. (Still, not ruling out threading badness, just on principle).

The last time there were problems of this sort (cascades of assertion deaths due to FFT size stuff) it was because object-initialization-time maxima weren't getting set properly, so algos would end up allocating too little space (which then gets picked up later by the assertions).

So, another thing to check is what mMaxSize is at the assertion site, and whether it is what one expects it to be.

[tremblap]

2 bulk answers:
@AlexHarker yes I can on Mac too but the message was less clear than Windows. In both cases it always needs serious wiggling and is unpredictable for stftpass. on the helpfile of melband or sines it is quite more immediate and easier to reproduce.

@weefuzzy no it didn't use to happen but I wonder if we tested that extensively in debug on windows. I mean, this needs serious wiggling in debug to crash, hence the team finding it (we roll that way now to find more of these and understand the framework using debugger to spy on states)

1 question: should we recompile in debug on the last public release(s) until it stops crashing as recommended or are you further down the rabbit hole?

[weefuzzy]

I'm nowhere near the rabbit hole, and will only be able to issue vague advice, which is:

• yes, debug builds back through time until the badness vanishes. git-bisect is helpful for this, and you can just build a single errant object whilst you're going, to speed things up
• the badness still needs to be diagnosed as well, so looking at what the values are when the assertion fires will be helpful for reasoning about this. (e.g. maybe when I thought I'd fixed this last time, I hadn't, which would point to init-issues). So, run with a debugger attached when reproducing

[tremblap]

I was implying @AlexHarker might have been near the whole, not you dear @weefuzzy - sorry if that wasn't clear. vague advice is most welcome in all cases.