Skip to content

Regression with Armadillo 15.0.0 -- and 14.6.* under Intel #474

@eddelbuettel

Description

@eddelbuettel

The eBsc package package is experiencing a very strange regression when going from Armadillo 14.6.* (either .0 or .3) to 15.0.0. So this looks like a change to worse. Oddly, the same error also appears under the Intel compiler where (copying here in case the link disappears) we currently see

* checking examples ... [9s/11s] ERROR
Running examples in ‘eBsc-Ex.R’ failed
The error most likely occurred in:

> ### Name: plot.eBsc
> ### Title: Plot fitted components
> ### Aliases: plot.eBsc
> ### Keywords: plot
> 
> ### ** Examples
> 
> 
> library(eBsc)
> n <- 250
> sigma <- 0.05
> Basis <- list()
> for(i in 1:6) Basis[[i]] <- drbasis(nn = n, qq = i)
Error: rank(): failed
Execution halted

Here the example for plot.eBsc fails with insuffcient rank(). We can trace this back to (R function) drbasis() which calls C++ function drbasis(). For arguments nn=250, qq=5 this fails deterministically.

We isolated the core of the qq=5 code (which is rather neatly inside a case statement) having it return the matrix for which the rank() fails, and have it print the final ten column of the first ten rows. The resulting C++ file has zero dependencies on R or R packages. No changes were made apart from reindentation, and removing of extra code not needed for a minimally viable replicable example as provided here.

For convenience we just point at the include directories provided by what is currently PR #473 which has both Armadillo 15.0.0 (for compilations under C++14 or newer) and Armadillo 14.6.3 (for C++11 compilations as more than two hundred CRAN packages still hardcode).

Note that the following example can be reproduced anywhere with both Armadillo 15.0.0 or equivalent, and the older Armadillo 14.6.* by adjusting the -I... location. current is the released upstream version 15.0.0, legacy the preceding 14.6.3. We do not need to link with -larmadillo or -lblas or -llapack etc as we execute no such functionality as only vector / matrix code is used.

edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ g++ -Wall -pedantic \   # indented for display
    -I/usr/local/lib/R/site-library/RcppArmadillo/include/legacy  \
    -o ebsc_rank_issue_main ebsc_rank_issue_main.cpp -lm 
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ ./ebsc_rank_issue_main 
M[0:9, 240:249]
   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000   0.2000
   0.0698   0.0704   0.0710   0.0717   0.0722   0.0728   0.0734   0.0739   0.0744   0.0749
  -0.0796  -0.0804  -0.0811  -0.0818  -0.0823  -0.0829  -0.0834  -0.0838  -0.0841  -0.0844
   0.0803   0.0816   0.0829   0.0840   0.0849   0.0858   0.0865   0.0871   0.0876   0.0879
  -0.0759  -0.0781  -0.0802  -0.0821  -0.0838  -0.0852  -0.0865  -0.0875  -0.0883  -0.0888
   0.0688   0.0722   0.0754   0.0783   0.0809   0.0831   0.0850   0.0866   0.0878   0.0887
  -0.0602  -0.0650  -0.0695  -0.0735  -0.0772  -0.0804  -0.0831  -0.0854  -0.0871  -0.0884
   0.0505   0.0568   0.0626   0.0680   0.0729   0.0772   0.0809   0.0839   0.0863   0.0881
  -0.0397  -0.0476  -0.0550  -0.0618  -0.0680  -0.0735  -0.0783  -0.0823  -0.0854  -0.0876
   0.0283   0.0377   0.0467   0.0550   0.0626   0.0695   0.0754   0.0804   0.0843   0.0871
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ g++ -Wall -pedantic \   # indented for display
    -I/usr/local/lib/R/site-library/RcppArmadillo/include/current  \
    -o ebsc_rank_issue_main ebsc_rank_issue_main.cpp -lm 
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ ./ebsc_rank_issue_main 
M[0:9, 240:249]
      nan      nan      nan      nan      nan      nan      nan      nan      nan      nan
   0.0698      nan      nan      nan      nan      nan      nan      nan      nan      nan
  -0.0796  -0.0804      nan      nan      nan      nan      nan      nan      nan      nan
   0.0803   0.0816   0.0829      nan      nan      nan      nan      nan      nan      nan
  -0.0759  -0.0781  -0.0802  -0.0821      nan      nan      nan      nan      nan      nan
   0.0688   0.0722   0.0754   0.0783   0.0809      nan      nan      nan      nan      nan
  -0.0602  -0.0650  -0.0695  -0.0735  -0.0772  -0.0804      nan      nan      nan      nan
   0.0505   0.0568   0.0626   0.0680   0.0729   0.0772   0.0809      nan      nan      nan
  -0.0397  -0.0476  -0.0550  -0.0618  -0.0680  -0.0735  -0.0783  -0.0823      nan      nan
   0.0283   0.0377   0.0467   0.0550   0.0626   0.0695   0.0754   0.0804   0.0843      nan
edd@paul:~/git/rcpparmadillo/local(feature/arma_15.0)$ 

The self-contained C++ code is below.

Details
// The eBsc package on CRAN (at version 4.17) reports an error under the Intel compiler.
// This error now also appears with RcppArmadillo 15.0.0. In the example for plot.eBsc,
// the function drbasis() is called with n=250 and a second parameter p that varies from
// 1 to 6. In the case of 5, NA values return leading an rank failure. The drbasis()
// function calls an underlying C function drbasis from file drbasisC.cpp which has case
// statements for the differenct values of p.  We extrace the one for 5 here.

#include <armadillo>

arma::mat compute(int nn) {
    using namespace arma;
    using namespace std;
    const double E = exp(1);
    arma::vec t(nn);
    arma::vec k(nn);
    typedef complex<double> dcomp;
    double Pi = M_PI;
    complex<double> comp0 (0,1);
    complex<double> comp1 (-1,0);
    complex<double> comp2 (0,-0.5);
    complex<double> comp3 (1,-1);
    complex<double> comp4 (-1,1);
    complex<double> comp5 (0,0.5);
    complex<double> comp6 (0,0.25);
    int const0 = 1;
    double const1 = -sqrt(3);
    double const2 = 2*sqrt(3);
    double const3 = -sqrt(5);
    double const4 = 6*sqrt(5);
    double const5 = -6*sqrt(5);
    double const6 = -sqrt(7);
    double const7 = 12*sqrt(7);
    double const8 = -30*sqrt(7);
    double const9 = 20*sqrt(7);
    double const10 = 3;
    double const11 = -60;
    double const12 = 270;
    double const13 = -420;
    double const14 = 210;
    /*
    double const15 = -sqrt(11);
    double const16 = 30*sqrt(11);
    double const17 = -210*sqrt(11);
    double const18 = 560*sqrt(11);
    double const19 = 630*sqrt(11);
    double const20 = 252*sqrt(11);
    */

    for (int j=0;j<nn;j++) {
        t(j)= j/double((nn-1));
    }
    for (int j=0;j<nn;j++) {
        k(j)= j+1;
    }

    arma::mat M5(nn,nn);
    for (int i=0;i<nn; i++) {
        for (int j=0;j<nn;j++) {
            if (j==0) {
                M5(i,j)=const0;
            } else if (j==1) {
                M5(i,j)= const1 + const2*t(i);
            } else if (j==2) {
                M5(i,j)= const3 + const4*t(i) + const5* pow(t(i),2);
            } else if (j==3) {
                M5(i,j)= const6 + const7*t(i) + const8* pow(t(i),2) + const9* pow(t(i),3);
            } else if (j==4) {
                M5(i,j)= const10 + const11*t(i) + const12* pow(t(i),2) + const13* pow(t(i),3) + const14* pow(t(i),4);
            } else {
                dcomp ans = (sqrt(2)*(1 + sqrt(5)/2) -
                             (comp5*(sqrt(10 - 2*sqrt(5)) +
                                     sqrt(2*(5 + sqrt(5)))))/sqrt(2))*(pow(comp1,1 + k(j))/pow(E,pow(comp1,0.1)* (dcomp(-3 + k(j)))*Pi*(1 - t(i))) + pow(E,-(pow(comp1,0.1)*(dcomp(-3 + k(j)))* Pi*t(i)))) +
                    (-(1/sqrt(2)) -
                     (comp5*(sqrt(10 - 2*sqrt(5)) + sqrt(2*(5 + sqrt(5)))))/
                     sqrt(2))*(pow(comp1,1 + k(j))/pow(E,pow(comp1,0.3)*(dcomp (-3 + k(j)))*Pi*(1 - t(i))) + pow(E,-(pow(comp1,0.3)*(dcomp (-3 + k(j)))*Pi*t(i)))) +
                    (sqrt(2)*(1 + sqrt(5)/2) + (comp5*(sqrt(10 - 2*sqrt(5)) + sqrt(2*(5 + sqrt(5)))))/sqrt(2))
                    *(pow(comp1,1 + k(j))/ pow(E,(sqrt(0.625 + sqrt(5)/8) - comp6*(-1 + sqrt(5)))*(dcomp (-3 + k(j)))*Pi*
                                               (1 - t(i))) +pow(E,-((sqrt(0.625 + sqrt(5)/8.) - comp6*(-1 + sqrt(5)))*(dcomp (-3 + k(j)))*
                                                                    Pi*t(i)))) +(-(1/sqrt(2)) + (comp5*(sqrt(10 - 2*sqrt(5)) + sqrt(2*(5 + sqrt(5)))))/sqrt(2))
                    *(pow(comp1,1 + k(j))/pow(E,(dcomp(sqrt(0.625 - sqrt(5)/8)) - comp6*(1 + sqrt(5)))*
                                              (dcomp (-3 + k(j)))*Pi*(1 - t(i))) +pow(E,-((sqrt(0.625 - sqrt(5)/8) - comp6*(1 + sqrt(5)))* (dcomp (-3 + k(j)))*Pi*t(i))))- sqrt(2)*cos((-3 + k(j))*Pi*t(i));
                M5(i,j) =ans.real();
            }
            M5(i,j) = M5(i,j)/sqrt(nn);
        }
    }
    return M5;
}

int main(int argc, char *argv[]) {
    constexpr int nn = 250;
    arma::mat M = compute(nn);
    M.submat(0, 240, 9, 249).print("M[0:9, 240:249]");
    exit(0);
}

I have not found any compiler switches, or compilation standard changes, affecting this. As the result also appears under the Intel compiler for the older Armadillo, I suspect the package has code with a latent frailty revealed by either the Intel compiler, or the newer Armadillo. The eBsc maintainer has been contacted by email a few days ago, I have yet to hear back.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions