Test error

[Cody-G]

I'm getting this test error in Julia 0.5. I also tried starting from a fresh empty package directory and still get the same error.

ERROR: LoadError: LoadError: BoundsError: attempt to access 6-element UnitRange{Int64} at index [-2:13]
 in throw_boundserror(::UnitRange{Int64}, ::Tuple{UnitRange{Int64}}) at ./abstractarray.jl:355
 in checkbounds at ./abstractarray.jl:284 [inlined]
 in getindex at ./range.jl:504 [inlined]
 in unsafe_reindex at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:437 [inlined]
 in extraunsafe_view at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:433 [inlined]
 in safe_get!(::Array{Float64,2}, ::SubArray{Float64,2,Array{Float64,2},Tuple{UnitRange{Int64},UnitRange{Int64}},false}, ::Tuple{UnitRange{Int64},UnitRange{Int64}}, ::Float64) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:387
 in #mismatch!#33(::Symbol, ::Function, ::CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}}, ::RegisterMismatch.CMStorage{Float64,2}, ::SubArray{Float64,2,Array{Float64,2},Tuple{UnitRange{Int64},UnitRange{Int64}},false}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatch.jl:136
 in (::RegisterMismatch.#kw##mismatch!)(::Array{Any,1}, ::RegisterMismatch.#mismatch!, ::CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}}, ::RegisterMismatch.CMStorage{Float64,2}, ::SubArray{Float64,2,Array{Float64,2},Tuple{UnitRange{Int64},UnitRange{Int64}},false}) at ./<missing>:0
 in #mismatch_apertures!#35(::Symbol, ::Function, ::Array{CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}},2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::RegisterMismatch.CMStorage{Float64,2}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatch.jl:230
 in (::RegisterMismatch.#kw##mismatch_apertures!)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures!, ::Array{CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}},2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::RegisterMismatch.CMStorage{Float64,2}) at ./<missing>:0
 in #mismatch_apertures#34(::Symbol, ::UInt32, ::Array{Any,1}, ::Function, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::Tuple{Float64,Float64}, ::Tuple{Int64,Int64}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatch.jl:207
 in (::RegisterMismatch.#kw##mismatch_apertures)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::Tuple{Float64,Float64}, ::Tuple{Int64,Int64}) at ./<missing>:0
 in #mismatch_apertures#5(::Array{Any,1}, ::Function, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:20
 in (::RegisterMismatch.#kw##mismatch_apertures)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}) at ./<missing>:0
 in #mismatch_apertures#3(::Array{Any,1}, ::Function, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Vararg{Tuple{Int64,Int64},N}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:13
 in (::RegisterMismatch.#kw##mismatch_apertures)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}) at ./<missing>:0
 in include_from_node1(::String) at ./loading.jl:488
 in runtest(::String) at /home/cody/.julia/v0.5/BlockRegistration/test/runtests.jl:6
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /home/cody/.julia/v0.5/BlockRegistration/test/register_mismatch.jl, in expression starting on line 104
while loading /home/cody/.julia/v0.5/BlockRegistration/test/runtests.jl, in expression starting on line 17
WARNING: Forcibly interrupting busy workers
WARNING: rmprocs: process 1 not removed
=========================================================[ ERROR: BlockRegistration ]==========================================================

failed process: Process(`/home/cody/git/julia/usr/bin/julia -Cnative -J/home/cody/git/julia/usr/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /home/cody/.julia/v0.5/BlockRegistration/test/runtests.jl`, ProcessExited(1)) [1]

===============================================================================================================================================
ERROR: BlockRegistration had test errors
 in #test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:740
 in (::Base.Pkg.Entry.#kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./<missing>:0
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:59
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31
 in (::Base.Pkg.Dir.#kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./<missing>:0
 in #test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
 in test(::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258

[Cody-G]

Maybe I should be running the release-0.5 branch of Julia? Currently on the tagged v0.5.0

[Cody-G]

Oh I forgot that we are using a patched Julia on cannon. I'll replicate the patch and see if that fixes it.

[Cody-G]

Hmm I'm still getting the same error when I run the latest "release-0.5" branch with the patch that we have on cannon now

[Cody-G]

Actually I'm getting the same error on cannon too, and it was also triggered by the latest Travis build

[mdhe1248]

I haven't run registration for a while. I can try a test run tonight to see if I can replicate the error. Best, Jerry

…

On Feb 21, 2017 4:59 PM, "Cody Greer" ***@***.***> wrote: Actually I'm getting the same error on cannon too. Has anyone else ran into this? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMvfFW68bo2_pPaoYtuRL8iVO3Z2YoEWks5re2xUgaJpZM4MH6nL> .

[timholy]

If I use the version of BlockRegistration that's installed at /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/BlockRegistration/test (which is what you use if you don't have it installed in your own private v0.5 directory), then the register_mismatch.jl test passes for me. Looks like I missed a depwarn or two in RegisterMismatchCuda, though.

[Cody-G]

I will check that out. Do you think we should merge that version into master? Trying to make sure I can run it on other machines.

…

On Feb 21, 2017 5:33 PM, "Tim Holy" ***@***.***> wrote: If I use the version of BlockRegistration that's installed at /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/ BlockRegistration/test (which is what you use if you *don't* have it installed in your own private v0.5 directory), then the register_mismatch.jl test passes for me. Looks like I missed a depwarn or two in RegisterMismatchCuda, though. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE78hBg7B-lyMBHY5u_g1Rbh64YvipByks5re3QtgaJpZM4MH6nL> .

[timholy]

I think it is, but it's worth investigating further. I'm in the middle of fixing the depwarn and I'm using that version (I shouldn't do that), so right now it's ahead of master.

[Cody-G]

I think the Travis failure on the AffineTransforms fix PR suggests it's not in master, right?

…

On Feb 21, 2017 5:41 PM, "Tim Holy" ***@***.***> wrote: I think it is, but it's worth investigating further. I'm in the middle of fixing the depwarn and I'm using that version (I shouldn't do that), so right now it's ahead of master. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE78hKqzVJEToKLUoxlx5U4XidW89b_5ks5re3Y_gaJpZM4MH6nL> .

[timholy]

Travis is running 0.5.0.

[Cody-G]

Very strange, I just got the same error when using the packages installed in /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/. The test that fails is register_mismatch.jl. What could we be doing differently?

Also in order to use those packages I had to run julia as super user, or else I get an error when trying to use precompiled stuff:

ERROR: LoadError: LoadError: SystemError: opening file /usr/local/julia/julia-0.5/usr/share/julia/site/lib/v0.5/Reexport.ji: Permission denied
 in #systemerror#51 at ./error.jl:34 [inlined]
 in systemerror(::String, ::Bool) at ./error.jl:34
 in open(::String, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./iostream.jl:89
 in open(::String, ::String) at ./iostream.jl:101
 in stale_cachefile(::String, ::String) at ./loading.jl:659
 in _require_search_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:214
 in require(::Symbol) at ./loading.jl:371
 in include_from_node1(::String) at ./loading.jl:488
 in eval(::Module, ::Any) at ./boot.jl:234
 in require(::Symbol) at ./loading.jl:415
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/BlockRegistration/src/BlockRegistration.jl, in expression starting on line 5
while loading /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/BlockRegistration/test/runtests.jl, in expression starting on line 1

[Cody-G]

Here's my Pkg.status() when I use the shared packages on cannon. I guess we should confirm that yours looks the same...

24 required packages:
 - AxisArrays                    0.0.3
 - CUDArt                        0.2.3+             standalone
 - ColorTypes                    0.3.4
 - CoordinateTransformations     0.4.0
 - DSP                           0.1.1
 - Documenter                    0.9.0
 - GLPlot                        0.0.4
 - IJulia                        1.4.1
 - ImageMagick                   0.2.1              963ac699 (dirty)
 - Images                        0.7.0
 - Interact                      0.4.3
 - IntervalSets                  0.0.3
 - LightGraphs                   0.7.2
 - MultivariateStats             0.3.1
 - NRRD                          0.2.1
 - OffsetArrays                  0.2.14
 - PlotlyJS                      0.5.2
 - Plots                         0.10.3
 - ProfileView                   0.1.5
 - PyPlot                        2.3.1
 - Rotations                     0.3.4
 - TestImages                    0.1.3
 - Unitful                       0.0.4
 - Winston                       0.12.1
145 additional packages:
 - AffineTransforms              0.2.1
 - AutoHashEquals                0.1.1
 - AxisAlgorithms                0.1.6
 - BaseTestNext                  0.2.2
 - BinDeps                       0.4.7
 - Blink                         0.5.1
 - BlockRegistration             0.0.0-             master (unregistered)
 - BlockRegistrationScheduler    0.0.0-             master (unregistered)
 - Blosc                         0.2.0
 - BoxTrees                      0.0.0-             master (unregistered)
 - BufferedStreams               0.3.0
 - CUFFT                         0.0.4
 - Cairo                         0.2.35
 - CalciumImaging                0.0.0-             master (unregistered)
 - Calculus                      0.2.0
 - CatIndices                    0.0.2
 - CellSegmentation              0.0.0-             master (unregistered)
 - Codecs                        0.2.0
 - ColorVectorSpace              0.4.0
 - Colors                        0.7.3
 - Combinatorics                 0.3.2
 - Compat                        0.19.0
 - Compose                       0.4.5
 - ComputationalResources        0.0.2
 - Conda                         0.5.1
 - Contour                       0.2.0
 - CustomUnitRanges              0.0.4
 - DFoFArrays                    0.0.0-             master (unregistered)
 - DataArrays                    0.3.12
 - DataFrames                    0.8.5
 - DataStructures                0.5.2
 - Dates                         0.4.4
 - DiffBase                      0.0.4
 - Distances                     0.4.0
 - Distributions                 0.12.1
 - DocStringExtensions           0.3.1
 - DualNumbers                   0.2.3
 - FFTViews                      0.0.1
 - FileIO                        0.3.0
 - FixedPointNumbers             0.3.4
 - FixedSizeArrays               0.2.5
 - Formatting                    0.2.0
 - ForwardDiff                   0.3.4
 - GLAbstraction                 0.3.1
 - GLFW                          1.2.2
 - GLText                        0.0.4
 - GLWindow                      0.3.2
 - GZip                          0.2.20
 - Gadfly                        0.5.3
 - GeometryTypes                 0.2.2
 - Graphics                      0.1.3
 - Graphs                        0.7.1
 - Gtk                           0.10.4
 - GtkUtilities                  0.2.0
 - HDF5                          0.7.3
 - Hexagons                      0.0.4
 - Hiccup                        0.1.1
 - HttpCommon                    0.2.6
 - HttpParser                    0.2.0
 - HttpServer                    0.1.7
 - ImageAxes                     0.1.0
 - ImageCore                     0.1.4
 - ImageFiltering                0.1.2
 - ImageMetadata                 0.2.0
 - ImagePlayer                   0.0.0-             master (unregistered)
 - Immerse                       0.0.12
 - ImmutableArrays               0.0.12
 - IndirectArrays                0.1.1
 - IniFile                       0.2.5
 - Interpolations                0.3.7
 - Ipopt                         0.2.6
 - IterativeSolvers              0.2.2
 - Iterators                     0.3.0
 - JLD                           0.6.9
 - JSON                          0.8.3
 - JuMP                          0.15.1
 - Juno                          0.2.6
 - KernelDensity                 0.3.0
 - LaTeXStrings                  0.2.0
 - LabShare                      0.0.0-             master (unregistered)
 - Lazy                          0.11.5
 - LegacyStrings                 0.2.0
 - Libz                          0.2.4
 - LightXML                      0.4.0
 - LineSearches                  0.1.5
 - Loess                         0.1.0
 - MAT                           0.3.1
 - MacroTools                    0.3.4
 - MappedArrays                  0.0.6
 - MathProgBase                  0.5.10
 - MbedTLS                       0.4.3
 - Measures                      0.0.3
 - Media                         0.2.5
 - ModernGL                      0.1.1
 - Munkres                       0.1.0
 - Mustache                      0.1.3
 - Mux                           0.2.3
 - NLsolve                       0.9.1
 - NaNMath                       0.2.2
 - NeighborhoodClustering        0.0.0-             master (unregistered)
 - Nettle                        0.3.0
 - Optim                         0.3.1+             teh/constrained
 - PDMats                        0.5.6
 - ParserCombinator              1.7.11
 - PlotThemes                    0.1.1
 - PlotUtils                     0.3.0
 - Polynomials                   0.1.2
 - PositiveFactorizations        0.0.4
 - ProgressMeter                 0.3.3
 - PyCall                        1.10.0
 - QuadGK                        0.1.1
 - Quaternions                   0.1.1
 - RFFT                          0.0.0-             master (unregistered)
 - RangeArrays                   0.1.2
 - Ranges                        0.0.1
 - Ratios                        0.0.4
 - Reactive                      0.3.7
 - RecipesBase                   0.1.0
 - Reexport                      0.0.3
 - Requires                      0.3.0
 - ReverseDiffSparse             0.6.0
 - Rmath                         0.1.6
 - SHA                           0.3.1
 - SIUnits                       0.1.0
 - ShowItLikeYouBuildIt          0.0.1
 - Showoff                       0.0.7
 - SimpleTraits                  0.3.0
 - SortingAlgorithms             0.1.0
 - SpecialFunctions              0.1.1
 - StaticArrays                  0.2.1
 - StatsBase                     0.13.1
 - StatsFuns                     0.4.0
 - TexExtensions                 0.0.3
 - TileTrees                     0.0.0-             master (unregistered)
 - TileTreesGUI                  0.0.0-             master (unregistered)
 - TileTreesIO                   0.0.0-             master (unregistered)
 - TiledIteration                0.0.2
 - Tk                            0.4.0
 - URIParser                     0.1.8
 - VolumeRaycasting              0.0.0-             master (unregistered)
 - WebSockets                    0.2.1
 - WoodburyMatrices              0.2.2
 - ZMQ                           0.4.1
 - ZipFile                       0.3.0
 - lib                           0.0.0-             non-repo (unregistered)

[Cody-G]

So this is really bizarre. I just noticed that I get the above error when running Pkg.test but not when I directly run the tests by include("runtests.jl"). Any ideas why that could be happening? Thanks!

[timholy]

Ah, that's interesting. I think Pkg.test by default uses the equivalent of julia --check-bounds=yes (see the command line options with julia -h). When it's set to yes, @inbounds has no effect---array bounds are always checked. Failing to check array bounds can be dangerous: you can get a segfault if you try to access memory that's not actually allocated for the array. But in this case it's throwing an error for getindex on a UnitRange, which doesn't even have allocated memory: the core operation (omitting the bounds check) is

function getindex{T}(v::UnitRange{T}, i::Integer)
    convert(T, first(v) + i - 1)
end

which is just a simple computation, not a memory-access.

Looking at the line corresponding to the error, it's in a part of the code that is deliberately violating bounds. In block registration, of our blocks are of the same size, and each block is centered on a grid position; the grid extends all the way to the corners of the image (this makes fine-scale interpolation for warping much cleaner, we don't have to extrapolate u beyond the region where it was defined). That means the blocks at the edges of images are missing data (in 2d, half the data along an edge and 3/4 of the data at a corner). We fill those slots with NaN. But the way the code is written, it sometimes temporarily ignores their absence for the purposes of index-computation and then uses a get!(block, view(img, inds...), NaN) mechanism to extract values. It's probably throwing that error on the computation of inds.

In other words, the error you're getting does not appear to be anything to worry about: it's how the code is supposed to work. Perhaps we should rewrite that code so you can use Pkg.test (it would be nice for catching non-intentional bounds errors), but if you don't want to tackle that, for now I think you can ignore it.

[Cody-G]

I see, that all makes sense, thanks! So I guess it's conventional that Travis calls the runtests.jl script directly rather than calling Pkg.test?

I don't think I have time at the moment to update the code for Pkg.test compatibility but may return to it at some point. Would you prefer that I leave this open as a reminder? If not feel free to close. Thanks!