cindex "panic: no progress" in v1.3.0-rc1

[jamessanford]

In the changes between v1.2.0 and v1.3.0-rc1 (index version 2), cindex will now frequently panic with "no progress"

Here's a reproduction against the codesearch code itself, initializing with cindex /home/test/codesearch and then running cindex three times.

$ git clone https://github.com/google/codesearch
$ cd codesearch/cmd/cindex
$ git checkout -b bug v1.3.0-rc.1
$ go version
go version go1.25.5 X:nodwarf5 linux/amd64
$ go build 
$ ./cindex -list
cindex: 2025/12/19 16:47:52 open /home/test/.csearchindex: no such file or directory
$ ./cindex /home/test/codesearch
cindex: 2025/12/19 16:47:58 index /home/test/codesearch
cindex: 2025/12/19 16:47:58 flush index 
cindex: 2025/12/19 16:47:58 152741 data bytes, 152880 index bytes
cindex: 2025/12/19 16:47:58 done
$ ./cindex 
cindex: 2025/12/19 16:48:00 index /home/test/codesearch
cindex: 2025/12/19 16:48:00 flush index 
cindex: 2025/12/19 16:48:00 152741 data bytes, 152880 index bytes
cindex: 2025/12/19 16:48:00 merge /home/test/.csearchindex /home/test/.csearchindex~
cindex: 2025/12/19 16:48:00 done
$ ./cindex 
cindex: 2025/12/19 16:48:02 index /home/test/codesearch
cindex: 2025/12/19 16:48:02 flush index 
cindex: 2025/12/19 16:48:02 152741 data bytes, 152880 index bytes
cindex: 2025/12/19 16:48:02 merge /home/test/.csearchindex /home/test/.csearchindex~
cindex: 2025/12/19 16:48:02 done
$ ./cindex 
cindex: 2025/12/19 16:48:02 index /home/test/codesearch
cindex: 2025/12/19 16:48:02 flush index 
cindex: 2025/12/19 16:48:02 152741 data bytes, 152880 index bytes
cindex: 2025/12/19 16:48:02 merge /home/test/.csearchindex /home/test/.csearchindex~
panic: no progress

goroutine 1 [running]:
github.com/google/codesearch/index.Merge({0xc0000205e0, 0x1a}, {0xc000018048?, 0x5248f8?}, {0xc000020080, 0x19})
        /home/test/codesearch/index/merge.go:201 +0x1005
main.main()
        /home/test/codesearch/cmd/cindex/cindex.go:170 +0x8d7

[BourgeoisBear]

same here

[idrassi]

I reproduced same issue on my side on Ubuntu 24.04.

I implemented a fix and I opened a PR: #101

The root cause appears to be repeated v2 index merges accumulating duplicate serialized posting-list sentinels. After enough reindexes, merge can no longer make progress through those duplicate sentinel entries and panics with panic: no progress.

The fix makes merge use invalidTrigram consistently for sentinel handling, restores lazy posting-list writes so empty non-sentinel posting lists are omitted and adds a repeated-merge regression test.

I reran the reproduction with repeated no-argument cindex runs: the patched binary completed successfully, cindex -check -list passed and the final posting index contained exactly one 0xffffff sentinel.