Add libfuse 3 support with libfuse 2 fallback#603
Open
matejk wants to merge 5 commits into
Open
Conversation
This was referenced Jun 11, 2026
matejk
force-pushed
the
fuse3-port
branch
2 times, most recently
from
3068ef8 to
051a223
Compare
matejk
changed the title
matejk
force-pushed
the
fuse3-port
branch
2 times, most recently
from
793bd14 to
9371bee
Compare
xattr.h included fuse.h without ltfs_fuse_version.h. libfuse 2 headers default to an old API level when FUSE_USE_VERSION is undefined, but libfuse 3 headers reject it, breaking every translation unit that pulls in xattr.h. The release branch received the same change as part of the FreeBSD build fix (b3e3355).
_unified_insert_new_request copies at most one cache block (the tape block size) but returned the full requested count, so the append loop in unified_write advanced past data that was never stored. Writes larger than the block size silently lost everything after the first block while reporting success. Latent with libfuse 2, which caps requests at 128 KiB, below the default 512 KiB block; reachable today through the I/O scheduler API and triggered by FUSE 3 request sizes. Return the number of bytes actually stored. A regression test (multi-block single writes of random data, content verified) accompanies the integration test suite. Fixes LinearTapeFileSystem#591
configure prefers fuse3 (>= 3.4.0) on Linux and falls back to fuse2 when the fuse3 development files are absent, so existing build environments keep working; --with-fuse2 forces the legacy API and remains the default on macOS, FreeBSD, and NetBSD. FUSE_USE_VERSION becomes 31 for fuse3 builds. README documents the new prerequisites and mount options. API changes for FUSE 3: - getattr/truncate absorb fgetattr/ftruncate via the fuse_file_info argument; chmod, chown, and utimens gain the argument and use the file handle when the path is NULL (possible under nullpath_ok). - rename handles flags: RENAME_NOREPLACE returns EEXIST when the target exists, RENAME_EXCHANGE is rejected with EINVAL. - readdir and the directory filler gain flag arguments. - init receives struct fuse_config; use_ino, hard_remove, and nullpath_ok move there from mount options. - FUSE_CAP_ASYNC_READ is cleared in init to keep tape reads ordered; FUSE 3 removed the -o sync_read option and enables asynchronous reads by default. - big_writes is gone (always enabled); fuse_parse_cmdline uses struct fuse_cmdline_opts. The fuse2 code paths are unchanged and selected with --with-fuse2.
Three independently measurable improvements, each with an integration test that the test-suite harness discovers automatically: - Request sizes up to 1 MiB: init sets conn->max_write (tunable with -o max_write=N) and libfuse >= 3.6 negotiates the matching max_pages with the kernel. Measured 1 MiB writes and O_DIRECT reads reaching the daemon, against 128 KiB with fuse2 big_writes; one request now carries two default-size tape blocks. - -o direct_io: sets FOPEN_DIRECT_IO on every open, so reads and writes bypass the kernel page cache and arrive at the application's I/O size. Streaming large archives no longer fills the page cache and data is not buffered twice; mmap does not work on files opened this way and small-block applications lose readahead, so the option is off by default. When fuse_file_info has parallel_direct_writes it is set as well; the field appeared in libfuse 3.15, so it is detected at configure time instead of via version checks (which broke on ubuntu-24.04's libfuse 3.14). - readdirplus: ltfs_fsops_readdir_attr hands each entry's attributes to the filler straight from the in-memory index, and init clears FUSE_CAP_READDIRPLUS_AUTO so every listing chunk carries attributes. ls -l over 100 files needs 2 getattr requests instead of 102. The kernel prefill is only effective with libfuse >= 3.17, so the test asserts the suppression conditionally.
macFUSE 5 ships a libfuse 3 whose operation signatures default to Darwin-specific types (struct fuse_darwin_attr, struct statfs, a Darwin directory filler). Define FUSE_DARWIN_ENABLE_EXTENSIONS=0 in ltfs_fuse_version.h so the upstream-compatible signatures are used; the library exports both symbol flavors. The macFUSE position argument of the xattr handlers exists only in the fuse2 API and the Darwin-extension mode, so the FUSE 3 build now uses the upstream xattr signatures (LTFS_XATTR_POSITION). Verified on macOS 26 (arm64) against the macFUSE 5.2.0 SDK (libfuse 3.18.2): autotools (--with-fuse2=no) and CMake (-DLTFS_WITH_FUSE2=OFF) both build without new warnings, link libfuse3.4.dylib, and the binaries run; parallel_direct_writes is detected. The macOS fuse2 builds and the Linux test suites (autotools make check and ctest, 14/14) are unaffected.
This was referenced Jun 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port of LTFS to libfuse 3 with a compile-time libfuse 2 fallback, plus the FUSE 3 performance features. This PR is based directly on
main(5 commits: xattr.h include fix, scheduler data-loss fix, the port, the performance features, macOS support); the test suite, the remaining bug fixes found during this work, and the base CMake build are filed as separate independent PRs.FUSE 3 support
--with-fuse2forces the legacy API (still the default on macOS and the BSDs).getattr/truncateabsorb the handle variants,chmod/utimensuse the file handle when the path is NULL (nullpath_ok),renamehandlesRENAME_NOREPLACE/RENAME_EXCHANGE,use_ino/hard_remove/nullpath_okmove intofuse_config, andFUSE_CAP_ASYNC_READis cleared to keep tape reads ordered (-o sync_readno longer exists).fuse_file_info.parallel_direct_writesis feature-detected (the field appeared in libfuse 3.15; version checks broke on ubuntu-24.04's 3.14).FUSE_DARWIN_ENABLE_EXTENSIONS=0; macOS keeps fuse2 as its default.Performance
-o max_write=N): measured 1 MiB writes and O_DIRECT reads reaching the daemon, vs 128 KiB on fuse2.-o direct_io: all I/O bypasses the kernel page cache; ~70 % higher sequential write throughput on the file backend and a flat page cache during large streaming jobs.ls -lover 100 files needs 2 getattr requests instead of 102 (effective with libfuse >= 3.17).Tests
The performance commit ships its integration tests (
tests/t/10–12: request sizes, direct_io, readdirplus). The scripts are inert until the test-suite PR (#611) merges — its harness discoverst/*.shautomatically (wildcard in automake, glob in ctest), so no registration conflicts arise in either merge order.Relationship to the other PRs
xattr.hinclude-order fix from Build and CLI fixes #608 (a compile prerequisite); git drops it automatically when both are merged.Verification
With #611 merged alongside: all 14 integration tests pass with fuse3 and fuse2, via
make checkandctest, against libfuse 3.14 and 3.17; zero compiler warnings at-Wall -Wextra; macOS compile-verified for both APIs against the macFUSE 5.2 SDK.Issues
pthread_yieldhalf is in Build and CLI fixes #608).