-
Notifications
You must be signed in to change notification settings - Fork 13
Open
Description
We've had a CephFS system indicate that they are seeing strongly negative effects of relatively small writes being emitted by XRootD. The thinking is they are seeing a large number of 16KB write syscalls and a large number of IOPS in Ceph. This is a bit surprising to me - my assumption is that buffered IO is going to be smoothed out at the OS level.
Regardless -- because we default to sequential writes, it should be trivial to do some application-side buffering as well:
- Add a configuration knob that allows the user to specify write buffering size (defaults to 0).
- When writes occur, if they are smaller than the buffer size (or the total buffered data is less than the buffer size), buffer. Otherwise, write to the next layer down. Use a mutex to ensure concurrent writes don't muck up the buffer.
- If an out-of-sequential write (or close) is encountered, dump the remaining buffer and turn off buffering for the remainder of the open file.
Reactions are currently unavailable