On 11/30/22 20:41, Michael Loftis wrote:
ZFS snapshots don’t typically have much if any performance impact versus not having a snapshot (and already being on ZFS) because it’s already doing COW style semantics.
Hi Michael,
I am not sure that such statement holds water. When a snapshot is taken, the amount of necessary I/O requests goes up dramatically. For every block that snapshot points to, it is necessary to read the block, write it to the spare location and then overwrite it, if you want to write to a block pointed by snapshot. That gives 3 I/O requests for every block written. NetApp is trying to optimize it by using 64MB blocks, but ZFS on Linux cannot do that, they have to use standard CoW because they don't have the benefit of their own hardware and OS. And the standard CoW is tripling the number of I/O requests for every write to the blocks pointed to by the snapshot, for every snapshot. CoW is a very expensive animal, with horns.
I am not an expert in this area, but we have zfs for specific instances which have timeseries/event log data, and we also need compression.
One day, there was a need to snapshot a 35tb zfs pool and send it across the network to a relplica, coz both the disks in the mirror degraded around same time, I do not recall zfs snapshots took anything resource intensive, and it was quick.ill ask around for actual time though.
We have more than 500 of these type of nodes with zfs (each having 20 disks in mirror each 8tb) for event log with compression, and zfs works just fine. This is a special setup where the data is assumed to be cold storage, hence compression, so it was designed for heavy writes and occasional reads queries only for debugging.