Re: Adding skip scan (including MDAM style range skip scan) to nbtree

On Fri, May 9, 2025 at 2:04 PM Tomas Vondra <tomas@vondra.me> wrote:
> Yes, I'm sure it's doing index only scan

Looks that way, from the pair of flame graphs you sent. Thanks for that.

> did you update "bid" or did
> you leave it as generated by "pgbench -i"?.

I didn't bother with updating, or running VACUUM FULL. I did run
VACUUM ANALYZE, though (can confirm no heap accesses for the
index-only scans).

> In fact, all of the malloc() calls seem to happen in AllocSetAllocLarge,
> which matches the guess that something tripped over allocChunkLimit. Not
> sure what, though.

While there are way too many AllocSetAllocLarge calls here, I don't
think that that can be blamed on the skip scan work. Note that commit
92fe23d9 didn't even touch the BTScanOpaqueData struct. Its size did
change a bit, in other nearby commits, but it was already so large
that I don't think that it could matter here. Besides, you said that
the problem clearly starts in commit 92fe23d9.

The AllocSetAllocLarge calls that I see from gdb look like the slow
ones from your flame graph. They're for the BTScanOpaqueData
allocation, and for the BLCKSZ * 2 used by index-only scans (though
not index scans). These allocations happen once per rescan/index scan.
So, again, too many large allocations take place here, but it doesn't
look like anything that can be attributed to skip scan.

The difference shown by your flame graph is absolutely enormous --
that's *very* surprising to me. btbeginscan and btrescan go from being
microscopic to being very prominent. But skip scan simply didn't touch
either function, at all, directly or indirectly. And neither function
has really changed in any significant way in recent years. So right
now I'm completely stumped.

--
Peter Geoghegan