On 2016-07-22 14:33:32 +0100, Greg Stark wrote:
> On Fri, Jul 22, 2016 at 2:56 AM, Andres Freund <andres@anarazel.de> wrote:
> >
> > But I think the bigger issue than the above is actually that we're just
> > performing a lot of useless work in a number of common scenarios. We're
> > always deforming all columns up to the one needed. Very often that's a
> > lot of useless work. I've experimented with selectively replacing
> > slot_getattr calls heap_getattr(), and for some queries that can yield
> > massive speedups. And obviously significant slowdowns in others. That's
> > the case even when preceding columns are varlena and/or contain nulls.
> > I.e. a good chunk of the problem is storing the results of deforming,
> > not accessing the data.
>
> As I said when we chatted I'm a bit puzzled.
I think that's a separate slant on slots than what I/we talked around
pgcon.
> My understanding was that
> the whole point of the "slots" thing was precisely to avoid this kind
> of extra work. So I would love to learn where my understanding
> diverges from reality. I wonder if there's some simple bug in the code
> leading to it not doing what it's intended to do.
>
> My understanding is that the idea of slots is that when we decode the
> Nth attribute we decode every attribute up until that attribute and
> have somewhere to put it -- in the slot. When we later refer to an
> earlier attribute we can just fetch it from the slot directly. This is
> the difference between using slot_getattr and heap_getattr directly.
Sure, it's intentional. But not working that well. The problem is that
very often we actually don't ever access the preceding
columns. Sometimes that knowledge would let us skip directly to the
interesting column(s) (fixed width, no nulls columns preceding). But
even if not, just the fact that we don't have to copy the column data
for columns never accessed can make for a *significant* speed bump. For
!varlena datatypes we don't ever have to access the data, just the NULL
bitmap. And for varlena ones, we just need to check the varlena header
for the width, not make variable width copy.
So what I'm saying is, that we should pre-compute the set of columns
we're interested in, and deform just those, but still in bulk.
Andres