I'm trying to contribute some improvements to the implementation of a gist index that is part of an open source project and it would be really nice if anyone could help me answer some questions.
I would like to use different data structures to represent the internal and leaf entries. I think I have found several examples in the postgresql source code where a variable length array is used to pack different data consistently with the value of a "type" attribute. Is this the suggested way to proceed, or would it be also possible to use two actually different data structures? I at first considered this latter approach, but some erroneous behavior suggested that maybe I am not fully clear about which data structure should be produced or expected by some parts of the implementation.
Could you please also help me understand the difference (if any) between using the GIST_LEAF macro or the leafkey attribute of the GISTENTRY data structure?