Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Mar 19, 2025 at 6:25 PM Sutou Kouhei <kou@clear-code.com> wrote:
>
> Hi,
>
> In <CAKFQuwaMAFMHqxDXR=SxA0mDjdmntrwxZd2w=nSruLNFH-OzLw@mail.gmail.com>
>   "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 19 Mar 2025 17:49:49 -0700,
>   "David G. Johnston" <david.g.johnston@gmail.com> wrote:
>
> >> And could someone help (take over if possible) writing a
> >> document for this feature? I'm not good at writing a
> >> document in English... 0009 in the attached v37 patch set
> >> has a draft of it. It's based on existing documents in
> >> doc/src/sgml/ and *.h.
> >>
> >>
> > I haven't touched the innards of the structs aside from changing
> > programlisting to synopsis.  And redoing the two section opening paragraphs
> > to better integrate with the content in the chapter opening.
> >
> > The rest I kinda went to town on...
>
> Thanks!!! It's very helpful!!!
>
> I've applied your patch. 0009 is only changed.

FYI I've implemented an extension to add JSON Lines format as a custom
COPY format[1] to check the usability of the COPY format APIs. I think
that the exposed APIs are fairly simple and minimum. I didn't find the
deficiency and excess of exposed APIs for helping extensions but I
find that it would be better to describe what the one-row callback
should do to utilize the abstracted destination. For example, in order
to use CopyToStateFlush() to write out to the destination, extensions
should write the data to cstate->fe_msgbuf. We expose
CopyToStateFlush() but not for any functions to write data there such
as CopySendString(). It was a bit inconvenient to me but I managed to
write the data directly there by #include'ing copyto_internal.h.

Regards,

[1] https://github.com/MasahikoSawada/pg_copy_jsonlines

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com