> You can use COPY from a stored procedure, but only to and from files.
I think that's in the chocolate fireguard realm though as far as
efficiency for this sort of scenario goes, even if its handled by
retaining an mmap'd file as workspace.
>
>> If SPI provided a way to perform a copy to a temp table and then some callback on an iterator that yields rows to
it,that would do the trick I guess.
>
> SPI is useful, but it's certainly possible to avoid its use. After all, that what almost the whole backend does,
includingthe COPY code. Of course, it's a lot harder to write that way, which is part of why SPI exists. Efficiency has
itsprice.
So it is possible to use a lower level interface from a C stored proc?
SPI is the (only) documented direct function extension API isn't it?
Is the issue with using the JSON data-to-record set that the parsing can
be costly? Perhaps it can be achieved with B64 of compressed protobuf,
or such. I don't mind if it seems a bit messy - the code can be
generated from the table easily enough, especially if I can use C++. I
guess an allocator that uses SPI_palloc would solve issues with memory
management on error?