Обсуждение: Crash on SRF execution

<div dir="ltr">Hi<br /> <br />I'm attempting to program a simple SRF function but it constantly crashes (details and
codebelow).<br /> <br />Any idea why?<br /> <br />Thanks!<br /> <br />-Itai<br
/>-------------------------------------------------------------<br/>Environment<br
/>-------------------------------------------------------------<br/>Ubunto: 14.04.2 (server)<br />PG Ver: PostgreSQL
9.4.1on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2, 64-bit<br />Install: deb <a
href="http://apt.postgresql.org/pub/repos/apt/">http://apt.postgresql.org/pub/repos/apt/</a>trusty-pgdg main<br />Dev:
postgresql-server-dev-9.4(9.4.1-1.pgdg14.04+1)<br />-------------------------------------------------------------<br
/>Execution<br/>-------------------------------------------------------------<br />select * from pg_srf();<br />INFO: 
-----------data source -----------<br />INFO:  value: 1203000000<br />INFO:  is_even: 1<br />INFO:  value:
1203000001<br/>INFO:  is_even: 0<br />...<br />INFO:  value: 1203003998<br />INFO:  is_even: 1<br />INFO:  value:
1203003999<br/>INFO:  is_even: 0<br />INFO:  ----------- data context -----------<br />INFO:  call_cntr: 0<br />INFO: 
value:1203000000<br />INFO:  is_even: 1<br />INFO:  call_cntr: 1<br />INFO:  value: 1203000001<br />INFO:  is_even:
0<br/>INFO:  call_cntr: 2<br />server closed the connection unexpectedly<br /> This probably means the server
terminatedabnormally<br /> before or while processing the request.<br />The connection to the server was lost.
Attemptingreset: Failed.<br />!> <br />-------------------------------------------------------------<br
/>pg_srf.h<br/>-------------------------------------------------------------<br />#ifndef PGSRF_H<br />#define
PGSRF_H<br/>#include "fmgr.h"<br />// using int as bool due to a different issue (one prob. at a time)<br />typedef
structNumber_tag<br />{<br /> int value;<br /> int is_even;<br />} Number;<br />typedef struct NumberList_tag<br />{<br
/> intlength;<br /> Number ** pp_numbers;<br />} NumberList;<br />extern Datum pg_srf(PG_FUNCTION_ARGS);<br />#endif<br
/>-------------------------------------------------------------<br/>pg_srf.c<br
/>-------------------------------------------------------------<br/>#include <postgres.h><br />#include
<access/htup_details.h><br/>#include <catalog/pg_type.h><br />#include <funcapi.h><br />#include
"pg_srf.h"<br/>#ifdef PG_MODULE_MAGIC<br />PG_MODULE_MAGIC;<br />#endif<br />PG_FUNCTION_INFO_V1(pg_srf);<br />Datum<br
/>pg_srf(PG_FUNCTION_ARGS)<br/>{<br /> int call_cntr, i, length, base_num;<br /> Number * num;<br /> NumberList *
list;<br/> HeapTuple rettuple;<br /> FuncCallContext *funcctx;<br /> MemoryContext oldcontext;<br /> if
(SRF_IS_FIRSTCALL())<br/> {<br />  length = 4000;<br />  base_num = 1203000000;<br />  list = (NumberList
*)palloc(sizeof(NumberList));<br/>  list->pp_numbers = (Number **)palloc(sizeof(Number*) * length);<br
/>  list->length= length;<br />  i = 0;<br />  for (; i < length; i++)<br />  {<br />   num = (Number
*)palloc(sizeof(Number));<br/>   num->value = base_num + i;<br />   num->is_even = ((base_num + i) % 2 == 0) ? 1
:0;<br />   list->pp_numbers[i] = num;<br />  }<br />  ereport(INFO, (errmsg("----------- data source
-----------")));<br/>  i = 0;<br />  for (; i < length; i++)<br />  {<br />   ereport(INFO, (errmsg("value: %d",
list->pp_numbers[i]->value)));<br/>   ereport(INFO, (errmsg("is_even: %d",
list->pp_numbers[i]->is_even)));   <br/>  }<br />  <br />  funcctx = SRF_FIRSTCALL_INIT();<br />  oldcontext =
MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);<br/>  funcctx->user_fctx = list;<br
/>  funcctx->max_calls= list->length;<br />  if (get_call_result_type(fcinfo, NULL, &funcctx->tuple_desc)
!=TYPEFUNC_COMPOSITE)<br />   ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),<br />   errmsg("check if sql
functiondefinition returns SETOF record")));<br /><br />  BlessTupleDesc(funcctx->tuple_desc);<br
/>  MemoryContextSwitchTo(oldcontext);<br/> }<br /> funcctx = SRF_PERCALL_SETUP();<br /> list =
funcctx->user_fctx;<br/> call_cntr = funcctx->call_cntr;<br /> if (call_cntr < funcctx->max_calls)<br
/> {<br/>  Datum retvals[2];<br />  bool retnulls[2];<br />  <br />  if (call_cntr == 0) <br />  {<br
/>   ereport(INFO,(errmsg("----------- data context -----------")));<br />  }<br />  ereport(INFO, (errmsg("call_cntr:
%d",call_cntr)));<br />  ereport(INFO, (errmsg("value: %d", list->pp_numbers[call_cntr]->value)));<br
/>  retvals[0]= Int32GetDatum(list->pp_numbers[call_cntr]->value);<br />  ereport(INFO, (errmsg("is_even: %d",
list->pp_numbers[call_cntr]->is_even)));<br/>  retvals[1] =
Int32GetDatum(list->pp_numbers[call_cntr]->is_even);<br/>  retnulls[0] = false;<br />  retnulls[1] = false;<br
/>  rettuple= heap_form_tuple(funcctx->tuple_desc, retvals, retnulls);<br />  SRF_RETURN_NEXT(funcctx,
HeapTupleGetDatum(rettuple));<br/> }<br /> else<br /> {<br />  SRF_RETURN_DONE(funcctx);<br /> }<br />}<br
/>-------------------------------------------------------------<br/>Makefile<br
/>-------------------------------------------------------------<br/>MODULES = pg_srf<br />OBJS = pg_srf.o<br
/>PG_CONFIG= pg_config<br />PGXS := $(shell $(PG_CONFIG) --pgxs)<br />include $(PGXS)<br
/>-------------------------------------------------------------<br/>SQL<br
/>-------------------------------------------------------------<br/>CREATE OR REPLACE FUNCTION<br />  pg_srf(OUT value
integer,OUT is_even integer)<br />RETURNS <br /> SETOF record<br />AS<br />  'pg_srf.so', 'pg_srf'<br />LANGUAGE<br /> 
C<br/>STRICT<br />IMMUTABLE;<br /></div> 

Itai <itaid@outlook.com> writes:
> I'm attempting to program a simple SRF function but it constantly crashes (details and code below).
> Any idea why?

Looks like you're pallocing some stuff in the calling context (ie, a
short-lived context) during the first execution and expecting it to
still be there in later executions.  You'd need to allocate those
data structures in the multi_call_memory_ctx instead.
        regards, tom lane

Hi,

On 2015-03-15 17:40:11 +0200, Itai wrote:
> I'm attempting to program a simple SRF function but it constantly crashes (details and code below).
>  
> Any idea why?

>  if (SRF_IS_FIRSTCALL())
>  {
>   length = 4000;
>   base_num = 1203000000;
>   list = (NumberList *)palloc(sizeof(NumberList));
>   list->pp_numbers = (Number **)palloc(sizeof(Number*) * length);
>   list->length = length;
>   i = 0;
>   for (; i < length; i++)
>   {
>    num = (Number *)palloc(sizeof(Number));
>    num->value = base_num + i;
>    num->is_even = ((base_num + i) % 2 == 0) ? 1 : 0;
>    list->pp_numbers[i] = num;
>   }
>   ereport(INFO, (errmsg("----------- data source -----------")));
>   i = 0;
>   for (; i < length; i++)
>   {
>    ereport(INFO, (errmsg("value: %d", list->pp_numbers[i]->value)));
>    ereport(INFO, (errmsg("is_even: %d", list->pp_numbers[i]->is_even)));   
>   }
>   
>   funcctx = SRF_FIRSTCALL_INIT();
>   oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
>   funcctx->user_fctx = list;
>   funcctx->max_calls = list->length;
>   if (get_call_result_type(fcinfo, NULL, &funcctx->tuple_desc) != TYPEFUNC_COMPOSITE)
>    ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
>    errmsg("check if sql function definition returns SETOF record")));
> 
>   BlessTupleDesc(funcctx->tuple_desc);
>   MemoryContextSwitchTo(oldcontext);
>  }

The palloc() for list above is in the per call memory context, but you
use it across several calls. You should allocate it in the multi call
context you use some lines below.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services

Hi,

On 2015-03-15 17:59:39 +0200, Itai wrote:
> Thanks for the quick response!! :)
> But I don't get it... isn't:
> if (SRF_IS_FIRSTCALL()){
> }
> the iterator one time
> "initialization block" where I setup the data to be iterated
> upon? 
> 
> Can you please elaborate on how should I fix this? I'm probably missing something basic...

> > >  if (SRF_IS_FIRSTCALL())
> > >  {
> > >   length = 4000;
> > >   base_num = 1203000000;
> > >   list = (NumberList *)palloc(sizeof(NumberList));
> > >   list->pp_numbers = (Number **)palloc(sizeof(Number*) * length);

You allocate memory in the per call context here.

> > >   list->length = length;
> > >   i = 0;
> > >   for (; i < length; i++)
> > >   {
> > >    num = (Number *)palloc(sizeof(Number));
> > >    num->value = base_num + i;
> > >    num->is_even = ((base_num + i) % 2 == 0) ? 1 : 0;
> > >    list->pp_numbers[i] = num;
> > >   }
> > >   ereport(INFO, (errmsg("----------- data source -----------")));
> > >   i = 0;
> > >   for (; i < length; i++)
> > >   {
> > >    ereport(INFO, (errmsg("value: %d", list->pp_numbers[i]->value)));
> > >    ereport(INFO, (errmsg("is_even: %d", list->pp_numbers[i]->is_even)));   
> > >   }
> > >   
> > >   funcctx = SRF_FIRSTCALL_INIT();
> > >   oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);

Because you only switch the memory context here. Move this up, to the
beginning of the SRF_IS_FIRSTCALL block. Before the palloc()s.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services

Thanks Tom.

Assuming the SRF had a parameter, would this be a correct approach
(considering the iterative model) to bail-out early?

if (SRF_IS_FIRSTCALL())
{int i;if (get_call_result_type(fcinfo, NULL, &funcctx->tuple_desc) !=
TYPEFUNC_COMPOSITE) {    ereport(ERROR,    (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),     errmsg("Check if sql function
definitionreturns SETOF
 
record")));            return;}
if (PG_ARGISNULL(0)) {    ereport(ERROR,    (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),     errmsg("Null value not allow
for...")));     return;}     if((i = PG_GETARG_INT32(0)) != 'WHATEVER') {    ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),    errmsg("Null value not allow for ...")));     return;}
 

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us] 
Sent: Sunday, March 15, 2015 5:50 PM
To: Itai
Cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Crash on SRF execution

Itai <itaid@outlook.com> writes:
> I'm attempting to program a simple SRF function but it constantly crashes
(details and code below).
> Any idea why?

Looks like you're pallocing some stuff in the calling context (ie, a
short-lived context) during the first execution and expecting it to still be
there in later executions.  You'd need to allocate those data structures in
the multi_call_memory_ctx instead.
        regards, tom lane