Обсуждение: BUG #16812: Logical decoding error

Поиск
Список
Период
Сортировка

BUG #16812: Logical decoding error

От
PG Bug reporting form
Дата:
The following bug has been logged on the website:

Bug reference:      16812
Logged by:          Mayur B.
Email address:      mayur555b@protonmail.com
PostgreSQL version: 12.3
Operating system:   Ubuntu 16.04.6 LTS
Description:

Hi,
We started receiving the logical decoding error :
ERROR:  could not map filenode "base/16437/140990927" to relation OID

Database is PG 12.3 and Ubuntu 16.04.6 LTS. There is no "vacuum full" or
CLUSTER happening. What could be other sources of this error?

Thanks


Re: BUG #16812: Logical decoding error

От
mayur
Дата:
Sorry, Operating system is Ubuntu 18.04. We are using pipelinewise to consume wal stream and replicate tables to
snowflake.YML config file consist of large number of tables so it is difficult to figure out exact table quickly. 

Documentation about pipelinewise
https://transferwise.github.io/pipelinewise/


Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, 7 January 2021 10:57, PG Bug reporting form <noreply@postgresql.org> wrote:

> The following bug has been logged on the website:
>
> Bug reference: 16812
> Logged by: Mayur B.
> Email address: mayur555b@protonmail.com
> PostgreSQL version: 12.3
> Operating system: Ubuntu 16.04.6 LTS
> Description:
>
> Hi,
> We started receiving the logical decoding error :
> ERROR: could not map filenode "base/16437/140990927" to relation OID
>
> Database is PG 12.3 and Ubuntu 16.04.6 LTS. There is no "vacuum full" or
> CLUSTER happening. What could be other sources of this error?
>
> Thanks





Re: BUG #16812: Logical decoding error

От
Amit Kapila
Дата:
On Thu, Jan 7, 2021 at 2:55 PM PG Bug reporting form
<noreply@postgresql.org> wrote:
>
> The following bug has been logged on the website:
>
> Bug reference:      16812
> Logged by:          Mayur B.
> Email address:      mayur555b@protonmail.com
> PostgreSQL version: 12.3
> Operating system:   Ubuntu 16.04.6 LTS
> Description:
>
> Hi,
> We started receiving the logical decoding error :
> ERROR:  could not map filenode "base/16437/140990927" to relation OID
>
> Database is PG 12.3 and Ubuntu 16.04.6 LTS. There is no "vacuum full" or
> CLUSTER happening. What could be other sources of this error?
>

It is not clear to me how else this error can happen. However, I think
this can happen due to some re-write of mapped relations as we don't
maintain historic-view for mapped relations. Having said that, as far
as I understand this shouldn't cause any problem because we anyway
don't decode updates to catalog relations and we would have anyway
skipped this operation. I am not sure maybe this could have been a LOG
instead of ERROR.

One way to figure what is going on here is to add a LOG message in
apply_map_update to see the change relationId and then we might be
able to track which exact operation has changed this. The other
possibility could be to use pg_waldump to figure out the WAL record
which leads to this error (maybe with the help of LSN, we can identify
that).

-- 
With Regards,
Amit Kapila.



Re: BUG #16812: Logical decoding error

От
mayur
Дата:
Thanks. This was happening because Patroni was removing inactive replication slot.


Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, 12 January 2021 07:06, Amit Kapila <amit.kapila16@gmail.com> wrote:

> On Thu, Jan 7, 2021 at 2:55 PM PG Bug reporting form
> noreply@postgresql.org wrote:
>
> > The following bug has been logged on the website:
> > Bug reference: 16812
> > Logged by: Mayur B.
> > Email address: mayur555b@protonmail.com
> > PostgreSQL version: 12.3
> > Operating system: Ubuntu 16.04.6 LTS
> > Description:
> > Hi,
> > We started receiving the logical decoding error :
> > ERROR: could not map filenode "base/16437/140990927" to relation OID
> > Database is PG 12.3 and Ubuntu 16.04.6 LTS. There is no "vacuum full" or
> > CLUSTER happening. What could be other sources of this error?
>
> It is not clear to me how else this error can happen. However, I think
> this can happen due to some re-write of mapped relations as we don't
> maintain historic-view for mapped relations. Having said that, as far
> as I understand this shouldn't cause any problem because we anyway
> don't decode updates to catalog relations and we would have anyway
> skipped this operation. I am not sure maybe this could have been a LOG
> instead of ERROR.
>
> One way to figure what is going on here is to add a LOG message in
> apply_map_update to see the change relationId and then we might be
> able to track which exact operation has changed this. The other
> possibility could be to use pg_waldump to figure out the WAL record
> which leads to this error (maybe with the help of LSN, we can identify
> that).
>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> With Regards,
> Amit Kapila.





Re: BUG #16812: Logical decoding error

От
Amit Kapila
Дата:
On Wed, Jan 13, 2021 at 2:05 PM mayur <mayur555b@protonmail.com> wrote:
>
> Thanks. This was happening because Patroni was removing inactive replication slot.
>

It is not apparent to me how removing the inactive replication slot by
Patroni causes this error. Can you please explain what's going on?

-- 
With Regards,
Amit Kapila.