I found the following error message in my logfiles.
Oct 24 04:05:57 db-app02 postgres[24640]: [2-1] user=nutricate,db=admin_production ERROR: missing chunk number 0 for toast value 2411466 in pg_toast_2619
Oct 24 04:05:57 db-app02 postgres[24640]: [2-2] user=nutricate,db=admin_production STATEMENT: SELECT "devices"."id", "devices"."name", "devices"."info", "devices"."pos_id", "devices"."device_group_id", "devices"."header_id", "devices"."footer_id", "devices"."device_type_id", "devices"."auth_code", "devices"."pos_comm_id", "devices"."printer_comm_id", "devices"."sw_version", "devices"."hw_version", "devices"."device_status", "devices"."entity_id", "devices"."address", "devices"."created_by", "devices"."create_method", "devices"."created_on", "devices"."modified_by", "devices"."updated_on", "devices"."version_id", "devices"."unique_id", "devices"."hw_key", "devices"."config_status", "devices"."activated_on", "devices"."store_id", "devices"."last_status_update", "devices"."loaded_on", "devices"."header2_id", "devices"."footer2_id", "devices"."timezone_id", "devices"."scanner_comm_id", "devices"."public_address", "devices"."hostname", "devices"."update_sw_version", "devices"."proxy_address", "devices"."proxy_type_id", "devices"."build_error",
Oct 24 04:05:57 db-app02 postgres[24640]: [2-3] "devices"."local_info", "devices"."associated_on" FROM "devices" WHERE ("devices"."entity_id" IN (SELECT U0."id" FROM "entities" U0 WHERE ((U0."lft" > 280 AND U0."rgt" < 2597 ) OR U0."id" = 140 )) AND "devices"."auth_code" = E'0063' )
I tried reproducing it with
SELECT * FROM devices WHERE ("devices"."entity_id" IN (SELECT U0."id" FROM "entities" U0 WHERE ((U0."lft" > 280 AND U0."rgt" < 2597 ) OR U0."id" = 140 )) AND "devices"."auth_code" = E'0063' )
This returned no rows and did not reproduce the error. I tried reproducing with an even broader approach
CREATE TEMP TABLE foo AS SELECT * FROM devices;
CREATE TEMP TABLE bar AS SELECT * FROM entities;
Still no repro. This is the second time I've seen this error, and the last time it also did not reproduce. I'm wondering a couple of things. First, is there some other way to reproduce this error? Second, would simply deleting the row cause the problem to go away? I wouldn't think so, but why else is it not reproducing? Third, what is the recommended course of action here. We have hourly backups, but this is a production database, so I would prefer not to have to restore. I tested a restore from an hour ago and it ran successfully.
A