Re: Remaining beta blockers

Could we please stop the ad-hominem stuff from all sides? We want to
solve the issue not to make it bigger.

On 2013-04-30 04:29:26 -0700, Kevin Grittner wrote:
> Let's look at the "corner" this supposedly paints us into.  If a
> later major release creates a better mechanism, current pg_dump and
> load will already use it, based on the way matviews are created
> empty and REFRESHed by pg_dump.  Worst case, we need to modify the
> behavior of pg_dump running with the switch used by pg_upgrade to
> use a new ALTER MATERIALIZED VIEW SET (populated); (or whatever
> syntax is chosen) -- a command we would probably want at that point
> anyway.  I'm not seeing cause for panic here.

I don't think panic is appropriate either, but I think there are some
valid concerns around this.

1) vacuum can truncate the table to an empty relation already if there is  no data returned by the view's query which
thenleads to the wrong  scannability state. 
  S1: CREATE MATERIALIZED VIEW matview_empty AS SELECT false WHERE random() < -1;  S2: S2: SELECT * FROM matview_empty;
--works  S1: VACUUM matview_empty;  S2: SELECT * FROM matview_empty; -- still works  S3: SELECT * FROM matview_empty;
--errors out 
  So we need to make vacuum skip cleaning out the last page. Once we  get incrementally updated matviews there are more
situationsto get  into this than just a query not returning anything.  I remember this being discussed somewhere
already,but couldn't find  it quickly enough. 
  Imo this is quite an indicator that the idea of using the filesize is  just plain wrong. Adding logic to vacuum not
totruncate data because  its a flag for matview scannability is quite the layer violation and  a sign for a bad design
decision.Such a hack has already been added  to copy_heap_data(), while not as bad, shows that it is hard to find  all
theplaces where we need to add it. 

2) Since we don't have a metapage to represent scannability in 9.3 we  cannot easily use one in 9.4+ without pg_upgrade
emptyingall  matviews unless we only rely on the catalogs which we currently  cannot. This will either slow down
developmentor make users  unhappy. Alternatively we can add yet another fork, but that has its  price (quite a bit more
openfiles during normal operation, slower  CREATE DATABASE). 
  This is actually an argument for not releasing matviews without such  an external state. Going from disk-based state
tocatalog is easy,  the other way round: not so much. 

3) Using the filesize as a flag will make other stuff like preallocating  on-disk data in bigger chunks and related
thingsmore complicated. 

4) doing the check for scannability in the executor imo is a rather bad  idea. Note that we e.g. don't see an error
abouta matview which  won't be scanned because of constraint exclusion or one-time  filters. 
  S1: CREATE MATERIALIZED VIEW matview_unit_false AS SELECT false WHERE false WITH NO DATA;  S1: SELECT * FROM
matview_unit_false;

You can get there in far more reasonable cases than WHERE false.

5) I have to agree with Kevin that the scannability is an important thing  to track though.
  a) we cannot remove support for WITH NO DATA because of pg_dump order     and unlogged relations. So even without
unloggedrelations the     problem exists although its easier to represent.  b) Just giving wrong responses is bad [tm].
Outdateddata is something     completely different (it has existed in that state in the (near)     past) than giving an
emptyresponse (might never have been a visible     state, and more likely not so in any reasonably near     past).
Consideran application behind a pooler suddently getting     an empty response from a SELECT * FROM unlogged_matview;
Itwon't     notice anything without a unscannable state since it probably     won't even notice the database restart. 

Not sure what the consequence of this is. The most reasonable solution
seems to be to introduce a metapage somewhere *now* which sucks, but ...

Greetings,

Andres Freund

--Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services