Re: pg_upgrade not able to cope with pg_largeobject being in a different tablespace

 

 

On Thu, Oct 13, 2016 at 7:35 AM, Andreas Joseph Krogh <andreas@visena.com> wrote:

På torsdag 13. oktober 2016 kl. 16:09:34, skrev Bruce Momjian <bruce@momjian.us>:

On Thu, Oct 13, 2016 at 10:14:08AM +0200, Andreas Joseph Krogh wrote:
> I would assume that having pg_largeobject in a separate tablespace is more and
> more common these days, having real-cheap SAN vs. fast-SSD for normal tables/
> indexes/wal.

So common that no one has ever asked for this feature before?

 

 

Sometimes one gets the feeling that one is the only one in the universe doing something one considers "quite common":-)

 

> So - I'm wondering if we can fund development of pg_upgrade to cope with this
> configuration or somehow motivate to getting this issue fixed?
>  
> Would any of the PG-companies (2ndQ, EDB, PgPro) take a stab at this?
>  
> Any feedback welcome, thanks.

You would need to get buy-in that that community wants the relocation of
pg_largeobject to be supported via an SQL command, and at that point
pg_upgrade would be modified to support that.  It is unlikely pg_upgrade
is going to be modified to support something that isn't supported at the
SQL level.  Of course, you can create a custom version of pg_upgrade to
do that.

 

Doesn't "ALTER TABLE pg_largeobject SET TABLESPACE myspace_lo" count as being "at the SQL-level"?

 

Well, it requires that you set allow_system_table_mods on, which is documented as a developer option. It's documented with things like "The following parameters are intended for work on the PostgreSQL source code, and in some cases to assist with recovery of severely damaged databases. There should be no reason to use them on a production database.".

Perhaps we should add another disclaimer there saying that if you make changes in this mode, tools like pg_upgrade (or for that matter, pg_dump or pretty much anything at all) may not work anymore?

 

 

The whole problem seems to come from the fact that BLOBs are stored in pg_largeobject which for some reason is implemented as a system-catalogue in PG, which imposes all kinds of weird problems, from a DBA-perspective.

 

Yes, there are several issues related to how lo style large objects work. I've often gone to similar implementations but in userspace on top of custom tables to work around those.

 

 

Can we pay you at EDB for making such a custom version of pg_upgrade for 9.6?

 

 

You're assuming pg_upgrade is the only potential problem. If you are willing to spend towards it, it would probably be better to spend towards the "upper layer" problem which would be to make it possible to move pg_largeobject to a different tablespace *without* turning on system_table_mods.

 

That said, I cannot comment to the complexity of either doing that *or* doing a custom pg_upgrade that would support it. But solving a long-term problem seems better than solving a one-off one.

Andreas Joseph Krogh

CTO / Partner - Visena AS

Mobile: +47 909 56 963

andreas@visena.com

www.visena.com