Обсуждение: Suggestion: pg_dump self-cleanup if out-of-disk
Having recently had a pg_dump error out due to not having enough disk it occurs to me that it would be nice for pg_dump to remove the partial dump file it was creating (if possible/known) instead of having it sit around taking up that last bit of available space and itself being unusable for restore purposes anyway. Given these tend to be large the benefit to cleanup seems quite strong and fairly direct to accomplish since we just created the file only a short while previous. Call it a beginner or part-time-dba usability feature. David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/Suggestion-pg-dump-self-cleanup-if-out-of-disk-tp5778842.html Sent from the PostgreSQL - general mailing list archive at Nabble.com.
On Sun, Nov 17, 2013 at 11:12 PM, David Johnston <polobo@yahoo.com> wrote:
Having recently had a pg_dump error out due to not having enough disk it
occurs to me that it would be nice for pg_dump to remove the partial dump
file it was creating (if possible/known) instead of having it sit around
taking up that last bit of available space and itself being unusable for
restore purposes anyway. Given these tend to be large the benefit to
cleanup seems quite strong and fairly direct to accomplish since we just
created the file only a short while previous. Call it a beginner or
part-time-dba usability feature.
I don't think I like this. If I unexpectedly filled up a partition with a dump file, I think the first thing I would want to do is 'head' or 'more' the partial dump to see what is making it so big--e.g. that I am dumping the wrong cluster/database/schema. I wouldn't want the software to hide the evidence.
Cheers,
Jeff
Jeff Janes wrote > On Sun, Nov 17, 2013 at 11:12 PM, David Johnston < > polobo@ > > wrote: > >> Having recently had a pg_dump error out due to not having enough disk it >> occurs to me that it would be nice for pg_dump to remove the partial dump >> file it was creating (if possible/known) instead of having it sit around >> taking up that last bit of available space and itself being unusable for >> restore purposes anyway. Given these tend to be large the benefit to >> cleanup seems quite strong and fairly direct to accomplish since we just >> created the file only a short while previous. Call it a beginner or >> part-time-dba usability feature. >> > > I don't think I like this. If I unexpectedly filled up a partition with a > dump file, I think the first thing I would want to do is 'head' or 'more' > the partial dump to see what is making it so big--e.g. that I am dumping > the wrong cluster/database/schema. I wouldn't want the software to hide > the evidence. So yeah, my situation was I had too many "archive dumps" maintained so that while each dump was just fine in totality they took up too much space and the last one aborted but left that space occupied. I'll admit my situation is novice-level admin work but I wouldn't put it past others to encounter this situation. Leaving the target drive full just seems hostile. Forensics should be do-able from knowing the command that was used. And maybe upon such an abort STDERR could be used to output relevant diagnostic information (e.g., host, port, user, the result of a version() call, etc...) instead of leaving gigabytes of a partial database dump around which might not even have enough detail. This would be a useful addition in its own right since what pg_dump sees does not always directly match the command that was issued. David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/Suggestion-pg-dump-self-cleanup-if-out-of-disk-tp5778842p5779030.html Sent from the PostgreSQL - general mailing list archive at Nabble.com.
David Johnston wrote > > Jeff Janes wrote >> On Sun, Nov 17, 2013 at 11:12 PM, David Johnston < >> polobo@ >> > wrote: >> >>> Having recently had a pg_dump error out due to not having enough disk it >>> occurs to me that it would be nice for pg_dump to remove the partial >>> dump >>> file it was creating (if possible/known) instead of having it sit around >>> taking up that last bit of available space and itself being unusable for >>> restore purposes anyway. Given these tend to be large the benefit to >>> cleanup seems quite strong and fairly direct to accomplish since we just >>> created the file only a short while previous. Call it a beginner or >>> part-time-dba usability feature. >>> >> >> I don't think I like this. If I unexpectedly filled up a partition with >> a >> dump file, I think the first thing I would want to do is 'head' or 'more' >> the partial dump to see what is making it so big--e.g. that I am dumping >> the wrong cluster/database/schema. I wouldn't want the software to hide >> the evidence. > So yeah, my situation was I had too many "archive dumps" maintained so > that while each dump was just fine in totality they took up too much space > and the last one aborted but left that space occupied. > > I'll admit my situation is novice-level admin work but I wouldn't put it > past others to encounter this situation. Leaving the target drive full > just seems hostile. Forensics should be do-able from knowing the command > that was used. And maybe upon such an abort STDERR could be used to > output relevant diagnostic information (e.g., host, port, user, the result > of a version() call, etc...) instead of leaving gigabytes of a partial > database dump around which might not even have enough detail. This would > be a useful addition in its own right since what pg_dump sees does not > always directly match the command that was issued. > > David J. I doubt anyone is likely to work on this at the moment but it seems like a decent starter project for someone interested in contributing. As such, IMO, it would make a decent ToDo item generally even if additional features are needed to maintain enough forensic data to figure out if the specific pg_dump call was incorrect or if other environmental factors were at play. Anyone agree enough to add this to the wiki in the pg_dump/pg_restore section? David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/Suggestion-pg-dump-self-cleanup-if-out-of-disk-tp5778842p5779031.html Sent from the PostgreSQL - general mailing list archive at Nabble.com.