Re: Complete data erasure

Greetings,

* asaba.takanori@fujitsu.com (asaba.takanori@fujitsu.com) wrote:
> From: Stephen Frost <sfrost@snowman.net>
> > * asaba.takanori@fujitsu.com (asaba.takanori@fujitsu.com) wrote:
> > > This feature erases data area just before it is returned to the OS (“erase”
> > means that overwrite data area to hide its contents here)
> > > because there is a risk that the data will be restored by attackers if it is returned
> > to the OS without being overwritten.
> > > The erase timing is when DROP, VACUUM, TRUNCATE, etc. are executed.
> >
> > Looking at this fresh, I wanted to point out that I think Tom's right-
> > we aren't going to be able to reasonbly support this kind of data
> > erasure on a simple DROP TABLE or TRUNCATE.
> >
> > > I want users to be able to customize the erasure method for their security
> > policies.
> >
> > There's also this- but I think what it means is that we'd probably have
> > a top-level command that basically is "ERASE TABLE blah;" or similar
> > which doesn't operate during transaction commit but instead marks the
> > table as "to be erased" and then perhaps "erasure in progress" and then
> > "fully erased" (or maybe just back to 'normal' at that point).  Making
> > those updates will require the command to perform its own transaction
> > management which is why it can't be in a transaction itself but also
> > means that the data erasure process doesn't need to be done during
> > commit.
> >
> > > My idea is adding a new parameter erase_command to postgresql.conf.
> >
> > Yeah, I don't think that's really a sensible option or even approach.
>
> I think erase_command can also manage the state of a table.
> The exit status of a configured command shows it.( 0 is "fully erased" or "normal", 1 is "erasure in progress")
> erase_command is executed not during a transaction but when unlink() is executed.

I really don't see what the advantage of having this be configurable is.
In addition, an external command's actions wouldn't be put through the
WAL meaning that replicas would have to be dealt with in some other way
beyind regular WAL and that seems like it'd just be ugly.

> (for example, after a transaction that has done DROP TABLE)

We certainly can't run external commands during transaction COMMIT, so
this can't be part of a regular DROP TABLE.

> > > When erase_command is set, VACUUM does not truncate a file size to non-zero
> > > because it's safer for users to return the entire file to the OS than to return part
> > of it.
> >
> > There was discussion elsewhere about preventing VACUUM from doing a
> > truncate on a file because of the lock it requires and problems with
> > replicas..  I'm not sure where that ended up, but, in general, I don't
> > think this feature and VACUUM should really have anything to do with
> > each other except for the possible case that a user might be told to
> > configure their system to not allow VACUUM to truncate tables if they
> > care about this case.
>
> I think that if ftruncate(fd, 0) is executed in VACUUM,
> data area allocated to a file is returned to the OS, so that area must be overwritten.

As I mentioned, there was already talk of making that disallowed in
VACUUM, so that would just need to be configured (and be able to be
configured) when running in an environment which requires this.

> > As mentioned elsewhere, you do also have to consider that the sensitive
> > data will end up in the WAL and on replicas.  I don't believe that means
> > this feature is without use, but it means that users of this feature
> > will also need to understand and be able to address WAL and replicas
> > (along with backups and such too, of course).
>
> I see.
> I can't think of it right away, but I will deal with it.

It's something that will need to be understood and dealt with.  A simple
answer might be "the user must also destroy all WAL and all backups in
an approved manner, and ensure all replicas have replayed all WAL or
been destroyed".

Thanks,

Stephen