Обсуждение: New copyright program

Folks,

I noticed that src/tools/copyright looks like it can only be run on
Bruce's machine, so this translation to Perl is intended:

1.  To make the script idempotent, which allows its safe use in
automated tools that might run it many times.

2.  To get the script to run on any machine a PostgreSQL developer
would be using, as Perl is already required.

3.  To make the script more efficient.  As the copyright notice we
need to munge only appears once per file, it stops once it has made a
substitution.

Please find attached a patch implementing same.

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter wrote:
> Folks,
> 
> I noticed that src/tools/copyright looks like it can only be run on
> Bruce's machine, so this translation to Perl is intended:
> 
> 1.  To make the script idempotent, which allows its safe use in
> automated tools that might run it many times.
> 
> 2.  To get the script to run on any machine a PostgreSQL developer
> would be using, as Perl is already required.
> 
> 3.  To make the script more efficient.  As the copyright notice we
> need to munge only appears once per file, it stops once it has made a
> substitution.
> 
> Please find attached a patch implementing same.

Thanks.  Applied to HEAD.  I never liked putting scripts in git that
only I could run, but I thought if something happened to me, it would be
good to record what I did.  The Perl solution is perfect.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +

On 8/19/2011 10:51 AM, Bruce Momjian wrote:
> David Fetter wrote:
>> [Here's a new copyright program.]
>
> Thanks.  Applied to HEAD.  I never liked putting scripts in git that
> only I could run, but I thought if something happened to me, it would be
> good to record what I did.  The Perl solution is perfect.
>

For me this fails with:

Bareword "Tie::File" not allowed while "strict subs" in use  at 
/home/jurka/pg/server/postgresql/src/tools/copyright.pl line 28.

Using perl -v:
This is perl 5, version 12, subversion 3 (v5.12.3) built for 
x86_64-linux-gnu-thread-multi

Additionally it would be nice if this file was marked executable in git.

Kris Jurka

On Fri, 19 Aug 2011, Kris Jurka wrote:

> For me this fails with:
> 
> Bareword "Tie::File" not allowed while "strict subs" in use  at
> /home/jurka/pg/server/postgresql/src/tools/copyright.pl line 28.
> 

This fixes things for me.  The copyright matching wasn't working for me 
either without escaping the parentheses.

Kris Jurka

On Fri, Aug 19, 2011 at 05:34:01PM -0400, Kris Jurka wrote:
> On Fri, 19 Aug 2011, Kris Jurka wrote:
> 
> > For me this fails with:
> > 
> > Bareword "Tie::File" not allowed while "strict subs" in use  at
> > /home/jurka/pg/server/postgresql/src/tools/copyright.pl line 28.
> 
> This fixes things for me.  The copyright matching wasn't working for me 
> either without escaping the parentheses.

Thanks for fixing this :)

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Kris Jurka wrote:
>
>
> On Fri, 19 Aug 2011, Kris Jurka wrote:
>
> > For me this fails with:
> >
> > Bareword "Tie::File" not allowed while "strict subs" in use  at
> > /home/jurka/pg/server/postgresql/src/tools/copyright.pl line 28.
> >
>
> This fixes things for me.  The copyright matching wasn't working for me
> either without escaping the parentheses.

Was able to reproduce the error you reported with Perl 5.10.  I then
tried the single-quote idea I got from Googling, but then got an error
about TIEARRAY being missing, so I recoded it as a simple file
open/close.  I also incorported your regex fix.  Path attached and
applied.  Thanks.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/src/tools/copyright.pl b/src/tools/copyright.pl
new file mode 100755
index 96b1f22..91f73e3
*** a/src/tools/copyright.pl
--- b/src/tools/copyright.pl
*************** use warnings;
*** 13,19 ****
  use File::Find;

  my $pgdg = 'PostgreSQL Global Development Group';
! my $cc = 'Copyright (c) ';
  # year-1900 is what localtime(time) puts in element 5
  my $year = 1900 + ${[localtime(time)]}[5];

--- 13,19 ----
  use File::Find;

  my $pgdg = 'PostgreSQL Global Development Group';
! my $cc = 'Copyright \(c\) ';
  # year-1900 is what localtime(time) puts in element 5
  my $year = 1900 + ${[localtime(time)]}[5];

*************** print "Using current year:  $year\n";
*** 22,33 ****
  find({wanted => \&wanted, no_chdir => 1}, '.');

  sub wanted {
!     return unless -f $File::Find::name;

!     my @lines;
!     tie @lines, Tie::File, $File::Find::name;

!     foreach my $line (@lines) {
          # We only care about lines with a copyright notice.
          next unless $line =~ m/$cc.*$pgdg/;
          # We stop when we've done one substitution.  This is both for
--- 22,35 ----
  find({wanted => \&wanted, no_chdir => 1}, '.');

  sub wanted {
!     my $filename = $File::Find::name;

!     # only regular files
!     return if ! -f $filename;

!     open(my $FILE, '<', $filename) or die "Cannot open $filename";
!
!     foreach my $line (<$FILE>) {
          # We only care about lines with a copyright notice.
          next unless $line =~ m/$cc.*$pgdg/;
          # We stop when we've done one substitution.  This is both for
*************** sub wanted {
*** 37,43 ****
          last if $line =~ s/($cc\d{4})(, $pgdg)/$1-$year$2/;
          last if $line =~ s/($cc\d{4})-\d{4}(, $pgdg)/$1-$year$2/;
      }
!     untie @lines;
  }

  print "Manually update doc/src/sgml/legal.sgml and src/interfaces/libpq/libpq.rc.in too\n";
--- 39,45 ----
          last if $line =~ s/($cc\d{4})(, $pgdg)/$1-$year$2/;
          last if $line =~ s/($cc\d{4})-\d{4}(, $pgdg)/$1-$year$2/;
      }
!     close($FILE) or die "Cannot close $filename";
  }

  print "Manually update doc/src/sgml/legal.sgml and src/interfaces/libpq/libpq.rc.in too\n";

On Fri, 19 Aug 2011, Bruce Momjian wrote:

> Was able to reproduce the error you reported with Perl 5.10.  I then
> tried the single-quote idea I got from Googling, but then got an error
> about TIEARRAY being missing, so I recoded it as a simple file
> open/close.  I also incorported your regex fix.  Path attached and
> applied.  Thanks.
> 

Did you also try the "use Tie::File" addition in my fix because your 
current coding doesn't work at all.  The tie operation is key to actually 
writing out the modified copyright notice.  Your version just updates the 
copyright year in memory, but never gets it back to the file.

Kris Jurka

Kris Jurka wrote:
>
>
> On Fri, 19 Aug 2011, Bruce Momjian wrote:
>
> > Was able to reproduce the error you reported with Perl 5.10.  I then
> > tried the single-quote idea I got from Googling, but then got an error
> > about TIEARRAY being missing, so I recoded it as a simple file
> > open/close.  I also incorported your regex fix.  Path attached and
> > applied.  Thanks.
> >
>
> Did you also try the "use Tie::File" addition in my fix because your
> current coding doesn't work at all.  The tie operation is key to actually
> writing out the modified copyright notice.  Your version just updates the
> copyright year in memory, but never gets it back to the file.

Ah, that did fix it;  thanks.  I am attached the updated committed
copyright.pl.  I also skipped symlinks.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +
#!/usr/bin/perl
#################################################################
# copyright.pl -- update copyright notices throughout the source tree, idempotently.
#
# Copyright (c) 2011, PostgreSQL Global Development Group
#
# src/tools/copyright.pl
#################################################################

use strict;
use warnings;

use File::Find;
use Tie::File;

my $pgdg = 'PostgreSQL Global Development Group';
my $cc = 'Copyright \(c\) ';
# year-1900 is what localtime(time) puts in element 5
my $year = 1900 + ${[localtime(time)]}[5];

print "Using current year:  $year\n";

find({wanted => \&wanted, no_chdir => 1}, '.');

sub wanted {
    return if ! -f $File::Find::name || -l $File::Find::name;

    my @lines;
    tie @lines, "Tie::File", $File::Find::name;

    foreach my $line (@lines) {
        # We only care about lines with a copyright notice.
        next unless $line =~ m/$cc.*$pgdg/;
        # We stop when we've done one substitution.  This is both for
        # efficiency and, at least in the case of this program, for
        # correctness.
        last if $line =~ m/$cc.*$year.*$pgdg/;
        last if $line =~ s/($cc\d{4})(, $pgdg)/$1-$year$2/;
        last if $line =~ s/($cc\d{4})-\d{4}(, $pgdg)/$1-$year$2/;
    }
    untie @lines;
}

print "Manually update doc/src/sgml/legal.sgml and src/interfaces/libpq/libpq.rc.in too\n";

On 08/19/2011 07:31 PM, Bruce Momjian wrote:
> Kris Jurka wrote:
>> Did you also try the "use Tie::File" addition in my fix because your
>> current coding doesn't work at all.  The tie operation is key to actually
>> writing out the modified copyright notice.  Your version just updates the
>> copyright year in memory, but never gets it back to the file.
> Ah, that did fix it;  thanks.  I am attached the updated committed
> copyright.pl.  I also skipped symlinks.
>

It probably doesn't matter that much in this context, but I should point 
out that Tie::File is not universally available. Some years ago I had to 
revert its use in the buildfarm code for that reason. In general we 
should try to avoid adding extra dependencies wherever possible.

cheers

andrew

Andrew Dunstan wrote:
> 
> 
> On 08/19/2011 07:31 PM, Bruce Momjian wrote:
> > Kris Jurka wrote:
> >> Did you also try the "use Tie::File" addition in my fix because your
> >> current coding doesn't work at all.  The tie operation is key to actually
> >> writing out the modified copyright notice.  Your version just updates the
> >> copyright year in memory, but never gets it back to the file.
> > Ah, that did fix it;  thanks.  I am attached the updated committed
> > copyright.pl.  I also skipped symlinks.
> >
> 
> It probably doesn't matter that much in this context, but I should point 
> out that Tie::File is not universally available. Some years ago I had to 
> revert its use in the buildfarm code for that reason. In general we 
> should try to avoid adding extra dependencies wherever possible.

Oh, great.  :-(

I can easily change this to rewrite files that contain copyright changes
--- should I?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +

On Fri, 19 Aug 2011, Bruce Momjian wrote:

> Andrew Dunstan wrote:
> > 
> > It probably doesn't matter that much in this context, but I should point 
> > out that Tie::File is not universally available. Some years ago I had to 
> > revert its use in the buildfarm code for that reason. In general we 
> > should try to avoid adding extra dependencies wherever possible.
> 
> 
> I can easily change this to rewrite files that contain copyright changes
> --- should I?
> 

No.  We don't need a super portable copyright year changing script.

Kris Jurka

On Fri, Aug 19, 2011 at 07:37:29PM -0400, Andrew Dunstan wrote:
> 
> 
> On 08/19/2011 07:31 PM, Bruce Momjian wrote:
> >Kris Jurka wrote:
> >>Did you also try the "use Tie::File" addition in my fix because your
> >>current coding doesn't work at all.  The tie operation is key to actually
> >>writing out the modified copyright notice.  Your version just updates the
> >>copyright year in memory, but never gets it back to the file.
> >Ah, that did fix it;  thanks.  I am attached the updated committed
> >copyright.pl.  I also skipped symlinks.
> >
> 
> It probably doesn't matter that much in this context, but I should
> point out that Tie::File is not universally available. Some years
> ago I had to revert its use in the buildfarm code for that reason.
> In general we should try to avoid adding extra dependencies wherever
> possible.

Tie::File ships as part of core Perl for all non-EOL versions, so I
really can't consider this as a problem.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On 08/19/2011 09:02 PM, David Fetter wrote:
> On Fri, Aug 19, 2011 at 07:37:29PM -0400, Andrew Dunstan wrote:
>>
>> On 08/19/2011 07:31 PM, Bruce Momjian wrote:
>>> Kris Jurka wrote:
>>>> Did you also try the "use Tie::File" addition in my fix because your
>>>> current coding doesn't work at all.  The tie operation is key to actually
>>>> writing out the modified copyright notice.  Your version just updates the
>>>> copyright year in memory, but never gets it back to the file.
>>> Ah, that did fix it;  thanks.  I am attached the updated committed
>>> copyright.pl.  I also skipped symlinks.
>>>
>> It probably doesn't matter that much in this context, but I should
>> point out that Tie::File is not universally available. Some years
>> ago I had to revert its use in the buildfarm code for that reason.
>> In general we should try to avoid adding extra dependencies wherever
>> possible.
> Tie::File ships as part of core Perl for all non-EOL versions, so I
> really can't consider this as a problem.



Which are those?

cheers

andrew

On Fri, Aug 19, 2011 at 09:22:03PM -0400, Andrew Dunstan wrote:
> On 08/19/2011 09:02 PM, David Fetter wrote:
> >On Fri, Aug 19, 2011 at 07:37:29PM -0400, Andrew Dunstan wrote:
> >>
> >>On 08/19/2011 07:31 PM, Bruce Momjian wrote:
> >>>Kris Jurka wrote:
> >>>>Did you also try the "use Tie::File" addition in my fix because your
> >>>>current coding doesn't work at all.  The tie operation is key to actually
> >>>>writing out the modified copyright notice.  Your version just updates the
> >>>>copyright year in memory, but never gets it back to the file.
> >>>Ah, that did fix it;  thanks.  I am attached the updated committed
> >>>copyright.pl.  I also skipped symlinks.
> >>>
> >>It probably doesn't matter that much in this context, but I should
> >>point out that Tie::File is not universally available. Some years
> >>ago I had to revert its use in the buildfarm code for that reason.
> >>In general we should try to avoid adding extra dependencies
> >>wherever possible.
> >Tie::File ships as part of core Perl for all non-EOL versions, so I
> >really can't consider this as a problem.
> 
> Which are those?

5.12 and 5.14 are still supported.  5.10 and earlier are EOL.

http://news.perlfoundation.org/2011/05/perl-514.html

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter wrote:
> On Fri, Aug 19, 2011 at 09:22:03PM -0400, Andrew Dunstan wrote:
> > On 08/19/2011 09:02 PM, David Fetter wrote:
> > >On Fri, Aug 19, 2011 at 07:37:29PM -0400, Andrew Dunstan wrote:
> > >>
> > >>On 08/19/2011 07:31 PM, Bruce Momjian wrote:
> > >>>Kris Jurka wrote:
> > >>>>Did you also try the "use Tie::File" addition in my fix because your
> > >>>>current coding doesn't work at all.  The tie operation is key to actually
> > >>>>writing out the modified copyright notice.  Your version just updates the
> > >>>>copyright year in memory, but never gets it back to the file.
> > >>>Ah, that did fix it;  thanks.  I am attached the updated committed
> > >>>copyright.pl.  I also skipped symlinks.
> > >>>
> > >>It probably doesn't matter that much in this context, but I should
> > >>point out that Tie::File is not universally available. Some years
> > >>ago I had to revert its use in the buildfarm code for that reason.
> > >>In general we should try to avoid adding extra dependencies
> > >>wherever possible.
> > >Tie::File ships as part of core Perl for all non-EOL versions, so I
> > >really can't consider this as a problem.
> > 
> > Which are those?
> 
> 5.12 and 5.14 are still supported.  5.10 and earlier are EOL.
> 
> http://news.perlfoundation.org/2011/05/perl-514.html

Odd the Tie works in my Perl 5.10;  I wonder if I installed it somehow.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +

On 08/19/2011 09:39 PM, David Fetter wrote:
>>>
>>> Tie::File ships as part of core Perl for all non-EOL versions, so I
>>> really can't consider this as a problem.
>> Which are those?
> 5.12 and 5.14 are still supported.  5.10 and earlier are EOL.
>
> http://news.perlfoundation.org/2011/05/perl-514.html
>
>

Well, they need to get their story straight. 
<http://www.cpan.org/src/README.html> says:
   Please note that branches earlier than 5.8 are no longer supported,   though fixes for urgent issues, for example
severesecurity   problems, may still be issued.
 

What is more, ignoring 5.10 and older would simply foolish. Those 
versions are live on many of the platforms we build Postgres on, and in 
quite modern distributions too, for example in RHEL 6.

Anyway, they point seems moot in this context.

cheers

andrew

On Fri, Aug 19, 2011 at 09:49:35PM -0400, Bruce Momjian wrote:
> David Fetter wrote:
> > On Fri, Aug 19, 2011 at 09:22:03PM -0400, Andrew Dunstan wrote:
> > > On 08/19/2011 09:02 PM, David Fetter wrote:
> > > >On Fri, Aug 19, 2011 at 07:37:29PM -0400, Andrew Dunstan wrote:
> > > >>
> > > >>On 08/19/2011 07:31 PM, Bruce Momjian wrote:
> > > >>>Kris Jurka wrote:
> > > >>>>Did you also try the "use Tie::File" addition in my fix because your
> > > >>>>current coding doesn't work at all.  The tie operation is key to actually
> > > >>>>writing out the modified copyright notice.  Your version just updates the
> > > >>>>copyright year in memory, but never gets it back to the file.
> > > >>>Ah, that did fix it;  thanks.  I am attached the updated committed
> > > >>>copyright.pl.  I also skipped symlinks.
> > > >>>
> > > >>It probably doesn't matter that much in this context, but I should
> > > >>point out that Tie::File is not universally available. Some years
> > > >>ago I had to revert its use in the buildfarm code for that reason.
> > > >>In general we should try to avoid adding extra dependencies
> > > >>wherever possible.
> > > >Tie::File ships as part of core Perl for all non-EOL versions, so I
> > > >really can't consider this as a problem.
> > > 
> > > Which are those?
> > 
> > 5.12 and 5.14 are still supported.  5.10 and earlier are EOL.
> > 
> > http://news.perlfoundation.org/2011/05/perl-514.html
> 
> Odd the Tie works in my Perl 5.10;  I wonder if I installed it somehow.

Sorry for being unclear.  I didn't mean to imply that earlier versions
of Perl didn't ship with File::Tie.  Just that all supported versions
do.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate