Re: [DOCS] Viability of text HISTORY/INSTALL/regression README files (was Re: [COMMITTERS] pgsql: Document a few more regression test hazards.)

I wrote:
> Gavin Flower <GavinFlower@archidevsys.co.nz> writes:
>> How about adding URL's for the online versions of HISTORY & README's (or
>> their rough equivalents - perhaps the online version of the latest
>> 'Appendix E. Release Notes' would be sufficient?) to the INSTALL file?

> Actually, what I had in mind was to replace the dynamically-generated
> HISTORY and README files with small text files that contain those
> URL references.

Here's a proposed patch against HEAD for this.  It also gets rid of some
rather quaint instructions for using Netscape to construct these files ;-)

Barring objection, I'd like to update all the live branches this way
before the upcoming releases.  I'm tired of having to worry about
whether the release notes will build as plain text; but that worry
won't go away unless we nuke the text output in all the branches.

            regards, tom lane

diff --git a/GNUmakefile.in b/GNUmakefile.in
index 40ab280..a573880 100644
*** a/GNUmakefile.in
--- b/GNUmakefile.in
*************** distdir:
*** 103,112 ****
        fi || exit; \
      done
      $(MAKE) -C $(distdir) distprep
!     $(MAKE) -C $(distdir)/doc/src/sgml/ HISTORY INSTALL regress_README
!     cp $(distdir)/doc/src/sgml/HISTORY $(distdir)/
      cp $(distdir)/doc/src/sgml/INSTALL $(distdir)/
-     cp $(distdir)/doc/src/sgml/regress_README $(distdir)/src/test/regress/README
      $(MAKE) -C $(distdir) distclean
      rm -f $(distdir)/README.git

--- 103,110 ----
        fi || exit; \
      done
      $(MAKE) -C $(distdir) distprep
!     $(MAKE) -C $(distdir)/doc/src/sgml/ INSTALL
      cp $(distdir)/doc/src/sgml/INSTALL $(distdir)/
      $(MAKE) -C $(distdir) distclean
      rm -f $(distdir)/README.git

diff --git a/HISTORY b/HISTORY
index ...360c7f6 .
*** a/HISTORY
--- b/HISTORY
***************
*** 0 ****
--- 1,6 ----
+ Release notes for all versions of PostgreSQL can be found on-line at
+ http://www.postgresql.org/docs/devel/static/release.html
+
+ In a distribution file set, release notes for the current version can be
+ found prebuilt under doc/src/sgml/html/.  Visit the index.html file with
+ an HTML browser, then consult the "Release Notes" appendix.
diff --git a/README b/README
index 49d55af..302028e 100644
*** a/README
--- b/README
*************** See the file INSTALL for instructions on
*** 17,24 ****
  PostgreSQL.  That file also lists supported operating systems and
  hardware platforms and contains information regarding any other
  software packages that are required to build or run the PostgreSQL
! system.  Changes between all PostgreSQL releases are recorded in the
! file HISTORY.  Copyright and license information can be found in the
  file COPYRIGHT.  A comprehensive documentation set is included in this
  distribution; it can be read as described in the installation
  instructions.
--- 17,23 ----
  PostgreSQL.  That file also lists supported operating systems and
  hardware platforms and contains information regarding any other
  software packages that are required to build or run the PostgreSQL
! system.  Copyright and license information can be found in the
  file COPYRIGHT.  A comprehensive documentation set is included in this
  distribution; it can be read as described in the installation
  instructions.
diff --git a/README.git b/README.git
index d5378b4..0bf2b56 100644
*** a/README.git
--- b/README.git
***************
*** 1,12 ****
  (This file does not appear in release tarballs.)

! In a release or snapshot tarball of PostgreSQL, documentation files named
! INSTALL and HISTORY will appear in this directory.  However, these files are
! not stored in git and so will not be present if you are using a git checkout.
! If you are using git, you can view the most recent install instructions at:
      http://www.postgresql.org/docs/devel/static/installation.html
- and the current release notes at:
-     http://www.postgresql.org/docs/devel/static/release.html

  Users compiling from git will also need compatible versions of Bison, Flex,
  and Perl, as discussed in the install documentation.  These programs are not
--- 1,12 ----
  (This file does not appear in release tarballs.)

! In a release or snapshot tarball of PostgreSQL, a documentation file named
! INSTALL will appear in this directory.  However, this file is not stored in
! git and so will not be present if you are using a git checkout.
!
! If you are using a git checkout, you can view the most recent installation
! instructions at:
      http://www.postgresql.org/docs/devel/static/installation.html

  Users compiling from git will also need compatible versions of Bison, Flex,
  and Perl, as discussed in the install documentation.  These programs are not
diff --git a/doc/src/sgml/.gitignore b/doc/src/sgml/.gitignore
index e1b84b4..2f0329c 100644
*** a/doc/src/sgml/.gitignore
--- b/doc/src/sgml/.gitignore
***************
*** 6,14 ****
  /man7/
  /man-stamp
  # Other popular build targets
- /HISTORY
  /INSTALL
- /regress_README
  /postgres-US.pdf
  /postgres-A4.pdf
  /postgres.html
--- 6,12 ----
***************
*** 22,30 ****
  /HTML.index
  # Assorted byproducts from building the above
  /postgres.xml
- /HISTORY.html
  /INSTALL.html
- /regress_README.html
  /postgres-US.aux
  /postgres-US.log
  /postgres-US.out
--- 20,26 ----
diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile
index a6a6ef3..304b362 100644
*** a/doc/src/sgml/Makefile
--- b/doc/src/sgml/Makefile
*************** JADE.text = $(JADE) $(JADEFLAGS) $(SGMLI
*** 215,247 ****
  ICONV = iconv
  LYNX = lynx

! # The release notes may contain non-ASCII characters (for contributor
! # names), which lynx converts to the encoding determined by the
! # current locale.  The get output that is deterministic and easily
! # readable by everyone, we make lynx produce LATIN1 and then convert
! # that to ASCII with transliteration for the non-ASCII characters.
! # Official releases are currently built on FreeBSD, which has limited
  # locale support and is very picky about locale name spelling.  The
  # below has been finely tuned to run on FreeBSD and Linux/glibc.
! INSTALL HISTORY regress_README: % : %.html
      $(PERL) -p -e 's/<H(1|2)$$/<H\1 align=center/g' $< | LC_ALL=en_US.ISO8859-1 $(LYNX) -force_html -dump -nolist
-stdin| $(ICONV) -f latin1 -t us-ascii//TRANSLIT > $@ 

  INSTALL.html: standalone-install.sgml installation.sgml version.sgml
      $(JADE.text) -V nochunks standalone-install.sgml installation.sgml > $@

- HISTORY.html: generate_history.pl $(wildcard $(srcdir)/release*.sgml)
-     $(PERL) $< "$(srcdir)" release.sgml >tempfile_HISTORY.sgml
-     $(JADE.text) -V nochunks tempfile_HISTORY.sgml > $@
-     rm tempfile_HISTORY.sgml
-
- regress_README.html: regress.sgml
-     ( echo '<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook V4.2//EN" ['; \
-       echo '<!ENTITY % standalone-ignore "IGNORE">'; \
-       echo '<!ENTITY % standalone-include "INCLUDE"> ]>'; \
-       cat $< ) >tempfile_regress_README.sgml
-     $(JADE.text) -V nochunks tempfile_regress_README.sgml > $@
-     rm tempfile_regress_README.sgml
-

  ##
  ## XSLT processing
--- 215,234 ----
  ICONV = iconv
  LYNX = lynx

! # The documentation may contain non-ASCII characters (mostly for
! # contributor names), which lynx converts to the encoding determined
! # by the current locale.  To get text output that is deterministic and
! # easily readable by everyone, we make lynx produce LATIN1 and then
! # convert that to ASCII with transliteration for the non-ASCII characters.
! # Official releases were historically built on FreeBSD, which has limited
  # locale support and is very picky about locale name spelling.  The
  # below has been finely tuned to run on FreeBSD and Linux/glibc.
! INSTALL: % : %.html
      $(PERL) -p -e 's/<H(1|2)$$/<H\1 align=center/g' $< | LC_ALL=en_US.ISO8859-1 $(LYNX) -force_html -dump -nolist
-stdin| $(ICONV) -f latin1 -t us-ascii//TRANSLIT > $@ 

  INSTALL.html: standalone-install.sgml installation.sgml version.sgml
      $(JADE.text) -V nochunks standalone-install.sgml installation.sgml > $@


  ##
  ## XSLT processing
*************** check-tabs:
*** 393,405 ****
  # This allows removing some files from the distribution tarballs while
  # keeping the dependencies satisfied.
  .SECONDARY: postgres.xml $(GENERATED_SGML) HTML.index
! .SECONDARY: INSTALL.html HISTORY.html regress_README.html
  .SECONDARY: %-A4.tex-ps %-US.tex-ps %-A4.tex-pdf %-US.tex-pdf

  clean:
  # text --- these are shipped, but not in this directory
!     rm -f INSTALL HISTORY regress_README
!     rm -f INSTALL.html HISTORY.html regress_README.html
  # single-page output
      rm -f postgres.html postgres.txt
  # print
--- 380,392 ----
  # This allows removing some files from the distribution tarballs while
  # keeping the dependencies satisfied.
  .SECONDARY: postgres.xml $(GENERATED_SGML) HTML.index
! .SECONDARY: INSTALL.html
  .SECONDARY: %-A4.tex-ps %-US.tex-ps %-A4.tex-pdf %-US.tex-pdf

  clean:
  # text --- these are shipped, but not in this directory
!     rm -f INSTALL
!     rm -f INSTALL.html
  # single-page output
      rm -f postgres.html postgres.txt
  # print
diff --git a/doc/src/sgml/docguide.sgml b/doc/src/sgml/docguide.sgml
index c9c9862..3cc1d60 100644
*** a/doc/src/sgml/docguide.sgml
--- b/doc/src/sgml/docguide.sgml
*************** save_size.pdfjadetex = 15000
*** 942,967 ****
     <title>Plain Text Files</title>

     <para>
!     Several files are distributed as plain text, for reading during
!     the installation process. The <filename>INSTALL</filename> file
      corresponds to <xref linkend="installation">, with some minor
      changes to account for the different context.  To recreate the
      file, change to the directory <filename>doc/src/sgml</filename>
!     and enter <userinput>gmake INSTALL</userinput>.  This will create
!     a file <filename>INSTALL.html</filename> that can be saved as text
!     with <productname>Netscape Navigator</productname> and put into
!     the place of the existing file.
!     <productname>Netscape</productname> seems to offer the best
!     quality for <acronym>HTML</acronym> to text conversions (over
!     <application>lynx</application> and
!     <application>w3m</application>).
     </para>

     <para>
!     The file <filename>HISTORY</filename> can be created similarly,
!     using the command <userinput>gmake HISTORY</userinput>.  For the
!     file <filename>src/test/regress/README</filename> the command is
!     <userinput>gmake regress_README</userinput>.
     </para>
    </sect2>

--- 942,960 ----
     <title>Plain Text Files</title>

     <para>
!     The installation instructions are also distributed as plain text,
!     in case they are needed in a situation where better reading tools
!     are not available.  The <filename>INSTALL</filename> file
      corresponds to <xref linkend="installation">, with some minor
      changes to account for the different context.  To recreate the
      file, change to the directory <filename>doc/src/sgml</filename>
!     and enter <userinput>gmake INSTALL</userinput>.
     </para>

     <para>
!     In the past, the release notes and regression testing instructions
!     were also distributed as plain text, but this practice has been
!     discontinued.
     </para>
    </sect2>

diff --git a/doc/src/sgml/generate_history.pl b/doc/src/sgml/generate_history.pl
index 1d90c67..e69de29 100644
*** a/doc/src/sgml/generate_history.pl
--- b/doc/src/sgml/generate_history.pl
***************
*** 1,65 ****
- #! /usr/bin/perl -w
-
- # generate_history.pl -- flatten release notes for use as HISTORY file
- #
- # Usage: generate_history.pl srcdir release.sgml >output.sgml
- #
- # The main point of this script is to strip out <link> references, which
- # generally point into the rest of the documentation and so can't be used
- # in a standalone build of the release notes.  To make sure this is done
- # everywhere, we have to fold in the sub-files of the release notes.
- #
- # doc/src/sgml/generate_history.pl
-
- use strict;
-
- my $srcdir = shift;
- die "$0: missing required argument: srcdir\n" if !defined($srcdir);
- my $infile = shift;
- die "$0: missing required argument: inputfile\n" if !defined($infile);
-
- # Emit DOCTYPE header so that the output is a self-contained SGML document
- print "<!DOCTYPE appendix PUBLIC \"-//OASIS//DTD DocBook V4.2//EN\">\n";
-
- process_file($infile);
-
- exit 0;
-
- sub process_file
- {
-     my $filename = shift;
-
-     local *FILE;    # need a local filehandle so we can recurse
-
-     my $f = $srcdir . '/' . $filename;
-     open(FILE, $f) || die "could not read $f: $!\n";
-
-     while (<FILE>)
-     {
-
-         # Recursively expand sub-files of the release notes
-         if (m/^&(release-.*);$/)
-         {
-             process_file($1 . ".sgml");
-             next;
-         }
-
-         # Remove <link ...> tags, which might span multiple lines
-         while (m/<link/)
-         {
-             if (s/<link\s+linkend[^>]*>//)
-             {
-                 next;
-             }
-
-             # incomplete tag, so slurp another line
-             $_ .= <FILE>;
-         }
-
-         # Remove </link> too
-         s|</link>||g;
-
-         print;
-     }
-     close(FILE);
- }
--- 0 ----
diff --git a/doc/src/sgml/release.sgml b/doc/src/sgml/release.sgml
index 0980c6e..8cfd57d 100644
*** a/doc/src/sgml/release.sgml
--- b/doc/src/sgml/release.sgml
*************** non-ASCII characters            find usi
*** 34,42 ****

  wrap long lines

! For new features, add links to the documentation sections.  Use </link>
! not just </> so that generate_history.pl can remove it, so HISTORY.html
! can be created without links to the main documentation.  Don't use <xref>.

  -->

--- 34,40 ----

  wrap long lines

! For new features, add links to the documentation sections.

  -->

*************** can be created without links to the main
*** 71,77 ****

  <!--
    To add a new major-release series, add an entry here and in filelist.sgml.
-   Follow the naming convention, or you'll confuse generate_history.pl.

    The reason for splitting the release notes this way is so that appropriate
    subsets can easily be copied into back branches.
--- 69,74 ----
diff --git a/doc/src/sgml/standalone-install.sgml b/doc/src/sgml/standalone-install.sgml
index c94b397..1942f9d 100644
*** a/doc/src/sgml/standalone-install.sgml
--- b/doc/src/sgml/standalone-install.sgml
***************
*** 2,22 ****

  <!--
  This file helps in generating the INSTALL text file that lives in the
! top level directory of the distribution. The exact process is like
! this:
!
! 1. Paste together with installation.sgml
!
! 2. Process with jade to HTML (use -V nochunks)
!
! 3. Remove "Chapter 1" heading
!
! 4. Save as text file in Netscape
!
! 5. Put in place of old INSTALL file
!
! Running 'make INSTALL' in the doc/src/sgml directory will do 1 through
! 3 for you.
  -->

  <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [
--- 2,8 ----

  <!--
  This file helps in generating the INSTALL text file that lives in the
! top level directory of the distribution.
  -->

  <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [
diff --git a/src/test/regress/README b/src/test/regress/README
index ...0cbf3b6 .
*** a/src/test/regress/README
--- b/src/test/regress/README
***************
*** 0 ****
--- 1,3 ----
+ Documentation concerning how to run these regression tests and interpret
+ the results can be found in the PostgreSQL manual, in the chapter
+ "Regression Tests".
diff --git a/src/tools/RELEASE_CHANGES b/src/tools/RELEASE_CHANGES
index 321f37c..79a3e19 100644
*** a/src/tools/RELEASE_CHANGES
--- b/src/tools/RELEASE_CHANGES
*************** For All Releases (major, minor, beta, RC
*** 10,16 ****
      o update doc/src/sgml/release.sgml
      o run spellchecker on result
      o add SGML markup
-     o check if 'gmake HISTORY.html' works for <link>s

  * Update timezone data to match latest zic database and new
    Windows releases, if any (see src/timezone/README)
--- 10,15 ----