Re: Updating the intro for packages - improve usability, reduce new user confusion

Hello, Craig.

You wrote:

CR> TL;DR: It's time to update the docs to make package-based and
CR> installer-based PostgreSQL the assumed environment our users are
CR> working with. Or at least put it on an equal footing as a first-class
CR> citizen, not relegate it to the dark corners of notes and appendices.
CR> The same is true for client drivers.

CR> I'll put my hand up to start the work, with a gradual series of small
CR> incremental patches, but only if I can get broad agreement here that
CR> it's the right thing to do first. Otherwise it's just wasting time.

I like your proposal. You have my +1.

CR> Why?

CR> I see a lot of confusion on Stack Overflow, chat groups, forums etc
CR> that all stems from a common cause:

CR> The PostgreSQL manual, including the intro/tutorial, is written with
CR> the assumption you installed from source and are using pg_ctl, etc
CR> directly. Very few of our users do this.

CR> The reality is that most users are immediately faced with:

CR>     psql: FATAL:  role "myname" does not exist

CR> as their first PostgreSQL experience, or

CR> psql: could not connect to server: No such file or directory
CR> Is the server running locally and accepting
CR> connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

CR> I think it's time to admit that most users first experience PostgreSQL
CR> via packages or installers, and moreover that we should encourage
CR> that. Joe "Never Used Linux Before" Bloggs and Julie "Is Terminal
CR> where I put the commands?" Smith should not be dealing with configure,
CR> make, macports, etc.

CR> We shouldn't leave them in the dark to figure out they want "systemctl
CR> enable --now postgresql-11.service" and that they need to
CR> "/usr/pgsql-10/bin/postgresql-10-setup" first. Or are they on Debian?
CR> Oh, then they need `pg_ctlcluster`, and the cluster is autocreated. We
CR> let the poor confused user sort this mess out.

CR> We, PGDG, ship these packages. We shouldn't entirely ignore them in
CR> our own documentation. That sucks for users. PostgreSQL has a
CR> reputation as a great DBMS that's hard to get started with. That
CR> reputation is deserved. Some trivial changes can ease that learning
CR> curve a lot.

CR> So of course we get people installing Pg from packages, then
CR> initdb'ing and pg_ctl'ing their own clusters instead of using
CR> pg_createcluster or working with systemctl. Sigh. Then they complain
CR> that postgres "crashed" when they restarted, because of course they
CR> didn't integrate it with their init system. How would they? We didn't
CR> tell them how, or that they had to. It's such a mess and I have better
CR> things to do than deal with these unnecessary horrors when I work with
CR> clients on real problems.

CR> Look at what some popular systems do. Slick docs that get any idiot
CR> straight to running the system, even if perhaps they shouldn't. I
CR> don't want to go that far, with the sort of

CR>    curl https://get.postgresql.org/11 | sudo bash

CR> horror that I see many places. But we can do better than we do now
CR> without encouraging hideously insecure and unmaintainable deployments.

CR> (It's bad enough that we nearly totally ignore the client drivers
CR> everyone actually uses - come on, how many of you write direct libpq
CR> code on a day to day basis? Good luck finding PgJDBC, nPgSQL,
CR> psycopg2, DBD::Pg or the Pg gem in our manual; hell, they most don't
CR> even appear to be mentioned in the SOURCE TREE anywhere. psqlODBC is
CR> mentioned - once - in "external projects".).

CR> So I propose to restructure our intro docs to emphasise common package
CR> based deployments, with a focus on the entrypoints at:

CR> * https://www.postgresql.org/docs/current/tutorial-start.html
CR> * https://www.postgresql.org/docs/current/tutorial-install.html
CR> * https://www.postgresql.org/docs/current/installation.html

CR> I realise we cannot cover everything, but we should:

CR> * Move "installation from source code" from "installation.html" to a subhead
CR> * Instead have the "installation" page list:

CR>   - Installation from Packages on Linux
CR>     - Fedora, RHEL, CentOS (mention PGDG and distro packages)
CR>     - Debian and Ubuntu (mention PGDG and distro packages)
CR>     - Other options (-> www.postgresql.org download page)
CR>   - Installation with an installer program
CR>     - (-> download page)
CR>    - Installation from Ports on *BSD
CR>    - Installation from source distribution
CR>      - git clone (lets face it, if you use source you use git now)
CR>      - tarball

CR> Keep a section of the manual dedicated to common distribution and
CR> install methods, where we can collect and maintain this info so it's
CR> not scattered. Cross-reference it from elsewhere. So for example under

CR> https://www.postgresql.org/docs/current/server-start.html

CR> we'd xref the distro/OS specific part, and one of those dist/OS
CR> specific parts would cover source installs. Right now when you read
CR> the docs, it's all about the source install, with distro/packaged
CR> stuff a half-considered afterthought mentioned after the manual stuff.
CR> A user should be able to easily open the "Fedora/RHEL/CentOS" section
CR> to get taken to the "Starting PostgreSQL on RHEL, CentOS or Fedora"
CR> section of the distro-specific info in the manual. With, of course, a
CR> note there saying that this applies if you installed from packages,
CR> and xrefs for manual source installs and installer-binary installs.

CR> We should have prominent links in the tutorial install section that link to:

CR> * Linux: apt.postgresql.org and yum.postgresql.org
CR> * the postgresql download page
CR> * https://help.ubuntu.com/community/PostgreSQL for package-related
CR> info on the Debian and Ubuntu variants
CR> * https://fedoraproject.org/wiki/PostgreSQL for Fedora/RHEL/CentOS
CR> package-related info

CR> These are long lived, stable, well known links, and we're long past
CR> the days where someone's reading the docs on a printout. Hell, most
CR> people don't even realise we have docs packages rather than just the
CR> website!

CR> We should also have a short, simple "concepts" section in the tutorial
CR> (or tweak the architecture fundamentals section) that explains a few
CR> key points of frequent confusion:

CR> * Explicitly mention that on most package installs you use `sudo -u
CR> postgres psql`  to get a "postgres" superuser shell. Give the Mac and
CR> Windows equivalents too.
CR> * Explain that PostgreSQL is a client/server system. The psql client
CR> connects to a server, either on the same computer or elsewhere via a
CR> network. The server is "postgres", the usual client is "psql" but
CR> there are others.
CR> * Explain that whenever we write "postgres=#" we mean "run this as a
CR> postgres superuser"
CR> * The "psql" shell is a separate program to the "sh" or "bash" shell.
CR> It takes SQL commands and special backslash-commands. You cannot run
CR> SQL on the system shell, and you cannot run system commands on the
CR> psql shell.
CR> * There are GUIs like PgAdmin4, OmniDB, HeidiSQL, DBeaver, etc that
CR> some people will prefer. These tools may NOT support the psql
CR> backslash commands the user manual describes. The manual is written
CR> with the assumption that you use psql. In particular \COPY will NOT
CR> work on many of them.
CR> * The "createuser", "createdb" etc commands are command-line ("Command
CR> Prompt", Terminal, etc) commands, NOT psql commands. They have sql
CR> equivalents like CREATE USER, CREATE DATABASE, etc.
CR> * Give a simple recipe for how to make your own user account with your
CR> own db and set up peer auth for it, so you can get started simply,
CR> like

CR> sudo -u postgres psql -c "CREATE USER ${USER} WITH CREATEDB
CR> CREATEROLE; CREATE DATABASE $user WITH OWNER ${USER};"

CR> It's a shame we don't have an ALTER SYSTEM equivalent for pg_hba.conf,
CR> but most packages ship with 'peer' enabled, so it should Just Work.


CR> Anyway. I know that was long, but I wanted to actually explain WHAT I
CR> want to change in the docs and HOW, not just "lets change some stuff
CR> ok?".

CR> I'll have extremely limited time to do this, but I hope to deliver a
CR> stream of small and simple patches to do it piece-by-piece.




-- 
With best wishes,
 Pavel                          mailto:pavel@gf.microolap.com