Обсуждение: Containerized development environment for pgweb
Hello pgweb team, Attaching a patch to simplify pgweb dev env setup. Dockerfile - Main container with Django and PostgreSQL client docker-compose.yml - to create proper dependency between python and postgres container settings_local.py - Development setting, mounted inside container to override pgweb/settings.py django_setup_local.sh - this script is executed at start of container and to setup pgweb specific env Updated README.rst with commands. Tested on Fedora 40 with podman as rootless containers. No custom setup was required. start - `podman compose --file docker-compose.yml up --detach` stop - `podman compose --file docker-compose.yml down` -- Kind Regards, Yogesh Sharma PostgreSQL, Linux, and Networking Expert Open Source Enthusiast and Advocate PostgreSQL Contributors Team @ RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
Вложения
On 6/21/24 9:08 AM, Yogesh Sharma wrote: > Hello pgweb team, > > Attaching a patch to simplify pgweb dev env setup. > > Dockerfile - Main container with Django and PostgreSQL client > docker-compose.yml - to create proper dependency between python and > postgres container > settings_local.py - Development setting, mounted inside container to > override pgweb/settings.py > django_setup_local.sh - this script is executed at start of container > and to setup pgweb specific env > > Updated README.rst with commands. > > Tested on Fedora 40 with podman as rootless containers. No custom setup > was required. > start - `podman compose --file docker-compose.yml up --detach` > stop - `podman compose --file docker-compose.yml down` This generally looks good to me. A few comments: * Authentication - are we able to use "peer" auth instead of requiring a password? Though IIRC that may be challenging if we're using composable containers. * Otherwise, do we have a way to dynamically/securely generate the password? Thanks, Jonathan
Вложения
On 2024-06-21 23:08, Yogesh Sharma wrote: <snip> > docker-compose.yml - to create proper dependency between python and > postgres container This one's using PG 16: image: postgres:16 Does anyone know if the PG infrastructure uses PG 16 too? Asking because it'd make sense for them to match, just in case. :) And if it doesn't matter what version runs, then using the auto-upgrade container might be worth considering: https://hub.docker.com/repository/docker/pgautoupgrade/pgautoupgrade --- Bear in mind that I'm not personally involved in the PostgreSQL WWW side of things, so none of the above should be seen as anything more than just ideas. :) Regards and best wishes, Justin Clift
Hi
On Wed, 26 Jun 2024 at 09:07, Justin Clift <justin@postgresql.org> wrote:
On 2024-06-21 23:08, Yogesh Sharma wrote:
<snip>
> docker-compose.yml - to create proper dependency between python and
> postgres container
This one's using PG 16:
image: postgres:16
Does anyone know if the PG infrastructure uses PG 16 too?
15 at present.
Asking because it'd make sense for them to match, just in case. :)
And if it doesn't matter what version runs, then using the
auto-upgrade container might be worth considering:
https://hub.docker.com/repository/docker/pgautoupgrade/pgautoupgrade
---
Bear in mind that I'm not personally involved in the PostgreSQL WWW
side of things, so none of the above should be seen as anything more
than just ideas. :)
Regards and best wishes,
Justin Clift
Dave Page
pgAdmin: https://www.pgadmin.org
On Wed, Jun 26, 2024 at 10:07 AM Justin Clift <justin@postgresql.org> wrote:
On 2024-06-21 23:08, Yogesh Sharma wrote:
<snip>
> docker-compose.yml - to create proper dependency between python and
> postgres container
This one's using PG 16:
image: postgres:16
Does anyone know if the PG infrastructure uses PG 16 too?
Asking because it'd make sense for them to match, just in case. :)
It also is on python 3.7 which should be 3.9 to match what's deployed. In general, I'd recommend starting off the debian bullseye container instead of the python specific one -- that's what's used in the production deployments, so it makes sense to keep to the same. That way it could also be using the same versions of the debian packages for all requirements (basically everything except Django itself), to ensure it's exactly the same.
//Magnus
On 2024-06-26 02:21, Jonathan S. Katz wrote: <snip> > * Otherwise, do we have a way to dynamically/securely generate the > password? Maybe having something call out to pwgen (or similar) would be feasible? Regards and best wishes, Justin Clift
On 6/26/24 04:23, Magnus Hagander wrote: > On Wed, Jun 26, 2024 at 10:07 AM Justin Clift <justin@postgresql.org> > wrote: > > On 2024-06-21 23:08, Yogesh Sharma wrote: > <snip> > > docker-compose.yml - to create proper dependency between python and > > postgres container > > This one's using PG 16: > > image: postgres:16 > > Does anyone know if the PG infrastructure uses PG 16 too? > > Asking because it'd make sense for them to match, just in case. :) > > > It also is on python 3.7 which should be 3.9 to match what's deployed. > In general, I'd recommend starting off the debian bullseye container > instead of the python specific one -- that's what's used in the > production deployments, so it makes sense to keep to the same. > That way it could also be using the same versions of the debian > packages for all requirements (basically everything except Django > itself), to ensure it's exactly the same. > > //Magnus Which Debian release is used by PG infrastructure? -- Kind Regards, Yogesh Sharma PostgreSQL, Linux, and Networking Expert Open Source Enthusiast and Advocate PostgreSQL Contributors Team @ RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
On Thu, Jun 27, 2024 at 2:25 PM Yogesh Sharma <yogesh.sharma@catprosystems.com> wrote:
On 6/26/24 04:23, Magnus Hagander wrote:
> On Wed, Jun 26, 2024 at 10:07 AM Justin Clift <justin@postgresql.org>
> wrote:
>
> On 2024-06-21 23:08, Yogesh Sharma wrote:
> <snip>
> > docker-compose.yml - to create proper dependency between python and
> > postgres container
>
> This one's using PG 16:
>
> image: postgres:16
>
> Does anyone know if the PG infrastructure uses PG 16 too?
>
> Asking because it'd make sense for them to match, just in case. :)
>
>
> It also is on python 3.7 which should be 3.9 to match what's deployed.
> In general, I'd recommend starting off the debian bullseye container
> instead of the python specific one -- that's what's used in the
> production deployments, so it makes sense to keep to the same.
> That way it could also be using the same versions of the debian
> packages for all requirements (basically everything except Django
> itself), to ensure it's exactly the same.
>
> //Magnus
Which Debian release is used by PG infrastructure?
Currently, bullseye.
On 6/27/24 09:34, Magnus Hagander wrote: > > > It also is on python 3.7 which should be 3.9 to match what's > deployed. > > In general, I'd recommend starting off the debian bullseye > container > > instead of the python specific one -- that's what's used in the > > production deployments, so it makes sense to keep to the same. > > That way it could also be using the same versions of the debian > > packages for all requirements (basically everything except Django > > itself), to ensure it's exactly the same. > > > > //Magnus > > Which Debian release is used by PG infrastructure? > > > Currently, bullseye. When using python 3.9 PyYAML 3.13 (requirements.txt) fails to compile. --- ext/_yaml.c: In function ‘__Pyx_modinit_type_init_code’: ext/_yaml.c:25698:25: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’ 25698 | __pyx_type_5_yaml_Mark.tp_print = 0; | ^ ext/_yaml.c:25718:28: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’ 25718 | __pyx_type_5_yaml_CParser.tp_print = 0; | ^ ext/_yaml.c:25732:29: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’ 25732 | __pyx_type_5_yaml_CEmitter.tp_print = 0; | ^ --- I assume, PyYAML was updated on system but not requirements.txt. Possible to verify following package versions from the PG Infra server? psycopg2==2.8.5 pycryptodomex>=3.4.7,<3.5 Markdown==3.0.1 requests-oauthlib==1.0.0 cvss==2.2 pytidylib==0.3.2 pycodestyle==2.4.0 pynliner==0.8.0 Babel==2.6.0 bleach==3.1.4 PyYAML==3.13 -- Kind Regards, Yogesh Sharma PostgreSQL, Linux, and Networking Expert Open Source Enthusiast and Advocate PostgreSQL Contributors Team @ RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
On Thu, Jun 27, 2024 at 5:26 PM Yogesh Sharma <yogesh.sharma@catprosystems.com> wrote:
On 6/27/24 09:34, Magnus Hagander wrote:
>
> > It also is on python 3.7 which should be 3.9 to match what's
> deployed.
> > In general, I'd recommend starting off the debian bullseye
> container
> > instead of the python specific one -- that's what's used in the
> > production deployments, so it makes sense to keep to the same.
> > That way it could also be using the same versions of the debian
> > packages for all requirements (basically everything except Django
> > itself), to ensure it's exactly the same.
> >
> > //Magnus
>
> Which Debian release is used by PG infrastructure?
>
>
> Currently, bullseye.
When using python 3.9 PyYAML 3.13 (requirements.txt) fails to compile.
---
ext/_yaml.c: In function ‘__Pyx_modinit_type_init_code’:
ext/_yaml.c:25698:25: error: ‘PyTypeObject’ {aka ‘struct
_typeobject’} has no member named ‘tp_print’
25698 | __pyx_type_5_yaml_Mark.tp_print = 0;
| ^
ext/_yaml.c:25718:28: error: ‘PyTypeObject’ {aka ‘struct
_typeobject’} has no member named ‘tp_print’
25718 | __pyx_type_5_yaml_CParser.tp_print = 0;
| ^
ext/_yaml.c:25732:29: error: ‘PyTypeObject’ {aka ‘struct
_typeobject’} has no member named ‘tp_print’
25732 | __pyx_type_5_yaml_CEmitter.tp_print = 0;
| ^
---
I assume, PyYAML was updated on system but not requirements.txt.
Possible to verify following package versions from the PG Infra server?
psycopg2==2.8.5
pycryptodomex>=3.4.7,<3.5
Markdown==3.0.1
requests-oauthlib==1.0.0
cvss==2.2
pytidylib==0.3.2
pycodestyle==2.4.0
pynliner==0.8.0
Babel==2.6.0
bleach==3.1.4
PyYAML==3.13
We get all dependencies from Debian, not from pip, when possible. And we use the standard versions on debian bullseye, so as long as you haven't configured backports or unstable or something like that, you should get the correct one automatically. The only things we get from pip is django itself and asgiref.
//Magnus