Solving the problems with wheel packages
От | Daniele Varrazzo |
---|---|
Тема | Solving the problems with wheel packages |
Дата | |
Msg-id | CA+mi_8bd6kJHLTGkuyHSnqcgDrJ1uHgQWvXCKQFD3tPQBUa2Bw@mail.gmail.com обсуждение исходный текст |
Ответы |
Re: Solving the problems with wheel packages
(Karsten Hilbert <Karsten.Hilbert@gmx.net>)
|
Список | psycopg |
Hello, since psycopg version 2.7, together with the classic source distribution, we have released psycopg as a wheel binary package on PyPI [1]. This allows people to install psycopg using: pip install psycopg2 without the usual build (Python and Postgres header files, pg_config) and runtime (libpq) requirements. The resulting package largely works, but it has a problem: among libpq dependencies there is libssl, which is included in the wheel package. But the system libssl version may also be imported for other reason by the Python process (e.g. using 'urllib' to open an https resource). When this happens, in concurrent situations, the process may segfault. I don't think the situation is easily fixable: the libssl ABI is not stable so it cannot be excluded from the wheel package. Unfortunately the libpq calls the libssl initialisation function at every connection (unnecessarily, AFAICS), and such function is non-reentrant; the call is guarded by a lock, which probably doesn't guard from the concurrent initialisation of the system libssl... or some mess like that. While the conditions for the segfault are relatively rare, the situation of a psycopg package unstable in combination with part of the standard library makes me less than happy. On the other hand I'm reluctant to throw the baby out with the bathwater: the binary packages have proven quite handy for a number of users. What I think is that the wheel packages should really be an opt-in feature, but AFAIK there is no such possibility [3]. In order to make up one, my idea is to release a new psycopg package called 'psycopg2-binary', and leave the basic psycopg2 package as source only ('psycopg2-binary' is only the name of the package on PyPI: the name of the Python package is still 'psycopg2'). My plan for the next releases is then: - release psycopg 2.7.4 as a normal 2.7.x, both as source and binary packages; - also release a package called psycopg2-binary version 2.7.4; - importing the wheel package installed from 'pip install psycopg2' will result in a warning: The psycopg2 wheel package will be renamed from release 2.8; to keep using the binary package please install 'psycopg2-binary' instead. - importing the wheel package installed from 'pip install psycopg2-binary' will cause no such warning; - starting from release 2.8, the binary packages will only be released under psycopg2-binary. This way, who prefers to keep on using the binary package may change their requirements to install 'psycopg2-binary'; who will opt to build from source can keep on doing as documented (e.g. installing with '--no-binary'). Until the release of psycopg 2.8 everything will keep on working as expected, but in order to silence the warning it will be necessary to opt in or out of the binary package; after the release 2.8 'pip install psycopg2' will build from source again. I have already updated the build system to create the packages as described above [4], and I have uploaded them on testpipy [5,6], so they can be obtained using: pip install --pre -i https://testpypi.python.org/simple psycopg2 pip install --pre -i https://testpypi.python.org/simple psycopg2-binary I would be happy to have your feedback. Thank you very much. -- Daniele [1] https://pypi.python.org/pypi/psycopg2 [2] https://github.com/psycopg/psycopg2/issues/543 [3] https://github.com/pypa/wheel/issues/215 [4] https://github.com/psycopg/psycopg2-wheels/commit/cf03a558 [5] https://testpypi.python.org/pypi/psycopg2 [6] https://testpypi.python.org/pypi/psycopg2-binary
В списке psycopg по дате отправления: