Обсуждение: pgsql: Transforms for jsonb to PL/Python
Transforms for jsonb to PL/Python Add a new contrib module jsonb_plpython that provide a transform between jsonb and PL/Python. jsonb values are converted to appropriate Python types such as dicts and lists, and vice versa. Author: Anthony Bykov <a.bykov@postgrespro.ru> Reviewed-by: Aleksander Alekseev <a.alekseev@postgrespro.ru> Reviewed-by: Nikita Glukhov <n.gluhov@postgrespro.ru> Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/3f44e3db72ad4097aae078c075a9b3cb3d6b761b Modified Files -------------- contrib/Makefile | 4 +- contrib/jsonb_plpython/.gitignore | 6 + contrib/jsonb_plpython/Makefile | 39 ++ contrib/jsonb_plpython/expected/jsonb_plpython.out | 347 ++++++++++++++++ contrib/jsonb_plpython/jsonb_plpython.c | 453 +++++++++++++++++++++ contrib/jsonb_plpython/jsonb_plpython2u--1.0.sql | 19 + contrib/jsonb_plpython/jsonb_plpython2u.control | 6 + contrib/jsonb_plpython/jsonb_plpython3u--1.0.sql | 19 + contrib/jsonb_plpython/jsonb_plpython3u.control | 6 + contrib/jsonb_plpython/jsonb_plpythonu--1.0.sql | 19 + contrib/jsonb_plpython/jsonb_plpythonu.control | 6 + contrib/jsonb_plpython/sql/jsonb_plpython.sql | 218 ++++++++++ doc/src/sgml/json.sgml | 15 + 13 files changed, 1155 insertions(+), 2 deletions(-)
On 3/28/18 08:39, Peter Eisentraut wrote: > Transforms for jsonb to PL/Python Needs fixes for Windows build. I can give it a blind attempt, but help would be welcome. I'm looking into the regression test failures that appear to be minor output differences. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes: > I'm looking into the regression test failures that appear to be minor > output differences. gaur (python 2.5) is showing a different output pathology: *** 246,253 **** return x $$; SELECT test1inf(); ! ERROR: could not convert value "inf" to jsonb ! CONTEXT: while creating return value PL/Python function "test1inf" -- test python -infinity -> jsonb CREATE FUNCTION test2inf() RETURNS jsonb --- 246,255 ---- return x $$; SELECT test1inf(); ! ERROR: ValueError: invalid literal for float(): inf ! CONTEXT: Traceback (most recent call last): ! PL/Python function "test1inf", line 2, in <module> ! x = float('inf') PL/Python function "test1inf" -- test python -infinity -> jsonb CREATE FUNCTION test2inf() RETURNS jsonb *************** https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gaur&dt=2018-03-31%2002%3A50%3A13 regards, tom lane
I wrote: > gaur (python 2.5) is showing a different output pathology: A bit of google research indicates that Python did not have reliable handling of infinity or NaN floats until 2.6; before that, you could get failures or platform-specific results from code like float('inf'). So this test ain't gonna work as-is on old versions. I see a few options: 1. Just remove the tests involving inf/nan. As far as I can see, there's no explicit handling of these cases in jsonb_plpython, so we're not really buying any test coverage of our own code anyway. 2. Maintain additional expected-output files. One problem with this is it's not clear how many we'd need. 3. Somehow tweak the test cases in a different way, so that they're not exercised before 2.6. 4. Move our minimum supported Python version up to 2.6 (from 2.4 where it is now). #4 seems like an overreaction at first sight, but on the other hand, I wonder how many people really care about pre-2.6 anymore anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only reason I'm running 2.5 on gaur/pademelon is we claim to support it. regards, tom lane
Hi, On Sun, 2018-04-01 at 18:07 -0400, Tom Lane wrote: > 4. Move our minimum supported Python version up to 2.6 (from 2.4 > where it is now). > > #4 seems like an overreaction at first sight, but on the other > hand, I wonder how many people really care about pre-2.6 anymore > anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only > reason I'm running 2.5 on gaur/pademelon is we claim to support it. FWIW, 2.6 is also the version that RHEL 6 has, so we still can support it. Regards, -- Devrim Gündüz EnterpriseDB: https://www.enterprisedb.com PostgreSQL Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes: > On Sun, 2018-04-01 at 18:07 -0400, Tom Lane wrote: >> 4. Move our minimum supported Python version up to 2.6 (from 2.4 >> where it is now). >> >> #4 seems like an overreaction at first sight, but on the other >> hand, I wonder how many people really care about pre-2.6 anymore >> anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only >> reason I'm running 2.5 on gaur/pademelon is we claim to support it. > FWIW, 2.6 is also the version that RHEL 6 has, so we still can support it. I poked around in the buildfarm logs to see what versions we're actually testing. It's hard to tell in some cases. (I notice that configure does actually extract python_version, but doesn't display it ... should we make it do so?) However, it's clear from version numbers embedded in places like library paths that we are testing with python 2.4.x on aholehole, arapaima, and prairiedog, and with 2.5.x on gaur/pademelon and locust. aholehole and arapaima are evidently using the platform-supplied version on RHEL 5, and locust the platform-supplied version on macOS 10.5.8. So while those are surely trailing-edge platforms, I'm no longer thinking that we can realistically claim there's zero remaining real-world interest in these Python versions, and so option #4 doesn't seem workable. I think option #1 (drop the tests with inf/nan) is the most attractive of the remaining choices. It's surely the least work, and AFAICS we aren't actually buying any code coverage with these test cases. (It is interesting though that we have four other pre-2.6 platforms that are passing these tests; apparently the pre-2.6 behavior wasn't broken everywhere.) Unless someone volunteers to undertake one of the other options, I'll go remove these tests shortly. regards, tom lane
Hi, On Mon, 2018-04-02 at 10:59 -0400, Tom Lane wrote: > aholehole and arapaima are evidently using the platform-supplied version > on RHEL 5, and locust the platform-supplied version on macOS 10.5.8. Yeah, I'm running arapaima and aholehole, they have stock RHEL 5 packages (Python 2.4) > So while those are surely trailing-edge platforms, I'm no longer thinking > that we can realistically claim there's zero remaining real-world interest > in these Python versions, and so option #4 doesn't seem workable. RHEL 5 is already EOL, last supported (community) RPM release was 9.6, so it won't hurt to drop Python 2.4 support from RPM point of view. Regards, -- Devrim Gündüz EnterpriseDB: https://www.enterprisedb.com PostgreSQL Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
On 4/1/18 18:07, Tom Lane wrote: > 1. Just remove the tests involving inf/nan. As far as I can see, > there's no explicit handling of these cases in jsonb_plpython, > so we're not really buying any test coverage of our own code anyway. That makes sense. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes: > On 4/1/18 18:07, Tom Lane wrote: >> 1. Just remove the tests involving inf/nan. As far as I can see, >> there's no explicit handling of these cases in jsonb_plpython, >> so we're not really buying any test coverage of our own code anyway. > That makes sense. OK, done. regards, tom lane