Обсуждение: Re: [pgjdbc] Implement JDBC specs via pre-processor step (#435)

I have serious doubts about preprocessing and the preprocessing tool
maturity -- and I strongly feel the pgjdbc goes the wrong non-java way.

Was other possibilities for preprocessing considered?  I've talked to java
guys (I'm not one, I just work on packaging pgjdbc plugin), but all the
guys told me that this is generally suboptimal java approach.  Usually,
when C developer wants to use preprocessor, java guy should go the
sub-classing way.

From the latest *7 release, there has been added lot more preprocessor
constructs.

So is this really wanted?  I haven't had time to study the background
carefully.

Pavel

On Friday 27 of November 2015 06:41:16 Dave Cramer wrote:
> That's awesome. Very minimal preprocessing then.
>
> Dave Cramer
>
> On 27 November 2015 at 09:39, Vladimir Sitnikov <notifications@github.com>
> wrote:
>
> > I'm a bit surprised, however I had to do *no*
> > modifications/pre-processing to support jdk6.
> > So, the only pre-processing required so far is commenting usage of SQLType
> > for java8.
> >
> > —
> > Reply to this email directly or view it on GitHub
> > <https://github.com/pgjdbc/pgjdbc/pull/435#issuecomment-160152281>.
> >
>
>
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/pgjdbc/pgjdbc/pull/435#issuecomment-160152501

> Was other possibilities for preprocessing considered?

Other possibilities were explored and it was agreed that preprocessing
was the least harmful one.

> I just work on packaging pgjdbc plugin

Are you running into some issue?

Vladimir

Hello Vladimir,

On Thursday 07 of January 2016 18:33:19 Vladimir Sitnikov wrote:
> > Was other possibilities for preprocessing considered?
>
> Other possibilities were explored and it was agreed that preprocessing
> was the least harmful one.

have you links to those discussions/threads?

> > I just work on packaging pgjdbc plugin
>
> Are you running into some issue?

Yes, as we disccussed recently [1], we used to provide postgresql jdbc
plugin for years, and those changes are adding new dependencies at least,
which are not standard (because those are not available in distributions).

The issue could be maintenance, we need to keep an eye on bigger set of
packages, etc.  It might be needed, or might not - I haven't decided
myself yet.  But why others java projects don't need pre-processing?  Java
is not new language, so I'm asking what makes pgjdbc to be different
project compared to other serious/long-existing/very-stable/mature Java
projects.

[1] http://www.postgresql.org/message-id/378969680.26311404.1449041058684.JavaMail.zimbra@redhat.com

Pavel

On Thursday 07 of January 2016 11:13:54 Dave Cramer wrote:
> Hi Pavel,
>
> Other java projects to not attempt to support 3 versions of an API with one
> piece of code.
>
> FYI, the PostgreSQL packaging team was able to package rpms for JDBC.
>
> See
> http://postgresql.nabble.com/Are-pgrpm-changes-for-JDBC-discussed-here-before-submission-td5879553.html

Ah, I missed thanks!  I'll look at it.

Pavel

Pavel, summary is here:
http://www.postgresql.org/message-id/565750BE.9090804@lawinegevaar.nl

Other threads are: https://github.com/pgjdbc/pgjdbc/pull/435 and
http://www.postgresql.org/message-id/flat/000c01d0a7b6$97571ee0$c6055ca0$@eu#000c01d0a7b6$97571ee0$c6055ca0$@eu

There was a point by Mark on using delegation,
http://www.postgresql.org/message-id/d3d1281d44e89ed904b245293a85e765@imap.procolix.com,
however it solves "accept java.time in setObject(Object)" problem
only.
The other question on "how to implement
statement.setObject(...java.sql.SQLType)" is harder to solve by mere
delegation".

Vladimir

On Thursday 07 of January 2016 17:30:01 Pavel Raiskup wrote:
> On Thursday 07 of January 2016 11:13:54 Dave Cramer wrote:
> > Hi Pavel,
> >
> > Other java projects to not attempt to support 3 versions of an API with one
> > piece of code.
> >
> > FYI, the PostgreSQL packaging team was able to package rpms for JDBC.
> >
> > See
> > http://postgresql.nabble.com/Are-pgrpm-changes-for-JDBC-discussed-here-before-submission-td5879553.html
>
> Ah, I missed thanks!  I'll look at it.

Hmms:

  + /usr/bin/rm -f pgjdbc-REL9.4.1207/.gitignore
  + /usr/bin/rm -f pgjdbc-REL9.4.1207/.travis.yml
  + find -name '*.jar' -or -name '*.class'
  + xargs /usr/bin/rm -f
  + exit 0
  Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.MYDZRI
  + umask 022
  + cd /home/praiskup/rh/projects/pgrpms/rpm/redhat/9.5/postgresql-jdbc/F-23
  + cd pgjdbc-REL9.4.1207
  + export CLASSPATH=
  + CLASSPATH=
  + mvn -DskipTests -P release-artifacts clean package
  [INFO] Scanning for projects...
  Downloading: https://repo.maven.apache.org/maven2/org/apache/apache/10/apache-10.pom
  Downloaded: https://repo.maven.apache.org/maven2/org/apache/apache/10/apache-10.pom (15 KB at 36.9 KB/sec)
  Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/2.0.6/maven-artifact-2.0.6.pom
  Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/2.0.6/maven-artifact-2.0.6.pom (2 KB
at24.8 KB/sec) 
  Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven/2.0.6/maven-2.0.6.pom
  Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven/2.0.6/maven-2.0.6.pom (9 KB at 140.2 KB/sec)
  Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model/2.0.6/maven-model-2.0.6.pom
  Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model/2.0.6/maven-model-2.0.6.pom (3 KB at
46.5KB/sec) 
  Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/2.0.6/maven-core-2.0.6.pom
  Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/2.0.6/maven-core-2.0.6.pom (7 KB at
107.4KB/sec) 
  ....

This is wrong :/ basically equivalent to downloading pre-compiled '*.jar',
or what am I missing here? (IOW: we don't see what code we use).

Pavel

On Thursday 07 of January 2016 11:13:54 Dave Cramer wrote:
> Other java projects to not attempt to support 3 versions of an API with
> one piece of code.

Is that really that rare?  Or those do split the code into many files?
That sounds weird.

Pavel

> basically equivalent to downloading pre-compiled '*.jar',

It is equivalent to downloading a *compiler*. You need to have a
compiler in order to build a tool, don't you?

Vladimir

maybe, but right after pom files are downloaded jars. Aren't they?

----- Original Message -----
From: "Vladimir Sitnikov" <sitnikov.vladimir@gmail.com>
To: "Pavel Raiskup" <praiskup@redhat.com>
Cc: "Dave Cramer" <pg@fastcrypt.com>, "List" <pgsql-jdbc@postgresql.org>, "Pavel Kajaba" <pkajaba@redhat.com>, "Dave
Cramer"<notifications@github.com>, "Devrim GÜNDÜZ" <devrim@gunduz.org> 
Sent: Thursday, January 7, 2016 5:54:04 PM
Subject: Re: [JDBC] [pgjdbc] Implement JDBC specs via pre-processor step (#435)

> basically equivalent to downloading pre-compiled '*.jar',

It is equivalent to downloading a *compiler*. You need to have a
compiler in order to build a tool, don't you?

Vladimir


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

On Thursday 07 of January 2016 11:58:29 Pavel Kajaba wrote:
> maybe, but right after pom files are downloaded jars. Aren't they?

Ah, I've killed the build process on my box as soon as I noticed it
downloads something from Internet.  That is not acceptable in FOSS
packaging for distributions -- there can be dependencies, but those must
be built from source, too.  The tarball should be self-standing with
requirements installed on the system, without even trying to connect to
Internet.

Pavel



> ----- Original Message -----
> From: "Vladimir Sitnikov" <sitnikov.vladimir@gmail.com>
> To: "Pavel Raiskup" <praiskup@redhat.com>
> Cc: "Dave Cramer" <pg@fastcrypt.com>, "List" <pgsql-jdbc@postgresql.org>, "Pavel Kajaba" <pkajaba@redhat.com>, "Dave
Cramer"<notifications@github.com>, "Devrim GÜNDÜZ" <devrim@gunduz.org> 
> Sent: Thursday, January 7, 2016 5:54:04 PM
> Subject: Re: [JDBC] [pgjdbc] Implement JDBC specs via pre-processor step (#435)
>
> > basically equivalent to downloading pre-compiled '*.jar',
>
> It is equivalent to downloading a *compiler*. You need to have a
> compiler in order to build a tool, don't you?
>
> Vladimir
>
>
>

On Thursday 07 of January 2016 12:10:04 Dave Cramer wrote:
> Before this change we used ant, plus a number of plugins. Which would have
> had to have been built from source. Is this somehow different ?
>
> Dave Cramer
> davec@postgresintl.com

This problem existed before (with ant) too.  Due to that, we haven't
updated postgresql-jdbc package very-very long ago, so haven't Debian
AFAIK.

I don't complain! :) Your work is very appreciated.  But unfortunately, we
are not able to distribute your software ATM and we are trying to change
that.

Pavel

On Thursday 07 of January 2016 13:35:13 Dave Cramer wrote:
> When was the driver last upgraded ?
>
> debian has 1201 which was built with ant.

9.3-1201 or 9.4.1201?  Fedora has 9.4-1200, per [1].  I would be
interested how Debian solved the packaging issues.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1196445

Pavel

> Dave Cramer
>
> davec@postgresintl.com
> www.postgresintl.com
>
> On 7 January 2016 at 12:24, Pavel Raiskup <praiskup@redhat.com> wrote:
>
> > On Thursday 07 of January 2016 12:10:04 Dave Cramer wrote:
> > > Before this change we used ant, plus a number of plugins. Which would
> > have
> > > had to have been built from source. Is this somehow different ?
> > >
> > > Dave Cramer
> > > davec@postgresintl.com
> >
> > This problem existed before (with ant) too.  Due to that, we haven't
> > updated postgresql-jdbc package very-very long ago, so haven't Debian
> > AFAIK.
> >
> > I don't complain! :) Your work is very appreciated.  But unfortunately, we
> > are not able to distribute your software ATM and we are trying to change
> > that.
> >
> > Pavel
> >
> >

ant is not problem.

there was problem with some ant-maven plugin. Plugin was added because new dependencies (my guess is OSGi and
waffle-jna).

Correct me if I am wrong - I am not java hacker.

Pavel Kajaba

----- Original Message -----
From: "Dave Cramer" <pg@fastcrypt.com>
To: "Pavel Raiskup" <praiskup@redhat.com>
Cc: "Pavel Kajaba" <pkajaba@redhat.com>, "Vladimir Sitnikov" <sitnikov.vladimir@gmail.com>, "List"
<pgsql-jdbc@postgresql.org>,"Dave Cramer" <notifications@github.com>, "Devrim GÜNDÜZ" <devrim@gunduz.org> 
Sent: Thursday, January 7, 2016 7:56:20 PM
Subject: Re: [JDBC] [pgjdbc] Implement JDBC specs via pre-processor step (#435)

9.3.1201 which was still built using ant

Dave Cramer

davec@postgresintl.com
www.postgresintl.com

OK, that makes sense. So here are the options AFAICT.

I guess you could use make and create a makefile to compile it. It should not be difficult.

I gather you would also have to compile the preprocessing tool as well ?

How are other java projects that use maven ever included in a distro ? Tomcat ?

Dave Cramer

davec@postgresintl.com

www.postgresintl.com

On 8 January 2016 at 08:32, Pavel Kajaba <pkajaba@redhat.com> wrote:

ant is not problem.

there was problem with some ant-maven plugin. Plugin was added because new dependencies (my guess is OSGi and waffle-jna).

Correct me if I am wrong - I am not java hacker.

Pavel Kajaba

----- Original Message -----
From: "Dave Cramer" <pg@fastcrypt.com>
To: "Pavel Raiskup" <praiskup@redhat.com>
Cc: "Pavel Kajaba" <pkajaba@redhat.com>, "Vladimir Sitnikov" <sitnikov.vladimir@gmail.com>, "List" <pgsql-jdbc@postgresql.org>, "Dave Cramer" <notifications@github.com>, "Devrim GÜNDÜZ" <devrim@gunduz.org>
Sent: Thursday, January 7, 2016 7:56:20 PM
Subject: Re: [JDBC] [pgjdbc] Implement JDBC specs via pre-processor step (#435)

9.3.1201 which was still built using ant

Dave Cramer

davec@postgresintl.com
www.postgresintl.com

Hi,

On Thu, 2016-01-07 at 12:00 -0500, Dave Cramer wrote:
> pretty sure Devrim and John built maven from source and then built
> the driver.

Mostly. I added apache-maven to EL-6 repo of us. Pavel, you can check
pgrpms git repo for details.

Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

On Wednesday 13 of January 2016 20:23:39 Devrim GÜNDÜZ wrote:
> On Thu, 2016-01-07 at 12:00 -0500, Dave Cramer wrote:
> > pretty sure Devrim and John built maven from source and then built
> > the driver.
>
> Mostly. I added apache-maven to EL-6 repo of us. Pavel, you can check
> pgrpms git repo for details.

Hi Devrim, not entirely, see this scratch build from Koji [1], after I
applied this patch to the PGRPMS package:

  -BuildRequires: ant
  +BuildRequires: ant, maven

The build failed because builers of Fedora packages are refused to contact
internet.

[1] https://kojipkgs.fedoraproject.org//work/tasks/8072/12548072/build.log

Pavel

Hi,

On Thu, 2016-01-14 at 17:15 +0100, Pavel Raiskup wrote:
> The build failed because builers of Fedora packages are refused to
> contact internet.
>
> [1] https://kojipkgs.fedoraproject.org//work/tasks/8072/12548072/buil
> d.log

So yes, maven builds would be impossible :(

Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

How does it fetch the initial tarball then?

Vladimir

Hi,

On Mon, 2016-01-18 at 01:17 +0300, Vladimir Sitnikov wrote:
> How does it fetch the initial tarball then?

Fedora uses git to store the tarballs, so initial tarball is available
through there, AFAIK.

Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

It is really similar with Fedora (maven part).

However we also miss waffle-jna. It's rather easy to patch, but do you think it would be possible to make it optional?

For example with jcp (we have already packaged it). I have tried it, but I am not able to pass OS condition into if
part.

What do you think? Both Fedora and Debian packagers would profit from generic patch.

Pavel Kajaba.

----- Original Message -----
From: "Stephen Nelson" <stephen@eccostudio.com>
To: pgsql-jdbc@postgresql.org
Sent: Monday, January 18, 2016 12:10:42 AM
Subject: Re: [JDBC] [pgjdbc] Implement JDBC specs via pre-processor step (#435)

On Sun, Jan 17, 2016 at 10:54 PM Dave Cramer <pg@fastcrypt.com> wrote:

> So what you are saying is that anything that is built using maven can't be
> used in any distro ?
>
>
I have some experience packaging for Debian in the Java team. The consensus
is that Maven is preferred, but each dependency a package has, must itself
be built from source and therefore available as a Debian package.

The only dependency not available in Debian is waffle-jna. Given this is an
authentication provider for Windows it will never get packaged, so the
maintainer will have to patch this out of pgjdbc in order to package the
latest version of the driver.

> I have tried it, but I am not able to pass OS condition into if part

I do not think "os" condition is a way to go there: "build" OS might
be different from "target use" OS.
I think maven profile can be added to exclude waffle from the build.

Vladimir

> I think maven profile can be added to exclude waffle from the build.

Maven profile would not help, because waffle is used in core/v3/ConnectionFactoryImpl.java class.

So basically if waffle was excluded it would not compile.

However I have tried to create patch [1], which uses Java reflection API to load different classes during runtime.

What do you think about this approach?

Pavel Kajaba.


[1] https://pkajaba.fedorapeople.org/0001-Disable-using-SSPI-under-Linux.patch

>What do you think about this approach?

As long as <dependency>...</depencency> is there, mvn would complain
on "missing dependency".
The only sane way of excluding dependency "on demand" is maven profiles.

Vladimir

Yes, you are right.

But in Fedora there is tooling where I can exclude dependency.

Even if we created maven profile we would have to patch source code somehow.

I think it's possible to do it way I proposed or rewrite ConnectionFactoryImpl class.

Pavel
----- Original Message -----
From: "Vladimir Sitnikov" <sitnikov.vladimir@gmail.com>
To: "Pavel Kajaba" <pkajaba@redhat.com>
Cc: "Stephen Nelson" <stephen@eccostudio.com>, "List" <pgsql-jdbc@postgresql.org>, "Pavel Raiskup"
<praiskup@redhat.com>
Sent: Tuesday, January 19, 2016 1:29:34 PM
Subject: Re: [JDBC] [pgjdbc] Implement JDBC specs via pre-processor step (#435)

>What do you think about this approach?

As long as <dependency>...</depencency> is there, mvn would complain
on "missing dependency".
The only sane way of excluding dependency "on demand" is maven profiles.

Vladimir

> Well to start with there are other operating systems besides Linux. I'd
> suggest reversing the logic to test for Windows.

Ok, it's trivial change.

> Is there some reason not to use github ?

Just a habbit and I wanted you to show it before committing to github.

> The advantage of github pull requests is that travis will build it for you
and make sure it compiles etc. Nothing gets committed.

Here [1] is my pull request. Coding style test failed but it's easy fix.

[1] https://github.com/pgjdbc/pgjdbc/pull/495

>Here [1] is my pull request

‎How does it address 'fedora no internet' problem?

Vladimir

‎> How does it address 'fedora no internet' problem?

All required maven dependencies are BuildDependencies in spec file.

It means that everything is in fedora repositories in form of RPMS.

So all maven dependencies are downloaded before build.

This solves problem with missing waffle.

Why do you need #495 change then?

Vladimir

On Tuesday 19 of January 2016 23:54:39 Vladimir Sitnikov wrote:
> Why do you need #495 change then?

You are asking why we don't add waffle into Fedora?  The Windows specific
'jar' files can't be distributed in Fedora, most probably neither built
from source on/for Fedora.  But please, substitute Fedora with any open
source GNU/Linux distribution,

regards,
Pavel

>You are asking why we don't add waffle into Fedora?

No. I am not.

I am asking: "can you kindly provide a build log of a failed due to
waffle build?"
In other words, "a fedora build log before #495, and a fedora build
log after #495 is merged in".

Alternative: can you provide a test case that *fails* without #495?

Unfortunately, current #495 has no tests behind. All tests pass
without #495, thus the change does almost nothing.

Vladimir

> Why do you need #495 change then?

There were 3 maven dependencies which missed from Fedora.
java-comment-preprocessor, osgi-enterprise and waffle-jna.

About java-comment-preprocessor we agreed that you will continue used it so we added it in form of RPM into Fedora.
About osgi-enterprise I simply delete content of folder pgjdbc/src/main/java/org/postgresql/osgi and It works without
it.

waffle-jna was the hardest because I had to patch the code. I could have just patch out code wich use waffle, but this
way

if you made some changes patch would not apply update of postgresql-jdbc. So I tried to create this patch which can
workwithout problem 
with official release (aka it's not a hack, java decide which class should load during runtime)
and compiler is OK when I delete pgjdbc/src/main/java/org/postgresql/sspi/SSPIClient.java.

Pavel Kajaba.

Hi Vladimir, I'm kindly reconstructing the way-too-much trimmed conversation in
this thread:

Vladimir wrote:
> Pavel R. wrote:
> > Vladimir wrote:
> > > Pavel K. wrote:
> > > > Vladimir wrote:
> > > > > Pavel K. wrote:
> > > > > > Here [1] is my pull request
> > > > >
> > > > > How does it address 'fedora no internet' problem?
> > > >
> > > > All required maven dependencies are BuildDependencies in spec file.
> > > >
> > > > It means that everything is in fedora repositories in form of RPMS.
> > > >
> > > > So all maven dependencies are downloaded before build.
> > > >
> > > > This solves problem with missing waffle.
> > >
> > > Why do you need #495 change then?
> >
> > You are asking why we don't add waffle into Fedora?  The Windows specific
> > 'jar' files can't be distributed in Fedora, most probably neither built
> > from source on/for Fedora.  But please, substitute Fedora with any open
> > source GNU/Linux distribution,
>
> No. I am not.
>
> I am asking: "can you kindly provide a build log of a failed due to
> waffle build?"
> In other words, "a fedora build log before #495, and a fedora build
> log after #495 is merged in".
>
> Alternative: can you provide a test case that *fails* without #495?
>
> Unfortunately, current #495 has no tests behind. All tests pass
> without #495, thus the change does almost nothing.

Those questions sound much more concrete ;).  Pavel, can we provide failed
build logs or other info?  I've posted issues with maven in this thread [1],
I bet with waffle it is similar situation -- from quick look at Pavel's --
it looks he tries to make the waffle "optional" dependancy -- but I
haven't done review.

[1] http://www.postgresql.org/message-id/4027547.QFGZ3nUSVM@nb.usersys.redhat.com

Pavel

> About osgi-enterprise I simply delete content of folder pgjdbc/src/main/java/org/postgresql/osgi and It works without
it.

That is a time bomb.
Why do you think "pgjdbc/src/main/java/org/postgresql/osgi" will ever
be named like that?
What if package gets renamed?

Fedora's build scripts should not delete files & folders at random.
Especially, when you are collaborating with live community.

Vladimir

On Wed, 2016-01-20 at 12:42 +0300, Vladimir Sitnikov wrote:
> > About osgi-enterprise I simply delete content of folder
> > pgjdbc/src/main/java/org/postgresql/osgi and It works without it.
>
> That is a time bomb.
> Why do you think "pgjdbc/src/main/java/org/postgresql/osgi" will ever
> be named like that?
> What if package gets renamed?
>
> Fedora's build scripts should not delete files & folders at random.
> Especially, when you are collaborating with live community.
>
> Vladimir
>
>

Why do you think it's a time bomb?

If someone rename that folder I will try delete something what is not
there and it won't build because osgi end up somewhere else.

Can you think about better solution? Can maven profile say "hey don't
build these files"?

If yes, It would be great. However in case of waffle class which uses
it, would have to be refactored into two (I can't think about different
solution).

Pavel Kajaba

>Why do you think it's a time bomb?
> it won't build because osgi end up somewhere else.

It is a time bomb because it does not ensure it achieves "exclude all
osgi classes" goal.
For instance, suppose in a week I add a *test* class to validate that
OSGi does work in pgjdbc as designed?
That would add some new files to ".../src/test/java/...".
You would miss to delete them, thus your build will fail.

Once again: if you delete/add/modify random files from source
distribution, you are building on sand.
Expect random failures in that case.

If current pgsql's build procedure somehow does not suit you, go ahead
and raise the discussion on mailing list and/or GitHub's issue.

>Can you think about better solution?

A proper solution starts from "gathering the requirements".
I have not seen your requirements on the build procedure except "no internet".
Well, at some point there was a statement like "maven cannot be used at all".

I perfectly understand, that distribution of "waffle" in linux world
makes little sense.
So, I could understand the requirement of "there should be a build
parameter that excludes waffle from dependencies".
It is easy to test, so we setup just another Travis job that builds
"no waffle" variant and it will ensure the build would not get
corrupted by some random accident here or there.

On the other hand, you go right into the implementation details and
claim that if you change some "direct call" to "reflective call", then
it would magically solve the problem for you.
That does not work.
Even if that fix somehow gets merged in, I expect a pull request in a
day or two with exactly reverse change set: "improve performance of
waffle calls..."

That is why I strongly advice you to present your requirements in a
clear & testable way.
Does that make sense?

Vladimir

On Wednesday 20 of January 2016 14:58:02 Vladimir Sitnikov wrote:
> >Why do you think it's a time bomb?
> > it won't build because osgi end up somewhere else.
>
> It is a time bomb because it does not ensure it achieves "exclude all
> osgi classes" goal.
> For instance, suppose in a week I add a *test* class to validate that
> OSGi does work in pgjdbc as designed?
> That would add some new files to ".../src/test/java/...".
> You would miss to delete them, thus your build will fail.

I was told that osgi sources are mine field for open-source world, no
clear licencing, source code, etc.  Each dependency needs to be taken with
*serious carefulness* -- case-by-case, is this documented somewhere in
pgjdbc code (licence implications)?

INAL, but this sounds like osgi support is something which can make newer
jdbc plugin GNU/Linux incompatible.  But also, because PostgreSQL license
is not against bundling with proprietary software -- you may expect that
two proprietary licenses will beat each other.

> Once again: if you delete/add/modify random files from source
> distribution, you are building on sand.  Expect random failures in that
> case.

This is usual case, we are used to deal with similar issues, don't worry.

> If current pgsql's build procedure somehow does not suit you, go ahead
> and raise the discussion on mailing list and/or GitHub's issue.

Vladimir, I guess you've missed the fact that we are talking on upstream
mailing list.

> >Can you think about better solution?
>
> A proper solution starts from "gathering the requirements".
> I have not seen your requirements on the build procedure except "no
> internet".  Well, at some point there was a statement like "maven cannot
> be used at all".

We may use maven as build tool, but third-party binary software blobs
*cannot be used at all*, it does not matter where those come from (maven
repositories or other RPM repositories).

> I perfectly understand, that distribution of "waffle" in linux world
> makes little sense.
> So, I could understand the requirement of "there should be a build
> parameter that excludes waffle from dependencies".
> It is easy to test, so we setup just another Travis job that builds
> "no waffle" variant and it will ensure the build would not get
> corrupted by some random accident here or there.

Do you have documentation for such approach, please?  We will be happy to
work on that.  And I don't think preprocessing in Java is an answer.

> On the other hand, you go right into the implementation details and
> claim that if you change some "direct call" to "reflective call", then
> it would magically solve the problem for you.
> That does not work.

This is very poor argument, to call this "issue", please say how much
slower it is and why.  If that is really issue, lets discuss how we can
make sure we'll fix the performance issues.

> Even if that fix somehow gets merged in, I expect a pull request in a
> day or two with exactly reverse change set: "improve performance of
> waffle calls..."

Be concrete Vladimir.  Who is going to merge this and who is going to
revert that later.  Will that be after serious discussion?  I wouldn't be
that much offensive..  let's make sure that we *all* advocate open source
principles, please.

> That is why I strongly advice you to present your requirements in a
> clear & testable way.  Does that make sense?

That's what we work towards.

Pavel

> Each dependency needs to be taken with
>*serious carefulness* -- case-by-case, is this documented somewhere in
>pgjdbc code (licence implications)?

I'm not sure what is license clearance status of pgjdbc.

>Be concrete Vladimir.  Who is going to merge this and who is going to
>revert that later.

Fixes are merged by committers. Is that what you are asking?

> Will that be after serious discussion?

Fixes are "merged in" provided they have tests for the new functionality.
That means, "#495 as of now" is NOT going to be merged in.

What I am saying is people come and go. Do not expect everybody (or
even a single person) on the list to remember all the peculiarities
regarding to Fedora builds.

>, but third-party binary software blobs *cannot be used at all*,

Can you define how to tell if a particular *.java or *.xml file is a
"binary blob" or not?

Vladimir> So, I could understand the requirement of "there should be a build
Vladimir> parameter that excludes waffle from dependencies".
Pavel>Do you have documentation for such approach, please?

There is no such feature in pgjdbc yet.

>And I don't think preprocessing in Java is an answer.

Don't go to implementation details until you have requirements at hand.
I think all you are asking can be implemented with no problems.
On the other hand, I am not sure I understand your requirements, so I
do not want to guess it.

>This is very poor argument, to call this "issue", please say how much
>slower it is and why.

One can easily rant on that like You-Know-Who does (see [1])
The strong argument is: it is extremely easy to forget the "Fedora's
requirement", and commit some non-relevant change that would
unexpectedly break Fedora.

I'm -1 on committing any changes to pgjdbc that cannot be tested in pgjdbc's CI.
Well, there might be exceptions, but this particular case is obviously
not an exception.
No tests -- no commits here.
Agreed?

Vladimir> you to present your requirements in a
Vladimir> clear & testable way
Pavel>That's what we work towards.

I'm all ears. No kidding.

[1]: http://shipilev.net/blog/2015/black-magic-method-dispatch/

Vladimir

On Wednesday 20 of January 2016 16:48:17 Vladimir Sitnikov wrote:
> > Each dependency needs to be taken with
> >*serious carefulness* -- case-by-case, is this documented somewhere in
> >pgjdbc code (licence implications)?
>
> I'm not sure what is license clearance status of pgjdbc.

That is unlucky .. who should I ask?  At least adding new dependency is
big issue every committer should take into account.  I'm not sure about
licensing too -- and thats why we can't provide jdbc with osgi support.

> > Will that be after serious discussion?
>
> Fixes are "merged in" provided they have tests for the new functionality.
> That means, "#495 as of now" is NOT going to be merged in.

Thats your opinion, or project status?  So we are going to support only
those architectures which are in the CI github provides?

> What I am saying is people come and go. Do not expect everybody (or
> even a single person) on the list to remember all the peculiarities
> regarding to Fedora builds.

I've told you this about 4 times:  This is not Fedora only.  This is
matter of open source re-distribution model.

> >, but third-party binary software blobs *cannot be used at all*,
>
> Can you define how to tell if a particular *.java or *.xml file is a
> "binary blob" or not?

Yes, we do manual reviews.  By "software binary blob" I meant code (which
can be executed) which may, e.g., infect my machine.

> Vladimir> So, I could understand the requirement of "there should be a build
> Vladimir> parameter that excludes waffle from dependencies".
> Pavel>Do you have documentation for such approach, please?
>
> There is no such feature in pgjdbc yet.

I'm not asking about pgjdbc, I'm asking you - as Java hacker - how this is
_normally_ done.

> >And I don't think preprocessing in Java is an answer.
>
> Don't go to implementation details until you have requirements at hand.

I won't have all requirements to build on my system.

> I think all you are asking can be implemented with no problems.

Pavel sent you an approach.  Except the testsuite issue?

> On the other hand, I am not sure I understand your requirements, so I
> do not want to guess it.

See above^.  Dependencies.

> >This is very poor argument, to call this "issue", please say how much
> >slower it is and why.
>
> One can easily rant on that like You-Know-Who does (see [1])
> The strong argument is: it is extremely easy to forget the "Fedora's
> requirement", and commit some non-relevant change that would
> unexpectedly break Fedora.

See above. .. revert != unexpectedly.

> I'm -1 on committing any changes to pgjdbc that cannot be tested in pgjdbc's CI.
> Well, there might be exceptions, but this particular case is obviously
> not an exception.

Why this can't be tested?  Please stop spreading FUD.  This is the first
time you said you want test-case.

> No tests -- no commits here.
> Agreed?

Please stop being offensive.

> Vladimir> you to present your requirements in a
> Vladimir> clear & testable way
> Pavel>That's what we work towards.
>
> I'm all ears. No kidding.

You are not.  I'll re-read the thread once more.

Pavel

>I'm asking you - as Java hacker - how this is
>_normally_ done.

Maven profile. See [1].
I do not understand why should I repeat that again and again.
I hope this time it would make the trick.

>That is unlucky .. who should I ask?  At least adding new dependency is
>big issue every committer should take into account.  I'm not sure about
>licensing too -- and thats why we can't provide jdbc with osgi support.

Frankly speaking, I've no idea how that is usually managed.
It looks like pgjdbc does not force to sign CLAs when accepting patches.

I'm not sure why you pay so much attention to osgi while completely
ignore the fact that pgjdbc includes lots of contributions (from
random people) itself.

The osgi jars in question are licensed under Apache 2.0.
jcp is Apache 2.0 as well.

>Thats your opinion, or project status?

That is my opinion. It seems to be in line with project development
for the recent year. I'm one of the committers if that matters you.

>So we are going to support only
>those architectures which are in the CI github provides?

Of course not. However, we do search for the possibilities to improve
testing coverage of pgjdbc (see [2]).
Current setup indeed limits us a lot. For instance, Travis job does
not test SSL. So we are having big troubles with that (lots of issues
from end-users). I does not mean we throw out SSL code. It just does
mean we are extra careful not to touch that code if that is possible.

>Yes, we do manual reviews.  By "software binary blob" I meant code (which
>can be executed) which may, e.g., infect my machine.

I do not want to stretch that, but I wonder if you call "sql statement
inside a java code" a "software binary blob" "which can be executed in
backend" and "infect the machine via integer overflow or whatever".
I mean: what if I convert osgi.jar into a java file into a form of
some select statements? Would it heal your case?

>Pavel sent you an approach.  Except the testsuite issue?

Yes. Without a testcase I cannot understand what that commit is trying
to accomplish.

>See above. .. revert != unexpectedly.

revert == unexpectedly.
I'm pretty sure we play well, and we are not going to explicitly delete that.

See [3] for an "innocent fix" that broke query batching just because
the same value was reused for a bit field.
Once again: a *major feature* was unexpectedly reverted and passed peer-review.

All what I was saying has happened to pgjdbc recently. So I am not
just raising "theoretical problems", but I'm highlighting the exact
way "a feature without testing" is going to have

>Why this can't be tested?

I did not mean "that particular commit cannot be tested".
I meant "if a patch does not include tests, it gets -1 for that".
If a patch does have tests, then it is worth reviewing as usual.
Sometimes, it is hard to add tests. Then the patch can still be
reviewed, however, it requires much more attention.

> Please stop being offensive.

I try to be explicit, so you can understand what I convey easier.
English is not my first language, so please accept my apologies if
that was offensive.
I'm trying to learn from grammar corrections, unfortunately I do not
receive many of them.

[1]: http://stackoverflow.com/a/167284/1261287
[2]: https://github.com/pgjdbc/pgjdbc/issues/461
[3]: https://github.com/pgjdbc/pgjdbc/commit/a6bd36faaedc779f932fa76f52bab9550f0fcd6d

Vladimir

On Wednesday 20 of January 2016 18:18:16 Vladimir Sitnikov wrote:
> >I'm asking you - as Java hacker - how this is
> >_normally_ done.
>
> Maven profile. See [1].

Thanks.

> >That is unlucky .. who should I ask?  At least adding new dependency is
> >big issue every committer should take into account.  I'm not sure about
> >licensing too -- and thats why we can't provide jdbc with osgi support.
>
> Frankly speaking, I've no idea how that is usually managed.
> It looks like pgjdbc does not force to sign CLAs when accepting patches.
>
> I'm not sure why you pay so much attention to osgi while completely
> ignore the fact that pgjdbc includes lots of contributions (from
> random people) itself.

Contributing to project with "PostgreSQL" license is clear;  using it is
clear either.  As a user, I can trust you (== software provider) that the
code you provide is really "PostgreSQL" licensed -- on distribution
level we must properly check/review that this is truth, but that is fine.

(hard) dependency on randomly licenced project OTOH can make otherwise
perfectly well licenced open source project unusable for me as open-source
user.  What would be worse:  Two dependencies on two proprietary licenses
might make the project (legally) unusable.  At least as far as I
understand.

That is why the licensing is important from user's perspective, and why SW
providers should be aware of that POV.

> The osgi jars in question are licensed under Apache 2.0.
> jcp is Apache 2.0 as well.

IANAL, but Apache 2.0 is fine license if we *have* the right source code
(which is not guaranteed by Apache 2.0 itself, neither osgi, afaik), but
again - IANAL.  Do we have source?

But to other point -- if I understand it right, only some kind of API for
"jar-file" propagation in osgi-capable system is used.
Is this too candidate for opt-out in maven profile?  That would simplify
things.

> >Yes, we do manual reviews.  By "software binary blob" I meant code
> >(which can be executed) which may, e.g., infect my machine.
>
> I do not want to stretch that, but I wonder if you call "sql statement
> inside a java code" a "software binary blob" "which can be executed in
> backend" and "infect the machine via integer overflow or whatever".

It is source code;  thus (if not obfuscated) nobody is blocked from
looking at the code and decide whether that "is"/"is not" malicious
software.  Or whether it has compatible license.  Pre-compiled software is
not human readable usually, similarly obfuscated code -- so that I meant
by "binary".

> I mean: what if I convert osgi.jar into a java file into a form of some
> select statements? Would it heal your case?

That sounds like obfuscation.

If you converted the osgi.jar file into properly licensed SQL source code,
that it would be probably fine :).

> > Pavel sent you an approach.  Except the testsuite issue?
>
> Yes. Without a testcase I cannot understand what that commit is trying
> to accomplish.

We try to move one dependency from 'hard' => 'optional', because that is
hard-to-get-into-linux (the right way).  But sounds fair, as I understand
it right -- in Travis is something like Ubuntu environment, right?  There
should be easy to remove '*.jar' (which is redundant anyway) file to make
such test.

Thanks Vladimir for the answer,
Pavel

     Hi all.

     I'm not going to reply to the low-level stuff mentioned here, as
there are many bits to analyze. But I still want to give my general opinion.

     pgjdbc has improved in the last year or so significantly, and a lot
of infrastructure has made it possible, like new features, CI and
Maven-ization. Thanks to everyone involved. So now it's not a bad time
to look at other issues/priorities, other than the "usual suspects",
like performance and new features.

     And I think packaging is definitely one of them. It blows my mind
that RHEL, Fedora or Debian users are using significantly outdated
versions of the driver because of a packaging issue. Why do we develop
so many cool stuff for other users not to have a simple means for using
it? It's probably nobody's specific fault, but this is something that
should be fixed with high priority, IMHO.

     So please lets sit down and try to constructively fix this. As far
as I understand, we (all) should mainly work on:

- Understand more clearly packaging requirements (what Vladimir referred
to as gathering the requirements). Is there a guide, document or other
reference which we may look at?

- Analyze our pom's dependencies and study them one by one, and their
transitive dependencies, to check:    (for the specific versions used)
     * Whether they are already packaged in RH/Fedora
     * What's their license, check it is effectively open source, and
find if the source is available. And how it is built.

- Understand how other Maven-based projects are built for RH/Fedora

     And then, only then, figure out all the low-level technical details
to see how to fix them.

     Am I missing something else? What do you think?

     Cheers,

     Álvaro



On 20/01/16 14:03, Pavel Raiskup wrote:
> On Wednesday 20 of January 2016 18:18:16 Vladimir Sitnikov wrote:
>>> I'm asking you - as Java hacker - how this is
>>> _normally_ done.
>> Maven profile. See [1].
> Thanks.
>
>>> That is unlucky .. who should I ask?  At least adding new dependency is
>>> big issue every committer should take into account.  I'm not sure about
>>> licensing too -- and thats why we can't provide jdbc with osgi support.
>> Frankly speaking, I've no idea how that is usually managed.
>> It looks like pgjdbc does not force to sign CLAs when accepting patches.
>>
>> I'm not sure why you pay so much attention to osgi while completely
>> ignore the fact that pgjdbc includes lots of contributions (from
>> random people) itself.
> Contributing to project with "PostgreSQL" license is clear;  using it is
> clear either.  As a user, I can trust you (== software provider) that the
> code you provide is really "PostgreSQL" licensed -- on distribution
> level we must properly check/review that this is truth, but that is fine.
>
> (hard) dependency on randomly licenced project OTOH can make otherwise
> perfectly well licenced open source project unusable for me as open-source
> user.  What would be worse:  Two dependencies on two proprietary licenses
> might make the project (legally) unusable.  At least as far as I
> understand.
>
> That is why the licensing is important from user's perspective, and why SW
> providers should be aware of that POV.
>
>> The osgi jars in question are licensed under Apache 2.0.
>> jcp is Apache 2.0 as well.
> IANAL, but Apache 2.0 is fine license if we *have* the right source code
> (which is not guaranteed by Apache 2.0 itself, neither osgi, afaik), but
> again - IANAL.  Do we have source?
>
> But to other point -- if I understand it right, only some kind of API for
> "jar-file" propagation in osgi-capable system is used.
> Is this too candidate for opt-out in maven profile?  That would simplify
> things.
>
>>> Yes, we do manual reviews.  By "software binary blob" I meant code
>>> (which can be executed) which may, e.g., infect my machine.
>> I do not want to stretch that, but I wonder if you call "sql statement
>> inside a java code" a "software binary blob" "which can be executed in
>> backend" and "infect the machine via integer overflow or whatever".
> It is source code;  thus (if not obfuscated) nobody is blocked from
> looking at the code and decide whether that "is"/"is not" malicious
> software.  Or whether it has compatible license.  Pre-compiled software is
> not human readable usually, similarly obfuscated code -- so that I meant
> by "binary".
>
>> I mean: what if I convert osgi.jar into a java file into a form of some
>> select statements? Would it heal your case?
> That sounds like obfuscation.
>
> If you converted the osgi.jar file into properly licensed SQL source code,
> that it would be probably fine :).
>
>>> Pavel sent you an approach.  Except the testsuite issue?
>> Yes. Without a testcase I cannot understand what that commit is trying
>> to accomplish.
> We try to move one dependency from 'hard' => 'optional', because that is
> hard-to-get-into-linux (the right way).  But sounds fair, as I understand
> it right -- in Travis is something like Ubuntu environment, right?  There
> should be easy to remove '*.jar' (which is redundant anyway) file to make
> such test.
>
> Thanks Vladimir for the answer,
> Pavel
>
>
>

On Wed, 2016-01-20 at 14:58 +0300, Vladimir Sitnikov wrote:
> > Why do you think it's a time bomb?
> > it won't build because osgi end up somewhere else.
>
> It is a time bomb because it does not ensure it achieves "exclude all
> osgi classes" goal.
> For instance, suppose in a week I add a *test* class to validate that
> OSGi does work in pgjdbc as designed?
> That would add some new files to ".../src/test/java/...".
> You would miss to delete them, thus your build will fail.
>
> Once again: if you delete/add/modify random files from source
> distribution, you are building on sand.
> Expect random failures in that case.
>
> If current pgsql's build procedure somehow does not suit you, go
> ahead
> and raise the discussion on mailing list and/or GitHub's issue.
>
> > Can you think about better solution?
>
> A proper solution starts from "gathering the requirements".
> I have not seen your requirements on the build procedure except "no
> internet".
> Well, at some point there was a statement like "maven cannot be used
> at all".
>
> I perfectly understand, that distribution of "waffle" in linux world
> makes little sense.
> So, I could understand the requirement of "there should be a build
> parameter that excludes waffle from dependencies".
> It is easy to test, so we setup just another Travis job that builds
> "no waffle" variant and it will ensure the build would not get
> corrupted by some random accident here or there.
>
> On the other hand, you go right into the implementation details and
> claim that if you change some "direct call" to "reflective call",
> then
> it would magically solve the problem for you.
> That does not work.
> Even if that fix somehow gets merged in, I expect a pull request in a
> day or two with exactly reverse change set: "improve performance of
> waffle calls..."
>
> That is why I strongly advice you to present your requirements in a
> clear & testable way.
> Does that make sense?
>
Ok, I try to present them:

1) all pgjdbc maven dependencies have in Fedora in form of RPMs.
  - we are missing osgi-enterpise and waffle-jna

2) If We are not able to get any dependency into Fedora repositories we
need to find a way how to resolve it without any time bombs/hacks.
  - basicaly we need to find way how to remove/disable waffle-jna and
osgi-enterpise.

That's pretty much all we require.

I would like to shed some light into way how we work with maven. There
is tool called XMvn [1]. It works like maven but it is getting jar
files from RPMs which are already installed.

> Vladimir
>
>

[1] https://mizdebsk.fedorapeople.org/xmvn/site/

>I'm curious how may people actually use osgi? Can this be a sub-project that builds an osgi compatible jar ?

It cannot.
Having pgjdbc-jre7-osgi, pgjdbc-jre7-noosgi would be a nightmare.

Vladimir

Actually a lot of people are using OSGI. Almost any enterprise container is OSGI container. JBoss, Websphere, Fuse Fabric, you name it.
We are using felix bundle maven plugin to make OSGI bundle. Its open source. May be it's worth switching to it? It's really easy to use.

Best regards, Vitalii Tymchyshyn

>So it seems like one solution to osgi is just to copy the sources into the project. These are all interfaces, and a
minimalnumber of files, I think around 4 

What's the problem with packaging OSGi interfaces as usual rpms (or
whatever is used for packaging)?
As for me, "copy&pasting <<minimal number of files, I think around
4>>" does not solve legal issues if there are any.
On top of that, exposing OSGi interfaces as part of pgjdbc.jar would
probably void OSGi compliance. The jar would just not load due to
invalid contents.

Vladimir

On 21/01/16 02:03, Pavel Kajaba wrote:

> [...]
> Ok, I try to present them:
>
> 1) all pgjdbc maven dependencies have in Fedora in form of RPMs.
>    - we are missing osgi-enterpise and waffle-jna


     OK, so here's how to reproducible check the dependencies of the
maven project:

- Clone the repo from github
- Do a "mvn package install -DskipTests"
- Run "mvn dependency:list"

     For the pgjdbc (we don't care about the ubenchmark, that's not
going to get packaged) looks like this:

org.postgresql:postgresql:bundle:9.4.1208-SNAPSHOT
[INFO] +- junit:junit:jar:4.12:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.osgi:org.osgi.enterprise:jar:4.2.0:provided
[INFO] +- org.osgi:org.osgi.core:jar:4.3.1:provided
[INFO] +- xml-resolver:xml-resolver:jar:1.2:compile
[INFO] \- com.github.dblock.waffle:waffle-jna:jar:1.7:compile
[INFO]    +- net.java.dev.jna:jna:jar:4.1.0:compile
[INFO]    \- net.java.dev.jna:jna-platform:jar:4.1.0:compile

     Now go to search.maven org and perform queries like:

u:org.osgi g:org.osgi.enterprise

     And read the pom.xml. There the license, dependencies and possible
information to the source code repository or developers can be found.
Also the source jar.

     Now looking at some dependencies:

- org.osgi.core is already present in Fedora/RH, but a very old version
(1.4, pgjdbc is using 4.3.1):
http://koji.fedoraproject.org/koji/buildinfo?buildID=649030 But I guess
packaging a new version shouldn't be hard. Specially as this module does
not contain any transitive dependency. Which brings me to another point:
is it easy to just build it, as source code is available and there are
no further dependencies? May we contact this 1.4 version package
mantainer and ask about updating it or otherwise use that packages
building scripts?

- org.osgi.enterprise is not packaged for Fedora/RH but doesn't have any
other dependencies and source is available.

- junit is just a test dependency, it's not required to build the package

- xml-resolver: pretty old code, sources available, no dependencies.
Maybe we should look at using XML code from the JDK rather than this
library, but I haven't checked how big this impact may be. Opinions
welcomed.

- waffle: the real deal, as it doesn't make any sense on a linux build.


     Hope this information helps.

     Cheers,

     Álvaro





>
> 2) If We are not able to get any dependency into Fedora repositories we
> need to find a way how to resolve it without any time bombs/hacks.
>    - basicaly we need to find way how to remove/disable waffle-jna and
> osgi-enterpise.
>
> That's pretty much all we require.
>
> I would like to shed some light into way how we work with maven. There
> is tool called XMvn [1]. It works like maven but it is getting jar
> files from RPMs which are already installed.
>
>> Vladimir
>>
>>
> [1] https://mizdebsk.fedorapeople.org/xmvn/site/
>
>

Álvaro>Run "mvn dependency:list"

mvn dependency:analyze-report allows to capture "Used but undeclared
dependencies" and "Unused but declared dependencies" as well.

Álvaro>- xml-resolver: pretty old code

I've no idea what it is doing there. It is marked as compile-time
dependency, however the code compiles without it just fine.
I think it can be just dropped.

Álvaro>    OK, so here's how to reproducible check the dependencies of
the maven project:

What we miss is:
1) Fedora-like CI environment. It makes little sense optimizing
pgjdbc's dependencies if those dependencies are already present in
Fedora-like build boxes.
Is there a CI for Fedora builds?
How do we validate that "the specific changes to pgjdbc" indeed help
Fedora packaging?

2) Requirements. I'm sure the first dozen of builds will fail due to
packaging limitations. It is much better to know the limitations
beforehand than running into them at Fedora CI builds.

Vladimir

Hi all, and sorry for the delay.  And for long mail.

On Friday 22 of January 2016 10:42:49 Vladimir Sitnikov wrote:
> What we miss is:
> 1) Fedora-like CI environment. It makes little sense optimizing
> pgjdbc's dependencies if those dependencies are already present in
> Fedora-like build boxes.

Build boxes:
  RPM distributions usually build package in "mock" tool [1], which is
  basically chroot.  There is really minimal set of packages, but
  particular package build can request additional packages (if those are
  available in official/trusted package repositories).  Thus each package
  has slightly different builder configuration.

Dependencies:
  Just a note that IMO optimizing dependencies is always worth.
  Fedora/Ubuntu/Windows is just subset of platforms available in the wild.
  Anybody on platform "Foo" can take the source and build it itself and
  each dependency makes the work harder.
  IOW, as a user:
    1. I have JDBC installed and PostgreSQL, I need a glue -- pgjdbc
    2. Let's compile
    3. I can't compile, there is dependency (a), (b), (c)...

  That's why I think that things like osgi/waffle should be supported, but
  only as an optional dependency.  Neither jdbc nor PostgreSQL needs the
  dependency a b c, and as a connector between jdbc/postgresql any new
  hard dependency hurts :(.

> Is there a CI for Fedora builds?

Yes.  We can assist with this, having this in pgjdbc would be awesome.
Copr [2] build system should be able to take github repo branch and spin a
new build of RPM package.  This probably does not show "green OK icon" on
github page (yet), but anybody can receive fedmsg [3] about successful
build or check manually that the build is OK.

Testsuite possibilities are limited, because the build-environment is
not able to do administrator tasks or some hard integration testing, but
those should not be needed in case of jdbc driver.

> How do we validate that "the specific changes to pgjdbc" indeed help
> Fedora packaging?

We can tell you it helps, as Fedora packaging is still our responsibility.
But yes, we should setup CI.  And any interest of upstream in Fedora
packaging is huge success.

I would correct your question by s/Fedora/RPM/ pattern though;  all RPM
[4] distributions would benefit from changes we talking about.

> 2) Requirements. I'm sure the first dozen of builds will fail due to
> packaging limitations. It is much better to know the limitations
> beforehand than running into them at Fedora CI builds.

To answer this, I need to know what limits you mean probably.  Fedora
builds are turing complete, there are just intentional limits (which
encourage packagers to do the job right) like no superuser access.
In CI [2], you can opt-in the internet;  but it is generally a bad idea
(I admit that not everybody will agree with me) because it (a) makes me
lazy and (b) is not secure.

So
  * no internet in official build system (but that is separate from CI),
  * you have limited set of dependencies (usually new package gets added
    on demand -- once somebody needs it), usually available on demand and
    installed before the build starts
  * no superuser access

[1] https://fedoraproject.org/wiki/Mock
[2] https://copr.fedorainfracloud.org/coprs/
[3] http://www.fedmsg.com/en/latest/
[4] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions

Pavel

Hi Vitalii,

On Friday 22 of January 2016 00:39:17 Vitalii Tymchyshyn wrote:
> I did some research. The OSGi dependencies can be replaced to the OSS ones:
> osgi-core to org.apache.felix:osgi-core:1.4.0 that is already in Fedora
> osge-enterprise to org.ops4j.pax.jdbc:pax-jdbc-spec:0.6.0. This one is not
> there, but other OPS4J packages are in there, so I presume this one can be
> added too. Please confirm.

Let me check please, OSS alternative is definitely better!  But if we
could make this optional build/runtime dependancy, that would be better.

Pavel

> The only change other than dependency needed is due to a little older spec
> used:
>
> Index: pgjdbc/src/main/java/org/postgresql/osgi/PGBundleActivator.java
> IDEA additional info:
> Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
> <+>UTF-8
> ===================================================================
> --- pgjdbc/src/main/java/org/postgresql/osgi/PGBundleActivator.java
> (revision 0f979c3bbe1a36e6614e3d448ea43de7be27448b)
> +++ pgjdbc/src/main/java/org/postgresql/osgi/PGBundleActivator.java
> (revision )
> @@ -22,7 +22,7 @@
>   * This class is an OSGi Bundle Activator and should only be used
> internally by the OSGi Framework
>   */
>  public class PGBundleActivator implements BundleActivator {
> -  private ServiceRegistration<?> _registration;
> +  private ServiceRegistration _registration;
>
>    public void start(BundleContext context) throws Exception {
>      Dictionary<String, Object> properties = new Hashtable<String, Object>();
>
>
>
> Чт, 21 січ. 2016 о 17:41 Dave Cramer <pg@fastcrypt.com> пише:
>
> > Yes, Alvaro found one that should work
> > https://fedoraproject.org/wiki/PackagingDrafts/OSGiGuidelines, although I
> > don't see a ubuntu one
> >
> > Dave Cramer
> >
> > davec@postgresintl.com
> > www.postgresintl.com
> >
> > On 21 January 2016 at 17:04, Vladimir Sitnikov <
> > sitnikov.vladimir@gmail.com> wrote:
> >
> >> >So it seems like one solution to osgi is just to copy the sources into
> >> the project. These are all interfaces, and a minimal number of files, I
> >> think around 4
> >>
> >> What's the problem with packaging OSGi interfaces as usual rpms (or
> >> whatever is used for packaging)?
> >> As for me, "copy&pasting <<minimal number of files, I think around
> >> 4>>" does not solve legal issues if there are any.
> >> On top of that, exposing OSGi interfaces as part of pgjdbc.jar would
> >> probably void OSGi compliance. The jar would just not load due to
> >> invalid contents.
> >>
> >> Vladimir
> >>
> >
> >

On 22/01/16 07:42, Vladimir Sitnikov wrote:
> Álvaro>Run "mvn dependency:list"
>
> mvn dependency:analyze-report allows to capture "Used but undeclared
> dependencies" and "Unused but declared dependencies" as well.
>
> Álvaro>- xml-resolver: pretty old code
>
> I've no idea what it is doing there. It is marked as compile-time
> dependency, however the code compiles without it just fine.
> I think it can be just dropped.
  xml-resolver was added [1] as a dependency to support using an XML
catalogue with docbook.
  Since docbook has been removed [2] the xml-resolver dependency is safe
to remove [3] also.

Jeremy

[1] https://github.com/pgjdbc/pgjdbc/pull/284
[2]
https://github.com/pgjdbc/pgjdbc/commit/92b65f04de1240b24bd4f71e34903422656660c5
[3] https://github.com/pgjdbc/pgjdbc-parent-poms/pull/3
> Álvaro>    OK, so here's how to reproducible check the dependencies of
> the maven project:
>
> What we miss is:
> 1) Fedora-like CI environment. It makes little sense optimizing
> pgjdbc's dependencies if those dependencies are already present in
> Fedora-like build boxes.
> Is there a CI for Fedora builds?
> How do we validate that "the specific changes to pgjdbc" indeed help
> Fedora packaging?
>
> 2) Requirements. I'm sure the first dozen of builds will fail due to
> packaging limitations. It is much better to know the limitations
> beforehand than running into them at Fedora CI builds.
>
> Vladimir
>
>