Обсуждение: 8.2.23 packages?

All,

Any hope for getting 8.2.23 packages..?

    Thanks,

        Stephen

Stephen Frost [2012-04-04 23:09 -0400]:
> Any hope for getting 8.2.23 packages..?

Where would they live? 8.2 has never been in any Debian release, and
we certainly won't add them to unstable at this point.

If you wish, you could build some based on the old 8.2 branch:

  http://bazaar.launchpad.net/~pitti/postgresql/debian-8.2/changes

Martin
--
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

--On 5. April 2012 10:14:49 +0200 Martin Pitt <mpitt@debian.org> wrote:

> Where would they live? 8.2 has never been in any Debian release, and
> we certainly won't add them to unstable at this point.
>
> If you wish, you could build some based on the old 8.2 branch:
>
>   http://bazaar.launchpad.net/~pitti/postgresql/debian-8.2/changes

<http://pgapt.debian.net/> comes to my mind, too. 8.2.22 is present for lenny
and squeeze. Maybe we can trigger Christoph to offer new builds, if he gets
time?

--
Thanks

    Bernd

On Thursday, April 5, 2012, Bernd Helmle wrote:

--On 5. April 2012 10:14:49 +0200 Martin Pitt <mpitt@debian.org> wrote:

> Where would they live? 8.2 has never been in any Debian release, and
> we certainly won't add them to unstable at this point.
>
> If you wish, you could build some based on the old 8.2 branch:
>
>   http://bazaar.launchpad.net/~pitti/postgresql/debian-8.2/changes

<http://pgapt.debian.net/> comes to my mind, too. 8.2.22 is present for lenny
and squeeze. Maybe we can trigger Christoph to offer new builds, if he gets
time?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Hmm. I had actually replied earlier, but I cannot find any traces of
this in my mail logs. Apparently mutt ate the mail while still putting
a copy in the local folder.

Re: To Stephen Frost 2012-04-09 <20120409150853.GC3474@msgid.df7cb.de>
> Re: Stephen Frost 2012-04-05 <20120405030906.GF1267@tamriel.snowman.net>
> > All,
> >
> > Any hope for getting 8.2.23 packages..?
>
> Re: Bernd Helmle 2012-04-05 <1F5F12B3C5E3B4394648F1EF@apophis.local>
> > > If you wish, you could build some based on the old 8.2 branch:
> > >
> > >   http://bazaar.launchpad.net/~pitti/postgresql/debian-8.2/changes
> >
> > <http://pgapt.debian.net/> comes to my mind, too. 8.2.22 is present for lenny
> > and squeeze. Maybe we can trigger Christoph to offer new builds, if he gets
> > time?
>
> I did some porting of Martin's 8.2 branch and the result of that is at
> http://pgapt.debian.net/pool/main/p/postgresql-8.2/ .
> I only need to find some time to update this to .23. I'll announce it
> here once that has happened.
>
> Christoph
> --
> cb@df7cb.de | http://www.df7cb.de/



Re: Magnus Hagander 2012-04-10 <CABUevEy=RgLduGz6HKeq+FLvr_WjvmCUpmYfSkFOhbsAhRQV8g@mail.gmail.com>
> > What *is* the actual state of this?
>
> There was previously a discussion about setting up an
> apt.postgresql.org(when we created this list as a first step) - has
> this been completely
> replaced by the effort to do pgapt.debian.net?

I am talking with the backports.debian.org people to see if that would
be a good place to put the whole project (or at least the part for
8.4/9.0/9.1). There's a wiki page about it:

http://wiki.debian.org/pkg-postgresql/BackportsIntegration

The "9.0" part is currently a problem, because there's only 8.4 and
9.1 left in testing/unstable.

backports.d.o would have the big advantage that there's buildds that
would pick up building packages. There is also some plan to implement
PPAs officially on ftp-master.debian.org, but that will take at least
months to get started.

That said, I have only limited spare time, and I'd definitely need.

> How many people can actually build/push packages there, and what's the
> update policy?

Currently only me, but so far no one else has asked for access. (I
didn't ask very hard, though.)

> I'd really like to get this information up on www.postgresql.org under
> downloads to make it easier for people who don't already know where to go,
> but it needs to be "stabilized" before we can do that. And we need to be
> able to specify some level of support commitments. (Which means it would be
> very useful if >1 person can actually do it).
>
> We really need something that works the same way the pg community yum
> repository does for RPMs (I don't care if it's under postgresql.org or
> debian.net as long as it *works* that way). I've already had several
> customers drop debian permanently due to the whole "9.0 will be removed
> from backports" issue, and I'd rather not have more of that happening :S
> Particularly since I know we have plenty of people in the pg/debian
> communities who know how to fix this problem...

Nod. (Currently in a meeting, hopefully a longer answer later.)

Christoph
--
cb@df7cb.de | http://www.df7cb.de/

On Wed, Apr 11, 2012 at 08:52, Christoph Berg <cb@df7cb.de> wrote:
> Hmm. I had actually replied earlier, but I cannot find any traces of
> this in my mail logs. Apparently mutt ate the mail while still putting
> a copy in the local folder.

Wow. mutt failing. I'm not used to *that* happening :-)


> Re: Magnus Hagander 2012-04-10 <CABUevEy=RgLduGz6HKeq+FLvr_WjvmCUpmYfSkFOhbsAhRQV8g@mail.gmail.com>
>> > What *is* the actual state of this?
>>
>> There was previously a discussion about setting up an
>> apt.postgresql.org(when we created this list as a first step) - has
>> this been completely
>> replaced by the effort to do pgapt.debian.net?
>
> I am talking with the backports.debian.org people to see if that would
> be a good place to put the whole project (or at least the part for
> 8.4/9.0/9.1). There's a wiki page about it:
>
> http://wiki.debian.org/pkg-postgresql/BackportsIntegration
>
> The "9.0" part is currently a problem, because there's only 8.4 and
> 9.1 left in testing/unstable.

Yes. And the same problem is likely to happen again. So it would
really need to be something that's "officially blessed and can't be
changed around again later". But if that can happen, then sure - I'm
just worried here about something that's done and "yeah it works", but
is not fully endorsed by all people.


Also - putting it in debian backports doesn't solve the Ubuntu
problem, does it? We currently have Martin's PPA for that, but
wouldn't it be Kind Of Neat (TM) to have a single solution for both
these scenarios?


> backports.d.o would have the big advantage that there's buildds that
> would pick up building packages. There is also some plan to implement
> PPAs officially on ftp-master.debian.org, but that will take at least
> months to get started.

Could we in theory have our own buildds if we run this elsewhere? I
know very little about buildds, so I wouldn't know. And it might be
doable but just too much work - so please inform me :-)

I fully understand the gain from having that :-)

And for the record, I don't really like the concept of PPAs for this.
Not necessarily from a technical perspective, that works just fine.
But it's really annoying to have to explain to "enterprise" customers
that "yes, using a *personal* package archive is the proper way to get
your fully supported version".

> That said, I have only limited spare time, and I'd definitely need.

heh, yeah, pretty sure we're all in that position, eh? :-)


>> How many people can actually build/push packages there, and what's the
>> update policy?
>
> Currently only me, but so far no one else has asked for access. (I
> didn't ask very hard, though.)

Let's make sure the process is documented enough that it's easy to
scale, eh? ;) bus-factors are bad...


>> I'd really like to get this information up on www.postgresql.org under
>> downloads to make it easier for people who don't already know where to go,
>> but it needs to be "stabilized" before we can do that. And we need to be
>> able to specify some level of support commitments. (Which means it would be
>> very useful if >1 person can actually do it).
>>
>> We really need something that works the same way the pg community yum
>> repository does for RPMs (I don't care if it's under postgresql.org or
>> debian.net as long as it *works* that way). I've already had several
>> customers drop debian permanently due to the whole "9.0 will be removed
>> from backports" issue, and I'd rather not have more of that happening :S
>> Particularly since I know we have plenty of people in the pg/debian
>> communities who know how to fix this problem...
>
> Nod. (Currently in a meeting, hopefully a longer answer later.)

ok!

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On ons, 2012-04-11 at 10:14 +0200, Magnus Hagander wrote:
> Could we in theory have our own buildds if we run this elsewhere? I
> know very little about buildds, so I wouldn't know. And it might be
> doable but just too much work - so please inform me :-)

I don't think having autobuilders is really a priority for this.  As
long as we only support i386 and amd64, you might as well have someone
fill in the missing builds manually.  Having a full autobuilder
infrastructure is likely to be more work than that.

With that in mind, I am somewhat doubtful about the integration with
backports.d.o.  It sounds nice in general, but the goals and constraints
of either project are not exactly aligned, so this will lead to
permanent conflict.

I think taking the current reprepro-based architecture that Christoph
has already running is just fine (modulo some details, such as source
packages missing).  We just need to give it a permanent home, so people
can start using it.

On Sat, Apr 14, 2012 at 12:32, Peter Eisentraut <peter_e@gmx.net> wrote:
> On ons, 2012-04-11 at 10:14 +0200, Magnus Hagander wrote:
>> Could we in theory have our own buildds if we run this elsewhere? I
>> know very little about buildds, so I wouldn't know. And it might be
>> doable but just too much work - so please inform me :-)
>
> I don't think having autobuilders is really a priority for this.  As
> long as we only support i386 and amd64, you might as well have someone
> fill in the missing builds manually.  Having a full autobuilder
> infrastructure is likely to be more work than that.

Yeah; I doubt there are particularly large number of postgresql
installs on debian on more exotic platforms than that.

Having an automatic build environment is of course useful anyway. But
there are a lot of levels between a full infrastructure to do all of
it, and just a couple of scripts...


> With that in mind, I am somewhat doubtful about the integration with
> backports.d.o.  It sounds nice in general, but the goals and constraints
> of either project are not exactly aligned, so this will lead to
> permanent conflict.

That's what I'm worried about as well. It got very clear to me with
the decision to drop 9.0 from backports (regardless of what happens
with that longterm) just shows that the whole backports project is not
designed to deal with what at least my goal for this is - which is
providing a stable platform across combinations of postgresql and
debian versions, making it possible to move incrementally and
independently between versions depending on external requirements, not
on OS requirements.


> I think taking the current reprepro-based architecture that Christoph
> has already running is just fine (modulo some details, such as source
> packages missing).  We just need to give it a permanent home, so people
> can start using it.

That would work for me, plus I'd also really prefer it to be a team
rather than just one person, to make it less likely that updates get
delayed due to simple things like overbooking or illness... That's an
organization question though, and not a technical one.

FWIW, if we want the repos themselves to run on the postgresql
infrastructure, we have resources to deploy that really quickly. The
yum repository was moved to the main infrastructure a few months ago.
However, we do not currently have resources to host *build nodes*.
Devrim has a build box that EDB donated that's used to build the RPMs
on a multitude of differnet virtual machines - it's quite possible
that this machine could be used to build debian stuff as well, since
it's just a set of xen (I think, could be kvm) virtual machines after
all...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Magnus Hagander 2012-04-11 <CABUevEwTswYhj7CwtwVuT_cMoTB5-SVdwQ7JjjxjZTz9JJCe=Q@mail.gmail.com>
> On Wed, Apr 11, 2012 at 08:52, Christoph Berg <cb@df7cb.de> wrote:
> > Hmm. I had actually replied earlier, but I cannot find any traces of
> > this in my mail logs. Apparently mutt ate the mail while still putting
> > a copy in the local folder.
>
> Wow. mutt failing. I'm not used to *that* happening :-)

(I'm still wondering what happened there. There's no postfix syslog
entries for that full day, so the mail was either completely eaten by
mutt, or postfix. Both of which is weird.)

> Also - putting it in debian backports doesn't solve the Ubuntu
> problem, does it? We currently have Martin's PPA for that, but
> wouldn't it be Kind Of Neat (TM) to have a single solution for both
> these scenarios?

If it works for Debian backports, the same should probably work for
the Ubuntu backports. (The problem there might be that they have
different minimum versions, so while squeeze-backports might support
8.4-and-up, Ubuntu backports might only have 9.0-and-up.)

Generally, yes, Ubuntu should be supported by whatever solution. I
haven't created any Ubuntu distributions on pgapt.debian.net yet
because there's already enough targets for now.

> And for the record, I don't really like the concept of PPAs for this.
> Not necessarily from a technical perspective, that works just fine.
> But it's really annoying to have to explain to "enterprise" customers
> that "yes, using a *personal* package archive is the proper way to get
> your fully supported version".

Nod. What we could do is to use PPAs to get the buildd part covered,
and then pull packages from there to the "official" archive.

> > Currently only me, but so far no one else has asked for access. (I
> > didn't ask very hard, though.)
>
> Let's make sure the process is documented enough that it's easy to
> scale, eh? ;) bus-factors are bad...

I'll try to post some of my thoughts on the whole process here.


Re: Peter Eisentraut 2012-04-14 <1334399550.9019.37.camel@vanquo.pezone.net>
> On ons, 2012-04-11 at 10:14 +0200, Magnus Hagander wrote:
> > Could we in theory have our own buildds if we run this elsewhere? I
> > know very little about buildds, so I wouldn't know. And it might be
> > doable but just too much work - so please inform me :-)
>
> I don't think having autobuilders is really a priority for this.  As
> long as we only support i386 and amd64, you might as well have someone
> fill in the missing builds manually.  Having a full autobuilder
> infrastructure is likely to be more work than that.

From my POV, they are essential. There's so many targets to cover that
so far most of the time I spent on the project was just building
packages: For the server packages, that's 9.1/9.0/...,
sid/squeeze/lenny, amd64/i386 (add Debian/Ubuntu later), for the
modules packages that's sid/squeeze/lenny, amd64/i386 for every module
included.

> With that in mind, I am somewhat doubtful about the integration with
> backports.d.o.  It sounds nice in general, but the goals and constraints
> of either project are not exactly aligned, so this will lead to
> permanent conflict.

Agreed. I still think that the support for PG versions on
backports.d.o should become better, but the necessary changes will be
the same as for pgapt.d.n.

> I think taking the current reprepro-based architecture that Christoph
> has already running is just fine (modulo some details, such as source
> packages missing).  We just need to give it a permanent home, so people
> can start using it.

The missing source packages should be a thing of the past, I only did
that for builds where the only difference to some other version was a
new changelog entry and rebuilding the package.

For the permanent home, I first like to get it more in shape.
Imho, pgapt.debian.net is fine for the moment.


Re: Magnus Hagander 2012-04-14 <CABUevEzxkg4gOhZaBqUqm4aEWxeE9+r=pYS_ces_=3eWvLvfGg@mail.gmail.com>
> Having an automatic build environment is of course useful anyway. But
> there are a lot of levels between a full infrastructure to do all of
> it, and just a couple of scripts...

The messy part is figuring out what to actually build. Building that
is then just a script.

I do have some semi-ready sql queries for that which I should either
finish or explain to someone else what I think they should do.

> > With that in mind, I am somewhat doubtful about the integration with
> > backports.d.o.  It sounds nice in general, but the goals and constraints
> > of either project are not exactly aligned, so this will lead to
> > permanent conflict.
>
> That's what I'm worried about as well. It got very clear to me with
> the decision to drop 9.0 from backports (regardless of what happens
> with that longterm) just shows that the whole backports project is not
> designed to deal with what at least my goal for this is - which is
> providing a stable platform across combinations of postgresql and
> debian versions, making it possible to move incrementally and
> independently between versions depending on external requirements, not
> on OS requirements.

9.0 is still present on backports.debian.org. Though it will probably
require a written policy somewhere to make it stay there.

> That would work for me, plus I'd also really prefer it to be a team
> rather than just one person, to make it less likely that updates get
> delayed due to simple things like overbooking or illness... That's an
> organization question though, and not a technical one.

Yes, definitely.

> FWIW, if we want the repos themselves to run on the postgresql
> infrastructure, we have resources to deploy that really quickly. The
> yum repository was moved to the main infrastructure a few months ago.
> However, we do not currently have resources to host *build nodes*.
> Devrim has a build box that EDB donated that's used to build the RPMs
> on a multitude of differnet virtual machines - it's quite possible
> that this machine could be used to build debian stuff as well, since
> it's just a set of xen (I think, could be kvm) virtual machines after
> all...

I should be able to arrange build nodes. Putting the repos on
*.postgresql.org will be nice. But first let's get it really running.

Oh, btw: I've built and uploaded 8.2.23 for sid/i386. Now there's only
a few builds left to do... (I should finally "upgrade" the notebook to
amd64)

Christoph
--
cb@df7cb.de | http://www.df7cb.de/

On Sun, Apr 15, 2012 at 10:38, Christoph Berg <cb@df7cb.de> wrote:
> Re: Magnus Hagander 2012-04-11 <CABUevEwTswYhj7CwtwVuT_cMoTB5-SVdwQ7JjjxjZTz9JJCe=Q@mail.gmail.com>
>> On Wed, Apr 11, 2012 at 08:52, Christoph Berg <cb@df7cb.de> wrote:
>> Also - putting it in debian backports doesn't solve the Ubuntu
>> problem, does it? We currently have Martin's PPA for that, but
>> wouldn't it be Kind Of Neat (TM) to have a single solution for both
>> these scenarios?
>
> If it works for Debian backports, the same should probably work for
> the Ubuntu backports. (The problem there might be that they have
> different minimum versions, so while squeeze-backports might support
> 8.4-and-up, Ubuntu backports might only have 9.0-and-up.)

I wasn't aware backports.debian.org could be used for ubuntu :-) My
mistake in that case...

Also - neither one of those two is good enough of course, since
PostgreSQL 8.3 is still supported...


> Generally, yes, Ubuntu should be supported by whatever solution. I
> haven't created any Ubuntu distributions on pgapt.debian.net yet
> because there's already enough targets for now.

Right - it should be on the long term plan.


>> And for the record, I don't really like the concept of PPAs for this.
>> Not necessarily from a technical perspective, that works just fine.
>> But it's really annoying to have to explain to "enterprise" customers
>> that "yes, using a *personal* package archive is the proper way to get
>> your fully supported version".
>
> Nod. What we could do is to use PPAs to get the buildd part covered,
> and then pull packages from there to the "official" archive.

That we could certainly do: There are two steps to whatever we do -
one is the build, another is the distribution.


>> > Currently only me, but so far no one else has asked for access. (I
>> > didn't ask very hard, though.)
>>
>> Let's make sure the process is documented enough that it's easy to
>> scale, eh? ;) bus-factors are bad...
>
> I'll try to post some of my thoughts on the whole process here.

Maybe set up a wiki page? Or if there is want/need, I can set up a
project at redmine.postgresql.org (i know the RPM builders have their
own trac instance with tickets and such things, though not as much
documentation as one would want there either).


> Re: Peter Eisentraut 2012-04-14 <1334399550.9019.37.camel@vanquo.pezone.net>
>> On ons, 2012-04-11 at 10:14 +0200, Magnus Hagander wrote:
>> > Could we in theory have our own buildds if we run this elsewhere? I
>> > know very little about buildds, so I wouldn't know. And it might be
>> > doable but just too much work - so please inform me :-)
>>
>> I don't think having autobuilders is really a priority for this.  As
>> long as we only support i386 and amd64, you might as well have someone
>> fill in the missing builds manually.  Having a full autobuilder
>> infrastructure is likely to be more work than that.
>
> From my POV, they are essential. There's so many targets to cover that
> so far most of the time I spent on the project was just building
> packages: For the server packages, that's 9.1/9.0/...,
> sid/squeeze/lenny, amd64/i386 (add Debian/Ubuntu later), for the
> modules packages that's sid/squeeze/lenny, amd64/i386 for every module
> included.

I think once you start adding modules it does become more of a
necessity, yes. For the main server, not as bad, but sure, it
certainly doesn't *hurt* there either.


>> With that in mind, I am somewhat doubtful about the integration with
>> backports.d.o.  It sounds nice in general, but the goals and constraints
>> of either project are not exactly aligned, so this will lead to
>> permanent conflict.
>
> Agreed. I still think that the support for PG versions on
> backports.d.o should become better, but the necessary changes will be
> the same as for pgapt.d.n.

Well, if we move the responsibility for maintaining it to
postgresql.org instead of backports.org (ignore the domain names at
this point, I'm talkinga bout the organisations), that will make it
easier in the long run to always adhere to the PostgreSQL support
policy - which covers more than the backports one. If we do have a
proper working and fully supported pg repository there, is there any
point to keep postgres in debian backports *at all*? Well, they can be
kept there of course, but is there ever any reason to recommend it?


>> I think taking the current reprepro-based architecture that Christoph
>> has already running is just fine (modulo some details, such as source
>> packages missing).  We just need to give it a permanent home, so people
>> can start using it.
>
> The missing source packages should be a thing of the past, I only did
> that for builds where the only difference to some other version was a
> new changelog entry and rebuilding the package.
>
> For the permanent home, I first like to get it more in shape.
> Imho, pgapt.debian.net is fine for the moment.

If it's not part of a firm, long-term plan, I'm afraid it isn't.
Larger customers need to *know* that things aren't going to change
again...


> Re: Magnus Hagander 2012-04-14 <CABUevEzxkg4gOhZaBqUqm4aEWxeE9+r=pYS_ces_=3eWvLvfGg@mail.gmail.com>
>> Having an automatic build environment is of course useful anyway. But
>> there are a lot of levels between a full infrastructure to do all of
>> it, and just a couple of scripts...
>
> The messy part is figuring out what to actually build. Building that
> is then just a script.
>
> I do have some semi-ready sql queries for that which I should either
> finish or explain to someone else what I think they should do.

Or both :-)


>> > With that in mind, I am somewhat doubtful about the integration with
>> > backports.d.o.  It sounds nice in general, but the goals and constraints
>> > of either project are not exactly aligned, so this will lead to
>> > permanent conflict.
>>
>> That's what I'm worried about as well. It got very clear to me with
>> the decision to drop 9.0 from backports (regardless of what happens
>> with that longterm) just shows that the whole backports project is not
>> designed to deal with what at least my goal for this is - which is
>> providing a stable platform across combinations of postgresql and
>> debian versions, making it possible to move incrementally and
>> independently between versions depending on external requirements, not
>> on OS requirements.
>
> 9.0 is still present on backports.debian.org. Though it will probably
> require a written policy somewhere to make it stay there.

Yes. It is. But there is a written statement today saying *it will go away*.

I know it's there, but it's not something anybody can rely on.

>> FWIW, if we want the repos themselves to run on the postgresql
>> infrastructure, we have resources to deploy that really quickly. The
>> yum repository was moved to the main infrastructure a few months ago.
>> However, we do not currently have resources to host *build nodes*.
>> Devrim has a build box that EDB donated that's used to build the RPMs
>> on a multitude of differnet virtual machines - it's quite possible
>> that this machine could be used to build debian stuff as well, since
>> it's just a set of xen (I think, could be kvm) virtual machines after
>> all...
>
> I should be able to arrange build nodes. Putting the repos on
> *.postgresql.org will be nice. But first let's get it really running.

Sure.

Just to be clear - what's actually needed to run that? A simple http
server is all, right? And then Some Way (TM) of getting the packages
onto it, like rsync or just scp?

(FWIW, the infrastructure currently runs on squeeze, so if debian
specifics are necessary, that can certainly be dealt with)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On sön, 2012-04-15 at 10:38 +0200, Christoph Berg wrote:
> I should be able to arrange build nodes. Putting the repos on
> *.postgresql.org will be nice. But first let's get it really running.

Do you have some kind of task or todo list?

Re: Magnus Hagander 2012-04-16 <CABUevEzY6K_w8YjPgLAUjdqxYPCBZJWGhBBtRS6R742zFCS_dw@mail.gmail.com>
> I wasn't aware backports.debian.org could be used for ubuntu :-) My
> mistake in that case...

No it can't, but there are backport repositories on ubuntu.com.

> Also - neither one of those two is good enough of course, since
> PostgreSQL 8.3 is still supported...

Right. (Though from a Debian perspective, having >= 8.4 supported
solves kind of 90% of the problem.)

> > I'll try to post some of my thoughts on the whole process here.
>
> Maybe set up a wiki page? Or if there is want/need, I can set up a
> project at redmine.postgresql.org (i know the RPM builders have their
> own trac instance with tickets and such things, though not as much
> documentation as one would want there either).

I've put a rough TODO list at
http://wiki.debian.org/pkg-postgresql/PgaptTodo

> Well, if we move the responsibility for maintaining it to
> postgresql.org instead of backports.org (ignore the domain names at
> this point, I'm talkinga bout the organisations), that will make it
> easier in the long run to always adhere to the PostgreSQL support
> policy - which covers more than the backports one. If we do have a

Nod.

> proper working and fully supported pg repository there, is there any
> point to keep postgres in debian backports *at all*? Well, they can be
> kept there of course, but is there ever any reason to recommend it?

Ideally, the packages would be the same. Completely dropping backports
will probably not work, as there might be other backports depending on
something from our packages, and that needs to be there so
(build-)dependencies work.

> >> I think taking the current reprepro-based architecture that Christoph
> >> has already running is just fine (modulo some details, such as source
> >> packages missing).  We just need to give it a permanent home, so people
> >> can start using it.
> >
> > The missing source packages should be a thing of the past, I only did
> > that for builds where the only difference to some other version was a
> > new changelog entry and rebuilding the package.
> >
> > For the permanent home, I first like to get it more in shape.
> > Imho, pgapt.debian.net is fine for the moment.
>
> If it's not part of a firm, long-term plan, I'm afraid it isn't.
> Larger customers need to *know* that things aren't going to change
> again...

Sure. Let's try to find a plan that will work :)

> > 9.0 is still present on backports.debian.org. Though it will probably
> > require a written policy somewhere to make it stay there.
>
> Yes. It is. But there is a written statement today saying *it will go away*.

Btw, where?

> Just to be clear - what's actually needed to run that? A simple http
> server is all, right? And then Some Way (TM) of getting the packages
> onto it, like rsync or just scp?

Exactly. The repository is driven by reprepro, this could either also
run on this host, or there could be a different pgapt-master machine
that hosts the master copy which then gets pushed to the public
mirror(s).

> (FWIW, the infrastructure currently runs on squeeze, so if debian
> specifics are necessary, that can certainly be dealt with)

reprepro is heavily using BDB files, I don't think there would be any
portability problems, but being on Debian is of course even easier.
(There's probably going to be some "real" database too for the
autobuild infrastructure, but this could be even another separate
host.)

Christoph
--
cb@df7cb.de | http://www.df7cb.de/

On Thu, Apr 19, 2012 at 20:12, Christoph Berg <cb@df7cb.de> wrote:
> Re: Magnus Hagander 2012-04-16 <CABUevEzY6K_w8YjPgLAUjdqxYPCBZJWGhBBtRS6R742zFCS_dw@mail.gmail.com>
>> I wasn't aware backports.debian.org could be used for ubuntu :-) My
>> mistake in that case...
>
> No it can't, but there are backport repositories on ubuntu.com.

Ah, right.


>> Also - neither one of those two is good enough of course, since
>> PostgreSQL 8.3 is still supported...
>
> Right. (Though from a Debian perspective, having >= 8.4 supported
> solves kind of 90% of the problem.)

Yes - that's one of the reasons why I think we need something that
works from a *postgresql* perspective rather than a debian or ubuntu
one.


>> > I'll try to post some of my thoughts on the whole process here.
>>
>> Maybe set up a wiki page? Or if there is want/need, I can set up a
>> project at redmine.postgresql.org (i know the RPM builders have their
>> own trac instance with tickets and such things, though not as much
>> documentation as one would want there either).
>
> I've put a rough TODO list at
> http://wiki.debian.org/pkg-postgresql/PgaptTodo

Just a few quick notes:
I assume bzr is required in order to work well with the debian
buildbots and such? If not, why use a different scm than all other
postgres projects? (If that is the reason, then it's a good one, of
course)


>> Well, if we move the responsibility for maintaining it to
>> postgresql.org instead of backports.org (ignore the domain names at
>> this point, I'm talkinga bout the organisations), that will make it
>> easier in the long run to always adhere to the PostgreSQL support
>> policy - which covers more than the backports one. If we do have a
>
> Nod.
>
>> proper working and fully supported pg repository there, is there any
>> point to keep postgres in debian backports *at all*? Well, they can be
>> kept there of course, but is there ever any reason to recommend it?
>
> Ideally, the packages would be the same. Completely dropping backports
> will probably not work, as there might be other backports depending on
> something from our packages, and that needs to be there so
> (build-)dependencies work.

Good point.


>> >> I think taking the current reprepro-based architecture that Christoph
>> >> has already running is just fine (modulo some details, such as source
>> >> packages missing).  We just need to give it a permanent home, so people
>> >> can start using it.
>> >
>> > The missing source packages should be a thing of the past, I only did
>> > that for builds where the only difference to some other version was a
>> > new changelog entry and rebuilding the package.
>> >
>> > For the permanent home, I first like to get it more in shape.
>> > Imho, pgapt.debian.net is fine for the moment.
>>
>> If it's not part of a firm, long-term plan, I'm afraid it isn't.
>> Larger customers need to *know* that things aren't going to change
>> again...
>
> Sure. Let's try to find a plan that will work :)

Yup!


>> > 9.0 is still present on backports.debian.org. Though it will probably
>> > require a written policy somewhere to make it stay there.
>>
>> Yes. It is. But there is a written statement today saying *it will go away*.
>
> Btw, where?

http://www.piware.de/2011/09/dropping-postgresql-9-0-packages-for-debianubuntubackports/


>> Just to be clear - what's actually needed to run that? A simple http
>> server is all, right? And then Some Way (TM) of getting the packages
>> onto it, like rsync or just scp?
>
> Exactly. The repository is driven by reprepro, this could either also
> run on this host, or there could be a different pgapt-master machine
> that hosts the master copy which then gets pushed to the public
> mirror(s).

Ok. That's something we can trivially put up there.


>> (FWIW, the infrastructure currently runs on squeeze, so if debian
>> specifics are necessary, that can certainly be dealt with)
>
> reprepro is heavily using BDB files, I don't think there would be any
> portability problems, but being on Debian is of course even easier.
> (There's probably going to be some "real" database too for the
> autobuild infrastructure, but this could be even another separate
> host.)

Right. And for some strange reason, we have this "postgreeeee" thing
running on our infrastructure boxes, and it seems to run pretty well..
;)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Magnus Hagander 2012-04-20 <CABUevEyzAZHwxNzBaUn-MwD5WXRwh1iGHXD4xv=HS2Vmt8iR1Q@mail.gmail.com>
> Just a few quick notes:
> I assume bzr is required in order to work well with the debian
> buildbots and such? If not, why use a different scm than all other
> postgres projects? (If that is the reason, then it's a good one, of
> course)

That's because Martin is maintaining the packages using bzr. Using a
different VCS would just make pulling updates harder.

https://code.launchpad.net/~pitti/postgresql/

The repositories just contain a debian/ directory that will work on
top of the postgresql source tree.

(And because .git != .bzr you can even do that with git clones instead
of with tarballs. Two VCSes in one directory!)

Christoph
--
cb@df7cb.de | http://www.df7cb.de/

On Fri, Apr 20, 2012 at 17:27, Christoph Berg <cb@df7cb.de> wrote:
> Re: Magnus Hagander 2012-04-20 <CABUevEyzAZHwxNzBaUn-MwD5WXRwh1iGHXD4xv=HS2Vmt8iR1Q@mail.gmail.com>
>> Just a few quick notes:
>> I assume bzr is required in order to work well with the debian
>> buildbots and such? If not, why use a different scm than all other
>> postgres projects? (If that is the reason, then it's a good one, of
>> course)
>
> That's because Martin is maintaining the packages using bzr. Using a
> different VCS would just make pulling updates harder.

Ah. Well, there's the reason I was looking for - that makes perfect
sense, and it's more important to have the same one there than with
the main project.


> https://code.launchpad.net/~pitti/postgresql/
>
> The repositories just contain a debian/ directory that will work on
> top of the postgresql source tree.
>
> (And because .git != .bzr you can even do that with git clones instead
> of with tarballs. Two VCSes in one directory!)

Right.

So, I remain at two questions ;)

1) can we break down the TODO list into smaller pieces somehow? With
more direct listings of what needs to be done, so we can somehow
estimate when it can be done?

2) Do we want the project management somewhere more than a wiki page?
(and if we do want a wiki page, should we consider the pg wiki which
probably more people involved will have accounts at?). Much as I hate
it as a tool, if upstream is on launchpad, is there a point to using
that? Or as I offered before, a separate redmine setup on pg.org. Or
anything like that. I dont' really care about which one, but a plain
wiki page seems a bit clunky to me (says the guy representing a
project that refuses to have an issue tracker, but let's not get into
that discussion)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 04/16/2012 06:48 AM, Magnus Hagander wrote:
> On Sun, Apr 15, 2012 at 10:38, Christoph Berg<cb@df7cb.de>  wrote:
>> For the permanent home, I first like to get it more in shape.
>> Imho, pgapt.debian.net is fine for the moment.
>
> If it's not part of a firm, long-term plan, I'm afraid it isn't.
> Larger customers need to *know* that things aren't going to change
> again...

I expect demand for this to heat up enough this summer to pull Dimitri
back into helping with this again; maybe some other people too.  I hope
that pulls enough bodies in to nail down something permanent.

The business case that I expect will fund some of this is backporting
the extensions added/improved significantly in 9.2, so they're easier to
install on 9.1.  In core extensions and PGXN are all nice, but there's a
chunk of the market that will want their extensions installed via OS
packages instead.  And that problem crosses over heavily with this one.

--
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Hi all,

I somehow failed to see activity here and just read the whole thread,
sorry for jumping that late into it.

Greg Smith <greg@2ndQuadrant.com> writes:
> On 04/16/2012 06:48 AM, Magnus Hagander wrote:
>> On Sun, Apr 15, 2012 at 10:38, Christoph Berg<cb@df7cb.de>  wrote:
>>> For the permanent home, I first like to get it more in shape.
>>> Imho, pgapt.debian.net is fine for the moment.
>>
>> If it's not part of a firm, long-term plan, I'm afraid it isn't.
>> Larger customers need to *know* that things aren't going to change
>> again...

+1

I vote for creating and hosting the files mainly at apt.postgresql.org
and having pgapt.debian.net either our developer's site or a mirror.

> I expect demand for this to heat up enough this summer to pull Dimitri back
> into helping with this again; maybe some other people too.  I hope that
> pulls enough bodies in to nail down something permanent.

It seems that we are mainly missing two things now:

 - a complete build infrastructure with machines, chroots or VMs,
   scripts to run all the process

 - a team with enough time to actually drive the builds when that's
   necessary

Publishing the binaries seems only too well covered as we have already
two servers at two locations that want to do that.

I'm going to see how far I can go on my side to provide for more time
and better organization here, maybe answering (at least in part) those
two items.

> The business case that I expect will fund some of this is backporting the
> extensions added/improved significantly in 9.2, so they're easier to install
> on 9.1.  In core extensions and PGXN are all nice, but there's a chunk of
> the market that will want their extensions installed via OS packages
> instead.  And that problem crosses over heavily with this one.

Yeah, the real deal is OS level packaging + CREATE EXTENSION. Anything
less is not production grade. Come on we're debian users here, let's not
be shy.

Regards,
--
Dimitri Fontaine                                        06 63 07 10 78
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

On Wed, May 2, 2012 at 9:37 PM, Dimitri Fontaine <dimitri@2ndquadrant.fr> wrote:
> Hi all,
>
> I somehow failed to see activity here and just read the whole thread,
> sorry for jumping that late into it.

No worries, glad to have you here :D


> Greg Smith <greg@2ndQuadrant.com> writes:
>> On 04/16/2012 06:48 AM, Magnus Hagander wrote:
>>> On Sun, Apr 15, 2012 at 10:38, Christoph Berg<cb@df7cb.de>  wrote:
>>>> For the permanent home, I first like to get it more in shape.
>>>> Imho, pgapt.debian.net is fine for the moment.
>>>
>>> If it's not part of a firm, long-term plan, I'm afraid it isn't.
>>> Larger customers need to *know* that things aren't going to change
>>> again...
>
> +1
>
> I vote for creating and hosting the files mainly at apt.postgresql.org
> and having pgapt.debian.net either our developer's site or a mirror.

I think that sounds reasonable.

I'm ready to deploy a box for apt.postgresql.org whenever we're ready
to make some sort of move on it. Perhaps we should do it pre-emptively
and just not add the DNS alias for it yet :D

If so, do we have *any* estimate on amount of disk space? We can
expand as we need, but we need to make sure we put it on a box with
ample host disk space in the first place...


>> I expect demand for this to heat up enough this summer to pull Dimitri back
>> into helping with this again; maybe some other people too.  I hope that
>> pulls enough bodies in to nail down something permanent.
>
> It seems that we are mainly missing two things now:
>
>  - a complete build infrastructure with machines, chroots or VMs,
>   scripts to run all the process

Yeah.


>  - a team with enough time to actually drive the builds when that's
>   necessary

And a team to build said infrastructure ;)


> Publishing the binaries seems only too well covered as we have already
> two servers at two locations that want to do that.
>
> I'm going to see how far I can go on my side to provide for more time
> and better organization here, maybe answering (at least in part) those
> two items.

If we can put together a well broken-down list of what we need done,
and a basic location for diong those things, I can contribute to do
some of the work on that as well. I'm no expert in debian packaging,
though I've done it a fair bit for simple stuff, but I can certainly
help to chew away at small items. It's also something that I can do
"during work hours", since we have numerous customers using these
systems that would be very happy to see this resolved...


>> The business case that I expect will fund some of this is backporting the
>> extensions added/improved significantly in 9.2, so they're easier to install
>> on 9.1.  In core extensions and PGXN are all nice, but there's a chunk of
>> the market that will want their extensions installed via OS packages
>> instead.  And that problem crosses over heavily with this one.
>
> Yeah, the real deal is OS level packaging + CREATE EXTENSION. Anything
> less is not production grade. Come on we're debian users here, let's not
> be shy.

Indeed.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus@hagander.net> writes:
> I'm ready to deploy a box for apt.postgresql.org whenever we're ready
> to make some sort of move on it. Perhaps we should do it pre-emptively
> and just not add the DNS alias for it yet :D

Yeah, I think just do it is the right way forward here. Christoph, are
you ok with that URL having a mirror of your current distribution,
basically so that you can break pgapt anytime you want knowing that real
users are using apt.postgresql.org? :)

> If so, do we have *any* estimate on amount of disk space? We can
> expand as we need, but we need to make sure we put it on a box with
> ample host disk space in the first place...

Christoph? :)

>>  - a complete build infrastructure with machines, chroots or VMs,
>>   scripts to run all the process
>
> Yeah.
>
>
>>  - a team with enough time to actually drive the builds when that's
>>   necessary
>
> And a team to build said infrastructure ;)

I see you're volunteering. I know Christoph has some hand prepared VMs
to do the builds on, and I've been trying to automate using either
chroots or pbuilder images but failed both way. I didn't have as much
time as I wanted to though, so I guess it's not a dead-end.

> If we can put together a well broken-down list of what we need done,
> and a basic location for diong those things, I can contribute to do
> some of the work on that as well. I'm no expert in debian packaging,
> though I've done it a fair bit for simple stuff, but I can certainly
> help to chew away at small items. It's also something that I can do
> "during work hours", since we have numerous customers using these
> systems that would be very happy to see this resolved...

See if you can script the build setup maintenance, that needs to be able
to:

 - clone and review https://github.com/dimitri/apt.postgresql.org
 - note that you're register as a contributor there now
 - have build envs for lenny, squeeze, testing, sid
 - add in ubuntu releases such as natty, oeniric and following
 - automate building of all PG packages from Christoph in there

When you're able to build the VM/chroot/whatever then build the 5
PostgreSQL versions we want in there, then it's getting interesting.

 - automate build of all PG extensions in each environment
 - move the packages into a public testing place

Then maybe we should have some basic testing, but that's a later step.

I'd like for Christoph to comment on those items before to add them at
any wiki like reference place.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

On Thu, May 3, 2012 at 5:11 PM, Dimitri Fontaine <dimitri@2ndquadrant.fr> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> I'm ready to deploy a box for apt.postgresql.org whenever we're ready
>> to make some sort of move on it. Perhaps we should do it pre-emptively
>> and just not add the DNS alias for it yet :D
>
> Yeah, I think just do it is the right way forward here. Christoph, are
> you ok with that URL having a mirror of your current distribution,
> basically so that you can break pgapt anytime you want knowing that real
> users are using apt.postgresql.org? :)

Ok, I'll start the process for getting this box up and running.


>> If so, do we have *any* estimate on amount of disk space? We can
>> expand as we need, but we need to make sure we put it on a box with
>> ample host disk space in the first place...
>
> Christoph? :)

yes, Christoph? ;)


>>>  - a complete build infrastructure with machines, chroots or VMs,
>>>   scripts to run all the process
>>
>> Yeah.
>>
>>
>>>  - a team with enough time to actually drive the builds when that's
>>>   necessary
>>
>> And a team to build said infrastructure ;)
>
> I see you're volunteering. I know Christoph has some hand prepared VMs
> to do the builds on, and I've been trying to automate using either
> chroots or pbuilder images but failed both way. I didn't have as much
> time as I wanted to though, so I guess it's not a dead-end.

Well, I am volunteering to help, and to be part of the team :-) But
note that it still need to be a team :P

Is your github repo the authoritative repository?


>> If we can put together a well broken-down list of what we need done,
>> and a basic location for diong those things, I can contribute to do
>> some of the work on that as well. I'm no expert in debian packaging,
>> though I've done it a fair bit for simple stuff, but I can certainly
>> help to chew away at small items. It's also something that I can do
>> "during work hours", since we have numerous customers using these
>> systems that would be very happy to see this resolved...
>
> See if you can script the build setup maintenance, that needs to be able
> to:
>
>  - clone and review https://github.com/dimitri/apt.postgresql.org
>  - note that you're register as a contributor there now
>  - have build envs for lenny, squeeze, testing, sid
>  - add in ubuntu releases such as natty, oeniric and following
>  - automate building of all PG packages from Christoph in there
>
> When you're able to build the VM/chroot/whatever then build the 5
> PostgreSQL versions we want in there, then it's getting interesting.

:-)


>  - automate build of all PG extensions in each environment
>  - move the packages into a public testing place

I honestly think that those two can well happen in the other order.


> Then maybe we should have some basic testing, but that's a later step.
>
> I'd like for Christoph to comment on those items before to add them at
> any wiki like reference place.

+1


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Hi,

we are currently expecting our second child which means I will have lots of time off work, but of course there are other priorities then.

I've continued working on the SQL queries. They are kind of ready, though I had to leave just before committing things.

generally, I'd say having a host available would make things easier now. I'd propose to call it apt-master (or point that alias to the real host name) so we can have the public "apt" host elsewhere if needed.

The next few days will be very busy here, but I should be able to at least move the current setup there so other can start with it as well.

On Mon, May 7, 2012 at 11:33 AM, Christoph Berg <cb@df7cb.de> wrote:
> Hi,
>
> we are currently expecting our second child which means I will have lots of
> time off work, but of course there are other priorities then.

That is absolutely understood. Congratulations! :-)


> I've continued working on the SQL queries. They are kind of ready, though I
> had to leave just before committing things.
>
> generally, I'd say having a host available would make things easier now. I'd
> propose to call it apt-master (or point that alias to the real host name) so
> we can have the public "apt" host elsewhere if needed.

Well, the thing I've set wheels in motion are for the public apt repo :-)

Are you referring to a "build host"? If so, I can try to get that done
as well, but it won't run on the main infrastructure - but we can
probably co-exist with the RPM build machines. Do we have any kind of
rough specs on what we need for that? Is a simple Xen virtual machine
running squeeze enough, or do we have more specific requirements? (not
performance-wise, but software-wise, at this point)

> The next few days will be very busy here, but I should be able to at least
> move the current setup there so other can start with it as well.

Ok, cool.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

I meant the host where the repository is kept and where people upload new versions to and buildds new builds of existing source packages. Requirements for that are reprepro installed, a 9.1 PG database, and some way to upload stuff. That's most likely just ssh (and maybe rsync).

The web frontend is mostly plain files served, plus some CGI scripts (DBD::Pg). In case these are separate hosts, the files part would get rsynced over.

Buildds would need to talk to the DB using the queries I was mentioning (probably via a CGI) to figure out what needs building.

I think putting the repository part in place would be possible quite easily. We would then start by manually building things, and set up buildds as we like.

Btw, I'm writing this from the hospital - mother and new boy are well and we are very happy :-)

Christoph Berg <cb@df7cb.de> writes:
> Btw, I'm writing this from the hospital - mother and new boy are well
> and we are very happy :-)

Congrats, enjoy those very unique moments :)

--
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

On Mon, May 7, 2012 at 8:13 PM, Christoph Berg <cb@df7cb.de> wrote:
> I meant the host where the repository is kept and where people upload new
> versions to and buildds new builds of existing source packages. Requirements
> for that are reprepro installed, a 9.1 PG database, and some way to upload
> stuff. That's most likely just ssh (and maybe rsync).

So it's basically the repository server then? It doesn't *run* any
builds? That's the one I'm working on :-)


> The web frontend is mostly plain files served, plus some CGI scripts
> (DBD::Pg). In case these are separate hosts, the files part would get
> rsynced over.

Check.


> Buildds would need to talk to the DB using the queries I was mentioning
> (probably via a CGI) to figure out what needs building.

All google hits I get for "buildds" are misspellings, so I assumed
above that yours was too. Or is it the same as "buildd"? Do you have
some actual reference to some info for me, since google fails me?

Because the machine I have in mind does *not* have the resources to
actually build anything, we need to do that somewhere else...

> Btw, I'm writing this from the hospital - mother and new boy are well and we
> are very happy :-)

\o/

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

“buildds“ was just the plural of "buildd", sorry if that was confusing.

The master host wouldn't run any builds itself, or anything CPU-intensive except for some gpg signatures. I/O is also very low for the backend, plus http traffic.

(Still in hospital.)

On Tue, May 8, 2012 at 7:32 PM, Christoph Berg <cb@df7cb.de> wrote:
> “buildds“ was just the plural of "buildd", sorry if that was confusing.

Ah. It was for me - probably because I'm not too experienced in deb-building :-)


> The master host wouldn't run any builds itself, or anything CPU-intensive
> except for some gpg signatures. I/O is also very low for the backend, plus
> http traffic.

Ok. Then it matches, and I shall bug my fellow sysadmins some more
about actually responding to my questions :-)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/