Обсуждение: Downtime website

Поиск
Список
Период
Сортировка

Downtime website

От
Magnus Hagander
Дата:
We're going to be moving some of the services on www.postgresql.org to
a different host starting in a few minutes. Much of the website will
keep working during this time, but some will  be unavailable, such as
logging in.

We expect the downtime to be fairly short, but as will all such
things, one can never be entirely sure :) I'll make another post here
when we're done.

//Magnus



Re: Downtime website

От
Magnus Hagander
Дата:
On Fri, Mar 20, 2020 at 1:32 PM Magnus Hagander <magnus@hagander.net> wrote:
>
> We're going to be moving some of the services on www.postgresql.org to
> a different host starting in a few minutes. Much of the website will
> keep working during this time, but some will  be unavailable, such as
> logging in.
>
> We expect the downtime to be fairly short, but as will all such
> things, one can never be entirely sure :) I'll make another post here
> when we're done.

We should now be back up and running with hopefully no remaining issues.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Re: Downtime website

От
Julien Rouhaud
Дата:
Hi Magnus,

On Fri, Mar 20, 2020 at 1:54 PM Magnus Hagander <magnus@hagander.net> wrote:
>
> On Fri, Mar 20, 2020 at 1:32 PM Magnus Hagander <magnus@hagander.net> wrote:
> >
> > We're going to be moving some of the services on www.postgresql.org to
> > a different host starting in a few minutes. Much of the website will
> > keep working during this time, but some will  be unavailable, such as
> > logging in.
> >
> > We expect the downtime to be fairly short, but as will all such
> > things, one can never be entirely sure :) I'll make another post here
> > when we're done.
>
> We should now be back up and running with hopefully no remaining issues.

It seems that the search engine is broken since the update.  E.g.:

https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test

results in:

Error 503 Backend fetch failed

Backend fetch failed

Guru Meditation:

XID: 71244669

________________________________

Varnish cache server



Re: Downtime website

От
Magnus Hagander
Дата:
On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> Hi Magnus,
>
> On Fri, Mar 20, 2020 at 1:54 PM Magnus Hagander <magnus@hagander.net> wrote:
> >
> > On Fri, Mar 20, 2020 at 1:32 PM Magnus Hagander <magnus@hagander.net> wrote:
> > >
> > > We're going to be moving some of the services on www.postgresql.org to
> > > a different host starting in a few minutes. Much of the website will
> > > keep working during this time, but some will  be unavailable, such as
> > > logging in.
> > >
> > > We expect the downtime to be fairly short, but as will all such
> > > things, one can never be entirely sure :) I'll make another post here
> > > when we're done.
> >
> > We should now be back up and running with hopefully no remaining issues.
>
> It seems that the search engine is broken since the update.  E.g.:
>
> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
>
> results in:

Hi!

Argh, of course there was one more IP based access list that I missed :)

Thanks for the report, fixed now!

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Re: Downtime website

От
Julien Rouhaud
Дата:
On Fri, Mar 20, 2020 at 4:24 PM Magnus Hagander <magnus@hagander.net> wrote:
>
> On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
> >
> > Hi Magnus,
> >
> > On Fri, Mar 20, 2020 at 1:54 PM Magnus Hagander <magnus@hagander.net> wrote:
> > >
> > > On Fri, Mar 20, 2020 at 1:32 PM Magnus Hagander <magnus@hagander.net> wrote:
> > > >
> > > > We're going to be moving some of the services on www.postgresql.org to
> > > > a different host starting in a few minutes. Much of the website will
> > > > keep working during this time, but some will  be unavailable, such as
> > > > logging in.
> > > >
> > > > We expect the downtime to be fairly short, but as will all such
> > > > things, one can never be entirely sure :) I'll make another post here
> > > > when we're done.
> > >
> > > We should now be back up and running with hopefully no remaining issues.
> >
> > It seems that the search engine is broken since the update.  E.g.:
> >
> > https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
> >
> > results in:
>
> Hi!
>
> Argh, of course there was one more IP based access list that I missed :)
>
> Thanks for the report, fixed now!

Thanks!



Re: Downtime website

От
Tom Lane
Дата:
Magnus Hagander <magnus@hagander.net> writes:
> On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>> It seems that the search engine is broken since the update.  E.g.:
>> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test

> Argh, of course there was one more IP based access list that I missed :)
> Thanks for the report, fixed now!

It's still broken for me --- I'm seeing

"Error talking to search server: Forbidden"

            regards, tom lane



Re: Downtime website

От
"Jonathan S. Katz"
Дата:
On 3/20/20 4:21 PM, Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>>> It seems that the search engine is broken since the update.  E.g.:
>>> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
>
>> Argh, of course there was one more IP based access list that I missed :)
>> Thanks for the report, fixed now!
>
> It's still broken for me --- I'm seeing
>
> "Error talking to search server: Forbidden"

I also just hit one viewing a file on git.postgresql.org, though upon
refreshing it worked fine.

Jonathan


Вложения

Re: Downtime website

От
Tom Lane
Дата:
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 3/20/20 4:21 PM, Tom Lane wrote:
>> Magnus Hagander <magnus@hagander.net> writes:
>>> Argh, of course there was one more IP based access list that I missed :)
>>> Thanks for the report, fixed now!

>> It's still broken for me --- I'm seeing
>> "Error talking to search server: Forbidden"

> I also just hit one viewing a file on git.postgresql.org, though upon
> refreshing it worked fine.

Further to this --- it appears that the cfbot is wedged; I can see
no evidence that it's run any tests in the last day or so.  Given
the timing, I'm suspicious that it's also a casualty of those
changes you made.

            regards, tom lane



Re: Downtime website

От
Thomas Munro
Дата:
On Sun, Mar 22, 2020 at 7:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> "Jonathan S. Katz" <jkatz@postgresql.org> writes:
> > On 3/20/20 4:21 PM, Tom Lane wrote:
> >> Magnus Hagander <magnus@hagander.net> writes:
> >>> Argh, of course there was one more IP based access list that I missed :)
> >>> Thanks for the report, fixed now!
>
> >> It's still broken for me --- I'm seeing
> >> "Error talking to search server: Forbidden"
>
> > I also just hit one viewing a file on git.postgresql.org, though upon
> > refreshing it worked fine.
>
> Further to this --- it appears that the cfbot is wedged; I can see
> no evidence that it's run any tests in the last day or so.  Given
> the timing, I'm suspicious that it's also a casualty of those
> changes you made.

Oops, cfbot is back.  It should now catch up with recently posted
patches in roughly the order they were posted over the next couple of
hours.  It got stuck for its own reasons (apparently the temporary
clone filesystem it uses for applying patches got into a weird state
that hung, will look into that but in any case, after this CF is over
I will be switching to a new version that doesn't use that).



Re: Downtime website

От
Magnus Hagander
Дата:
On Fri, Mar 20, 2020 at 9:21 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Magnus Hagander <magnus@hagander.net> writes:
> > On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
> >> It seems that the search engine is broken since the update.  E.g.:
> >> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
>
> > Argh, of course there was one more IP based access list that I missed :)
> > Thanks for the report, fixed now!
>
> It's still broken for me --- I'm seeing
>
> "Error talking to search server: Forbidden"

Ah, I believe ilmari on irc just pointed out the magic keyword for me
-- list archives searches didn't work. And of course, I had only
tested regular web searches.

*sigh*. Sorry about that, fixed now.

//Magnus



Re: Downtime website

От
Magnus Hagander
Дата:
On Fri, Mar 20, 2020 at 9:28 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>
> On 3/20/20 4:21 PM, Tom Lane wrote:
> > Magnus Hagander <magnus@hagander.net> writes:
> >> On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
> >>> It seems that the search engine is broken since the update.  E.g.:
> >>> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
> >
> >> Argh, of course there was one more IP based access list that I missed :)
> >> Thanks for the report, fixed now!
> >
> > It's still broken for me --- I'm seeing
> >
> > "Error talking to search server: Forbidden"
>
> I also just hit one viewing a file on git.postgresql.org, though upon
> refreshing it worked fine.

Um. Surely you didn't get "Error talking to search server" on the git
server?! It should never do that..

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Re: Downtime website

От
Magnus Hagander
Дата:
On Sat, Mar 21, 2020 at 8:58 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Sun, Mar 22, 2020 at 7:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > "Jonathan S. Katz" <jkatz@postgresql.org> writes:
> > > On 3/20/20 4:21 PM, Tom Lane wrote:
> > >> Magnus Hagander <magnus@hagander.net> writes:
> > >>> Argh, of course there was one more IP based access list that I missed :)
> > >>> Thanks for the report, fixed now!
> >
> > >> It's still broken for me --- I'm seeing
> > >> "Error talking to search server: Forbidden"
> >
> > > I also just hit one viewing a file on git.postgresql.org, though upon
> > > refreshing it worked fine.
> >
> > Further to this --- it appears that the cfbot is wedged; I can see
> > no evidence that it's run any tests in the last day or so.  Given
> > the timing, I'm suspicious that it's also a casualty of those
> > changes you made.
>
> Oops, cfbot is back.  It should now catch up with recently posted
> patches in roughly the order they were posted over the next couple of
> hours.  It got stuck for its own reasons (apparently the temporary
> clone filesystem it uses for applying patches got into a weird state
> that hung, will look into that but in any case, after this CF is over
> I will be switching to a new version that doesn't use that).

Yeah, I don't see how any of this would've had an effect onthe CF bot,
given it doesn't (yet) use any of the internal APIs. It only talks to
the CF site (without logging in) and the list archives.

//Magnus



Re: Downtime website

От
Tom Lane
Дата:
Magnus Hagander <magnus@hagander.net> writes:
> Yeah, I don't see how any of this would've had an effect onthe CF bot,
> given it doesn't (yet) use any of the internal APIs. It only talks to
> the CF site (without logging in) and the list archives.

I suspected that the same problem that was breaking list archive
search might've affected the CF bot's archive access.  Evidently
that guess was wrong, but I don't think it was unreasonable.

            regards, tom lane



Re: Downtime website

От
Magnus Hagander
Дата:
On Sat, Mar 21, 2020 at 10:02 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Magnus Hagander <magnus@hagander.net> writes:
> > Yeah, I don't see how any of this would've had an effect onthe CF bot,
> > given it doesn't (yet) use any of the internal APIs. It only talks to
> > the CF site (without logging in) and the list archives.
>
> I suspected that the same problem that was breaking list archive
> search might've affected the CF bot's archive access.  Evidently
> that guess was wrong, but I don't think it was unreasonable.

Nope, not at all. That comment was mainly in direction of Thomas, who
does know the points where the cf bot "plugs in".


-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Re: Downtime website

От
"Jonathan S. Katz"
Дата:
On Mar 21, 2020, at 4:54 PM, Magnus Hagander <magnus@hagander.net> wrote:
>
> On Fri, Mar 20, 2020 at 9:28 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>>
>>> On 3/20/20 4:21 PM, Tom Lane wrote:
>>> Magnus Hagander <magnus@hagander.net> writes:
>>>> On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>>>>> It seems that the search engine is broken since the update.  E.g.:
>>>>> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
>>>
>>>> Argh, of course there was one more IP based access list that I missed :)
>>>> Thanks for the report, fixed now!
>>>
>>> It's still broken for me --- I'm seeing
>>>
>>> "Error talking to search server: Forbidden"
>>
>> I also just hit one viewing a file on git.postgresql.org, though upon
>> refreshing it worked fine.
>
> Um. Surely you didn't get "Error talking to search server" on the git
> server?! It should never do that..

It was from clicking on a blob view of a file. I got a backend fetch error. I didn’t think
to grab the URL and it worked on the next request.

Jonathan



Re: Downtime website

От
Magnus Hagander
Дата:
On Sun, Mar 22, 2020 at 12:59 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>
> On Mar 21, 2020, at 4:54 PM, Magnus Hagander <magnus@hagander.net> wrote:
> >
> > On Fri, Mar 20, 2020 at 9:28 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
> >>
> >>> On 3/20/20 4:21 PM, Tom Lane wrote:
> >>> Magnus Hagander <magnus@hagander.net> writes:
> >>>> On Fri, Mar 20, 2020 at 3:47 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
> >>>>> It seems that the search engine is broken since the update.  E.g.:
> >>>>> https://www.postgresql.org/search/?u=%2Fdocs%2F12%2F&q=test
> >>>
> >>>> Argh, of course there was one more IP based access list that I missed :)
> >>>> Thanks for the report, fixed now!
> >>>
> >>> It's still broken for me --- I'm seeing
> >>>
> >>> "Error talking to search server: Forbidden"
> >>
> >> I also just hit one viewing a file on git.postgresql.org, though upon
> >> refreshing it worked fine.
> >
> > Um. Surely you didn't get "Error talking to search server" on the git
> > server?! It should never do that..
>
> It was from clicking on a blob view of a file. I got a backend fetch error. I didn’t think
> to grab the URL and it worked on the next request.

Ah yeah, that's a totally different error, and lot more expected
there. That is basically "gitweb is slow and overloaded" in different
words...

--
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/