Обсуждение: "ago" times on buildfarm status page
I find the time displays like
01:03 ago
on the buildfarm status page unhelpful.
First, I can never tell whether this is hours-minutes or minutes-seconds
-- there is probably a less ambiguous format available.
But more importantly, the page doesn't say when it was generated, so a
relative time like this is meaningless. The page might have most
recently reloaded last night. That means when I look at the page, I
*always* have to reload it first to make sense of the times.
I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off? Or preferably convert to local time. I can much easier
make sense of an absolute local time: I can compare that to the clock in
the corner of the screen, and I can compare that, say, to a commit
timestamp.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
I find the time displays like
01:03 ago
on the buildfarm status page unhelpful.
First, I can never tell whether this is hours-minutes or minutes-seconds
-- there is probably a less ambiguous format available.
But more importantly, the page doesn't say when it was generated, so a
relative time like this is meaningless. The page might have most
recently reloaded last night. That means when I look at the page, I
*always* have to reload it first to make sense of the times.
I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off? Or preferably convert to local time. I can much easier
make sense of an absolute local time: I can compare that to the clock in
the corner of the screen, and I can compare that, say, to a commit
timestamp.
It used to be that the "ago" part was generated on the server, but Andrew changed that to the fixed timestamp + javascript to improve cachability and thus performance. Perhaps now that it's that it could be as easy as adding a checkbox to the page (which could remember your preference in a cookie) that switches between the two modes?
Magnus Hagander <magnus@hagander.net> writes: > On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut < > peter.eisentraut@2ndquadrant.com> wrote: > >> I find the time displays like >> >> 01:03 ago >> >> on the buildfarm status page unhelpful. >> >> First, I can never tell whether this is hours-minutes or minutes-seconds >> -- there is probably a less ambiguous format available. >> >> But more importantly, the page doesn't say when it was generated, so a >> relative time like this is meaningless. The page might have most >> recently reloaded last night. That means when I look at the page, I >> *always* have to reload it first to make sense of the times. >> >> I notice that the page source actually includes absolute times that are >> then converted to relative using some JavaScript. Could we perhaps just >> turn that off? Or preferably convert to local time. I can much easier >> make sense of an absolute local time: I can compare that to the clock in >> the corner of the screen, and I can compare that, say, to a commit >> timestamp. >> > > It used to be that the "ago" part was generated on the server, but Andrew > changed that to the fixed timestamp + javascript to improve cachability and > thus performance. Perhaps now that it's that it could be as easy as adding > a checkbox to the page (which could remember your preference in a cookie) > that switches between the two modes? The Javscript could also be made to update the "ago" part every minute, and show the absoulte time as a tooltip, which is what pretty much every other website does. - ilmari -- "A disappointingly low fraction of the human race is, at any given time, on fire." - Stig Sandbeck Mathisen
Magnus Hagander <magnus@hagander.net> writes:
> On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut <
> peter.eisentraut@2ndquadrant.com> wrote:
>> I find the time displays like
>> 01:03 ago
>> on the buildfarm status page unhelpful.
>>
>> I notice that the page source actually includes absolute times that are
>> then converted to relative using some JavaScript. Could we perhaps just
>> turn that off?
> It used to be that the "ago" part was generated on the server, but Andrew
> changed that to the fixed timestamp + javascript to improve cachability and
> thus performance. Perhaps now that it's that it could be as easy as adding
> a checkbox to the page (which could remember your preference in a cookie)
> that switches between the two modes?
FWIW, I'm used to the way it's shown and would not like a change, so
if this can be made user-settable as Magnus suggests, that would be
better IMO.
The real problem with that column though is that it relies on run start
times that are self-reported by the buildfarm clients, and some of them
have system clocks that are many hours off reality. What *I'd* like to
see is for the column to contain time of receipt of the buildfarm report
at the server, less the measured runtime of the test.
regards, tom lane
On 8/21/19 8:32 AM, Dagfinn Ilmari Mannsåker wrote: > Magnus Hagander <magnus@hagander.net> writes: > >> On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut < >> peter.eisentraut@2ndquadrant.com> wrote: >> >>> I find the time displays like >>> >>> 01:03 ago >>> >>> on the buildfarm status page unhelpful. >>> >>> First, I can never tell whether this is hours-minutes or minutes-seconds >>> -- there is probably a less ambiguous format available. >>> >>> But more importantly, the page doesn't say when it was generated, so a >>> relative time like this is meaningless. The page might have most >>> recently reloaded last night. That means when I look at the page, I >>> *always* have to reload it first to make sense of the times. >>> >>> I notice that the page source actually includes absolute times that are >>> then converted to relative using some JavaScript. Could we perhaps just >>> turn that off? Or preferably convert to local time. I can much easier >>> make sense of an absolute local time: I can compare that to the clock in >>> the corner of the screen, and I can compare that, say, to a commit >>> timestamp. >>> >> It used to be that the "ago" part was generated on the server, but Andrew >> changed that to the fixed timestamp + javascript to improve cachability and >> thus performance. Perhaps now that it's that it could be as easy as adding >> a checkbox to the page (which could remember your preference in a cookie) >> that switches between the two modes? > The Javscript could also be made to update the "ago" part every minute, > and show the absoulte time as a tooltip, which is what pretty much every > other website does. > The code for the page is here: <https://github.com/PGBuildFarm/server-code/blob/master/templates/status.tt> Patches welcome. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 8/21/19 9:55 AM, Tom Lane wrote: > > The real problem with that column though is that it relies on run start > times that are self-reported by the buildfarm clients, and some of them > have system clocks that are many hours off reality. What *I'd* like to > see is for the column to contain time of receipt of the buildfarm report > at the server, less the measured runtime of the test. That might be possible. I'll put it on my list of things to do. It's not happening any time soon, though. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 8/21/19 3:40 AM, Peter Eisentraut wrote: > I find the time displays like > > 01:03 ago > > on the buildfarm status page unhelpful. > > First, I can never tell whether this is hours-minutes or minutes-seconds > -- there is probably a less ambiguous format available. This is hours:minutes. days are always explicitly prepended if greater than 0. > > But more importantly, the page doesn't say when it was generated, so a > relative time like this is meaningless. The page might have most > recently reloaded last night. That means when I look at the page, I > *always* have to reload it first to make sense of the times. > > I notice that the page source actually includes absolute times that are > then converted to relative using some JavaScript. Could we perhaps just > turn that off? Or preferably convert to local time. I can much easier > make sense of an absolute local time: I can compare that to the clock in > the corner of the screen, and I can compare that, say, to a commit > timestamp. > Maybe. TBH the whole buildfarm UI is rather dated and clunky. But it needs work from a web monkey, which I am really not. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 8/21/19 11:07 AM, Andrew Dunstan wrote: > On 8/21/19 9:55 AM, Tom Lane wrote: >> The real problem with that column though is that it relies on run start >> times that are self-reported by the buildfarm clients, and some of them >> have system clocks that are many hours off reality. What *I'd* like to >> see is for the column to contain time of receipt of the buildfarm report >> at the server, less the measured runtime of the test. > > That might be possible. I'll put it on my list of things to do. It's not > happening any time soon, though. > > What I have done quickly is to store a measure of the clock skew. We already calculated it but we didn't store it. Initial indications are that only a few have significant skew. Still, if we simply added the skew to the snapshot time that might be enough to achieve what you want. That would be a one line change, I think. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> What I have done quickly is to store a measure of the clock skew. We
> already calculated it but we didn't store it. Initial indications are
> that only a few have significant skew.
Oh, I didn't know that the server had the ability to measure that.
(Yes, I agree that there are just a couple with big skews at any
one time.)
> Still, if we simply added the skew to the snapshot time that might be
> enough to achieve what you want. That would be a one line change, I think.
+1
regards, tom lane
On 8/21/19 4:16 PM, Tom Lane wrote: > Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes: >> What I have done quickly is to store a measure of the clock skew. We >> already calculated it but we didn't store it. Initial indications are >> that only a few have significant skew. > Oh, I didn't know that the server had the ability to measure that. > > (Yes, I agree that there are just a couple with big skews at any > one time.) > >> Still, if we simply added the skew to the snapshot time that might be >> enough to achieve what you want. That would be a one line change, I think. > +1 > > Done. It's only happening prospectively, so we'll need to wait a few days to see it flow through. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 8/21/19 10:52 AM, Andrew Dunstan wrote: >> >> The Javscript could also be made to update the "ago" part every minute, >> and show the absoulte time as a tooltip, which is what pretty much every >> other website does. >> > > The code for the page is here: > <https://github.com/PGBuildFarm/server-code/blob/master/templates/status.tt> > > > Patches welcome. > > I have done both of these things. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 8/21/19 4:16 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>>> Still, if we simply added the skew to the snapshot time that might be
>>> enough to achieve what you want. That would be a one line change, I think.
>> +1
> Done. It's only happening prospectively, so we'll need to wait a few
> days to see it flow through.
Hm, doesn't seem to have done the trick. The current dashboard page shows
(in the v12 branch)
mule ... 01:17 ago OK [97205d0] Config
loach ... 01:32 ago OK [97205d0] Config
dangomushi ... 02:11 ago OK [97205d0] Config
bowerbird ... 02:23 ago scriptsCheck [97205d0] Details
snapper ... 02:48 ago OK [63fc3b1] Config
caiman ... 03:04 ago OK [97205d0] Config
nightjar ... 03:17 ago recoveryCheck [97205d0] Details
chub ... 03:29 ago OK [97205d0] Config
clam ... 03:34 ago OK [97205d0] Config
demoiselle ... 03:45 ago OK [97205d0] Config
snapper is clearly out of line here: the commit it claims
to have fetched 2:48 ago was obsoleted around seven hours ago.
(Snapper is one of the machines that is typically inconsistent
in this way. I've been assuming that's because its system clock
is a few hours off ... but maybe there's something else going on?)
regards, tom lane
On 8/26/19 2:55 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> On 8/21/19 4:16 PM, Tom Lane wrote:
>>> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>>>> Still, if we simply added the skew to the snapshot time that might be
>>>> enough to achieve what you want. That would be a one line change, I think.
>>> +1
>> Done. It's only happening prospectively, so we'll need to wait a few
>> days to see it flow through.
> Hm, doesn't seem to have done the trick. The current dashboard page shows
> (in the v12 branch)
>
> mule ... 01:17 ago OK [97205d0] Config
> loach ... 01:32 ago OK [97205d0] Config
> dangomushi ... 02:11 ago OK [97205d0] Config
> bowerbird ... 02:23 ago scriptsCheck [97205d0] Details
> snapper ... 02:48 ago OK [63fc3b1] Config
> caiman ... 03:04 ago OK [97205d0] Config
> nightjar ... 03:17 ago recoveryCheck [97205d0] Details
> chub ... 03:29 ago OK [97205d0] Config
> clam ... 03:34 ago OK [97205d0] Config
> demoiselle ... 03:45 ago OK [97205d0] Config
>
> snapper is clearly out of line here: the commit it claims
> to have fetched 2:48 ago was obsoleted around seven hours ago.
>
> (Snapper is one of the machines that is typically inconsistent
> in this way. I've been assuming that's because its system clock
> is a few hours off ... but maybe there's something else going on?)
>
>
I think this is the problem:
'scmrepo' => '/home/pgbf/pgmirror.git',
Probably this isn't updated often enough. It probably has little to do with the clock settings.
This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repo
asthe base would avoid these problems.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: > I think this is the problem: > > 'scmrepo' => '/home/pgbf/pgmirror.git', > > Probably this isn't updated often enough. It probably has little to do with the clock settings. > > This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repoas the base would avoid these problems. We've discussed this before (see below). The configuration is intentionally like that. I specifically configured skate and snapper to build the exact same source,where skate builds with default buildfarm settings, while snapper builds with the settings actually used by Debiansource packages. These animals were set up to avoid cases we had in the past where Debian source packages failed to build on sparc, even thoughbuild animals running on Debian sparc were building fine: https://www.postgresql.org/message-id/000001d2f0c2%24e2d335a0%24a879a0e0%24%40turelinckx.be With snapper building the exact same source as skate (as it is now), we have some point of reference if snapper fails butskate succeeds. I could configure snapper to perform an update of the repo before building, but then we give up this comparabilityin exchange for a bit more clarity regarding timestamps. Best regards, Tom Turelinckx On Thu, Nov 9, 2017, at 8:54 PM, Andrew Dunstan wrote: > > The first line of the log file is always something like this (see end of https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=nightjar&dt=2017-11-09%2018%3A59%3A01) > > Last file mtime in snapshot: Thu Nov 9 17:56:07 2017 GMT > This should be the time of the last commit in the snapshot. > > On Mon, Nov 6, 2017 at 9:49 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> >> Tom Turelinckx <pgbf@twiska.com> writes: >> >> > On Fri, Nov 3, 2017, at 09:42 PM, Tom Lane wrote: >> >> Either that, or it's fallen through a wormhole ;-), but the results >> >> it's posting seem to be mis-timestamped by several hours, which is >> >> confusing. Please set its clock correctly. Maybe spin up ntpd? >> >> > The clock is correct, but the configuration may be unusual. >> >> > In fact, snapper runs on the same machine as skate, and it's using ntp. >> > At 7 AM (Western Europe), a local git repo is updated. In the morning, >> > skate builds from that local repo with the default buildfarm >> > configuration that most animals use. In the afternoon, snapper builds >> > from that local repo with the exact same configuration, per branch, that >> > the Debian source packages from the pgdg repo use on the same platform. >> > The local repo is updated only once per day to ensure that snapper and >> > skate build the same source with different settings, and they share the >> > git mirror and build root, as suggested in the build farm howto. >> >> Hm. So the issue really is that the build timestamp that the buildfarm >> client is reporting tells when it pulled from the local repo, not when >> that repo was last updated from the community server. Not sure if there's >> any simple way to improve that ... Andrew, any thoughts?
On 8/27/19 4:33 AM, Tom Turelinckx wrote: > On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >> I think this is the problem: >> >> 'scmrepo' => '/home/pgbf/pgmirror.git', >> >> Probably this isn't updated often enough. It probably has little to do with the clock settings. >> >> This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repoas the base would avoid these problems. > We've discussed this before (see below). > > > Hm. So the issue really is that the build timestamp that the buildfarm > client is reporting tells when it pulled from the local repo, not when > that repo was last updated from the community server. Not sure if there's > any simple way to improve that ... Andrew, any thoughts? Maybe we need an option to use the git commit time. instead of the snapshot time. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 8/27/19 8:45 AM, Andrew Dunstan wrote: > On 8/27/19 4:33 AM, Tom Turelinckx wrote: >> On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >>> I think this is the problem: >>> >>> 'scmrepo' => '/home/pgbf/pgmirror.git', >>> >>> Probably this isn't updated often enough. It probably has little to do with the clock settings. >>> >>> This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repoas the base would avoid these problems. >> We've discussed this before (see below). >> >> >> Hm. So the issue really is that the build timestamp that the buildfarm >> client is reporting tells when it pulled from the local repo, not when >> that repo was last updated from the community server. Not sure if there's >> any simple way to improve that ... Andrew, any thoughts? > > > Maybe we need an option to use the git commit time. instead of the > snapshot time. > > Scratch that - we use this to calculate the duration of the first stage, so mangling it would just create another error. It's tempting to say we should sort the dashboard by git reference time then snapshot - that should be fairly doable. But what if there isn't a git reference, as happens when there's a git failure for example. In those cases Maybe just use the snapshot time? Storing the git timestanp would involve a table change in our second largest table, so the team would need to discuss and plan it. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
"Tom Turelinckx" <pgbf@twiska.com> writes:
> On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote:
>> I think this is the problem:
>> 'scmrepo' => '/home/pgbf/pgmirror.git',
>> Probably this isn't updated often enough. It probably has little to do with the clock settings.
> The configuration is intentionally like that. I specifically configured skate and snapper to build the exact same
source,where skate builds with default buildfarm settings, while snapper builds with the settings actually used by
Debiansource packages.
TBH, I don't find that particularly important ... especially not for HEAD
builds, where building a many-hours-old snapshot is pretty much in the
category of "why bother?". On the whole, I think building from the latest
available source is the most useful policy. If there's some platform-
or configuration-specific issue, it usually takes more than one build
cycle for us to notice it anyway, so that ensuring two animals have exactly
comparable builds at any given instant isn't very helpful.
regards, tom lane
On 8/27/19 10:15 AM, Tom Lane wrote: > "Tom Turelinckx" <pgbf@twiska.com> writes: >> On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >>> I think this is the problem: >>> 'scmrepo' => '/home/pgbf/pgmirror.git', >>> Probably this isn't updated often enough. It probably has little to do with the clock settings. >> The configuration is intentionally like that. I specifically configured skate and snapper to build the exact same source,where skate builds with default buildfarm settings, while snapper builds with the settings actually used by Debiansource packages. > TBH, I don't find that particularly important ... especially not for HEAD > builds, where building a many-hours-old snapshot is pretty much in the > category of "why bother?". On the whole, I think building from the latest > available source is the most useful policy. If there's some platform- > or configuration-specific issue, it usually takes more than one build > cycle for us to notice it anyway, so that ensuring two animals have exactly > comparable builds at any given instant isn't very helpful. > > Yeah, point. snapper seems the more important box here. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services