Re: Proposal: Adding json logging
| От | David Arnold |
|---|---|
| Тема | Re: Proposal: Adding json logging |
| Дата | |
| Msg-id | CAH6vsW+Ze=ENCGB7abhYmCFOX8ZV0=U8+zFypeCJZNoRiP3AGg@mail.gmail.com обсуждение исходный текст |
| Ответ на | Re: Proposal: Adding json logging (David Arnold <dar@xoe.solutions>) |
| Ответы |
Re: Proposal: Adding json logging
(Christophe Pettus <xof@thebuild.com>)
|
| Список | pgsql-hackers |
Does everyone more or less agree with the following intermediate résumé?
1. Throughout this vivid discussion a good portion of support has already been manifested for the need of a more structured (machine readable) logging format. There has been no substantial objection to this need.
2. It has been proposed one JSON object per logging event and alternatively or complementary one logfmt object per event (although with much less resonance).
3. Doubts about space have been addressed by the following arguments:
- Data is short lived, as most likely consumed directly without much of a persistence
- GZIP compression could easily lift space requirements to a near deduplicacion format's requirements.
4. Doubts about standard conformance are still ongoing discussion, however there seems not to be in existence yet the ultimate logging standard which addresses all the shortcomings of existing approaches in an pareto optimal way. It's most likely something that requires a compromise or options for the user to choose from.
I hope this is a truthful account of the current state about this thread.
I'd propose, in order to move forward, to wait during a "call for objections" of a one week period until next Friday if there are any substantial objections to the direction of this discussion.
Afterwards, I'd like to get, hopefully not alone, some hands dirty.
I hope that's ok for everyone, please let me know if you think this mail is not a legitimate intention to take a step forward.
Best Regards, David Arnold
El dom., 15 abr. 2018, 10:49 a.m., David Arnold <dar@xoe.solutions> escribió:
> Exactly - arrays, maps, nested json objects. It's more organized and easier to reason about. As postgresql becomes more and more sophisticated over time, I see flat logging becoming more unwieldy. With tools like jq, reading and querying json on the command line is simple and user friendly, and using json for logging capture and aggregation is widely supporting and embraced.Existence and adoption of jq does certainly make a point here over grep friendly but less structured data.But I recommend reading over this blog post https://brandur.org/logfmtIt's got some strong arguments over a good log format.Basically, a good log format is both, human (tail stdaout) and machine (log aggregators) readable. Note how logrus solves the conflict.El dom., 15 abr. 2018, 10:27 a.m., Jordan Deitch <jd@rsa.pub> escribió:> > I would suggest that the community consider whether postgres will log
> multidimensional data. That will weigh into the decision of json vs.
> another format quite significantly. I am a fan of the json5 spec (
> https://json5.org/), though adoption of this is quite poor.
>
> What do you mean by multidimensional data? Arrays/maps?
>
> I think there is no advantage of multidimensional vs prefixed flat logging
> unless data structure gets really nastily nested.
>
> What case where you thinking of?
Exactly - arrays, maps, nested json objects. It's more organized and easier to reason about. As postgresql becomes more and more sophisticated over time, I see flat logging becoming more unwieldy. With tools like jq, reading and querying json on the command line is simple and user friendly, and using json for logging capture and aggregation is widely supporting and embraced.
В списке pgsql-hackers по дате отправления: