Обсуждение: Refectoring of receivelog.c
I was working on adding the tar streaming functionality we talked about at the developer meeting to pg_basebackup, and rapidly ran across the issue that Andres has been complaining about for a while. The code in receivelog.c just passes an insane number of parameters around. Adding or changing even a small thing ends up touching a huge number of places.
Here's an attempt to refactor the code to instead pass around a control structure. I think it's a definite win already now, and we can't just keep adding new functionality on top of the current one.
--
I'll proceed to work on the actual functionality I was working on to go on top of this separately, but would appreciate a review of this part independently. It's mostly mechanical, but there may definitely be mistakes - or thinkos in the whole idea...
Вложения
On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net> wrote:
I didn't find any mistakes, but I do admit my eyes started glazing over after a bit.
-- I was working on adding the tar streaming functionality we talked about at the developer meeting to pg_basebackup, and rapidly ran across the issue that Andres has been complaining about for a while. The code in receivelog.c just passes an insane number of parameters around. Adding or changing even a small thing ends up touching a huge number of places.
Other than the lack of comments on the fields in StreamCtl to indicate their functions I think this looks good.
I didn't find any mistakes, but I do admit my eyes started glazing over after a bit.
I'd rather not have StreamCtl as a typedef of an anonymous struct if it's exposed in a header though. It should have a name that can be used in forward declarations etc.
After recently working with the XLogReader I can't emphasise enough how important *useful* comments on the fields in these sorts of structs are - the XLogReaderState has ReadRecPtr, readSegNo + readOff + readLen, currRecPtr AND latestPagePtr. Which actually do have comments, just not super helpful ones.
On Mon, Feb 15, 2016 at 7:15 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net> wrote:I was working on adding the tar streaming functionality we talked about at the developer meeting to pg_basebackup, and rapidly ran across the issue that Andres has been complaining about for a while. The code in receivelog.c just passes an insane number of parameters around. Adding or changing even a small thing ends up touching a huge number of places.Other than the lack of comments on the fields in StreamCtl to indicate their functions I think this looks good.
I copied that lack of comments from the previous implementation :P
But yeah, I agree, it's probably a good idea to add them.
I didn't find any mistakes, but I do admit my eyes started glazing over after a bit.I'd rather not have StreamCtl as a typedef of an anonymous struct if it's exposed in a header though. It should have a name that can be used in forward declarations etc.
It's not exactly uncommon with anonymous ones either in our code, but I see no problem adding that.
Thanks!
--
> On 15 Feb 2016, at 14:46, Magnus Hagander <magnus@hagander.net> wrote: > > On Mon, Feb 15, 2016 at 7:15 AM, Craig Ringer <craig@2ndquadrant.com <mailto:craig@2ndquadrant.com>> wrote: > On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net <mailto:magnus@hagander.net>> wrote: > I was working on adding the tar streaming functionality we talked about at the developer meeting to pg_basebackup, andrapidly ran across the issue that Andres has been complaining about for a while. The code in receivelog.c just passesan insane number of parameters around. Adding or changing even a small thing ends up touching a huge number of places. > > Other than the lack of comments on the fields in StreamCtl to indicate their functions I think this looks good. > > I copied that lack of comments from the previous implementation :P > > But yeah, I agree, it's probably a good idea to add them. > > I didn't find any mistakes, but I do admit my eyes started glazing over after a bit. > > I'd rather not have StreamCtl as a typedef of an anonymous struct if it's exposed in a header though. It should have aname that can be used in forward declarations etc. > > It's not exactly uncommon with anonymous ones either in our code, but I see no problem adding that. Short review of this patch: The patch applies, and builds, cleanly on top of master as of today. No new functionality is introduced and thus no new tests or doc patches etc are applicable. The main point of the patch is to improve readability and reduce the number of parameters passed, goals which are reached. However, I agree with Craig that comments on the struct fields should be added to improve readability even further. The comment on ReceiveXlogStream() also now reads a bit strange since it describes fields inside the StreamCtl without referencing StreamCtl at all. For first time readers of the code it could perhaps be helpful with a brief note that the discussed parameters are in StreamCtl to avoid potential confusion. Overall I think this patch is an improvement on the current code and consider it ready for committer. cheers ./daniel
On Fri, Mar 11, 2016 at 9:40 AM, Daniel Gustafsson <daniel@yesql.se> wrote:
Thanks!
> On 15 Feb 2016, at 14:46, Magnus Hagander <magnus@hagander.net> wrote:
>
> On Mon, Feb 15, 2016 at 7:15 AM, Craig Ringer <craig@2ndquadrant.com <mailto:craig@2ndquadrant.com>> wrote:
> On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net <mailto:magnus@hagander.net>> wrote:
> I was working on adding the tar streaming functionality we talked about at the developer meeting to pg_basebackup, and rapidly ran across the issue that Andres has been complaining about for a while. The code in receivelog.c just passes an insane number of parameters around. Adding or changing even a small thing ends up touching a huge number of places.
>
> Other than the lack of comments on the fields in StreamCtl to indicate their functions I think this looks good.
>
> I copied that lack of comments from the previous implementation :P
>
> But yeah, I agree, it's probably a good idea to add them.
>
> I didn't find any mistakes, but I do admit my eyes started glazing over after a bit.
>
> I'd rather not have StreamCtl as a typedef of an anonymous struct if it's exposed in a header though. It should have a name that can be used in forward declarations etc.
>
> It's not exactly uncommon with anonymous ones either in our code, but I see no problem adding that.
Short review of this patch:
The patch applies, and builds, cleanly on top of master as of today. No new
functionality is introduced and thus no new tests or doc patches etc are
applicable.
The main point of the patch is to improve readability and reduce the number of
parameters passed, goals which are reached. However, I agree with Craig that
comments on the struct fields should be added to improve readability even
further. The comment on ReceiveXlogStream() also now reads a bit strange since
it describes fields inside the StreamCtl without referencing StreamCtl at all.
For first time readers of the code it could perhaps be helpful with a brief
note that the discussed parameters are in StreamCtl to avoid potential
confusion.
Overall I think this patch is an improvement on the current code and consider
it ready for committer.
Pushed with updated comments and a named stsruct.
Thanks!
On 03/11/2016 11:15 AM, Magnus Hagander wrote:
>
...
>
>
> Pushed with updated comments and a named stsruct.
Pretty sure this memset call in pg_basebackup.c is incorrect, as it
passes parameters in the wrong order:
MemSet(&stream, sizeof(stream), 0);
It seems benign, because we're setting all the fields explicitly, but
gcc is nagging about it.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sun, Mar 13, 2016 at 12:15 AM, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
On 03/11/2016 11:15 AM, Magnus Hagander wrote:...
Pushed with updated comments and a named stsruct.
Pretty sure this memset call in pg_basebackup.c is incorrect, as it passes parameters in the wrong order:
MemSet(&stream, sizeof(stream), 0);
It seems benign, because we're setting all the fields explicitly, but gcc is nagging about it.
Indeed,that's backwards. Interestingly enough I thought I did a c&p between that and the one in pg_receivexlog.c, but clearly I did not. And my gcc did not nag about it.
Fixed, thanks for the pointer!