Обсуждение: [HACKERS] On Complex Source Code Reading Strategy
Hello Dear PostgreSQL Hackers, I want to be PG hacker but it seems a complex beast to find my way out in it.So, can anyone suggest me from his experience/stylethe general approaches/techniques/strategies on how to read complex source code in general and PG in particular effectively. Can you remember your being novice to PostgreSQL hacking and write your experience/style to me and probably to other novices as well. There are some guidelines in PG wiki but they are far from the truth of down the rabbit hole. I believe that there is plethora of knowledge and skill sets in the PG hackers head not to be found in any kind of written material and thus, that It would be great if any PG hacker could write his experience/style of PG hacking so that is of useful asset for PG hacking beginners. Regards, Zeray
Zeray Kalayu wrote: > I want to be PG hacker but it seems a complex beast to find my way out in it. > So, can anyone suggest me from his experience/style the general > approaches/techniques/strategies on how to read complex source code in > general and PG in particular effectively. > > Can you remember your being novice to PostgreSQL hacking and write > your experience/style to me and probably to other novices as well. > > There are some guidelines in PG wiki but they are far from the truth > of down the rabbit hole. > > I believe that there is plethora of knowledge and skill sets in the PG > hackers head not to be found in any kind of written material and thus, > that It would be great if any PG hacker could write his > experience/style of PG hacking so that is of useful asset for PG > hacking beginners. I'm not the most competent hacker around, but maybe that qualifies me to answer your question :^) I assume that you have read the documentation and know your way around PostgreSQL. Find something that you need and want to work on - maybe some shortcoming or missing feature of PostgreSQL. The TODO list (https://wiki.postgresql.org/wiki/Todo) may give some inspiration, but be cautioned that it contains mostly things that nobody could reach consensus on or implement easily. Best is to work on something that serves your own need. You don't have to work on core PostgreSQL, it could also be a server extension. If you have some project, you have an angle from which to tackle the large body of code that PostgreSQL is. As always, start with a design. Ask the list before you start coding. Another good and commendable path into the source is to review patches (https://wiki.postgresql.org/wiki/Reviewing_a_Patch). This is higly appreciated, because there is always a shortage of reviewers, and you get to see how other people go about changing the code. There is a lot to learn this way! You will find that the PostgreSQL source is mostly well written, well documented and well structured. Yours, Laurenz Albe
On Tue, Jul 25, 2017 at 11:54 PM, Zeray Kalayu <tiggreen87@gmail.com> wrote: > I want to be PG hacker but it seems a complex beast to find my way out in it. > So, can anyone suggest me from his experience/style the general > approaches/techniques/strategies on how to read complex source code in > general and PG in particular effectively. I can only think of two things: 1. Get familiar with how cscope or a similar tool works. Figure out how to make it integrate well with your text editor. 2. Start somewhere. I have no idea where that should be, but it has to be some particular place that seems interesting to you. The trick, if there is one, is to find what you read in some way relevant, interesting, or useful, in the short term, so that a virtuous circle starts. I don't think that there's a right place to begin. If you find a way to learn that is sustainable, then you can eventually have a good understanding of the system as a whole. -- Peter Geoghegan
Peter Geoghegan <pg@bowt.ie> writes: > 2. Start somewhere. I have no idea where that should be, but it has to > be some particular place that seems interesting to you. Don't forget to start with the available documentation, ie https://www.postgresql.org/docs/devel/static/internals.html You should certainly read https://www.postgresql.org/docs/devel/static/overview.html and depending on what your interests are, there are probably other chapters of Part VII that are worth your time. Also keep an eye out for README files in the part of the source tree you're browsing in. regards, tom lane
On 07/27/2017 04:45 PM, Tom Lane wrote: > Peter Geoghegan <pg@bowt.ie> writes: >> 2. Start somewhere. I have no idea where that should be, but it has to >> be some particular place that seems interesting to you. > > Don't forget to start with the available documentation, ie > https://www.postgresql.org/docs/devel/static/internals.html > You should certainly read > https://www.postgresql.org/docs/devel/static/overview.html > and depending on what your interests are, there are probably other > chapters of Part VII that are worth your time. > > Also keep an eye out for README files in the part of the source > tree you're browsing in. The doxygen instance at doxygen.postgresql.org is also helpful in navigating dependencies as you start attacking a features. JD > > regards, tom lane > > -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.us ***** Unless otherwise stated, opinions are my own. *****
On 28 July 2017 at 07:45, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Peter Geoghegan <pg@bowt.ie> writes:
> 2. Start somewhere. I have no idea where that should be, but it has to
> be some particular place that seems interesting to you.
Don't forget to start with the available documentation, ie
https://www.postgresql.org/docs/devel/static/internals. html
You should certainly read
https://www.postgresql.org/docs/devel/static/overview. html
and depending on what your interests are, there are probably other
chapters of Part VII that are worth your time.
Also keep an eye out for README files in the part of the source
tree you're browsing in.
In fact, even though you won't initially understand much from some of them, reading most of
find src/ -name README\*
There are very useful comments at the start of some of the source files too. Unfortunately in some cases the really important explanation will be on some function that you won't know to look for, not the comment at the top of the file, so there's an element of discovery there.
I'd start with the docs as Tom suggested, then
(some of which need to be added to the "developer information" wiki page I think)
--
On Fri, Jul 28, 2017 at 6:23 AM, Craig Ringer <craig@2ndquadrant.com> wrote: > On 28 July 2017 at 07:45, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> >> Peter Geoghegan <pg@bowt.ie> writes: >> > 2. Start somewhere. I have no idea where that should be, but it has to >> > be some particular place that seems interesting to you. >> >> Don't forget to start with the available documentation, ie >> https://www.postgresql.org/docs/devel/static/internals.html >> You should certainly read >> https://www.postgresql.org/docs/devel/static/overview.html >> and depending on what your interests are, there are probably other >> chapters of Part VII that are worth your time. >> >> Also keep an eye out for README files in the part of the source >> tree you're browsing in. > > > In fact, even though you won't initially understand much from some of them, > reading most of > > find src/ -name README\* > > can be quite useful. It's nearly time for me to do that again myself; each > time I absorb more. > > There are very useful comments at the start of some of the source files too. > Unfortunately in some cases the really important explanation will be on some > function that you won't know to look for, not the comment at the top of the > file, so there's an element of discovery there. > > I'd start with the docs as Tom suggested, then > > * https://www.postgresql.org/developer/ > * https://momjian.us/main/presentations/internals.html > * https://wiki.postgresql.org/wiki/Development_information > * https://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer%3F > * https://wiki.postgresql.org/wiki/Developer_FAQ > > (some of which need to be added to the "developer information" wiki page I > think) > Thanks dear hackers. This is an enormous help for me. I think recommending specific techniques/tools like Cscope, find src/ -name README\* and others you might know that make life easy with PG hacking can be great help for the beginners. Regards, Zeray
On Fri, Jul 28, 2017 at 2:45 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Peter Geoghegan <pg@bowt.ie> writes: >> 2. Start somewhere. I have no idea where that should be, but it has to >> be some particular place that seems interesting to you. > > Don't forget to start with the available documentation, ie > https://www.postgresql.org/docs/devel/static/internals.html > You should certainly read > https://www.postgresql.org/docs/devel/static/overview.html > and depending on what your interests are, there are probably other > chapters of Part VII that are worth your time. > > Also keep an eye out for README files in the part of the source > tree you're browsing in. There is at least one book with more than 500 pages on "Code Reading" and I can see and sense that "Code Reading" competency is not a trivial problem nowadays(https://www.amazon.com/Code-Reading-Open-Source-Perspective/dp/0201799405). Dear Tom Lane, I know that you are one of the most competent PG hacker we ever have. You could write a book entitled: "The Art Of Code Reading: The Case Of PostgreSQL" so that many beginners struggling to find their space in PG would somehow shorten the amount of effort required to be capable PG hackers. I feel and think that you are not able to write a book on your art/style of writing and reading code simply because you are so busy taking PG forward . To reiterate: Code Reading has become increasingly important more than ever and thus, the problem of code reading needs to be be addressed comprehensively as it is being tried in https://www.amazon.com/Code-Reading-Open-Source-Perspective/dp/0201799405). Lastly, I strongly believe that Code is the ultimate truth and being able to understand complex and high quality code effectively and strategically is of paramount importance. BTW, I have taken all of the PG hackers advice and am studying the code base accordingly. Regards, Zeray
On Wed, Aug 2, 2017 at 7:24 AM, Zeray Kalayu <tiggreen87@gmail.com> wrote: > Lastly, I strongly believe that Code is the ultimate truth and being > able to understand complex and high quality code effectively and > strategically is of paramount importance. Documentation to understand how a system works from the user prospective, and comments in the code itself are also important properties of a code that can be considered as a good base. Postgres has both. -- Michael
On Wed, Aug 2, 2017 at 9:30 PM, Michael Paquier <michael.paquier@gmail.com> wrote: > On Wed, Aug 2, 2017 at 7:24 AM, Zeray Kalayu <tiggreen87@gmail.com> wrote: >> Lastly, I strongly believe that Code is the ultimate truth and being >> able to understand complex and high quality code effectively and >> strategically is of paramount importance. > > Documentation to understand how a system works from the user > prospective, and comments in the code itself are also important > properties of a code that can be considered as a good base. Postgres > has both. Michael,thanks and I am trying to take advantage of them. Actually, I have learned that being hacker of PG or other data management platforms like Greenplum is not a one-stop story or journey. It is the sum of deep understanding of computing sciences(the foundations like compatibility theory, computational complexity...), the engineering aspect of computing and the tech aspect of the same. Therefore, I feel and think that I am a bit in a hurry to be DB hacker. But given time, indefatigability and PG community support, I believe that I will manage to achieve my dream. Regards, Zeray
On Thu, Aug 3, 2017 at 1:55 AM, Zeray Kalayu <tiggreen87@gmail.com> wrote: > Therefore, I feel and think that I am a bit in a hurry to be DB > hacker. But given time, indefatigability and PG community support, I > believe that I will manage to achieve my dream. Indefatigability is key. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company