Обсуждение: add line number as prompt option to psql

Hi all,

The attached IWP patch is one prompt option for psql, which shows
current line number.
If the user made syntax error with too long SQL then psql outputs
message as following.

ERROR:  syntax error at or near "a"
LINE 250: hoge
                ^
psql teaches me where syntax error is occurred, but it is not kind
when SQL is too long.
We can use write SQL with ¥e(editor) command(e.g., emacs) , and we can
know line number.
but it would make terminal log dirty . It will make analyzing of log
difficult after error is occurred.
(I think that we usually use copy & paste)

After attached this patch, we will be able to use %l option as prompting option.

e.g.,
$ cat ~/.psqlrc
\set PROMPT2 '%/[%l]%R%# '
\set PROMPT1 '%/[%l]%R%# '
$ psql -d postgres
postgres[1]=# select
postgres[2]-# *
postgres[3]-# from
postgres[4]-# hoge;

The past discussion is following.
<http://www.postgresql.org/message-id/CAFj8pRC1ruPk6+chA1jpxPh3uS_zipaBDOvmcEex4wPbp2kZMQ@mail.gmail.com>

Please give me feedback.

Regards,

-------
Sawada Masahiko

<div dir="ltr">Hi Sawada Masahiko,<br /><br />I liked this feature. So I have reviewed it.<br /><br />Changes are
straightforward and looks perfect.<br />No issues found with make/make install/initdb/regression.<br /><br />However I
wouldsuggest removing un-necessary braces at if, as we have only<br /> one statement into it.<br /><br /><span
style="font-family:couriernew,monospace">    if (++cur_line >= INT_MAX)<br />    {<br />        cur_line = 1;<br
/>   }<br /></span><br />Also following looks wrong:<br /><br /><span style="font-family:courier
new,monospace">postgres[1]=#select<br /> postgres[2]-# *<br />postgres[3]-# from<br />postgres[4]-# tab;<br /> a <br
/>---<br/>(0 rows)<br /><br />postgres[1]=# select<br />*<br />from<br />tab<br />postgres[2]-# where t > 10;<br
/>ERROR: column "t" does not exist<br />LINE 5: where t > 10;<br />               ^</span><br /><br />Line number in
ERRORis 5 which is correct.<br />But line number in psql prompt is wrong.<br /><br />To get first 4 lines I have simply
usedup arrow followed by an enter for<br />which I was expecting 5 in psql prompt.<br /> But NO it was 2 which is
certainlywrong.<br /><br />Need to handle above carefully.<br /><br />Thanks<br /></div><div class="gmail_extra"><br
/><br/><div class="gmail_quote">On Thu, Jun 12, 2014 at 10:46 PM, Sawada Masahiko <span dir="ltr"><<a
href="mailto:sawada.mshk@gmail.com"target="_blank">sawada.mshk@gmail.com</a>></span> wrote:<br /><blockquote
class="gmail_quote"style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br /><br /> The
attachedIWP patch is one prompt option for psql, which shows<br /> current line number.<br /> If the user made syntax
errorwith too long SQL then psql outputs<br /> message as following.<br /><br /> ERROR:  syntax error at or near "a"<br
/>LINE 250: hoge<br />                 ^<br /> psql teaches me where syntax error is occurred, but it is not kind<br />
whenSQL is too long.<br /> We can use write SQL with ¥e(editor) command(e.g., emacs) , and we can<br /> know line
number.<br/> but it would make terminal log dirty . It will make analyzing of log<br /> difficult after error is
occurred.<br/> (I think that we usually use copy & paste)<br /><br /> After attached this patch, we will be able to
use%l option as prompting option.<br /><br /> e.g.,<br /> $ cat ~/.psqlrc<br /> \set PROMPT2 '%/[%l]%R%# '<br /> \set
PROMPT1'%/[%l]%R%# '<br /> $ psql -d postgres<br /> postgres[1]=# select<br /> postgres[2]-# *<br /> postgres[3]-#
from<br/> postgres[4]-# hoge;<br /><br /> The past discussion is following.<br /> <<a
href="http://www.postgresql.org/message-id/CAFj8pRC1ruPk6+chA1jpxPh3uS_zipaBDOvmcEex4wPbp2kZMQ@mail.gmail.com"
target="_blank">http://www.postgresql.org/message-id/CAFj8pRC1ruPk6+chA1jpxPh3uS_zipaBDOvmcEex4wPbp2kZMQ@mail.gmail.com</a>><br
/><br/> Please give me feedback.<br /><br /> Regards,<br /><br /> -------<br /> Sawada Masahiko<br /><br /><br /> --<br
/>Sent via pgsql-hackers mailing list (<a
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br/> To make changes to your
subscription:<br/><a href="http://www.postgresql.org/mailpref/pgsql-hackers"
target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/><br /></blockquote></div><br /><br clear="all"
/><br/>-- <br /><div dir="ltr">Jeevan B Chalke<br />Principal Software Engineer, Product Development<br />EnterpriseDB
Corporation<br/>The Enterprise PostgreSQL Company<br /><br />Phone: +91 20 30589500<br /><br />Website: <a
href="http://www.enterprisedb.com"target="_blank">www.enterprisedb.com</a><br />EnterpriseDB Blog: <a
href="http://blogs.enterprisedb.com/"target="_blank">http://blogs.enterprisedb.com/</a><br />Follow us on Twitter: <a
href="http://www.twitter.com/enterprisedb"target="_blank">http://www.twitter.com/enterprisedb</a><br /><br />This
e-mailmessage (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This
messagecontains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from
disclosureunder applicable law. If you are not the intended recipient or authorized to receive this for the intended
recipient,any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly
prohibited.If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete
thismessage.</div></div> 

On Fri, Jun 20, 2014 at 7:17 PM, Jeevan Chalke
<jeevan.chalke@enterprisedb.com> wrote:
> Hi Sawada Masahiko,
>
> I liked this feature. So I have reviewed it.
>
> Changes are straight forward and looks perfect.
> No issues found with make/make install/initdb/regression.
>
> However I would suggest removing un-necessary braces at if, as we have only
> one statement into it.
>
>     if (++cur_line >= INT_MAX)
>     {
>         cur_line = 1;
>     }
>
> Also following looks wrong:
>
> postgres[1]=# select
> postgres[2]-# *
> postgres[3]-# from
> postgres[4]-# tab;
>  a
> ---
> (0 rows)
>
> postgres[1]=# select
> *
> from
> tab
> postgres[2]-# where t > 10;
> ERROR:  column "t" does not exist
> LINE 5: where t > 10;
>               ^
>
> Line number in ERROR is 5 which is correct.
> But line number in psql prompt is wrong.
>
> To get first 4 lines I have simply used up arrow followed by an enter for
> which I was expecting 5 in psql prompt.
> But NO it was 2 which is certainly wrong.
>
> Need to handle above carefully.
>
> Thanks
>
>
> On Thu, Jun 12, 2014 at 10:46 PM, Sawada Masahiko <sawada.mshk@gmail.com>
> wrote:
>>
>> Hi all,
>>
>> The attached IWP patch is one prompt option for psql, which shows
>> current line number.
>> If the user made syntax error with too long SQL then psql outputs
>> message as following.
>>
>> ERROR:  syntax error at or near "a"
>> LINE 250: hoge
>>                 ^
>> psql teaches me where syntax error is occurred, but it is not kind
>> when SQL is too long.
>> We can use write SQL with ¥e(editor) command(e.g., emacs) , and we can
>> know line number.
>> but it would make terminal log dirty . It will make analyzing of log
>> difficult after error is occurred.
>> (I think that we usually use copy & paste)
>>
>> After attached this patch, we will be able to use %l option as prompting
>> option.
>>
>> e.g.,
>> $ cat ~/.psqlrc
>> \set PROMPT2 '%/[%l]%R%# '
>> \set PROMPT1 '%/[%l]%R%# '
>> $ psql -d postgres
>> postgres[1]=# select
>> postgres[2]-# *
>> postgres[3]-# from
>> postgres[4]-# hoge;
>>
>> The past discussion is following.
>>
>> <http://www.postgresql.org/message-id/CAFj8pRC1ruPk6+chA1jpxPh3uS_zipaBDOvmcEex4wPbp2kZMQ@mail.gmail.com>
>>
>> Please give me feedback.
>>
>> Regards,
>>
>> -------
>> Sawada Masahiko
>>
>>
>> --
>> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-hackers
>>
>
>
>
> --
> Jeevan B Chalke
> Principal Software Engineer, Product Development
> EnterpriseDB Corporation
> The Enterprise PostgreSQL Company
>
> Phone: +91 20 30589500
>
> Website: www.enterprisedb.com
> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> Follow us on Twitter: http://www.twitter.com/enterprisedb
>
> This e-mail message (and any attachment) is intended for the use of the
> individual or entity to whom it is addressed. This message contains
> information from EnterpriseDB Corporation that may be privileged,
> confidential, or exempt from disclosure under applicable law. If you are not
> the intended recipient or authorized to receive this for the intended
> recipient, any use, dissemination, distribution, retention, archiving, or
> copying of this communication is strictly prohibited. If you have received
> this e-mail in error, please notify the sender immediately by reply e-mail
> and delete this message.

Thank you for reviewing patch, and sorry for late response.

I have updated this patch, and attached it.

> postgres[1]=# select
> *
> from
> tab
> postgres[2]-# where t > 10;
> ERROR:  column "t" does not exist
> LINE 5: where t > 10;
Attached patch can handle this case.

Please give me feedback.

Regards,

-------
Sawada Masahiko

<div dir="ltr">Hi,<br /><br />Found two (A and B) issues with latest patch:<br /><br />A:<br />-- Set prompts<br
/>postgres=#\set PROMPT1 '%/[%l]%R%# '<br />postgres[1]=# \set PROMPT2 '%/[%l]%R%# '<br /><br />postgres[1]=# <br />
postgres[1]=#select<br />postgres[2]-# *<br />postgres[3]-# from<br />postgres[4]-# abc;<br />ERROR:  relation "abc"
doesnot exist<br />LINE 4: abc;<br />        ^<br /><br />Now I used \e to edit the source. Deleted last line i.e.
"abc;"and<br /> returned to prompt, landed at line 4, typed "abc;"<br /><br />postgres[1]=# \e<br />postgres[4]-#
abc;<br/>ERROR:  relation "abc" does not exist<br />LINE 5: abc;<br />        ^<br /><br />In above steps, error
messagesays "LINE 5", where as on prompt "abc" is at<br /> line 4.<br /><br />postgres[1]=# select<br />*<br />from<br
/>abc;<br/>ERROR:  relation "abc" does not exist<br />LINE 4: abc;<br />        ^<br /><br />Here I again see error at
line4. Something fishy. Please investigate.<br />Looks like bug in LINE numbering in error message, not sure though.<br
/>But with prompt line number feature, it should be sync to each other.<br /><br /><br />B:<br />However, I see that
youhave removed the code changes related to INT_MAX.<br />Why?<br />I have set cur_line to INT_MAX - 2 and then
observedthat after 2 lines I<br /> start getting negative numbers.<br /><br />Thanks<br /></div><div
class="gmail_extra"><br/><br /><div class="gmail_quote">On Sun, Jul 6, 2014 at 10:48 PM, Sawada Masahiko <span
dir="ltr"><<ahref="mailto:sawada.mshk@gmail.com" target="_blank">sawada.mshk@gmail.com</a>></span> wrote:<br
/><blockquoteclass="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div
class="HOEnZb"><divclass="h5">On Fri, Jun 20, 2014 at 7:17 PM, Jeevan Chalke<br /> <<a
href="mailto:jeevan.chalke@enterprisedb.com">jeevan.chalke@enterprisedb.com</a>>wrote:<br /> > Hi Sawada
Masahiko,<br/> ><br /> > I liked this feature. So I have reviewed it.<br /> ><br /> > Changes are straight
forwardand looks perfect.<br /> > No issues found with make/make install/initdb/regression.<br /> ><br /> >
HoweverI would suggest removing un-necessary braces at if, as we have only<br /> > one statement into it.<br />
><br/> >     if (++cur_line >= INT_MAX)<br /> >     {<br /> >         cur_line = 1;<br /> >     }<br
/>><br /> > Also following looks wrong:<br /> ><br /> > postgres[1]=# select<br /> > postgres[2]-# *<br
/>> postgres[3]-# from<br /> > postgres[4]-# tab;<br /> >  a<br /> > ---<br /> > (0 rows)<br /> ><br
/>> postgres[1]=# select<br /> > *<br /> > from<br /> > tab<br /> > postgres[2]-# where t > 10;<br />
>ERROR:  column "t" does not exist<br /> > LINE 5: where t > 10;<br /> >               ^<br /> ><br />
>Line number in ERROR is 5 which is correct.<br /> > But line number in psql prompt is wrong.<br /> ><br />
>To get first 4 lines I have simply used up arrow followed by an enter for<br /> > which I was expecting 5 in
psqlprompt.<br /> > But NO it was 2 which is certainly wrong.<br /> ><br /> > Need to handle above
carefully.<br/> ><br /> > Thanks<br /> ><br /> ><br /> > On Thu, Jun 12, 2014 at 10:46 PM, Sawada
Masahiko<<a href="mailto:sawada.mshk@gmail.com">sawada.mshk@gmail.com</a>><br /> > wrote:<br /> >><br />
>>Hi all,<br /> >><br /> >> The attached IWP patch is one prompt option for psql, which shows<br />
>>current line number.<br /> >> If the user made syntax error with too long SQL then psql outputs<br />
>>message as following.<br /> >><br /> >> ERROR:  syntax error at or near "a"<br /> >> LINE
250:hoge<br /> >>                 ^<br /> >> psql teaches me where syntax error is occurred, but it is not
kind<br/> >> when SQL is too long.<br /> >> We can use write SQL with ¥e(editor) command(e.g., emacs) , and
wecan<br /> >> know line number.<br /> >> but it would make terminal log dirty . It will make analyzing of
log<br/> >> difficult after error is occurred.<br /> >> (I think that we usually use copy & paste)<br
/>>><br /> >> After attached this patch, we will be able to use %l option as prompting<br /> >>
option.<br/> >><br /> >> e.g.,<br /> >> $ cat ~/.psqlrc<br /> >> \set PROMPT2 '%/[%l]%R%# '<br
/>>> \set PROMPT1 '%/[%l]%R%# '<br /> >> $ psql -d postgres<br /> >> postgres[1]=# select<br />
>>postgres[2]-# *<br /> >> postgres[3]-# from<br /> >> postgres[4]-# hoge;<br /> >><br />
>>The past discussion is following.<br /> >><br /> >> <<a
href="http://www.postgresql.org/message-id/CAFj8pRC1ruPk6+chA1jpxPh3uS_zipaBDOvmcEex4wPbp2kZMQ@mail.gmail.com"
target="_blank">http://www.postgresql.org/message-id/CAFj8pRC1ruPk6+chA1jpxPh3uS_zipaBDOvmcEex4wPbp2kZMQ@mail.gmail.com</a>><br
/>>><br /> >> Please give me feedback.<br /> >><br /> >> Regards,<br /> >><br /> >>
-------<br/> >> Sawada Masahiko<br /> >><br /> >><br /> >> --<br /> >> Sent via
pgsql-hackersmailing list (<a href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br />
>>To make changes to your subscription:<br /> >> <a href="http://www.postgresql.org/mailpref/pgsql-hackers"
target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/> >><br /> ><br /> ><br /> ><br
/>> --<br /> > Jeevan B Chalke<br /> > Principal Software Engineer, Product Development<br /> >
EnterpriseDBCorporation<br /> > The Enterprise PostgreSQL Company<br /> ><br /> > Phone: +91 20 30589500<br />
><br/> > Website: <a href="http://www.enterprisedb.com" target="_blank">www.enterprisedb.com</a><br /> >
EnterpriseDBBlog: <a href="http://blogs.enterprisedb.com/" target="_blank">http://blogs.enterprisedb.com/</a><br />
>Follow us on Twitter: <a href="http://www.twitter.com/enterprisedb"
target="_blank">http://www.twitter.com/enterprisedb</a><br/> ><br /> > This e-mail message (and any attachment)
isintended for the use of the<br /> > individual or entity to whom it is addressed. This message contains<br /> >
informationfrom EnterpriseDB Corporation that may be privileged,<br /> > confidential, or exempt from disclosure
underapplicable law. If you are not<br /> > the intended recipient or authorized to receive this for the intended<br
/>> recipient, any use, dissemination, distribution, retention, archiving, or<br /> > copying of this
communicationis strictly prohibited. If you have received<br /> > this e-mail in error, please notify the sender
immediatelyby reply e-mail<br /> > and delete this message.<br /><br /></div></div>Thank you for reviewing patch,
andsorry for late response.<br /><br /> I have updated this patch, and attached it.<br /><div class=""><br /> >
postgres[1]=#select<br /> > *<br /> > from<br /> > tab<br /> > postgres[2]-# where t > 10;<br /> >
ERROR: column "t" does not exist<br /> > LINE 5: where t > 10;<br /></div>Attached patch can handle this case.<br
/><divclass="HOEnZb"><div class="h5"><br /> Please give me feedback.<br /><br /> Regards,<br /><br /> -------<br />
SawadaMasahiko<br /></div></div></blockquote></div><br /><br clear="all" /><br />-- <br /><div dir="ltr">Jeevan B
Chalke<br/>Principal Software Engineer, Product Development<br />EnterpriseDB Corporation<br />The Enterprise
PostgreSQLCompany<br /><br />Phone: +91 20 30589500<br /><br />Website: <a href="http://www.enterprisedb.com"
target="_blank">www.enterprisedb.com</a><br/>EnterpriseDB Blog: <a href="http://blogs.enterprisedb.com/"
target="_blank">http://blogs.enterprisedb.com/</a><br/>Follow us on Twitter: <a
href="http://www.twitter.com/enterprisedb"target="_blank">http://www.twitter.com/enterprisedb</a><br /><br />This
e-mailmessage (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This
messagecontains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from
disclosureunder applicable law. If you are not the intended recipient or authorized to receive this for the intended
recipient,any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly
prohibited.If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete
thismessage.</div></div> 

On Mon, Jul 7, 2014 at 8:33 PM, Jeevan Chalke
<jeevan.chalke@enterprisedb.com> wrote:
> Hi,
>
> Found two (A and B) issues with latest patch:
>
> A:
> -- Set prompts
> postgres=# \set PROMPT1 '%/[%l]%R%# '
> postgres[1]=# \set PROMPT2 '%/[%l]%R%# '
>
> postgres[1]=#
> postgres[1]=# select
> postgres[2]-# *
> postgres[3]-# from
> postgres[4]-# abc;
> ERROR:  relation "abc" does not exist
> LINE 4: abc;
>         ^
>
> Now I used \e to edit the source. Deleted last line i.e. "abc;" and
> returned to prompt, landed at line 4, typed "abc;"
>
> postgres[1]=# \e
> postgres[4]-# abc;
> ERROR:  relation "abc" does not exist
> LINE 5: abc;
>         ^
>
> In above steps, error message says "LINE 5", where as on prompt "abc" is at
> line 4.
>
>
> postgres[1]=# select
> *
> from
> abc;
> ERROR:  relation "abc" does not exist
> LINE 4: abc;
>         ^
>
> Here I again see error at line 4. Something fishy. Please investigate.
> Looks like bug in LINE numbering in error message, not sure though.
> But with prompt line number feature, it should be sync to each other.
>
>
> B:
> However, I see that you have removed the code changes related to INT_MAX.
> Why?
> I have set cur_line to INT_MAX - 2 and then observed that after 2 lines I
> start getting negative numbers.
>

Thank you for reviewing the patch.
I have revised the patch, and attached.

> A:
> But with prompt line number feature, it should be sync to each other.

This patch can handle this case.

> B:
> However, I see that you have removed the code changes related to INT_MAX.
> Why?

I had mistake to remove them. I added them to latest patch.

Regards,

-------
Sawada Masahiko

<div dir="ltr">Hi,<br /><br />Found few more bugs in new code:<br /><br />A:<br />This got bad:<br /><br /><span
style="font-family:couriernew,monospace">jeevan@ubuntu:~/pg_master$ ./install/bin/psql postgres<br />psql (9.5devel)<br
/>Type"help" for help.<br /><br />postgres=# \set PROMPT1 '%/[%l]%R%# '<br />postgres[1]=# \set PROMPT2 '%/[%l]%R%#
'<br/>postgres[1]=# select<br />postgres[2]-# *<br />postgres[3]-# from<br />postgres[4]-# abc;<br />ERROR:  syntax
errorat or near "fromabc"<br /> LINE 1: select*fromabc;<br />               ^<br />postgres[1]=# <br />postgres[1]=#
<br/>postgres[1]=# \e<br />ERROR:  syntax error at or near "fromabc"<br />LINE 1: select*fromabc;<br />              
^<br/>postgres[1]=# select*fromabc;<br /> ERROR:  syntax error at or near "fromabc"<br />LINE 1: select*fromabc;<br
/>              ^<br />postgres[1]=# <br /></span><br /><br />See query text in LINE 1:. This is because, you have
removedaddition of<br />newline character. Related added_nl_pos. Need more investigation here.<br /> However I don't
thinkthese changes are relevant to what you wanted in this<br />feature.<br />Will you please explain the idea behind
thesechanges ?<br /><br />Moreover, if you don't want to add newline character, then I guess entire<br /> logic related
toadded_nl_pos is NO more required. You may remove this<br />variable and its logic altogether, not sure though. Also
makesure you<br />update the relevant comments while doing so. Here you have removed the code<br /> which is adding the
newlinebut the comment there still reads:<br /><span style="font-family:courier new,monospace">/* insert newlines into
querybuffer between source lines */</span><br /><br />Need more thoughts on this.<br /><br /><br />B:<br />postgres=#
\setPROMPT1 '%/[%l]%R%# '<br />postgres[1]=# \set PROMPT2 '%/[%l]%R%# '<br />postgres[1]=# \e<br
/>postgres[-2147483645]-#limit 1;<br />   relname    <br />--------------<br /> pg_statistic<br /> (1 row)<br /><br
/>postgres[1]=#<br />postgres[1]=# select<br />relname<br />from<br />pg_class<br />limit 1;<br /><br />Logic related
towrapping around the cur_line counter is wrong. Actually<br />issue is with newline variable. If number of lines in \e
editorgoes beyond<br /> INT_MAX (NOT sure about the practical use), then newline will be -ve which<br />then enforces
cur_lineto be negative. To mimic this I have initialized<br />newline = INT_MAX - 1.<br /><br />Thanks<br /><div
class="gmail_extra"><br/> -- <br /><div dir="ltr">Jeevan B Chalke<br />Principal Software Engineer, Product
Development<br/>EnterpriseDB Corporation<br />The Enterprise PostgreSQL Company<br /><br /></div></div></div> 

On Thu, Jul 10, 2014 at 8:35 PM, Jeevan Chalke
<jeevan.chalke@enterprisedb.com> wrote:
> Hi,
>
> Found few more bugs in new code:
>
> A:
> This got bad:
>
> jeevan@ubuntu:~/pg_master$ ./install/bin/psql postgres
> psql (9.5devel)
> Type "help" for help.
>
>
> postgres=# \set PROMPT1 '%/[%l]%R%# '
> postgres[1]=# \set PROMPT2 '%/[%l]%R%# '
> postgres[1]=# select
> postgres[2]-# *
> postgres[3]-# from
> postgres[4]-# abc;
> ERROR:  syntax error at or near "fromabc"
> LINE 1: select*fromabc;
>
>                ^
> postgres[1]=#
> postgres[1]=#
> postgres[1]=# \e
> ERROR:  syntax error at or near "fromabc"
> LINE 1: select*fromabc;
>                ^
> postgres[1]=# select*fromabc;
> ERROR:  syntax error at or near "fromabc"
> LINE 1: select*fromabc;
>                ^
> postgres[1]=#
>
>
> See query text in LINE 1:. This is because, you have removed addition of
> newline character. Related added_nl_pos. Need more investigation here.
> However I don't think these changes are relevant to what you wanted in this
> feature.
> Will you please explain the idea behind these changes ?
>
> Moreover, if you don't want to add newline character, then I guess entire
> logic related to added_nl_pos is NO more required. You may remove this
> variable and its logic altogether, not sure though. Also make sure you
> update the relevant comments while doing so. Here you have removed the code
> which is adding the newline but the comment there still reads:
> /* insert newlines into query buffer between source lines */
>
> Need more thoughts on this.
>
>
> B:
>
> postgres=# \set PROMPT1 '%/[%l]%R%# '
> postgres[1]=# \set PROMPT2 '%/[%l]%R%# '
> postgres[1]=# \e
> postgres[-2147483645]-# limit 1;
>    relname
> --------------
>  pg_statistic
> (1 row)
>
>
> postgres[1]=#
> postgres[1]=# select
> relname
> from
> pg_class
> limit 1;
>
> Logic related to wrapping around the cur_line counter is wrong. Actually
> issue is with newline variable. If number of lines in \e editor goes beyond
> INT_MAX (NOT sure about the practical use), then newline will be -ve which
> then enforces cur_line to be negative. To mimic this I have initialized
> newline = INT_MAX - 1.
>

Thank you for reviewing the patch with variable cases.
I have revised the patch, and attached latest patch.

> A:
> Will you please explain the idea behind these changes ?
I thought wrong about adding new to tail of query_buf.
The latest patch does not change related to them.

> B:
I added the condition of cur_line < 0.


Regards,

-------
Sawada Masahiko

On Fri, Jul 11, 2014 at 4:23 PM, Jeevan Chalke
<jeevan.chalke@enterprisedb.com> wrote:
> Hi,
>
> A.
>
> However, this introduced new bug. As I told, when editor number of lines
> reaches INT_MAX it starts giving negative number. You tried overcoming this
> issue by adding "< 0" check. But I guess you again fumbled in setting that
> correctly. You are setting it to INT_MAX - 1. This enforces each new line to
> show line number as INT_MAX - 1 which is incorrect.
>
> postgres=# \set PROMPT1 '%/[%l]%R%# '
> postgres[1]=# \set PROMPT2 '%/[%l]%R%# '
> postgres[1]=# \e
> postgres[2147483646]-# limit
> postgres[2147483646]-# 1;
>
>    relname
> --------------
>  pg_statistic
> (1 row)
>
> postgres[1]=# \e
> postgres[2147483646]-# =
> postgres[2147483646]-# '
> postgres[2147483646]'# abc
> postgres[2147483646]'# '
> postgres[2147483646]-# ;
>  relname
> ---------
> (0 rows)
>
>
> postgres[1]=# select
> relname
> from
> pg_class
> where
> relname
> =
> '
> abc
> '
> ;
>
>
> Again to mimic that, I have simply intialized newline to INT_MAX - 2.
> Please don't take me wrong, but it seems that your unit testing is not
> enough. Above issue I discovered by doing exactly same steps I did in
> reviewing previous patch. If you had tested your new patch with those steps
> I guess you have caught that yourself.
>

To my understating cleanly, you means that line number is not changed
when newline has reached to INT_MAX, is incorrect?
And the line number should be switched to 1 when line number has
reached to INT_MAX?

>
> B.
>
> +             /* Calculate the line number */
> +             if (scan_result != PSCAN_INCOMPLETE)
> +             {
> +                 /* The one new line is always added to tail of query_buf
> */
> +                 newline = (newline != 0) ? (newline + 1) : 1;
> +                 cur_line += newline;
> +             }
>
> Here in above code changes, in any case you are adding 1 to newline. i.e.
> when it is 0 you are setting it 1 (+1) and when > 0 you are setting nl + 1
> (again +1).
> So why can't you simply use"
>     if (scan_result != PSCAN_INCOMPLETE)
>         cur_line += (newline + 1);
>
> Or better, why can't you initialize newline with 1 itself and then directly
> assign cur_line with newline. That will eliminate above entire code block,
> isn't it?
> Or much better, simply get rid of newline, and use cur_line itself, will
> this work well for you?

this is better.  I will change code to this.

>
> C. Typos:
> 1.
> /* Count the number of new line for calculate ofline number */
>
> Missing space between 'of' and 'line'.
> However better improve that to something like (just suggestion):
> "Count the number of new lines to correctly adjust current line number"
>
> 2.
> /* Avoid cur_line and newline exceeds the INT_MAX */
>
> You are saying avoid cur_line AND newline, but there is no adjustment for
> newline in the code following the comment.
>
> Thanks
> --
> Jeevan B Chalke
> Principal Software Engineer, Product Development
> EnterpriseDB Corporation
> The Enterprise PostgreSQL Company
>

Thanks.
I will fix it.

-- 
Regards,

-------
Sawada Masahiko

Jeevan Chalke wrote:

> On Fri, Jul 11, 2014 at 3:13 PM, Sawada Masahiko <sawada.mshk@gmail.com>
> wrote:

> > And the line number should be switched to 1 when line number has
> > reached to INT_MAX?
> 
> Yes, when it goes beyond INT_MAX, wrap around to 1.
> 
> BTW, I wonder, can't we simply use unsigned int instead?

That was my thought also: let the variable be unsigned, and have it wrap
around normally.  So once you reach UINT_MAX, the next line number is
zero (instead of getting stuck at UINT_MAX, which would be rather
strange).  Anyway I don't think anyone is going to reach the UINT_MAX
limit ... I mean that would be one hell of a query, wouldn't it.  If
your query is upwards of a million lines, surely you are in deep trouble
already.

Does your text editor handle files longer than 4 billion lines?

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Fri, Jul 11, 2014 at 11:01 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Jeevan Chalke wrote:
>
>> On Fri, Jul 11, 2014 at 3:13 PM, Sawada Masahiko <sawada.mshk@gmail.com>
>> wrote:
>
>> > And the line number should be switched to 1 when line number has
>> > reached to INT_MAX?
>>
>> Yes, when it goes beyond INT_MAX, wrap around to 1.
>>
>> BTW, I wonder, can't we simply use unsigned int instead?
>
> That was my thought also: let the variable be unsigned, and have it wrap
> around normally.  So once you reach UINT_MAX, the next line number is
> zero (instead of getting stuck at UINT_MAX, which would be rather
> strange).  Anyway I don't think anyone is going to reach the UINT_MAX
> limit ... I mean that would be one hell of a query, wouldn't it.  If
> your query is upwards of a million lines, surely you are in deep trouble
> already.
>
> Does your text editor handle files longer than 4 billion lines?
>

As you said, if line number reached UINT_MAX then I think that this
case is too strange.
I think INT_MAX is enough for line number.

The v5 patch which Jeevan is created seems to good.
But one point, I got hunk when patch is applied to HEAD. (doc file)
So I have revised it and attached.

Regards,

-------
Sawada Masahiko

Sawada Masahiko wrote:

> As you said, if line number reached UINT_MAX then I think that this
> case is too strange.
> I think INT_MAX is enough for line number.

My point is not whether 2 billion is a better number than 4 billion as a
maximum value.  My point is that wraparound of signed int is, I think,
not even defined in C, whereas wraparound of unsigned int is well
defined.  cur_line should be declared as unsigned int.  I don't trust
that INT_MAX+2 arithmetic.

Please don't use cur_line as a name for a global variable.  Something
like PSQLLineNumber seems more appropriate if it's going to be exposed
through prompt.h.  However, note that MainLoop() keeps state in local
variables and notes that it is reentrant; what happens to your cur_line
when a file is read by \i and similar?  I wonder if it should be part of
PsqlScanStateData instead ...

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sat, Jul 12, 2014 at 2:19 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Sawada Masahiko wrote:
>
>> As you said, if line number reached UINT_MAX then I think that this
>> case is too strange.
>> I think INT_MAX is enough for line number.
>
> My point is not whether 2 billion is a better number than 4 billion as a
> maximum value.  My point is that wraparound of signed int is, I think,
> not even defined in C, whereas wraparound of unsigned int is well
> defined.  cur_line should be declared as unsigned int.  I don't trust
> that INT_MAX+2 arithmetic.
>
> Please don't use cur_line as a name for a global variable.  Something
> like PSQLLineNumber seems more appropriate if it's going to be exposed
> through prompt.h.  However, note that MainLoop() keeps state in local
> variables and notes that it is reentrant; what happens to your cur_line
> when a file is read by \i and similar?  I wonder if it should be part of
> PsqlScanStateData instead ...
>

Thank you for comment.
I restarted to make this patch again.

Attached patch is new version patch, and rebased.
pset structure has cur_lineno variable which shows current line number
as unsigned int64.

Regards,

-------
Sawada Masahiko

On Wed, Aug 20, 2014 at 9:00 PM, Jeevan Chalke
<jeevan.chalke@enterprisedb.com> wrote:
> Hi,
>
> I have reviewed this:
>
> I have initialize cur_lineno to UINTMAX - 2. And then observed following
> behaviour to check wrap-around.
>
> postgres=# \set PROMPT1 '%/[%l]%R%# '
> postgres[18446744073709551613]=# \set PROMPT2 '%/[%l]%R%# '
> postgres[18446744073709551613]=# select
> postgres[18446744073709551614]-# a
> postgres[18446744073709551615]-# ,
> postgres[0]-# b
> postgres[1]-# from
> postgres[2]-# dual;
>
> It is wrapping to 0, where as line number always start with 1. Any issues?
>
> I would like to ignore this as UINTMAX lines are too much for a input
> buffer to hold. It is almost NIL chances to hit this.
>
>
> However, I think you need to use UINT64_FORMAT while printing uint64
> number. Currently it is %u which wrap-around at UINT_MAX.
> See how pset.lineno is displayed.
>
> Apart from this, I didn't see any issues in my testing.
>
> Patch applies cleanly. make/make install/initdb/make check all are well.
>

Thank you for reviewing the patch!
Attached patch is latest version patch.
I modified the output format of cur_lineno.

Regards,

-------
Sawada Masahiko

Jeevan Chalke wrote:

> I have initialize cur_lineno to UINTMAX - 2. And then observed following
> behaviour to check wrap-around.
> 
> postgres=# \set PROMPT1 '%/[%l]%R%# '
> postgres[18446744073709551613]=# \set PROMPT2 '%/[%l]%R%# '
> postgres[18446744073709551613]=# select
> postgres[18446744073709551614]-# a
> postgres[18446744073709551615]-# ,
> postgres[0]-# b
> postgres[1]-# from
> postgres[2]-# dual;
> 
> It is wrapping to 0, where as line number always start with 1. Any issues?
> 
> I would like to ignore this as UINTMAX lines are too much for a input
> buffer to hold. It is almost NIL chances to hit this.

Yeah, most likely you will run out of memory before reaching that point,
or out of patience.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2014-08-21 11:43:48 +0900, Sawada Masahiko wrote:
> On Wed, Aug 20, 2014 at 9:00 PM, Jeevan Chalke
> <jeevan.chalke@enterprisedb.com> wrote:
> > Hi,
> >
> > I have reviewed this:
> >
> > I have initialize cur_lineno to UINTMAX - 2. And then observed following
> > behaviour to check wrap-around.
> >
> > postgres=# \set PROMPT1 '%/[%l]%R%# '
> > postgres[18446744073709551613]=# \set PROMPT2 '%/[%l]%R%# '
> > postgres[18446744073709551613]=# select
> > postgres[18446744073709551614]-# a
> > postgres[18446744073709551615]-# ,
> > postgres[0]-# b
> > postgres[1]-# from
> > postgres[2]-# dual;
> >
> > It is wrapping to 0, where as line number always start with 1. Any issues?
> >
> > I would like to ignore this as UINTMAX lines are too much for a input
> > buffer to hold. It is almost NIL chances to hit this.
> >
> >
> > However, I think you need to use UINT64_FORMAT while printing uint64
> > number. Currently it is %u which wrap-around at UINT_MAX.
> > See how pset.lineno is displayed.
> >
> > Apart from this, I didn't see any issues in my testing.
> >
> > Patch applies cleanly. make/make install/initdb/make check all are well.
> >
>
> Thank you for reviewing the patch!
> Attached patch is latest version patch.
> I modified the output format of cur_lineno.

I like the feature - and I wanted to commit it, but enough stuff turned
up that I needed to fix that it warrants some new testing.

Stuff I've changed:
* removed include of limits.h - that probably was a rememnant from a
  previous version
* removed a trailing whitespace
* expanded the documentation about %l. "The current line number" isn't
  very clear. Of a file? Of all lines ever entered in psql? It's now
  "The line number inside the current statement, starting from
  <literal>1</>."
* Correspondingly I've changed the variable's name to stmt_lineno.
* COPY FROM ... STDIN/PROMPT3 was broken because a) the promp was only generated
  once b) the lineno wasn't incremented.
* CTRL-C didn't reset the line number.
*  Unfortunately I've notice here that the prompting is broken in some
common cases:

postgres[1]=# SELECT 1,
postgres[2]-# '2
postgres[2]'# 2b
postgres[2]'# 2c
postgres[2]'# 2d',
postgres[3]-# 3;
┌──────────┬──────────┬──────────┐
│ ?column? │ ?column? │ ?column? │
├──────────┼──────────┼──────────┤
│        1 │ 2       ↵│        3 │
│          │ 2b      ↵│          │
│          │ 2c      ↵│          │
│          │ 2d       │          │
└──────────┴──────────┴──────────┘
(1 row)

postgres[1]=# SELECT 1,
'2
2b
2c
2d',
3
postgres[7]-#

That's rather inconsistent...


I've attached my version of the patch. Note that I've got rid of all the
PSCAN_INCOMPLETE checks... Please test!

Greetings,

Andres Freund

--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On Tue, Aug 26, 2014 at 10:23 PM, Andres Freund <andres@2ndquadrant.com> wrote:
> On 2014-08-21 11:43:48 +0900, Sawada Masahiko wrote:
>> On Wed, Aug 20, 2014 at 9:00 PM, Jeevan Chalke
>> <jeevan.chalke@enterprisedb.com> wrote:
>> > Hi,
>> >
>> > I have reviewed this:
>> >
>> > I have initialize cur_lineno to UINTMAX - 2. And then observed following
>> > behaviour to check wrap-around.
>> >
>> > postgres=# \set PROMPT1 '%/[%l]%R%# '
>> > postgres[18446744073709551613]=# \set PROMPT2 '%/[%l]%R%# '
>> > postgres[18446744073709551613]=# select
>> > postgres[18446744073709551614]-# a
>> > postgres[18446744073709551615]-# ,
>> > postgres[0]-# b
>> > postgres[1]-# from
>> > postgres[2]-# dual;
>> >
>> > It is wrapping to 0, where as line number always start with 1. Any issues?
>> >
>> > I would like to ignore this as UINTMAX lines are too much for a input
>> > buffer to hold. It is almost NIL chances to hit this.
>> >
>> >
>> > However, I think you need to use UINT64_FORMAT while printing uint64
>> > number. Currently it is %u which wrap-around at UINT_MAX.
>> > See how pset.lineno is displayed.
>> >
>> > Apart from this, I didn't see any issues in my testing.
>> >
>> > Patch applies cleanly. make/make install/initdb/make check all are well.
>> >
>>
>> Thank you for reviewing the patch!
>> Attached patch is latest version patch.
>> I modified the output format of cur_lineno.
>
> I like the feature - and I wanted to commit it, but enough stuff turned
> up that I needed to fix that it warrants some new testing.
>
> Stuff I've changed:
> * removed include of limits.h - that probably was a rememnant from a
>   previous version
> * removed a trailing whitespace
> * expanded the documentation about %l. "The current line number" isn't
>   very clear. Of a file? Of all lines ever entered in psql? It's now
>   "The line number inside the current statement, starting from
>   <literal>1</>."
> * Correspondingly I've changed the variable's name to stmt_lineno.
> * COPY FROM ... STDIN/PROMPT3 was broken because a) the promp was only generated
>   once b) the lineno wasn't incremented.
> * CTRL-C didn't reset the line number.
> *  Unfortunately I've notice here that the prompting is broken in some
> common cases:
>
> postgres[1]=# SELECT 1,
> postgres[2]-# '2
> postgres[2]'# 2b
> postgres[2]'# 2c
> postgres[2]'# 2d',
> postgres[3]-# 3;
> ┌──────────┬──────────┬──────────┐
> │ ?column? │ ?column? │ ?column? │
> ├──────────┼──────────┼──────────┤
> │        1 │ 2       ↵│        3 │
> │          │ 2b      ↵│          │
> │          │ 2c      ↵│          │
> │          │ 2d       │          │
> └──────────┴──────────┴──────────┘
> (1 row)
>
> postgres[1]=# SELECT 1,
> '2
> 2b
> 2c
> 2d',
> 3
> postgres[7]-#
>
> That's rather inconsistent...
>
>
> I've attached my version of the patch. Note that I've got rid of all the
> PSCAN_INCOMPLETE checks... Please test!
>

Thank you for review comment and improving the patch!
I tested it.
Your patch always increment line number even if there is no input line
as follows.

postgres[1]=#
postgres[2]=# select
postgres[3]-# ,
postgres[4]-# from
postgres[5]-# hoge;
ERROR:  syntax error at or near "," at character 8
STATEMENT:  select
,
from
hoge;
ERROR:  syntax error at or near ","
LINE 2: ,
        ^
Actually error syntax is in line 2 as postgres reported.
But it is inconsistent.
Attached patch is resolve above behavior based on your version patch.

Regards,

-------
Sawada Masahiko

On 2014-08-31 12:06:31 +0900, Sawada Masahiko wrote:
> Thank you for review comment and improving the patch!
> I tested it.
> Your patch always increment line number even if there is no input line
> as follows.
>
> postgres[1]=#
> postgres[2]=# select
> postgres[3]-# ,
> postgres[4]-# from
> postgres[5]-# hoge;
> ERROR:  syntax error at or near "," at character 8
> STATEMENT:  select
> ,
> from
> hoge;
> ERROR:  syntax error at or near ","
> LINE 2: ,
>         ^
> Actually error syntax is in line 2 as postgres reported.
> But it is inconsistent.

Hm. Right. That's clearly wrong.

> Attached patch is resolve above behavior based on your version patch.

I've looked a bit further and found two more broken things.

1)
postgres[1]=# SELECT 1; SELECT 2
postgres[1]=#

Note the 1 in the second line. Obviously wrong.

The fix for this is easy: Don't count a newline if there isn't one. But
check for PSCAN_EOL. That also gets rid of inconsistent pset.stmt_lineno
initializations (sometimes to 0, sometimes to 1).

2)
postgres[1]=# SELECT 1,
postgres[2]-# 2,
postgres[3]-# 3;
┌──────────┬──────────┬──────────┐
│ ?column? │ ?column? │ ?column? │
├──────────┼──────────┼──────────┤
│        1 │        2 │        3 │
└──────────┴──────────┴──────────┘
(1 row)

postgres[1]=# SELECT 1,
2,
3;
┌──────────┬──────────┬──────────┐
│ ?column? │ ?column? │ ?column? │
├──────────┼──────────┼──────────┤
│        1 │        2 │        3 │
└──────────┴──────────┴──────────┘
(1 row)

postgres[3]=#

Obviously the three in the last line is wrong.

The fix is slightly nontrivial. It's wrong to look at 'line' when
determining the number of lines to add - it may already be
executed. The, it seems to me, correct thing is to look at the data
that's appended to the query buffer. Alternatively we could always count
all lines in the query buffer, but that'd be O(lines^2)...


I've done both in the appended patch.


I've now used up a perfectly good glass of wine for this, so this is it
for today ;)

Greetings,

Andres Freund

--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On Tue, Sep 2, 2014 at 11:12 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> I've now used up a perfectly good glass of wine for this.
Red or white? From where? Useful tips for hacking in this area are
always useful.
-- 
Michael

On 2014-09-02 11:19:09 +0900, Michael Paquier wrote:
> On Tue, Sep 2, 2014 at 11:12 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > I've now used up a perfectly good glass of wine for this.
> Red or white? From where? Useful tips for hacking in this area are
> always useful.

Hah ;). Nothing special, but I rather like it: Bordeaux Superieur Cuvee,
Chateau Couronneau 2011.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services

On Tue, Sep 2, 2014 at 11:12 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> On 2014-08-31 12:06:31 +0900, Sawada Masahiko wrote:
>> Thank you for review comment and improving the patch!
>> I tested it.
>> Your patch always increment line number even if there is no input line
>> as follows.
>>
>> postgres[1]=#
>> postgres[2]=# select
>> postgres[3]-# ,
>> postgres[4]-# from
>> postgres[5]-# hoge;
>> ERROR:  syntax error at or near "," at character 8
>> STATEMENT:  select
>> ,
>> from
>> hoge;
>> ERROR:  syntax error at or near ","
>> LINE 2: ,
>>         ^
>> Actually error syntax is in line 2 as postgres reported.
>> But it is inconsistent.
>
> Hm. Right. That's clearly wrong.
>
>> Attached patch is resolve above behavior based on your version patch.
>
> I've looked a bit further and found two more broken things.
>
> 1)
> postgres[1]=# SELECT 1; SELECT 2
> postgres[1]=#
>
> Note the 1 in the second line. Obviously wrong.
>
> The fix for this is easy: Don't count a newline if there isn't one. But
> check for PSCAN_EOL. That also gets rid of inconsistent pset.stmt_lineno
> initializations (sometimes to 0, sometimes to 1).
>
> 2)
> postgres[1]=# SELECT 1,
> postgres[2]-# 2,
> postgres[3]-# 3;
> ┌──────────┬──────────┬──────────┐
> │ ?column? │ ?column? │ ?column? │
> ├──────────┼──────────┼──────────┤
> │        1 │        2 │        3 │
> └──────────┴──────────┴──────────┘
> (1 row)
>
> postgres[1]=# SELECT 1,
> 2,
> 3;
> ┌──────────┬──────────┬──────────┐
> │ ?column? │ ?column? │ ?column? │
> ├──────────┼──────────┼──────────┤
> │        1 │        2 │        3 │
> └──────────┴──────────┴──────────┘
> (1 row)
>
> postgres[3]=#
>
> Obviously the three in the last line is wrong.
>
> The fix is slightly nontrivial. It's wrong to look at 'line' when
> determining the number of lines to add - it may already be
> executed. The, it seems to me, correct thing is to look at the data
> that's appended to the query buffer. Alternatively we could always count
> all lines in the query buffer, but that'd be O(lines^2)...
>
>
> I've done both in the appended patch.
>
>
> I've now used up a perfectly good glass of wine for this, so this is it
> for today ;)
>

Thank you for updating the patch!
I tested it.
These fix looks good to me :)

Regards,

-------
Sawada Masahiko

On 2014-09-02 12:34:12 +0900, Sawada Masahiko wrote:
> On Tue, Sep 2, 2014 at 11:12 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > I've now used up a perfectly good glass of wine for this, so this is it
> > for today ;)
> >
> 
> Thank you for updating the patch!
> I tested it.
> These fix looks good to me :)

Committed. Thanks for the patch!

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services