Обсуждение: Finding error in long input file
I've a file with 488 rows to be input into a table. I run the script using psql with the `\i' option. After psql found a few missing commas I thought the script was clean. But, psql returned this error: bustrac=# \i scripts/insert-addrs.sql psql:scripts/insert-addrs.sql:488: ERROR: syntax error at or near ";" LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636'); ^ Am I correct that some rows were inserted into the table before this error was generated somewhere prior to the end of the file? If so, how do I determine which rows were inserted and what is the recommended procedure to locate just where that error is? I've learned from experience that using the old coding approach of dividing the file in half, then half again and again until the row with the error is found produces multiple rows with different PKs. Not what I want to clean up. A clue stick will be much appreciated. TIA, Rich
On 9 July 2024 23:14:33 Rich Shepard <rshepard@appl-ecosys.com> wrote:
I've a file with 488 rows to be input into a table. I run the script usingpsql with the `\i' option.After psql found a few missing commas I thought the script was clean. But,psql returned this error:bustrac=# \i scripts/insert-addrs.sqlpsql:scripts/insert-addrs.sql:488: ERROR: syntax error at or near ";"LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636');^ AmI correct that some rows were inserted into the table before this error wasgenerated somewhere prior to the end of the file? If so, how do I determinewhich rows were inserted and what is the recommended procedure to locatejust where that error is?
Did you run the entire thing inside a transaction? If so, then it will have been rolled back after the error, and no rows will have been inserted.
Ray.
I've learned from experience that using the old coding approach of dividingthe file in half, then half again and again until the row with the error isfound produces multiple rows with different PKs. Not what I want to cleanup.A clue stick will be much appreciated.TIA,Rich
On 7/9/24 15:14, Rich Shepard wrote: > I've a file with 488 rows to be input into a table. I run the script using > psql with the `\i' option. > > After psql found a few missing commas I thought the script was clean. But, > psql returned this error: > bustrac=# \i scripts/insert-addrs.sql > psql:scripts/insert-addrs.sql:488: ERROR: syntax error at or near ";" > LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636'); > > ^ Am > I correct that some rows were inserted into the table before this error was > generated somewhere prior to the end of the file? If so, how do I determine > which rows were inserted and what is the recommended procedure to locate > just where that error is? > > I've learned from experience that using the old coding approach of dividing > the file in half, then half again and again until the row with the error is > found produces multiple rows with different PKs. Not what I want to clean > up. > > A clue stick will be much appreciated. bustrac=#\e scripts/insert-addrs.sql 488 If that raises this error: environment variable PSQL_EDITOR_LINENUMBER_ARG must be set to specify a line number then: bustrac=#\e scripts/insert-addrs.sql If the editor is vi then: :488 > > TIA, > > Rich > > > -- Adrian Klaver adrian.klaver@aklaver.com
On Tue, 9 Jul 2024, Ray O'Donnell wrote: > Did you run the entire thing inside a transaction? If so, then it will have > been rolled back after the error, and no rows will have been inserted. Ray, When I tried using transactions they froze the application. Probably because I don't know to properly invoke them. Thanks Rich
On Tue, 9 Jul 2024, Adrian Klaver wrote: > bustrac=#\e scripts/insert-addrs.sql 488 > If that raises this error: > environment variable PSQL_EDITOR_LINENUMBER_ARG must be set to specify a line > number Adrian, It doesn't; it prints the contents of the file to the virtual terminal. Using my default small editor, joe, I scrolled back through the file displayed without seeing any error. Should I run BEGIN: at the psql prompt prior to entering the insert command? Would that tell me if any rows were entered and, if so, where that stopped? TIA, Rich
On Tue, 9 Jul 2024, Rich Shepard wrote: > Should I run BEGIN: at the psql prompt prior to entering the insert > command? Would that tell me if any rows were entered and, if so, where > that stopped? Began a transction, ran the script, checked the locations table for a couple of rows to be inserted. They weren't. Issued the rollback; command. How best should I proceed to isolate the source of the error at the end of the file? Rich
On 7/9/24 15:40, Rich Shepard wrote: > On Tue, 9 Jul 2024, Adrian Klaver wrote: > >> bustrac=#\e scripts/insert-addrs.sql 488 >> If that raises this error: >> environment variable PSQL_EDITOR_LINENUMBER_ARG must be set to specify >> a line number > > Adrian, > > It doesn't; it prints the contents of the file to the virtual terminal. > Using my default small editor, joe, I scrolled back through the file > displayed without seeing any error. The error: LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636') is giving you the line number and the data: a) Navigate to that line number using whatever method Joe has for that. b) Search for '85250 Red House Rd'. > > Should I run BEGIN: at the psql prompt prior to entering the insert > command? > Would that tell me if any rows were entered and, if so, where that stopped? > > TIA, > > Rich > > -- Adrian Klaver adrian.klaver@aklaver.com
Full error message from earlier in the thread:
> psql:scripts/insert-addrs.sql:488: ERROR: syntax error at or near ";"
> LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636');
> ^
> LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636');
> ^
The error:
LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636')
is giving you the line number and the data:
a) Navigate to that line number using whatever method Joe has for that.
b) Search for '85250 Red House Rd'.
The input file is 488 lines (presumably, since Rich said the file should insert 488 rows). It seems like too much of a coincidence that the last character of the last line is really the error. My guess is that there is an unmatched character, perhaps a parenthesis, that is throwing off the parser because it doesn't expect the statement to terminate yet. Maybe that unmatched char really is on the last line, but '85250 Red House Rd' doesn't seem like the issue. I don't know anything about the joe editor, but I'd hope that any decent editor with syntax highlighting would make it apparent where things went awry.
Craig
On 7/9/24 17:46, Craig McIlwee wrote: > Full error message from earlier in the thread: > > > psql:scripts/insert-addrs.sql:488: ERROR: syntax error at or near ";" > > LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636'); > > > ^ > > > The error: > > LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636') > > is giving you the line number and the data: > a) Navigate to that line number using whatever method Joe has for that. > b) Search for '85250 Red House Rd'. > > > > The input file is 488 lines (presumably, since Rich said the file should > insert 488 rows). It seems like too much of a coincidence that the last > character of the last line is really the error. My guess is that there This assumes that there where only INSERT lines and that each INSERT was only one line. I have bit by those assumptions before, hence my suggestion to actually find line 488. > is an unmatched character, perhaps a parenthesis, that is throwing off > the parser because it doesn't expect the statement to terminate yet. > Maybe that unmatched char really is on the last line, but '85250 Red > House Rd' doesn't seem like the issue. I don't know anything about the > joe editor, but I'd hope that any decent editor with syntax highlighting > would make it apparent where things went awry. > > Craig -- Adrian Klaver adrian.klaver@aklaver.com
> On 10 Jul 2024, at 06:58, Adrian Klaver <adrian.klaver@aklaver.com> wrote: > > On 7/9/24 17:46, Craig McIlwee wrote: >> Full error message from earlier in the thread: >> > psql:scripts/insert-addrs.sql:488: ERROR: syntax error at or near ";" >> > LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636'); >> > ^ >> The error: >> LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636') >> is giving you the line number and the data: >> a) Navigate to that line number using whatever method Joe has for that. >> b) Search for '85250 Red House Rd'. >> The input file is 488 lines (presumably, since Rich said the file should insert 488 rows). It seems like too much ofa coincidence that the last character of the last line is really the error. My guess is that there > > This assumes that there where only INSERT lines and that each INSERT was only one line. I have bit by those assumptionsbefore, hence my suggestion to actually find line 488. > >> is an unmatched character, perhaps a parenthesis, that is throwing off the parser because it doesn't expect the statementto terminate yet. Maybe that unmatched char really is on the last line, but '85250 Red House Rd' doesn't seem likethe issue. I don't know anything about the joe editor, but I'd hope that any decent editor with syntax highlightingwould make it apparent where things went awry. >> Craig > Is this a single INSERT statement with multiple tuples after VALUES? Then perhaps an earlier line (my bet would be on line487) accidentally ends with a semi-colon instead of a comma? Something like this: INSERT INTO table (col1, col2, ..., coln) VALUES (..., ..., ), (..., ..., ), (..., ..., ); -- <-- This terminates the INSERT (..., ..., ); -- <-- Now this line make no sense Alban Hertroys -- If you can't see the forest for the trees, cut the trees and you'll find there is no forest.
On Tue, 9 Jul 2024, Adrian Klaver wrote: > The error: > > LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636') > > is giving you the line number and the data: > a) Navigate to that line number using whatever method Joe has for that. > b) Search for '85250 Red House Rd'. Adrian, With the semicolon at the end of the line it's the last line in the file. That's why I asked for help in finding where, in all the lines above it, the actual error is located. FWIW, I use emacs, not vim. I'll try inserting the data a few lines at a time, each in a transaction. Thanks, Rich
On Tue, 9 Jul 2024, Craig McIlwee wrote: > The input file is 488 lines (presumably, since Rich said the file should > insert 488 rows). It seems like too much of a coincidence that the last > character of the last line is really the error. My guess is that there is > an unmatched character, perhaps a parenthesis, that is throwing off the > parser because it doesn't expect the statement to terminate yet. Maybe > that unmatched char really is on the last line, but '85250 Red House Rd' > doesn't seem like the issue. I don't know anything about the joe editor, > but I'd hope that any decent editor with syntax highlighting would make it > apparent where things went awry. Craig, et al., I use emacs for scripts and coding, joe's only for small jobs. I added a line to the file so the bottom line is now 489. The attached image shows that line is the only one terminated with a semicolon rather than a comma. psql would tell me if there was no closing parenthesis on a line, if the terminating comma was missing, or other similar error, and would tell me the number of the line or following line. Having the error marked at the end of the file does not tell _me_ just where the error actually is. Partial screenshot attached. Thanks all, Rich
Вложения
On Wednesday, July 10, 2024, Rich Shepard <rshepard@appl-ecosys.com> wrote:
Partial screenshot attached.
And what are the first few lines of the file? Use text, not screenshots.
David J.
If the file has these line breaks you show, then can make it to multiple 'INSERT INTO' instead.
Search for lines starting with parentese begin '(' and replace it with the correct INSERT and last comma to semi-colon:
cat i.sql | sed -e 's/^(/INSERT INTO foo VALUES(/' -e 's/,$/;/'
Does the file come from mysqldump? Then try option
--extended-insert=FALSE
On Wed, Jul 10, 2024 at 2:53 PM Rich Shepard <rshepard@appl-ecosys.com> wrote:
On Tue, 9 Jul 2024, Craig McIlwee wrote:
> The input file is 488 lines (presumably, since Rich said the file should
> insert 488 rows). It seems like too much of a coincidence that the last
> character of the last line is really the error. My guess is that there is
> an unmatched character, perhaps a parenthesis, that is throwing off the
> parser because it doesn't expect the statement to terminate yet. Maybe
> that unmatched char really is on the last line, but '85250 Red House Rd'
> doesn't seem like the issue. I don't know anything about the joe editor,
> but I'd hope that any decent editor with syntax highlighting would make it
> apparent where things went awry.
Craig, et al.,
I use emacs for scripts and coding, joe's only for small jobs.
I added a line to the file so the bottom line is now 489. The attached image
shows that line is the only one terminated with a semicolon rather than a
comma.
psql would tell me if there was no closing parenthesis on a line, if the
terminating comma was missing, or other similar error, and would tell me the
number of the line or following line. Having the error marked at the end of
the file does not tell _me_ just where the error actually is.
Partial screenshot attached.
Thanks all,
Rich
--
𝕳𝖆𝖓𝖘 𝕾𝖈𝖍𝖔𝖚
☏ ➁➁ ➅➃ ➇⓪ ➁⓪
On Wed, Jul 10, 2024 at 2:59 PM David G. Johnston <david.g.johnston@gmail.com> wrote:
And what are the first few lines of the file? Use text, not screenshots.
Yes the line with 'INSERT'
grep -ni 'INSERT INTO' scripts/insert-addrs.sql
--
𝕳𝖆𝖓𝖘 𝕾𝖈𝖍𝖔𝖚
☏ ➁➁ ➅➃ ➇⓪ ➁⓪
On Wed, 10 Jul 2024, David G. Johnston wrote: > And what are the first few lines of the file? Use text, not screenshots. David, insert into locations (company_nbr,loc_nbr,loc_name,addr1,city,state_code,postcode) values (2564,1,default,'4250 Hopkins Rd','Ontario','OR','97914'), (2565,1,default,'10695 Meridian Rd','Mount Angel','OR','97362'), (2566,1,default,'6963 Keene Rd NE','Gervais ','OR','97026'), (2567,1,default,'31250 S Bond Rd','Hubbard','OR','97032'), Rich
On Wed, 10 Jul 2024, Hans Schou wrote: > If the file has these line breaks you show, then can make it to multiple > 'INSERT INTO' instead. Hans, I thought of that, but forgot it. This makes more sense than dividing the file in small chunks. Thanks, Rich
On 7/10/24 05:30, Rich Shepard wrote: > On Tue, 9 Jul 2024, Adrian Klaver wrote: > >> The error: >> >> LINE 488: ...2832,1,default,'85250 Red House Rd','Paisley','OR','97636') >> >> is giving you the line number and the data: >> a) Navigate to that line number using whatever method Joe has for that. >> b) Search for '85250 Red House Rd'. > > Adrian, > > With the semicolon at the end of the line it's the last line in the file. True, I was fixated on the line number. > That's why I asked for help in finding where, in all the lines above it, > the > actual error is located. > > FWIW, I use emacs, not vim. > > I'll try inserting the data a few lines at a time, each in a transaction. If it where me I would make the data a CSV file and use \copy. That is an all or none operation and you would get a error message showing you where in the file the operation failed. > > Thanks, > > Rich > > -- Adrian Klaver adrian.klaver@aklaver.com
On Wednesday, July 10, 2024, Rich Shepard <rshepard@appl-ecosys.com> wrote:
On Wed, 10 Jul 2024, David G. Johnston wrote:And what are the first few lines of the file? Use text, not screenshots.
David,
insert into locations (company_nbr,loc_nbr,loc_name,addr1,city,state_code,postcode ) values
(2564,1,default,'4250 Hopkins Rd','Ontario','OR','97914'),
(2565,1,default,'10695 Meridian Rd','Mount Angel','OR','97362'),
(2566,1,default,'6963 Keene Rd NE','Gervais ','OR','97026'),
(2567,1,default,'31250 S Bond Rd','Hubbard','OR','97032'),
My first easy look for this setup is for any single quotes not adjacent to either a comma or a parenthesis. Syntax highlighting should ideally have caught this but I’d look anyway.
David J.
On Wed, 10 Jul 2024, David G. Johnston wrote: > My first easy look for this setup is for any single quotes not adjacent to > either a comma or a parenthesis. Syntax highlighting should ideally have > caught this but I’d look anyway. David, I found an error on line 21 that I missed seeing every time I slowly scanned the file: missing the terminal `);'. A common situation when I look at the file so many times. Rich
On Wed, 10 Jul 2024, Rob Sargent wrote: > I'm an emacs user too. Do you have show-paren enabled? This would show > that your file was ill-formed. M-p and M-n go previous/next matching > parentheses of all types. Rob, No, I haven't enabled show-paren. Thanks for the tip, Rich