Обсуждение: Issue import (COPY) data by between miss lines

Поиск
Список
Период
Сортировка

Issue import (COPY) data by between miss lines

От
phuongnh2
Дата:
Hi Admin,
I have issue import data (COPY). Data content between miss lines
Row 1: have 26 columns (DELIMITER BY ,)
Row 2: have 24 column  (DELIMITER BY ,)
How to skip and write to log file this line ??

sample:
Row1:

20150416ATM11429151780752,G6MOBILE,123PVTB,FARM,704450787,50000,10.30.81.2,704450787,MTgyIExlIERhaSBoYW5oIC0gUTExIC0gVFAgSENN,M,1/1/1970,0978233650,so6@vng.com.vn,Nạp
ruby cho

704450787.,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,G6MOBILEkiMicJopM34kcam,bbac3d3f2615fcd1f1f4a9fe3a2c0c45bac009ab,,1,123P1504161455496,https://online.banknetvn.com.vn/pg/transprocess?Trans_id=c98d0f1693a49312b9011615066346a8,d3c516fb43a889ffa233674676188ff76e2957dd,Tao
don hang thanh cong,2015-04-16 09:36:22

Row2:

20150416ATM21429151915479,G6MOBILE,123PBAB,FARM,50000,10.30.81.2,704443473,MTgyIExlIERhaSBoYW5oIC0gUTExIC0gVFAgSENN,M,01687302683,so6@vng.com.vn,Nạp
ruby cho

704443473.,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,G6MOBILEkiMicJopM34kcam,0b4459ff26e11f4bf8bede11b4ee0129e42a2e25,,6203,,,,Khong
ho tro ngan hang da chon,2015-04-16 09:38:35

20150416ATM21429151929774,G6MOBILE,123PVTB,FARM,704293984,50000,10.30.81.2,704293984,MTgyIExlIERhaSBoYW5oIC0gUTExIC0gVFAgSENN,M,1/1/1970,0972198329,so6@vng.com.vn,Nạp
ruby cho

704293984.,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,G6MOBILEkiMicJopM34kcam,53e3082c97f26fc7632247d905471ac679ecf39b,,1,123P1504161455511,https://online.banknetvn.com.vn/pg/transprocess?Trans_id=0c23fe272e3fb15460f9153899648853,f6f7bff5dfc7f2d9bcd4bd2a98761e1911cf8530,Tao
don hang thanh cong,2015-04-16 09:38:51

my code:
COPY raw_table FROM '/data/scribe_log/CREATE_ORDER-2015-04-16_00009' WITH
(FORMAT CSV, HEADER, DELIMITER E',', NULL '\n');

thanks,
phuongnh2







--
View this message in context: http://postgresql.nabble.com/Issue-import-COPY-data-by-between-miss-lines-tp5848263.html
Sent from the PostgreSQL - admin mailing list archive at Nabble.com.


Re: Issue import (COPY) data by between miss lines

От
phuongnh2
Дата:
I have retry this

COPY raw_table FROM '/data/scribe_log/CREATE_ORDER-2015-04-16_00009' WITH
(FORMAT CSV, HEADER, DELIMITER E',', NULL '\n',ERROR_LOGGING,
ERROR_LOGGING_SKIP_BAD_ROWS);

ERROR:  option "error_logging" not recognized

Current version: "PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu, compiled by
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11), 64-bit"

How to fix it ??

thanks,
phuongnh2




--
View this message in context:
http://postgresql.nabble.com/Issue-import-COPY-data-by-between-miss-lines-tp5848263p5848264.html
Sent from the PostgreSQL - admin mailing list archive at Nabble.com.


Re: Issue import (COPY) data by between miss lines

От
phuongnh2
Дата:
Hi Admin,
How to config postgres to result below ??

 CREATE TEMP TABLE foo (a bigint, b text);
-- input_file.txt --

 1    one
 2
 3    three    111
 four    4
 5    five
 error logging off
 COPY foo FROM 'input_file.txt';

 ERROR:  missing data for column "b"
 CONTEXT:  COPY foo, line 2: "2"

skip bad rows
 --skip bad rows
 COPY foo FROM 'input_file.txt' (ERROR_LOGGING,
ERROR_LOGGING_SKIP_BAD_ROWS);
 SELECT * FROM foo;

  a |  b
 ---+------
  1 | one
  5 | five
 (2 rows)




--
View this message in context:
http://postgresql.nabble.com/Issue-import-COPY-data-by-between-miss-lines-tp5848263p5848403.html
Sent from the PostgreSQL - admin mailing list archive at Nabble.com.


Re: Issue import (COPY) data by between miss lines

От
Guillaume Lelarge
Дата:
Hi,

2015-05-08 6:41 GMT+02:00 phuongnh2 <phuongnh2@vng.com.vn>:
Hi Admin,
How to config postgres to result below ??

 CREATE TEMP TABLE foo (a bigint, b text);
-- input_file.txt --

 1      one
 2
 3      three   111
 four    4
 5      five
 error logging off
 COPY foo FROM 'input_file.txt';

 ERROR:  missing data for column "b"
 CONTEXT:  COPY foo, line 2: "2"

skip bad rows
 --skip bad rows
 COPY foo FROM 'input_file.txt' (ERROR_LOGGING,
ERROR_LOGGING_SKIP_BAD_ROWS);
 SELECT * FROM foo;

  a |  b
 ---+------
  1 | one
  5 | five
 (2 rows)


You can't use COPY do to that. That feature does not exist with PostgreSQL itself. pgloader should be able to do that, you should take a look.


--