Обсуждение: psql copy filename path wildcard

Поиск
Список
Период
Сортировка

psql copy filename path wildcard

От
Mahesh Wadekar
Дата:
Hi Team,

I am copying data from S3 to local using psql copy command but file name path is daily changing. so i am trying to put '*' but its giving error.

I am trying below command from psql prompt.

copy tablename from 's3://S3-bucket-name//detailedconsumption/*/*/file.json' credentials     'aws_iam_role=arn:aws:iam::xxxxxxxx:role/RedshiftS3Role2' region 'xxxxx'     GZIP CSV IGNOREHEADER 1 TIMEFORMAT 'auto' manifest;

Below error message we got.

ERROR:  Problem reading manifest file - S3ServiceException:The specified key does not exist.,Status 404,Error NoSuchKey,Rid 81DF7C1A9B0BE7BF,ExtRid Cx+8oamynCoaG6IAMEbbgwGM1ngpCJSuJ/8az1nX/C2djLLlo2huhzLzQBAQrTORXoglc1zOO7E=,CanRetry 1


Please suggest method in place of '*'.


Regards,

Mahesh WADEKAR

Re: psql copy filename path wildcard

От
Guillaume Lelarge
Дата:
Le lun. 4 mai 2020 à 09:40, Mahesh Wadekar <maheshwadekar@gmail.com> a écrit :
Hi Team,

I am copying data from S3 to local using psql copy command but file name path is daily changing. so i am trying to put '*' but its giving error.

I am trying below command from psql prompt.

copy tablename from 's3://S3-bucket-name//detailedconsumption/*/*/file.json' credentials     'aws_iam_role=arn:aws:iam::xxxxxxxx:role/RedshiftS3Role2' region 'xxxxx'     GZIP CSV IGNOREHEADER 1 TIMEFORMAT 'auto' manifest;

Below error message we got.

ERROR:  Problem reading manifest file - S3ServiceException:The specified key does not exist.,Status 404,Error NoSuchKey,Rid 81DF7C1A9B0BE7BF,ExtRid Cx+8oamynCoaG6IAMEbbgwGM1ngpCJSuJ/8az1nX/C2djLLlo2huhzLzQBAQrTORXoglc1zOO7E=,CanRetry 1


Please suggest method in place of '*'.


Write a shell script to get the filepath and give it to psql. That's the simplest thing to do.


--
Guillaume.