Обсуждение: Question Regarding COPY Command Handling of Line Breaks in PostgreSQL

Поиск
Список
Период
Сортировка

Question Regarding COPY Command Handling of Line Breaks in PostgreSQL

От
gzh
Дата:
Dear,

I am encountering an issue with the COPY command in PostgreSQL regarding the handling of line breaks in data fields. My PostgreSQL instance is installed on a Linux system.

When I use the COPY command to export data to a CSV file and then import the same CSV file back into the database, all carriage return characters (CR) in the fields are automatically converted to line feed characters (LF). For example:

Field content before import: *** (CR) ***
Field content after import: *** (LF) ***
I would like to know if there is a way to use the COPY command while preserving the original line breaks (CR) in the data fields during import.

Thank you for your assistance.

Best regards,
gzh

Re: Question Regarding COPY Command Handling of Line Breaks in PostgreSQL

От
Ron Johnson
Дата:
On Fri, Jul 11, 2025 at 8:20 AM gzh <gzhcoder@126.com> wrote:
Dear,

I am encountering an issue with the COPY command in PostgreSQL regarding the handling of line breaks in data fields. My PostgreSQL instance is installed on a Linux system.

When I use the COPY command to export data to a CSV file and then import the same CSV file back into the database, all carriage return characters (CR) in the fields are automatically converted to line feed characters (LF). For example:

Field content before import: *** (CR) ***
Field content after import: *** (LF) ***
I would like to know if there is a way to use the COPY command while preserving the original line breaks (CR) in the data fields during import.
 
What PG version?
What's the full command (including both shell and SQL commands)?

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

Re: Question Regarding COPY Command Handling of Line Breaks in PostgreSQL

От
Adrian Klaver
Дата:
On 7/11/25 06:07, Ron Johnson wrote:
> On Fri, Jul 11, 2025 at 8:20 AM gzh <gzhcoder@126.com 
> <mailto:gzhcoder@126.com>> wrote:

> What PG version?
> What's the full command (including both shell and SQL commands)?

Also provide a sample of the original exported data and a sample of the 
data after it is imported.

-- 
Adrian Klaver
adrian.klaver@aklaver.com




Re:Question Regarding COPY Command Handling of Line Breaks in PostgreSQL

От
gzh
Дата:


The cause has been identified. In my VB program, I was using the COPY command. 

To prevent memory issues with large files, I changed the file reading method from reading the entire file at once to reading it line by line, as shown below. 

However, this caused line breaks within the data to be lost. This is not an issue with the COPY command itself. 

I tested reading the entire file at once, and it worked without any problems. My apologies for the confusion.


Reading the entire file at once:


Using reader As New StreamReader(inputFile, System.Text.Encoding.UTF8)

    Using copyIn = conn.BeginTextImport("COPY Schema.tableName FROM STDIN WITH CSV HEADER")

        copyIn.Write(reader.ReadToEnd())

    End Using

End Using


Reading the file line by line:


Using reader As New StreamReader(inputFile, System.Text.Encoding.UTF8)

    Using copyIn = conn.BeginTextImport("COPY Schema.tableName FROM STDIN WITH CSV HEADER")

        While Not reader.EndOfStream

            Dim line As String = reader.ReadLine()

            copyIn.Write(line & vbLf)

        End While

    End Using

End Using





At 2025-07-11 20:20:38, "gzh" <gzhcoder@126.com> wrote:

Dear,

I am encountering an issue with the COPY command in PostgreSQL regarding the handling of line breaks in data fields. My PostgreSQL instance is installed on a Linux system.

When I use the COPY command to export data to a CSV file and then import the same CSV file back into the database, all carriage return characters (CR) in the fields are automatically converted to line feed characters (LF). For example:

Field content before import: *** (CR) ***
Field content after import: *** (LF) ***
I would like to know if there is a way to use the COPY command while preserving the original line breaks (CR) in the data fields during import.

Thank you for your assistance.

Best regards,
gzh