Re: [Proposal] Adding TRIM_SPACE option to COPY
| От | 河田達也 |
|---|---|
| Тема | Re: [Proposal] Adding TRIM_SPACE option to COPY |
| Дата | |
| Msg-id | CAHza6qdABhLXpDo44PWYLDnvwTGHUW37Me-PSUe5Pf5vTw1kpQ@mail.gmail.com обсуждение исходный текст |
| Ответ на | Re: [Proposal] Adding TRIM_SPACE option to COPY (Tom Lane <tgl@sss.pgh.pa.us>) |
| Список | pgsql-hackers |
Hi all,
Thank you very much for the helpful feedback so far.
I would like to take a moment to discuss and confirm the overall direction
before preparing the next revision (v2).
My goal is to ensure we are aligned on the intended behavior, scope, and
design considerations for the proposed TRIM_SPACE option.
To summarize the discussion and the points I plan to address in the next action:
Thank you very much for the helpful feedback so far.
I would like to take a moment to discuss and confirm the overall direction
before preparing the next revision (v2).
My goal is to ensure we are aligned on the intended behavior, scope, and
design considerations for the proposed TRIM_SPACE option.
To summarize the discussion and the points I plan to address in the next action:
> I have a suggestion, would it be better to restrict the change to only "copy from"?
1. Scope (COPY FROM only)
I agree that the option should apply only to COPY FROM.
I will adjust the patch accordingly.
1. Scope (COPY FROM only)
I agree that the option should apply only to COPY FROM.
I will adjust the patch accordingly.
> Since the docs already notes that leading or trailing spaces in CSV values
> can cause errors during loading, at [1], we'd likely need to update
> that section as well if we proceed with this feature.
2. Documentation
I will update the CSV section mentioned earlier so that it reflects
the behavior when TRIM_SPACE is enabled.
> can cause errors during loading, at [1], we'd likely need to update
> that section as well if we proceed with this feature.
2. Documentation
I will update the CSV section mentioned earlier so that it reflects
the behavior when TRIM_SPACE is enabled.
> One question: should TRIM_SPACE remove only the literal space character (' '),
> or should it also trim other whitespace characters (e.g., tab, newline,
> those recognized by isspace())?
> or should it also trim other whitespace characters (e.g., tab, newline,
> those recognized by isspace())?
3. Characters to trim
I plan to trim only the ASCII space character (' ') to keep the behavior simple and avoid ambiguity.
Support for additional whitespace characters could be considered later if there is consensus.
> I'm kind of down on it, because it's inevitably going to add
> processing overhead to every COPY operation whether the feature
> is used or not. I don't find it likely to be sufficiently
> useful to justify that universal cost.
4. Performance / overhead concerns
Thank you for raising this point.
I fully agree that the feature must not introduce overhead when TRIM_SPACE is disabled.
The trimming logic will be executed only when the option is explicitly enabled,
so there will be just a single additional conditional check.
While this does technically add some processing overhead, it is expected to be slight in practice for typical CSV loads and unlikely to be a concern.
> COPY is not a general-purpose filter or ETL tool, and we try
> to make it one at our peril.
My intention is not to expand COPY into an ETL tool, but rather to
provide a small convenience option similar to FORCE_NULL or ON_ERROR,
to help users avoid common issues caused by unintended leading or
trailing spaces in CSV/text files.
I look forward to your thoughts and any additional feedback before preparing the next patch.
Thank you again for all the valuable comments.
Best regards,
Tatsuya Kawata
I plan to trim only the ASCII space character (' ') to keep the behavior simple and avoid ambiguity.
Support for additional whitespace characters could be considered later if there is consensus.
> I'm kind of down on it, because it's inevitably going to add
> processing overhead to every COPY operation whether the feature
> is used or not. I don't find it likely to be sufficiently
> useful to justify that universal cost.
4. Performance / overhead concerns
Thank you for raising this point.
I fully agree that the feature must not introduce overhead when TRIM_SPACE is disabled.
The trimming logic will be executed only when the option is explicitly enabled,
so there will be just a single additional conditional check.
While this does technically add some processing overhead, it is expected to be slight in practice for typical CSV loads and unlikely to be a concern.
> COPY is not a general-purpose filter or ETL tool, and we try
> to make it one at our peril.
My intention is not to expand COPY into an ETL tool, but rather to
provide a small convenience option similar to FORCE_NULL or ON_ERROR,
to help users avoid common issues caused by unintended leading or
trailing spaces in CSV/text files.
I look forward to your thoughts and any additional feedback before preparing the next patch.
Thank you again for all the valuable comments.
Best regards,
Tatsuya Kawata
В списке pgsql-hackers по дате отправления: