Hi hackers,
I'd like to propose adding a LIMIT option to COPY FROM, which limits
the number of rows to load.
With COPY TO, we can use the LIMIT clause in the query to restrict
output rows, but COPY FROM has no equivalent way to limit the number
of rows to load (except using the PROGRAM option with external tools
like head). This patch resolves that asymmetry.
Syntax example:
- COPY t FROM STDIN (LIMIT 100);
This feature is useful for:
- Loading only the first N rows from a huge CSV file to verify data or
table definitions before a full import
- Sampling production data for staging or testing environments
- Preventing unexpectedly large data loads in ETL pipelines
Design:
- The LIMIT count applies after WHERE filtering and ON_ERROR skipping,
so it represents the actual number of rows inserted.
- When the source is STDIN, the server reads and discards the
remaining stream to keep the frontend/backend protocol synchronized.
The patch is attached. Thoughts?
--
Best regards,
Shinya Kato
NTT OSS Center