Обсуждение: Import: function report
Hi all This mail will describe "import function" with some pictures. 1. Entry (table context menu): http://lh3.ggpht.com/_HsL2j8BRguc/SXboWADkDUI/AAAAAAAAAD0/1m7idvvyXHk/pgImport%201%20entry.JPG 2. Source type: http://lh5.ggpht.com/_HsL2j8BRguc/SXboWERWhSI/AAAAAAAAAD8/dYpJeleR85U/s512/pgImport%202%20src.JPG These items will be certer of wizard, don't worry about this now. The third option "Database". It is unnecessary and will be removed. Beacause the end users can do it with db-link. The ODBC option will be enabled only when wxWidgets is compiled with wxUSE_ODBC=1. Maybe some users need more kinds of source. This is the reason why separate them from others options. Yesterday, I thought import from archieve file (tar gz zip ...) may needed. The users often get a compressed file when there are large data. 3. Text file: http://lh4.ggpht.com/_HsL2j8BRguc/SXboWeCIj4I/AAAAAAAAAEE/socmOXfpu40/s512/pgImport%203%20file.JPG There are about 39 encoding. Delimiter can be multi-wchars. Width delimiter separeted by comma, like "10,15,,23" (empty means 0). Quote by single or double quotation. 4. Option: http://lh6.ggpht.com/_HsL2j8BRguc/SXboWV5GDLI/AAAAAAAAAEM/CJ1boJJANNQ/s512/pgImport%204%20opt.JPG The option "Set NULL on empty text" will be extends to other types. Just like: On empty text: Boolean : false Numberic: 0 Char : NULL and so forth New options will be added: Boolean true : (yes,y,1,on) Boolean false: (no, n,0,off) and more. 5 Format: http://lh3.ggpht.com/_HsL2j8BRguc/SXboWRc7WqI/AAAAAAAAAEU/vLm1sd-eIzY/s512/pgImport%205%20fmt.JPG The "Ignore formats" Option will try to insert data into DB without any change. New option "format style" will be added, user can select a style, ANSI C, PostgreSQL, RFC-822. If users would like to use COPY to insert into DB, them have to select a non-pgsql style. 6. Column http://lh5.ggpht.com/_HsL2j8BRguc/SXbqNfJJU-I/AAAAAAAAAE0/WoeAnjD4PCo/s400/pgImport%206%20col.JPG The target table's column info. The users can select source column for every table column and set its format. In my plan, there is a new col "option", user can control act when empty value incurred. (That be set in Option page will be default here.) If the user decide to insert data with INSERT, then "Expression" col can be used to set a single PostgreSQL expression, like "Column #1 || Columns #12". Grid in bottom is a preview. 7. Status http://lh3.ggpht.com/_HsL2j8BRguc/SXbqNWmvhEI/AAAAAAAAAE8/Kl2d27GZgdk/s512/pgImport%207%20sta.JPG Destination can be "create INSERT statments into file", "create COPY statments into file", "Import into database with INSERT", "Import into database with COPY", and so on. And more log option. (Save to log file, Only log error/warn.) This is my plan about import from text file. Most of code is not completed. You are welcome to give me any suggestions. It looks too complicated. Can I implement it? ( o.O ) Regards. ----------------------------------------------- Quan Zongliang quanzongliang@gmail.com CIT Japan: http://www.cit.co.jp CIT China: http://www.citbj.com.cn
Hi On Wed, Jan 21, 2009 at 10:54 AM, Quan Zongliang <quanzongliang@gmail.com> wrote: > 2. Source type: > http://lh5.ggpht.com/_HsL2j8BRguc/SXboWERWhSI/AAAAAAAAAD8/dYpJeleR85U/s512/pgImport%202%20src.JPG > These items will be certer of wizard, don't worry about this now. > The third option "Database". It is unnecessary and will be removed. > Beacause the end users can do it with db-link. > The ODBC option will be enabled only when wxWidgets is compiled with wxUSE_ODBC=1. > Maybe some users need more kinds of source. This is the reason why separate > them from others options. > Yesterday, I thought import from archieve file (tar gz zip ...) may needed. > The users often get a compressed file when there are large data. Yeah, could be useful I guess. I think the Database option is useful though - dblink is useful, but it's not as handy as having a nice GUI which can move and transform data. Certainly not a 'must have' item for v1.0 though. > 3. Text file: > http://lh4.ggpht.com/_HsL2j8BRguc/SXboWeCIj4I/AAAAAAAAAEE/socmOXfpu40/s512/pgImport%203%20file.JPG > There are about 39 encoding. > Delimiter can be multi-wchars. > Width delimiter separeted by comma, like "10,15,,23" (empty means 0). > Quote by single or double quotation. Can it cope with quoted strings with unquoted non-strings? Eg. 3232,43.22,"This is a string",999 iirc, some CSV formats look like that. > 4. Option: > http://lh6.ggpht.com/_HsL2j8BRguc/SXboWV5GDLI/AAAAAAAAAEM/CJ1boJJANNQ/s512/pgImport%204%20opt.JPG > The option "Set NULL on empty text" will be extends to other types. > Just like: > On empty text: > Boolean : false > Numberic: 0 > Char : NULL > and so forth > New options will be added: > Boolean true : (yes,y,1,on) > Boolean false: (no, n,0,off) > and more. What's the 'Import rows match' option for? It sounds like it might be a regexp filter over the raw row data. > 6. Column > http://lh5.ggpht.com/_HsL2j8BRguc/SXbqNfJJU-I/AAAAAAAAAE0/WoeAnjD4PCo/s400/pgImport%206%20col.JPG > The target table's column info. > The users can select source column for every table column and set its format. > In my plan, there is a new col "option", user can control act when empty value incurred. > (That be set in Option page will be default here.) > If the user decide to insert data with INSERT, then "Expression" col can be used to set > a single PostgreSQL expression, like "Column #1 || Columns #12". > > Grid in bottom is a preview. Should the format options from page 5. be included as selectable options on the grid? It would allow the import data to have different formats in different columns, plus it would remove an entire page from the wizard. > 7. Status > http://lh3.ggpht.com/_HsL2j8BRguc/SXbqNWmvhEI/AAAAAAAAAE8/Kl2d27GZgdk/s512/pgImport%207%20sta.JPG > Destination can be "create INSERT statments into file", "create COPY statments into file", > "Import into database with INSERT", "Import into database with COPY", and so on. Oooh, I like that. > This is my plan about import from text file. > Most of code is not completed. > You are welcome to give me any suggestions. > > It looks too complicated. Can I implement it? ( o.O ) I think it looks ambitious (though certainly achievable) for v1.0, given that this is unlikely to be in the next release. I didn't see any functionality or options that seemed complex for the sake of it - everything seemed to have a useful purpose. I think it's progressing well. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com
> Can it cope with quoted strings with unquoted non-strings? Eg. > > 3232,43.22,"This is a string",999 > > iirc, some CSV formats look like that. Yes, they can be commixed. > What's the 'Import rows match' option for? It sounds like it might be > a regexp filter over the raw row data. Yes, it is a filter. > Should the format options from page 5. be included as selectable > options on the grid? It would allow the import data to have different > formats in different columns, plus it would remove an entire page from > the wizard. Ok, right. ----------------------------------------------- Quan Zongliang quanzongliang@gmail.com CIT Japan: http://www.cit.co.jp CIT China: http://www.citbj.com.cn