Changes for Postgresql Copy Export:#68
Changes for Postgresql Copy Export:#68RobertBHamilton wants to merge 3 commits intoapache:trunkfrom RobertBHamilton:trunk
Conversation
RobertBHamilton
commented
Jan 22, 2019
- Support for empty null string --null-string ''
- Support for non-xml delim --fields-terminated-by'\0x1c'
- Added line buffering perf --batch
- optional TEXT mode instead of CSV -Dpostgresql.format.text=true
- Support for postgres Version 8x -Dpostgresql.targetdb.ver=8
- Optional disable escape sequences -Dpostgresql.input.israw=true
1. Support for empty null string --null-string '' 2. Support for non-xml delim --fields-terminated-by'\0x1c' 3. Added line buffering perf --batch 4. optional TEXT mode instead of CSV -Dpostgresql.format.text=true 5. Support for postgres Version 8 -Dpostgresql.targetdb.ver=8 6. Optional disable escape sequences -Dpostgresql.input.israw=true
|
This is in process. These changes were motivated by a project at GM to move significant data sets to Greenplum. Because Greenplum is currently still in version 8x of postgress we added support for the 8x syntax of the copy command. We also noticed that implementing line buffering in the mapper would significantly enhance performance. Also we conventionally use a char for field delim which happens to be invalid XML char so we added support for non-xml delims with direct mode. |
|
Hi Robert, Thanks for sharing these improvements! I haven't done an in-depth review yet but I noticed that the indentation and the formatting you used is quite different from the surrounding code so please try to follow those conventions, e.g.:
instead of Apart from this I have seen a few commented out code lines, please remove those as well. Regards, |
|
Rebasing onto master will fix the CI again. |