Sqoop – Import part 2
Sqoop Option file (Creditinals Save)
import
-Dorg.apache.sqoop.splitter.allow_text_splitter=true
–connect
jdbc:mysql://gator4148.hostgator.com/retail_db
–username
root
–password
password
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–table CustomerBase \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6
sqoop import in Textfile format
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–table CustomerBase \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6 \
–as-textfile
sqoop import in parquet file format
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–table CustomerBase \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6 \
–as-parquetfile
run custom queries using sqoop
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase where \$CONDITIONS” \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6
Handling NULL values part 1 (without any null handling)
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase_with_primarykey_with_null where \$CONDITIONS” \
–target-dir /user/password/sqoop/CustomerBase_with_primarykey_with_null \
–split-by “cust_id” \
–delete-target-dir \
–m 6
Handling NULL values part 1 (with null handling)
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase_with_primarykey_with_null where \$CONDITIONS” \
–target-dir /user/password/sqoop/CustomerBase_with_primarykey_with_null \
–split-by “cust_id” \
–delete-target-dir \
–null-string BLANK \
–m 6
Handling NULL values part 1 (with null handling for non string datatypes)
sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase_with_primarykey_with_null where \$CONDITIONS” \
–target-dir /user/password/sqoop/CustomerBase_with_primarykey_with_null \
–split-by “cust_id” \
–delete-target-dir \
–null-string BLANK \
–null-non-string 0 \
–m 6
