Sqoop – Import part 2

Estimated reading: 2 minutes 368 views

Sqoop Option file (Creditinals Save)


import
-Dorg.apache.sqoop.splitter.allow_text_splitter=true
–connect
jdbc:mysql://gator4148.hostgator.com/retail_db
–username
root
–password
password

sqoop \
–options-file /home/password/sqoop/sqoop-options \
–table CustomerBase \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6

sqoop import in Textfile format

sqoop \
–options-file /home/password/sqoop/sqoop-options \
–table CustomerBase \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6 \
–as-textfile

sqoop import in parquet file format

sqoop \
–options-file /home/password/sqoop/sqoop-options \
–table CustomerBase \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6 \
–as-parquetfile

run custom queries using sqoop

 sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase where \$CONDITIONS” \
–target-dir /user/password/sqoop/customer_base_with_no_pk \
–split-by “cust_id” \
–delete-target-dir \
–m 6

Handling NULL values part 1 (without any null handling)


sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase_with_primarykey_with_null where \$CONDITIONS” \
–target-dir /user/password/sqoop/CustomerBase_with_primarykey_with_null \
–split-by “cust_id” \
–delete-target-dir \
–m 6

Handling NULL values part 1 (with null handling)

sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase_with_primarykey_with_null where \$CONDITIONS” \
–target-dir /user/password/sqoop/CustomerBase_with_primarykey_with_null \
–split-by “cust_id” \
–delete-target-dir \
–null-string BLANK \
–m 6

Handling NULL values part 1 (with null handling for non string datatypes)

sqoop \
–options-file /home/password/sqoop/sqoop-options \
–query “select cust_id,customer_segment,age,customer_vintage_group from CustomerBase_with_primarykey_with_null where \$CONDITIONS” \
–target-dir /user/password/sqoop/CustomerBase_with_primarykey_with_null \
–split-by “cust_id” \
–delete-target-dir \
–null-string BLANK \
–null-non-string 0 \
–m 6

Leave a Comment

CONTENTS