车立方" type="application/atom+xml">

hahakubile Blog, Powered by 车立方

Welcome to hahakubile's blog, You should know him. Thanks to 车立方

Bulk Load Huge Data From Mysql to HBase Using Sqoop

MySQL->HBase:HBaseImportJob

MySQL->HBase:HBaseBulkImportJob

MySQL->HDFS(Avro files)–>HBase

Sqoop from MySQL into Avro files in HDFS, this will result in a bunch of avro files in your target directory. You can check that they’re there with hadoop fs -ls $TARGET_DIR.

sqoop import --connect jdbc:mysql://$MYSQL_SERVER/$DATABASE --driver com.mysql.jdbc.Driver --username $USER --password $PASSWORD --table $TABLE --target-dir $TARGET_DIR --as-avrodatafile