報(bào)錯(cuò)代碼:
20/11/14 00:36:50 ERROR tool.ImportTool: Import failed: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://localhost:9000/user/frog005/course already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:145)
at org.apache.hadoop.mapreduce.Job11.run(Job.java:1567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1889)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:200)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:173)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:270)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
發(fā)生場(chǎng)景:
在使用sqoop命令向hive中的Lee庫(kù)導(dǎo)入數(shù)據(jù)表(course)和數(shù)據(jù)后,drop掉course表。重新使用完全相同的sqoop命令重新導(dǎo)入數(shù)據(jù)表和數(shù)據(jù)。
20/11/14 00:36:50 ERROR tool.ImportTool: Import failed: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://localhost:9000/user/frog005/course already exists
解決方案:
這是因?yàn)槊看蜯R都會(huì)生成一個(gè)output目錄但是不能自動(dòng)刪除,所以我們就把HDFS的輸出目錄刪除:
- step 1
hadoop fs -ls -r hdfs://localhost:9000/user/frog005/
hadoop fs -rm -r hdfs://localhost:9000/user/frog005/course



-
step 2
重新運(yùn)行sqoop import命令即可。
sqoop import \
--hive-import \
--connect jdbc:mysql://106.xx.xxx.xxx:xxx/datafrog05_adventure \
--table course \
--username xxxx05 \
--password xxx'!'xxx \
--hive-database Lee \
--hive-overwrite \
--m 1
