sqoop2安装

int32位 posted @ Mar 22, 2016 03:42:06 PM in hadoop/spark , 2369 阅读
转载请注明：http://krystism.is-programmer.com/若有错误，请多多指正，谢谢！

 

1. 下载解压缩

此次安装版本为1.99.6

# Decompress Sqoop distribution tarball
tar -xvf sqoop-<version>-bin-hadoop<hadoop-version>.tar.gz


ln -s sqoop-<version>-bin-hadoop<hadoop version>.tar.gz sqoop

export SQOOP_HOME=`pwd`/sqoop
# Change working directory
cd $SQOOP_HOME

2. 配置服务

配置文件在$SQOOP_HOME/server/conf下，大多数默认配置即可。需要注意的配置是catalina.properties文件下common_loader需要正确配置hadoop库文件，包括hdfs、mapreduce、hive等所有jar包. 我的配置为:

common.loader=${catalina.base}/lib,\ 
${catalina.base}/lib/*.jar,\ 
${catalina.home}/lib,\ 
${catalina.home}/lib/*.jar,\ 
${catalina.home}/../lib/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop/lib/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop-hdfs/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/*.jar,\ /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop-yarn/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/*.jar,\ 
/opt/cloudera/parcels/CDH/lib/hive/lib/*.jar

sqoop.properties文件的org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/etc/hadoop/conf/配置hadoop 配置文件路径，默认为/etc/hadoop/conf

3. 测试配置是否正确

运行

sqoop2-tool verify

若正确,则输出为:

Verification was successful.
Tool class org.apache.sqoop.tools.tool.VerifyTool has finished correctly

4. 下载derby 包

自带derby版本过低运行时会由于不兼容而出现以下错误：

org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
    at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:85)
    at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)
    at org.apache.sqoop.repository.JdbcRepository.createOrUpgradeRepository(JdbcRepository.java:127)
    at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:123)
    at org.apache.sqoop.tools.tool.UpgradeTool.runToolWithConfiguration(UpgradeTool.java:39)
    at org.apache.sqoop.tools.ConfiguredTool.runTool(ConfiguredTool.java:35)
    at org.apache.sqoop.tools.ToolRunner.main(ToolRunner.java:75)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.sqoop.tomcat.TomcatToolRunner.main(TomcatToolRunner.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.catalina.startup.Tool.main(Tool.java:225)
Caused by: java.sql.SQLException: No suitable driver found for

解决办法为下载derby最新版本，先删除/sqoop-1.99.6-bin-hadoop200/server/webapps/sqoop/WEB-INF/lib下的derby旧包，然后把新下载的derby目录下的lib下的jar包拷贝到/sqoop-1.99.6-bin-hadoop200/server/webapps/sqoop/WEB-INF/lib

5. 验证

运行bin/sqoop2-shell,进入sqoop shell模式, 运行show version --all，若能正确输出server版本，则安装成功:

sqoop:000> show version --all
client version:
  Sqoop 1.99.6 source revision 07244c3915975f26f03d9e1edf09ab7d06619bb8
  Compiled by root on Wed Apr 29 10:40:43 CST 2015
server version:
  Sqoop 1.99.6 source revision 07244c3915975f26f03d9e1edf09ab7d06619bb8
  Compiled by root on Wed Apr 29 10:40:43 CST 2015
API versions:
  [v1]
转载请注明：http://krystism.is-programmer.com/若有错误，请多多指正，谢谢！
  • 无匹配
  • 无匹配
Small_feed 评论 (18)
1
[回复]
cleaning companies i 说:
2019年9月18日 16:11

Still, if you're looking for “regular housecleaning, ” you have to pick maid housecleaning services in which charge usually while using number about maids, and therefore the time requested. Maid housecleaning companies will probably price most of the services located at 25 to make sure you 35 AED each hour.

2
[回复]
handi sports 说:
2019年11月15日 01:10

Credit card debt financing is just about the most common solution to finance your organization. You sign up for a bank loan and consent to repay over a lot of time. The financial institution charges attention and tends to make a profit like that. The most frequent business lenders will be the following:

3
[回复]
star lite shopping m 说:
2020年3月24日 20:37

On the net shopping would be the perfect solution with the home limited and if you find shopping a challenging task. Online looking has turned out to be a boon if you live with rural regions too. Whether you will be buying directly at a business on the net retailer, anyone online looking site, or maybe an World-wide-web auction web page, shopping online is usually fun, uncomplicated, time protecting and cost effective.

4
[回复]
shopping batalha 说:
2020年3月24日 20:37

Accomplishing this prior to a exchange online is possibly the most critical into a shopping cart's achievements. If this buying practice causes annoyance, confusion or maybe insecurity, the user will abandon the shopping cart software package, never to return again.

5
[回复]
elimperio travel 说:
2020年3月24日 20:37

In order to make the the vast majority of your management and business travel funds, it is vital to cover leveraging ones program for everyone it might be priced at. Telling travelers to pick out the most competitive logical airfare is not enough. Here are classified as the elements to be considered as soon as planning or maybe evaluating ones travel method.

6
[回复]
travelling 2 peru 说:
2020年3月24日 20:38

An incredible option is to get a 'personalised' on the net booking tool that was custom made in line with you company's particular traveling policies in addition to needs. Although taking your own traveling tech possesses its strengths - like the ability to increase proficiency with inbuilt approval operations and coping with your travel whenever they want - here are a few disadvantages to be noted.

7
[回复]
rapidity news 说:
2020年3月24日 20:38

Telly is a different medium connected with news in addition to current affairs in a country or maybe locality; more reguarily news is it being brought dwell and real-time. However, internet may bring such advantages for news people and followers. Aside by full articles or blog posts on favorite news, you will discover websites which often enable viewers to see news including what they will find with TV.

8
[回复]
monthly maid service 说:
2020年4月29日 18:28

Also you can enquire towards previous buyers about if thez maid provides her unique vacuums, cleansing agents, mops, towels, pails in addition to polishes or it is advisable to provide these individuals. Make sure to supply a exclusive training or maybe advice n the best way to clean this antique pieces of furniture, oriental carpeting and rugs, and very sensitive show portions; also reveal them the way to handle with them.

9
[回复]
construction paintin 说:
2020年4月29日 18:28

Symbol painting manufactured with the effort in addition to talent connected with portrait painters. Without these individuals, this style of art can never exist. Really, portrait painters have many contribution from the art marketplace. Agree or maybe not, i am the people that harvest the goods of the skills. That's why, we experience the appearance connected with portraits particularly when they usually are imitated on the works connected with famous symbol painters.

10
[回复]
Emma 说:
2023年1月22日 20:22

Installing Sqoop2 is a straightforward process that requires you to download and decompress the Sqoop distribution tarball. Once completed, you will need to create a symbolic link benefits of consuming CBD oil capsules and then set the SQOOP_HOME environment variable. By following these steps, you will be well on your way to getting the most out of Sqoop2!

11
[回复]
Bushra 说:
2023年4月04日 00:04

Great article Lot's of information to Read...Great Man Keep Posting and update to People..Thanks Click Through the Following Webpage

12
[回复]
amarnath helicopter 说:
2024年6月05日 23:12

Hi! This is my first visit to your blog! We are a team of volunteers and new initiatives in the same niche. Blog gave us useful information to work. You have done an amazing job!

13
[回复]
amarnath helicopter 说:
2024年6月05日 23:25

Cool stuff you have and you keep overhaul every one of us

14
[回复]
tech 说:
2024年6月07日 11:23

Hi there, I found your blog via Google while searching for such kinda informative post and your post looks very interesting for me обязательно посетите древние города и достопримечательности Турции

15
[回复]
tech 说:
2024年7月25日 06:06

i am for the first time here. I found this board and I in finding It truly helpful & it helped me out a lot. I hope to present something back and help others such as you helped me. india visa types business visa

16
[回复]
tech 说:
2024年9月05日 06:51

Very good post. I wanted to let you know that I recently came across your site and that I have been very enjoying reading your entries. In the end, I'll be following your feed and hoping you post once more soon! индийска електронна виза за кубински граждани

17
[回复]
tech 说:
2024年10月16日 06:59

wonderful post; continue doing this fascinating stuff. Thank you for taking the time to talk about this; it's great to know that this website is also covering this subject! безопасно ли е да пътувате с влакове в Индия

18
[回复]
tech 说:
2024年12月02日 21:48

Good website, where did you find the details for this post? Still, I'm glad I found it, and I'll be returning soon to see what other postings you have. india visa eligibility india visa for djiboutian citizens


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter