Chém gió

Thứ Hai, 25 tháng 11, 2013

Hive in cooperate with hadoop.

Download Hive-0.12.0 from apache (at the time of this article can't access to apache :( )
extract it.
Some configurations:
1. hive-env.sh (hive-0.12.0/conf) add this line to point hive to hadoop (ref here for configuring hadoop)
HADOOP_HOME=/home/thientin/packages_container/hadoop-2.2.0 
from here we can run hive as the top application of hadoop. But metadata will be store in derby.log.
Below steps to store metadata in mysql (for security reason whatever.)
Install mysql(of course) create user in mysql pass accordingly to that user.(intend for hive using.), create database for hive:
$ mysql -u root -p
Enter password:
mysql> CREATE DATABASE metastore;
mysql> USE metastore;
mysql> SOURCE /home/hive-0.12.0/scripts/metastore/upgrade/mysql/hive-schema-0.10.0.mysql.sql;

mysql> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'mypassword';
...
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'localhost';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'localhost';
mysql> FLUSH PRIVILEGES;
mysql> quit;

2. create file hive-site.xml (if not exists)
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
 <property>
      <name>javax.jdo.option.ConnectionURL</name>
      <value>jdbc:mysql://localhost:3306/metastore</value>
      <description>the URL of the MySQL database</description>
 </property>

 <property>
      <name>hive.metastore.warehouse.dir</name>
      <value>/user/hive/warehouse</value>
 </property>


 <property>
      <name>javax.jdo.option.ConnectionDriverName</name>
      <value>com.mysql.jdbc.Driver</value>
 </property>

 <property>
      <name>javax.jdo.option.ConnectionUserName</name>
      <value>hive</value>
 </property>

 <property>
      <name>javax.jdo.option.ConnectionPassword</name>
      <value>mypassword</value>
 </property>


</configuration>
3. Copy mysql-connector-java-5.1.27.jar to /hive-0.12.0/lib/
4. Start hadoop services.
5. Test hive : 
> cd /home/hive-0.12.0/bin
> ./hive
screen like this:
try some query: 
>show databases;
>use default; show tables;create table tbl_1(key int, value string);
Done!

 

Không có nhận xét nào:

Đăng nhận xét