Apache Hive Install

Overview

Hive는 Hadoop에 저장된 데이터를 SQL을 사용해 조회할 수 있도록 도와주는 Mapreduce Wrapper이다.

설치 방법
Metastore schema 초기화
hive 실행.

설치 방법

MySQL 설치

mysql-server

sudo apt install mysql-server

mysql connector 설치

mysql-connector-official-download-site에서 설치된 mysql과 호환이 되는 버젼의 mysql-connector 다운로드

wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-java_8.0.30-1ubuntu22.04_all.deb

dpkg -x mysql-connector-java_8.0.30-1ubuntu22.04_all.deb ./
cp usr/share/java/mysql-connector-java-8.0.30.jar /usr/local/hive/lib/

Hive 설치

https://www.apache.org/dyn/closer.cgi/hive/ 에서 적절한 hive version을 찾아 download 한다.

wget https://dlcdn.apache.org/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
tar -zxvf apache-hive-3.1.3-bin.tar.gz
cd -R apache-hive-3.1.3-bin /usr/local/hive

~/.bashrc 수정

HIVE_HOME 환경변수 추가

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}
export HIVE_HOME=/usr/local/hive
PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$JAVA_HOME/bin:$HIVE_HOME/bin

hive-env.xh 수정

아래에 hadoop home directory 설정

HADOOP_HOME=/usr/local/hadoop

hive-site.xml 수정

처음에는 hive-site.xml 파일이 존재하지 않으므로 hive-default.xml.template 파일을 복사해서 hive-site.xml을 생성.

아래 내용 추가.

hive-site.xml

 <property>
    <name>system:java.io.tmpdir</name>
    <value>/tmp/hive/java</value>
  </property>
  <property>
    <name>system:user.name</name>
    <value>${user.name}</value>
  </property>

아래 내용 수정

hive-site.xml

  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://ubuntu01:3306/hive?createDatabaseIfNotExist=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>my_mysql_password</value>
    <description>password to use against metastore database</description>
  </property>
   <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>my_mysql_username</value>
    <description>Username to use against metastore database</description>
  </property>

Metastore schema 초기화

schematool -initSchema -dbType mysql -verbose

hive 실행.

/usr/local/hive/bin/hive