Skip to content

Instantly share code, notes, and snippets.

@giwa
Last active March 27, 2020 11:31
Show Gist options
  • Save giwa/ed13ac177c1e1a97fba0 to your computer and use it in GitHub Desktop.
Save giwa/ed13ac177c1e1a97fba0 to your computer and use it in GitHub Desktop.
Install hive on Mac with Homebrew ref: http://qiita.com/giwa/items/dabf0bb21ae242532423
$ brew update
$ brew install hive
# ~/.bashrc
export HADOOP_HOME=/usr/local/Cellar/hadoop/2.7.0
export HIVE_HOME=/usr/local/Cellar/hive/1.1.1/libexec
$ tar zxvf mysql-connector-java-5.1.35.tar.gz
$ sudo cp mysql-connector-java-5.1.35/mysql-connector-java-5.1.35-bin.jar /usr/local/Cellar/hive/1.1.1/libexec/lib/
$ mysql
mysql> CREATE DATABASE metastore;
mysql> USE metastore;
mysql> CREATE USER 'hiveuser'@'localhost' IDENTIFIED BY 'password';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE ON metastore.* TO 'hiveuser'@'localhost';
$ cd /usr/local/Cellar/hive/1.1.1/libexec/conf
$ cp hive-default.xml.template hive-site.xml
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/metastore</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>password</value>
</property>
<property>
<name>datanucleus.fixedDatastore</name>
<value>false</value>
</property>
<property>
<name>hive.exec.local.scratchdir</name>
<value>/tmp/hive</value>
<description>Local scratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/tmp/hive</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
<property>
<name>hive.querylog.location</name>
<value>/tmp/hive</value>
<description>Location of Hive run time structured log file</description>
</property>
$hive
hive > show tables;
@sriharsha-sammeta
Copy link

sriharsha-sammeta commented Sep 23, 2017

Changes as per the errors I faced:

Note:
wherever you see "hive/1.1.1" replace it with "hive/{version-number}"

in file2.txt
mysql to jdbc drive can be downloaded from:

https://dev.mysql.com/downloads/connector/j/

in file3.txt use:

For the first time users of mysql:
install mysql by: [First Time Users]
brew install mysql
then run the following [First Time Users]
brew services start mysql

[For all users]
in Place of:
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE ON metastore.* TO 'hiveuser'@'localhost';
use:
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE, REFERENCES, INDEX ON metastore.* TO 'hiveuser'@'localhost';

in file5.txt:
Also update this property:

<property>
    `<name>hive.metastore.schema.verification</name>`
    <value>false</value>
    <description>Some Description</description>
</property>

in file6.txt:
Also, before running final "hive" command we have to run
./schematool -initSchema -dbType mysql

@bradhowes
Copy link

Just to keep things going:

In file1.txt:
Instead of referring to specific versions, I use the links in /usr/local/opt:

export HADOOP_HOME=/usr/local/opt/hadoop
export HADOOP_COMMON_HOME=/usr/local/opt/hadoop
export HIVE_HOME=/usr/local/opt/hive/libexec
export HCAT_HOME=/usr/local/opt/hive/libexec/hcatalog

I also added HADOOP_COMMON_HOME and HCAT_HOME per warning I saw from brew install and an error from schematool

@amtsha09
Copy link

amtsha09 commented Nov 5, 2018

I am getting the following error while running "hive" after following all the steps:

WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME
Unable to determine Hadoop version information.
'hadoop version' returned:
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME

@skywhat
Copy link

skywhat commented Nov 27, 2018

hive

Unable to determine Hadoop version information.
'hadoop version' returned:

@huanzhang
Copy link

hive

Unable to determine Hadoop version information.
'hadoop version' returned:

This worked for me:
export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.1/libexec

@f911
Copy link

f911 commented Mar 27, 2020

just kill every detail:

  • chmod 777 /tmp/hive
  • cp $HADOOP_HOME/share/hadoop/common/lib/guava-27.0-jre.jar $HIVE_HOME/lib/ ; mv guava-19.0.jar guava-19.0.bak # fix version conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment