Last active
June 9, 2016 15:45
-
-
Save scicco/f722a7edc47a2883256c to your computer and use it in GitHub Desktop.
fixing Compression algorithm 'lzo' previously failed test.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd /etc/yum.repos.d/ | |
| wget http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/cloudera-gplextras5.repo | |
| yum check-update | |
| yum install -y hadoop-lzo | |
| dd if=/dev/random of=/tmp/test.bin bs=1000k count=10 | |
| su - hdfs | |
| hbase org.apache.hadoop.hbase.util.CompressionTest /tmp/test.bin lzo | |
| echo "remember to restart all services on the node to take effect" | |
| #ALTERNATIVE WAY | |
| #if this still not work change core-site.xml in cloudera configuration section (snippet) put this: | |
| https://www.cloudera.com/content/cloudera/en/documentation/core/v5-3-x/topics/cdh_ig_hdfs_cluster_deploy.html#concept_g4n_z23_gk_unique_1 (Configuring LZO paragraph) | |
| <property> | |
| <name>io.compression.codecs</name> | |
| <value>org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec, | |
| org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.LzoCodec, | |
| com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.SnappyCodec</value> | |
| </property> | |
| and restart cluster. | |
| try again with | |
| su - hdfs | |
| hbase org.apache.hadoop.hbase.util.CompressionTest /tmp/test.bin lzo | |
| echo "remember to restart all services on the node to take effect" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If you run this on local filesystem I belive 'file' preffix should be uses:
hbase org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.bin lzo