单机多卡的方案
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
viewfs://nsX/user/hive/warehouse/default.db/web/uigs/web_uigs_wapsearch/201404/20140408/2014040815/web_uigs_wapsearch.location_pointer.2014040815 -> hdfs://cloud1016128162.wd.sjs.ss.nop.sogou-op.org:8020/logdata/uigs/appsearch//201404/20140408/*2014{-,}04{-,}08{_15,15}* | |
middle of get splits , path is viewfs://nsX/user/hive/warehouse/default.db/web/uigs/web_uigs_wapsearch/201404/20140408/2014040815 | |
add hdfs://cloud1016128162.wd.sjs.ss.nop.sogou-op.org:8020/logdata/uigs/appsearch/201404/20140408/*2014{-,}04{-,}08{_15,15}* to input path | |
14/12/02 19:33:12 ERROR thriftserver.SparkExecuteStatementOperation: Error executing query: | |
java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "rsync.cloud-dev1013412128.web.sjs.vm.sogou-op.org/10.134.12.128"; destination host is: "cloud1016128162.wd.sjs.ss.nop.sogou-op.org":8020; | |
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764) | |
at org.apache.hadoop.ipc.Client.call(Client.java:1413) | |
at org |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4/12/04 18:38:35 ERROR thriftserver.SparkExecuteStatementOperation: Error executing query: | |
org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: viewfs://nsX/user/hive/warehouse/default.db/web/uigs/web_uigs_wapsearch/201312/20131201/2013120102 | |
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:251) | |
at com.hadoop.mapred.DeprecatedLzoTextInputFormat.listStatus(DeprecatedLzoTextInputFormat.java:72) | |
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:270) | |
at com.hadoop.mapred.DeprecatedLzoTextInputFormat.getSplits(DeprecatedLzoTextInputFormat.java:112) | |
at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:201) | |
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:205) | |
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:203) | |
at scala.Option.getOrElse(Option.scala:120) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
%global srcvers 3.0.0-beta-1 | |
%global srcname protobuf-%{srcvers} | |
%global _prefix /opt/%{srcname} | |
# | |
Summary: Protocol Buffers Google''s data interchange format | |
Name: protobuf3 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
warning: core file may not match specified executable file. | |
[New LWP 15712] | |
[New LWP 15799] | |
[New LWP 15713] | |
[New LWP 15801] | |
[New LWP 15802] | |
[New LWP 15711] | |
[New LWP 15800] | |
[New LWP 15691] | |
[Thread debugging using libthread_db enabled] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(gdb) bt | |
#0 0x00007fcec8f181d7 in raise () from /lib64/libc.so.6 | |
#1 0x00007fcec8f198c8 in abort () from /lib64/libc.so.6 | |
#2 0x00007fcec981c9d5 in __gnu_cxx::__verbose_terminate_handler() () from /lib64/libstdc++.so.6 | |
#3 0x00007fcec981a946 in ?? () from /lib64/libstdc++.so.6 | |
#4 0x00007fcec981a973 in std::terminate() () from /lib64/libstdc++.so.6 | |
#5 0x00007fcec981ab93 in __cxa_throw () from /lib64/libstdc++.so.6 | |
#6 0x00007fcec986fa17 in std::__throw_out_of_range(char const*) () from /lib64/libstdc++.so.6 | |
#7 0x0000000000412fd9 in std::vector<int, std::allocator<int> >::_M_range_check (this=0x14e7940, __n=1) at /usr/include/c++/4.8.2/bits/stl_vector.h:794 | |
#8 0x0000000000412c57 in std::vector<int, std::allocator<int> >::at (this=0x14e7940, __n=1) at /usr/include/c++/4.8.2/bits/stl_vector.h:830 |
这个很平常的代码其实本身还是有些技术含量的
NewerOlder