###如何导入原始数据
- 查看导入原始数据配置是否正确,配置应该存在于项目 preprocess/sql 目录下面,文件名叫 config.json
- 运行 preprocess 项目
- 点击
导入数据
- 选择存放原始数据文本文件的文件夹,就可以开始导入
- 原始数据文本文件的文件名需要和配置中写明的完全一致。
###如何修改原始数据文件编码
$ /usr/local/rvm/gems/jruby-1.6.7@global/gems/torquebox-server-2.0.0.cr1-java/jboss/bin/standalone.sh -b 0.0.0.0 | |
========================================================================= | |
JBoss Bootstrap Environment | |
JBOSS_HOME: /usr/local/rvm/gems/jruby-1.6.7@global/gems/torquebox-server-2.0.0.cr1-java/jboss | |
JAVA: java | |
JAVA_OPTS: -server -XX:+UseCompressedOops -XX:+TieredCompilation -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=standalone.xml |
[pid 6541] restart_syscall(<... resuming interrupted call ...> <unfinished ...> | |
[pid 8676] restart_syscall(<... resuming interrupted call ...> <unfinished ...> | |
[pid 6540] restart_syscall(<... resuming interrupted call ...> <unfinished ...> | |
[pid 6539] select(22, [19 21], NULL, NULL, NULL <unfinished ...> | |
[pid 6538] restart_syscall(<... resuming interrupted call ...> <unfinished ...> | |
[pid 6537] restart_syscall(<... resuming interrupted call ...> <unfinished ...> | |
[pid 6536] select(23, [22], NULL, NULL, {3, 567701} <unfinished ...> | |
[pid 6533] wait4(-1, <unfinished ...> | |
[pid 6537] <... restart_syscall resumed> ) = -1 ETIMEDOUT (Connection timed out) | |
[pid 6537] futex(0xcc33390, FUTEX_WAKE_PRIVATE, 1) = 0 |
subscriber_ids = Collection.find(4).likes.pluck(:user_id) | |
subscription = Subscription.find_by(source_identity: "4:collection") | |
id = subscription.id | |
def func1(subscriber_ids, id) | |
Sider.subscription.with do |conn| | |
subscriber_ids.each_slice(2000) do |batch| | |
conn.pipelined do | |
batch.each do |subscriber_id| | |
conn.set("users:#{subscriber_id}:total_unread_subscriptions_count", 1) |
// Use Gists to store code you would like to remember later on | |
console.log(window); // log the "window" object to the console |
jobs = [] | |
# Read from Old Sidekiq | |
ss = Sidekiq::ScheduledSet.new | |
ss.each do |job| | |
value = JSON.parse(job.value) | |
if value["class"] == "ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper" | |
# this is an active job |
{ | |
"tables": [ | |
{ | |
"txtFileName": "absolute_nav_data_fused.txt", | |
"tableName": "ac_nav_data_raw", | |
"skipLinesCount": 8, | |
"leadingFieldsCount": 18, | |
"nonRepeatedFields": { | |
"time": 1, | |
"msg_publish_time_ac_nav": 2, |
###如何导入原始数据
导入数据
###如何修改原始数据文件编码
编程的智慧 | |
简叔咖啡馆20151026-你看过哪些书名平(坑)平(爹),内容却非常精彩的书 ? | |
是情还是色?适合女生看的9部情色电影。 | |
那些脑洞大开创意满满的神奇短篇动画(第二弹) | |
为什么我不给孩子看国产动漫?看看这5部法国动漫,你就知道了! | |
推荐的10部长篇小说 | |
如何从一个空有上进心的人,变成行动上的巨人? | |
高效记笔记,你可以的 | |
上瘾五万年--读戴维•考特莱特《上瘾五百年》 | |
聊聊终生学习这件事 |
require 'goliath' | |
require 'grape' | |
module Myapp | |
class API < Grape::API | |
version 'v1' | |
format :json | |
get '/ping' do | |
{ ping: "pong" } |
doT: | |
executed in 7964 milliseconds VM510:24 | |
executed in 8042 milliseconds VM510:24 | |
executed in 7716 milliseconds | |
Eco: | |
executed in 20901 milliseconds VM228:24 | |
executed in 20539 milliseconds |