Skip to content

Instantly share code, notes, and snippets.

View nlpjoe's full-sized avatar
🎯
Focusing

Jzzhou nlpjoe

🎯
Focusing
293 followers · 42 following
View GitHub Profile
@nlpjoe
nlpjoe / main.md
Last active January 20, 2021 04:01
[tensorflow 1.4.0 api] #tf 
输入
tf.contrib.layers.input_from_feature_columns(features, self.ubb_column_dict[ubb_noclk_jfy_value_name]) 

计算op
tf.multiply(a, b) 返回逐元素的a * b。
tf.matmul(q, a, transpose_b=True)	将矩阵a与矩阵b相乘，得到a * b。
tf.div(a, b, name="scores") a除以b
@nlpjoe
nlpjoe / main.md
Last active April 29, 2021 03:31
[api调用]#sql
@nlpjoe
nlpjoe / main.md
Created August 26, 2020 06:34
[transformers使用] #pytorch

字典decode

from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
text = '''why isn't my card working'''
encoded = tokenizer.encode(text, add_special_tokens=False)
text_tokenized = tokenizer.decode(encoded, clean_up_tokenization_spaces=False)
print(text_tokenized)
@nlpjoe
nlpjoe / main.md
Last active August 18, 2020 07:12
[docker使用] #coding

sudo docker start ${HOME}

# 无需任何修改，直接执行这个命令即可；如要修改，请参考`docker启动参数的说明`中的说明
sudo docker run -d --name=${USER} --net=host -u `id -u`:`id -g`              \
                -v /etc/passwd:/etc/passwd:ro -v ${HOME}:${HOME}             \
                -w `pwd` -it reg.docker.alibaba-inc.com/dii/dii_alios7u2_dev \
                /bin/bash
# 默认我们使用了当前用户名，作为docker的container名称
# 通过以上命令启动的docker container已经在后台运行了，只要不停止和删除会一直存在
@nlpjoe
nlpjoe / main.md
Last active August 17, 2020 03:25
[oss访问] #coding 
bucket_name="q2q_data"
role_arn="acs:ram::1753758217554671:role/210313"
host="oss-cn-hangzhou-zmf.aliyuncs.com"
oss://210313/${bucket_name}/?role_arn=${role_arn}&host=${host}
@nlpjoe
nlpjoe / main.md
Last active September 3, 2020 08:05
[pai工具] #coding 
def str_normalize(s):
    if isinstance(s, bytes):
        s = s.decode("utf-8")
    out_s = s.encode("utf-8")
    if not isinstance(out_s, str):
        out_s = out_s.decode("utf-8")
    return out_s
@nlpjoe
nlpjoe / main.md
Last active August 7, 2020 03:23
[技巧] # tf

涉及字典的lookup操作，参阅tf.contrib.lookup

@nlpjoe
nlpjoe / main.md
Created July 31, 2020 07:52
[tunnel 上传下载数据] #sql

全命令参考： http://help.aliyun-inc.com/internaldoc/detail/27833.html?spm=a2c1f.8259794.2.1.706796d5idBQN4

upload

tunnel upload log.txt test_project.test_table/p1="b1",p2="b2" --fd "||";
-- 将log.txt中的数据上传至项目空间test_project的表test_table（二级分区表）中的p1="b1",p2="b2"分区
tunnel upload  log.txt  test_table --scan=only;
-- 将log.txt中的数据上传至表 test_table 中。--scan参数表示需要扫描log.txt中的数据是否符合 test_table 的定义，如果不符合报错，并停止上传数据。
@nlpjoe
nlpjoe / main.md
Created July 30, 2020 14:32
[odps 输出中文乱码] #sql

将python解释器改成python3

set odps.isolation.session.enable=true;
set odps.sql.python.version=cp37;
@nlpjoe
nlpjoe / main.md
Created July 28, 2020 16:03
[排序指标] #nlp