tips from guanxi.me

changelog.markdown

guanxi app 's changelog from francis

date: 2013-4-3

feature

• commend#index 支持基于 sql 的无限翻页
• 重新定义智能推荐逻辑
• 定义活跃度算法
• 定义了定时任务用于更新智能推荐和活跃用户

improve

• 提高了推荐模块的执行性能

commend_user_sort_distance_logintime.markdown

Some Solutions for sort commend user with distance and logintime

Assume

user
  fields:
    gps [x,y]
    :login_time, type: string
  attr
    :distance
    :distance_range #0..100, 100..500, 500..1000, 1000..5000

set commend user must logintime less than 2.hours

Solutions

1. 每个 distance_range 各取一部分, 拼接为 1000 count 的 array, 存放在内存中 按照 [:distance_range, :logintime] sort
2. 取 (0..2.hours.ago).cover? logintime and User.near(gps: gps).max_distance(0.1) << User.near(gps: gps).where(logintime: 2.weeks.ago..Time.now)
   • 这里有个问题如何实现前后两个集合之间的过渡是个问题因为 它们的个数都是不确定的, 也无法实现 sql 分页
3. 只取 User.near(gps: gps).where(logintime: 2.weeks.ago..Time.now), 对于当前页的 user sort by [:distance_range, :logintime]

Summary

1. 按照 [:distance_range, :logintime] sort 这种方式, _对于 user 量太小和太大来说_除了加大数据检索量和计算量外并没有多大的意义, 完全可以按照控制 logintime 范围来取得合适的用户

front_end.markdown

javascript

a = [12,1,2]
a.map(function(i, e) { console.log(i) })
12
1
2

b = $("a")
b.map(function(i, e) { console.log(i) })
0
1
2

improve_performance.markdown

# GxUser.only([:UserID, :GuanxiID, :UserName]).limit(1).first
GxUser.limit(10).collect{|u| st_time= Time.now;u.users_similar(u,[1]).only(:UserID).limit(200).entries.count;puts "take #{Time.now - st_time}'s"}
take 0.254902492's
take 0.293922163's
take 0.279348809's
take 0.284525304's
take 0.074807277's
take 0.05721699

parameter_agents.markdown

We should have an mechanism abount parameter agents(我们应该有一个基于 C/S) 的参数托管机制

resources

• http://open.weibo.com/wiki/Oauth2/authorize

how to use when use login from an oauth platform(authorization from server)

• 首先需要用户通过 server check if user exists?

• if user exists?

  • send request to oauth server(eg, weibo) and if sucess then return oauth token

• unless user exists?

  • send request(include an state = {new_user: true}) to oauth server and if sucess then return oauth token and an state parameter

• handle authorize callback

  • if has state with {new_user: true} then create new and navgiter to welcome page

  • else then navgiter to use main page

how it help us

例如, 我们的 自动消息(回复人数)的统计, 关系表情(新旧版本兼容)

• send and push an auto message invoke an parameter state = {auto_message: true}
• client receive an message(invoke an parameter state = {auto_message: true}), should know state is need by server
• when user reply the message client should send the reply message(and state if state exists ) to server
• server receive the request, know this is an reply message and an state, it mean the message is an reply messge for auto message, then handle it (just we want)

关系表情(新旧版本兼容)流程类似

这里 state 如同一个令牌在两个点间传递, 最终回到 sender 这里有个难点需要 state 的托管人知道这是一个 托管参数, 并且在 处理流程回到 sender 并且原样传回 state

tips_guanxi.md

aims within 3 month

1. 融入团队, 能高效的和团队沟通 done
2. 熟悉服务端代码, 能独立相关功能模块 done
3. 完成首页智能推荐相关模块 done
4. 完成系统后台管理基础功能 done
5. 协助团队进行代码重构 done
6. 配合系统后台的门户网站 done
7. 希望能用 git + capistrano 做 scm (管理后台代码) hung up
   • just guanxi_cms from myself suit
8. 推荐内部使用 gistub hung up

aims within guanxi

1. 融入团队, 能高效的和团队沟通 done
2. 熟悉服务端代码, 能独立相关功能模块 done
3. 完成首页智能推荐相关模块 done
4. 完成系统后台管理基础功能 done
5. 协助团队进行代码重构 done
6. 配合系统后台的门户网站 hung up
7. 希望能用 git + capistrano 做 scm (管理后台代码) hung up
   • just guanxi_cms from myself suit
8. 推荐内部使用 gistub hung up
9. migrate mongodb server from grandcloud to aliyun done
10. configurate server environment we need done

detail process

• Calculate 活跃度
  • status: done
  • updated_at: 2013-3-13
  • status: refactory
  • updated_at: 2013-3-15
  • status: done
  • updated_at: 2013-3-17
• 完成首页智能推荐相关模块
  • status: processing
  • updated_at: 2013-3-12
  • status: done
  • updated_at: 2013-3-21
• 搭建后台管理系统
  • status: checkout
  • updated_at: 2013-3-20
• 完成内容举报系统
• config and use redis, whenever, resque
  • status: done
  • updated_at: 2013-3-21
• bulit an git service
  • status: done
  • updated_at: 2013-3-21
  • note: how built an git server

remove at 2013-3-19

• 对于所有 有朋友 但是低或者不活跃(低于某个分值)的用户, 自动发送来自他们朋友的消息, 如: 'Hi'或者'你好' 两边同步发送, 但在消息记录里不显示(但是同一个用户不应该发送同样的消息给好友超过三次)
• 对于新用户, 5分钟内, 取活跃用户如果和新用户有高分值匹配, 自动发送他们一个'HI'之类(两边都能收到这个消息)
• 活跃用户, 并且最近一周没有更新资料的前10%用户, 发送提醒更新资料消息

guanxi cms

• 虚拟用户 模拟 实际的交互(小僵系统)
  • 一个人实体人 模拟 N 个角色
• 管理现有的用户
  • 举报系统
  • message
  • 用户 ...
• 统计系统
  • 每个 api 的请求次数
  • page_request
• notice for cms user
• authorization system user(role,permit) role(permit,assin_permit)(guest)
• ...
• main web manage interface
  • seo manage interface
  • ...

智能推荐

• date 2014-02-20:

  1. 要推前一日优质用户和新注册的活跃用户

  2. 对于老用户则按照关系度推荐用户

  3. 对于新用户则选取一定(200 个有相同 Hobbys and Lookfor 的)相关用户(计算静态关系分)来按照关系度推荐

  4. 尽量采用局部更新 relation score(当天有 action 的用户) 和 commend users(当天有 action 的用户), 以减少数据库压力

  5. 按照 desc(:commend_index, :relation_score) 排序,其中 commend_index 为优质用户(20) 新用户(5) 高关系分(0)

NOTE: 更新及迭代

以一个新注册用户开始:

1. 创建用户帐号(假定称之为 A), 选取 100个用户(他们有相同的 hobby 或则 lookfor), 计算 static_relation_score (对于新用户静态关系分也就是关系分) 存储在表 commend_users create user account(assume called A), pick up 100 user (which has same hobby or lookfor) caluate the static_relation_score (for new user same as relation score) store in to table commend_users
2. 当用户 A 跳转至 智能推荐, 参考 commend_users 返回推荐的用户 when user A navigate to 智能推荐, refer the commend_users and return commend users
3. (一天后)在后端队列计算 A 的 关系分, 收集 commend_users 中推荐的(100)个用户 以及 随机的 100 优质用户(不足则以活跃新用户代替)计算与 A 的关系分存到数据库 (after 1 day) caluate the A's relation score on backend queue, collect the commend users(100) in commend_users and top random 100 recommend_users (or activly newer as instead) caluate relation score and caluate relation score with A store in db
4. (一天后)计算 A 的推荐用户列表, 如果在数据库中有关系分记录, 选取前 100 个高分用户, 否则跳转至第 1 步 (after 1 day)caluate the A's commend users list, if have relation_score record in db, pick user which is 1-100'th high score, otherwise jump to step 1

logic for commend_user

• generate user's commend_users, for new user who first time visit referred, or user who have some trouble on update commend_users task (can also alise new user commend, main handle method name UserCommendable#generate_commend_users and UserCommendable#generate_statis_commend_users ) 生成用户的智能推荐, 适用于新用户第一次访问 referred, 或者在更新用户智能推荐的时候发生故障

  • normal case: commend activly user who have similar hobbys or look_for, and calculate static_relation_score 一般场景: 推送具有相同的 hobby 或者 look_for, 并且计算静态关系分
  • an alternate case: commend activly user, and provider a random relation_score during in(6..9.9) 备用场景: 推送活跃用户, 提供一个随机 6..9.9 的关系分

• update user's commend_users (can also alise old user commend, main handle method named generate_commend_user#update_commend_user) 更新用户的智能推荐

  • constitutional or struct 结构或组成
  • high_relation_users(sort :relation_score as desc) but exclude friends(100..150) 高关系用户非朋友
  • active_users(current glob day's users by sort :active_score as desc and equal slice 100 length but choose each chunk's first 10) (0..100) 优质用户(or activly newer)

logic for commend_user first page index(CommendCalculate#quickly_commend_users)

• date 2014-02-20:

  1. do not exists in commend_users list which recode in gx_commend_users
  2. an collect users which from users who have earned highly grade_score today and whose level greater than 1 (GxGradeScoreDaily.daily_speedup_rank), and split to male_recommend_user_ids and female_recommend_user_ids.
  3. gender ratio female(1): f:m = 8:7 male(0): f:m = 6:9
  4. if no daily_speedup_rank matched use GxUser.active_new_users instead
  5. call GxCommendUser.try_new_user_recommend_for(user) else call repair_commend_users for current_user

call generate for new user GxCommendUser.try_new_user_recommend_for(user)

1. launch recommend for new user when have no commend_users for this user(GxCommendUser#launch_recommend_for_new_user(user_arg))
2. should only exec launch_recommend_for_new_user 1 times during 1 days

repair incorrent commend_user data for user GxCommendUser.repair_commend_users(user_id)

1. run with sidekiq worker, and limit can only enqueue 1 times during 20.minutes
2. should only exec really data update operate 1 times every 10.minutes
3. will drop dup commend_user and update relation_score relevantly

some tips

智能推荐用户组成:
  当天与用户非好友并且关系分最高的前 `100..150` 个用户
  优质用户
优质用户组成:
  美女帅哥
活跃新用户组成:
  当天注册的活跃新用户
关系用户集合(可以理解为与用户有关系的用户的集合)组成:
  有相互访问, 并且在 nearby 列表中的用户
  更新 智能推荐 添加的时候插入的活跃用户

relation score

class GxRelationScore
  include Mongoid::Document
  include Mongoid::Timestamps

  store_in session: "gxrecommend"

  # field :userids, type: String
  # field :user_ids, type: Array
  # 约定 较小的为 user_id, 较大的为 relation_user_id
  field :user_id, type: String
  field :relation_user_id, type: String

  # ...

  field :score, type: Float #总分
  field :bak_score, type: Float #备份分数，用于分数更新
  field :last_update_time, type: Time, default: -> { Time.now }

  index({user_id: 1, relation_user_id: 1}, {background: true})
  index({user_id: 1, be_friends: 1, score: -1}, {background: true})
  index({relation_user_id: 1, be_friends: 1, score: -1}, {background: true})
  index({ updated_at: 1 }, { expire_after_seconds: 3 * 24 * 3600, background: true })
end

1. every relation_score document should only live 3 days if no update operate
2. 获取用户分数列表,返回推荐用户ID列表 RelationScoreable#get_relation_users
3. when update relation_score document should use atomic operate
4. when user first login should fresh all friend user relation_score

considerations

• should consider more how to sort commend_users desc(:commend_index, :relation_score)
• when every times commend user by CommendCalculate#quickly_commend_users will fetch all commend_user_ids exists in collection gx_commend_users
• keep data only used by current day's commend_user in database will reduce db io usage
• update_commend_users cycle is every day's 2:00 am 更新周期每天 2:00 am
• update range, during recent day glob user who has Logintime(mean user who opend us app) 更新范围

tips_suggestion.markdown

some tips

guanxi on android platform

date: 2013-3-8

1. no icon on phone's status bar, so if I want return back to guanxi app, can only click the icon on app darsh board(not so comfortably) 在手机状态上面没有 app 的图标, 因此如果想要切回后台运行的 app 中不是那么方便. (SLOVED)

date: 2013-4-8

update user_location

1. 十分钟上传一次定位也有可能造成重复比如说学生上课45 分钟甚至一天都不会动, 按照现有的逻辑在没有改动一上午可能会多出几十条多余的 user_location 记录, 可以考虑如果 gps 一直在一定范围内 不