Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save allwefantasy/fc8280006f466ddc860c842ac8abf2c0 to your computer and use it in GitHub Desktop.
Save allwefantasy/fc8280006f466ddc860c842ac8abf2c0 to your computer and use it in GitHub Desktop.
支持在流式计算中进行ALS推荐
{
"als_predict": {
"desc": "测试",
"strategy": "streaming.core.strategy.SparkStreamingStrategy",
"algorithm": [],
"ref": [],
"compositor": [
{
"name": "streaming.core.compositor.spark.streaming.source.MockJsonCompositor",
"params": [
{
"a": "http://123.com/path?userId=1&itemId=2"
},
{
"a": "http://123.com/path?userId=2&itemId=3"
},
{
"a": "http://123.com/path?userId=2&itemId=2"
},
{
"a": "http://123.com/path?userId=3&itemId=2"
}
]
},
{
"name": "streaming.core.compositor.spark.streaming.transformation.JSONTableCompositor",
"params": [
{
"tableName": "test"
}
]
},
{
"name": "streaming.core.compositor.spark.streaming.transformation.SQLCompositor",
"params": [
{
"sql": [
"select",
"split(split(split(a,'\\?')[1],'&')[0],'=')[1] as user, ",
"split(split(split(a,'\\?')[1],'&')[1],'=')[1] as item,",
"split(split(split(a,'\\?')[1],'&')[0],'=')[1] as label,",
"'1.0' as rating",
"from test"
],
"outputTableName": "test2"
}
]
},
{
"name": "streaming.core.compositor.spark.streaming.transformation.SQLCompositor",
"params": [
{
"sql": [
"select",
"user,",
"item",
"from test2"
],
"outputTableName": "test3"
}
]
},
{
"name": "streaming.core.compositor.spark.streaming.transformation.AlgorithmCompositor",
"params": [
{
"path": "file:///tmp/als_log",
"algorithm": "als",
"outputTableName": "test4",
"recommendUsersForProductsNum": 1
}
]
},
{
"name": "streaming.core.compositor.spark.streaming.transformation.SQLCompositor",
"params": [
{
"sql": "select * from test4"
}
]
},
{
"name": "streaming.core.compositor.spark.streaming.output.SQLPrintOutputCompositor",
"params": [
{
"num": 10
}
]
}
],
"configParams": {
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment