Skip to content

Instantly share code, notes, and snippets.

@philschmid
Last active September 6, 2019 05:49
Show Gist options
  • Save philschmid/feafb4d24d7082897b6721cdfb1733f0 to your computer and use it in GitHub Desktop.
Save philschmid/feafb4d24d7082897b6721cdfb1733f0 to your computer and use it in GitHub Desktop.
Example Job json for Talos
[{
"userCompanyId": "c26ce5b0-9e3b-4146-9637-3047d5b92c56",
"jobId": "General Electric-Purchase-2019-09-06T04:51:11.720Z-Siemens",
"userId": "da957012-7329-4135-b278-e250810f5484",
"userCompany" :"General Electric",
"employee": "Stefan",
"department": "Purchase",
"startDate": "2019-09-06T04:51:11.720Z",
"endDate" : "2020-09-06T04:51:11.720Z",
"searchCompany": "Siemens",
"type": "subscribe",
"apis" : [
{"id": "GN",
"name": "Google News",
"updateDate": "1970-01-01T00:00:00.000Z"},
{"id": "FN",
"name": "Focus News",
"updateDate": "1970-01-01T00:00:00.000Z"}],
"nlpModules":["swot","sentiment"],
"intervall": 15,
"createdAt": "2019-09-05T04:51:11.720Z"
},
{
"userCompanyId": "c26ce5b0-9e3b-4146-9637-3047d5b92c56",
"jobId": "General Electric-Purchase-2019-09-06T04:51:11.720Z-BMW",
"userId": "0d834695-3e0b-4824-8dd4-015c91f16e44",
"userCompany" :"General Electric",
"employee": "Max",
"department": "Sales",
"startDate": "2019-09-06T04:51:11.720Z",
"endDate" : "2020-09-06T04:51:11.720Z",
"searchCompany": "BMW",
"type": "adhoc",
"apis" : [
{"id": "GN",
"name": "Google News",
"updateDate": "1970-01-01T00:00:00.000Z"},
],
"createdAt": "2019-09-05T04:51:11.720Z"}
{
"userCompanyId": "314be7f8-f38e-49ae-b054-bcf14fbf73a7",
"jobId": "Telekom-Purchase-2019-09-06T04:51:11.720Z-Siemens",
"userId": "7b0522e2-1414-49cd-b5ad-5620d10fa446",
"userCompany" :"Telekom",
"employee": "Philipp",
"department": "RnD",
"startDate": "2019-09-06T04:51:11.720Z",
"endDate" : "2020-09-06T04:51:11.720Z",
"searchCompany": "Siemens",
"type": "subscribe",
"apis" : [{"id": "GN",
"name": "Google News",
"updateDate": "1970-01-01T00:00:00.000Z"},
{"id": "FN",
"name": "Focus News",
"updateDate": "1970-01-01T00:00:00.000Z"}],
"nlpModules":["swot","sentiment"],
"intervall": 15,
"createdAt": "2019-09-05T04:51:11.720Z"
},
]
@philschmid
Copy link
Author

philschmid commented Sep 6, 2019

*after var name signals requirement

if department is missing then range key is built with the remaining var

userCompanyId*: uuid **hashkey**(once created per Company, saved in Paymenttable)
jobId*:  userCompany-department-startDate-SearchCompany **rangeKey**
userId*: uuid (created after signUp saved in coginto as custom field)
userCompany*: string (name of the company)
employee: string (name of Employee)
department: string (name of department)
startDate*: ISOstring (first crawl date, _default_ now)
endDate*:  ISOstring (first crawl datem _default_ 01.01.9999)
searchCompany*: string (company name to search for)
type*: string (atm adhoc or subscribe ) 
apis*: Object[] (array of objects for api crawl identified through id, after crawl updateDate has to be modified)
nlpModules*: string[] (String array for nlp modules)
intervall: number (repeat intervall for craw, _default_ 60l)
createdAt: ISOstring (job creation date, _default_ now)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment