If we are going to allow users to save requests we should create a resouce Search, if not we will allow users to bookmark query.
We can use Mongo query language in this manner
Actual example of API with uses Mongo query syntax
? I don't know how we will validate those queries ?
pros
- Search can be saved as bookmark
- Search is cached
cons
- If we want to save filers we we need to create a source anyway
source.repository.full_name != "main/repo" AND state = "OPEN" AND reviewers.username = "evzijst" AND destination.branch.name = "master"
/2.0/repositories/main/repo/pullrequests?q=source.repository.full_name+%21%3D+%22main%2Frepo%22+AND+state+%3D+%22OPEN%22+AND+reviewers.username+%3D+%22evzijst%22+AND+destination.branch.name+%3D+%22master%22
http://localhost:8080/confluence/rest/api/search?cql=creator=currentUser()&type%20in%20(space,page,user)&cqlcontext={%22spaceKey%22:%22TST%22, %22contentId%22:%2255%22}
&filters=ga:browser%3D~%5EFirefox
- oData - there is no pyhton parser fot his one
http://services.odata.org/OData/OData.svc/Products?$filter=Name eq 'Milk' and Price lt '2.55M' (Requests all products with the Name 'Milk' that also have a Price less than 2.55).
?fieldList=name,id&filter=name contains 'smith'
/api/rest/v1/products?search={ATTIBUTE_CODE:[{"operator":OPERATOR,"value":VALUE,"locale":LOCALE_CODE,"scope":CHANNEL_CODE}]}
?{element1}__{operator1}={value1}&{element2}__{operator2}={value2}&searchOperator=OR
http://MY-SERVER/api/2.3/sites/9a8b7c6d-5e4f-3a2b-1c0d-9e8f7a6b5c4d/workbooks?filter=tags:in:[stocks,market]
pros
- we won't have a huge request string and filters will not be exposed
cons
- users won't be able to save query in bookmarks, only by clicking on "Save query
Atlassian Jira Uses Filter Resources
Jira saves URL as a resource filed and can return this field back to user. Example
We can use ElasticDSL + Elastic + MongoDb
MongoDB is a data storage not a search engine, to make our solution scalable we need to look at search engines integration.
MONGO:
- how efficient is that? - if we use regex on index that will be the same as looking for the whole string in index
- look at sparse index and index intersection - sparse index will not help in long perspective
We can try $text elastic index
Elastic: ?
What operators do we need:
- is
- is not
- contains (regex of full text)
- is empty
- is not empty
- starts with (regex of full text)
- ends with (regex of full text)
- before
- after
- less than
- greater than ...
Mongo vs Elastic on filtering task for network scanner
Elastic returns top information based on score and Mongo will return every match
for elastic:
- how do we map types? Elastics maps all fiealds automatically
- how do we address security questions?
TODO:
- test how solution works with elastic: test queries
- compare search engines
- run on production Mongo DB dump
How does database indexing work?- SO
Flask reqparse is deprecated
- WEBARGS | webargs custom parser
- pyparser
- elastic search query parser
- pasring query expressions in python
Official Security Sheld is not free
TL;DR
- bind Elsatic to PRIVATE IP
- use proxy
- disable scripting
- use free plugins
:checked: In order to use Elastic we will need to convert mongoDB into replica set
elastic river is depricated