Skip to content

Instantly share code, notes, and snippets.

@macbre

macbre/dataflow.tsv

Last active June 24, 2017 08:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save macbre/ddf5742b8293062cc78b767fccb5197b to your computer and use it in GitHub Desktop.
Save macbre/ddf5742b8293062cc78b767fccb5197b to your computer and use it in GitHub Desktop.
Download ZIP
elecena.pl data flow
Raw
dataflow.tsv
We can make this file beautiful and searchable if this error is corrected: It looks like row 3 should actually have 1 column, instead of 5. in line 2.
# elecena.pl
# Log entries analized: 1227179
mysql:shops _select mq/request.php 0.0166 QPS: 0.1205
mysql:products run ShopsUpdateScript 0.0065 QPS: 0.0476
sphinx:parameters newFromCode Parameters 0.0788 QPS: 0.5734
mysql:parameters exists Parameters 0.0221 QPS: 0.1608
sphinx:parameters getSimilar Parameters 0.0092 QPS: 0.0666
currency.php _ mysql:currencies 0.0001 QPS: 0.0008
sphinx:products getLastChanges StatsController 0.0002 QPS: 0.0017
mysql:producers _select mq/request.php 0.0824 QPS: 0.5996
ShopsUpdateScript run mysql:shops 0.0022 QPS: 0.0158
sphinx:products search Elecena\Services\Sphinx 0.0045 QPS: 0.0324
sphinx:products getKeywords Elecena\Services\Sphinx 0.0500 QPS: 0.3639
sphinx:parameters getSuggestions Parameters 0.0023 QPS: 0.0171
ImageBot fetchImage mysql:products 0.0037 QPS: 0.0268
sphinx:products getProductsMatching ShopParamsScript 0.0001 QPS: 0.0000
sphinx:parameters getDatabaseCount Parameters 0.0002 QPS: 0.0013
mysql:shops index PanelController 0.0001 QPS: 0.0002
sphinx:parameters getCategoriesTree Parameters 0.0001 QPS: 0.0001
mysql:products _describe mq/request.php 0.0004 QPS: 0.0029
sphinx:products getProductsForTags Elecena\Services\Sphinx 0.0268 QPS: 0.1952
mysql:products newFromIds Elecena\Models\Product 0.1535 QPS: 1.1168
mysql:shops run ShopsUpdateScript 0.0001 QPS: 0.0001
mysql:shops_stats _describe mq/request.php 0.0005 QPS: 0.0033
sphinx:datasheets onGetStats DatasheetController 0.0001 QPS: 0.0008
sphinx:shops newFromQuery Elecena\Models\Shop 0.0023 QPS: 0.0170
mysql:products getLastChanges StatsController 0.0002 QPS: 0.0017
mq/request.php _update mysql:shops 0.0161 QPS: 0.1169
mq/request.php _insert mysql:producers 0.0001 QPS: 0.0001
sphinx:products getSnippet Elecena\Services\Sphinx 0.0702 QPS: 0.5110
mysql:datasheets newFromIds Elecena\Models\Datasheet 0.0426 QPS: 0.3099
mq/request.php _insert mysql:products 0.0063 QPS: 0.0459
sphinx:shops newFromLocation Elecena\Models\Shop 0.0005 QPS: 0.0035
mq/request.php _delete mysql:shops_stats 0.0005 QPS: 0.0033
mysql:shops newFromDB Elecena\Models\Shop 0.0012 QPS: 0.0089
sphinx:parameters getFromCategory Parameters 0.0001 QPS: 0.0005
mq/request.php _update mysql:products 1.0000 QPS: 7.2778
mysql:suggest getSuggestions Elecena\Services\Sphinx 0.0024 QPS: 0.0172
sphinx:datasheets search Elecena\Services\Sphinx 0.1071 QPS: 0.7794
mysql:products _select mq/request.php 0.0312 QPS: 0.2268
mysql:parameters newFromTop Parameters 0.0006 QPS: 0.0045
mq/request.php _insert mysql:shops_stats 0.0036 QPS: 0.0265
sphinx:products getIndexCount Elecena\Services\Sphinx 0.0001 QPS: 0.0008
mysql:producers _describe mq/request.php 0.0001 QPS: 0.0000
mysql:datasheets checkIfExists DatasheetsUpdateScript 0.1081 QPS: 0.7864
mysql:datasheets getOtherCodes Elecena\Models\Datasheet 0.0020 QPS: 0.0147
mysql:currencies _ currency.php 0.0001 QPS: 0.0008
mysql:products getImagesToFetch ImageBot 0.0008 QPS: 0.0057
mysql:datasheets onSitemapGetLinks DatasheetController 0.0001 QPS: 0.0000
sphinx:products products Elecena\Services\Search 0.0762 QPS: 0.5547
mysql:shops checkBotsQueue Elecena\ElecenaHealthcheck 0.0044 QPS: 0.0321
sphinx:products rebuildStats StatsController 0.0003 QPS: 0.0025
mysql:products getIdsBatch ProductController 0.0147 QPS: 0.1073
# Redis log entries
bots:docker-bjornoya push redis:products 0.3988 166.8 messages/hour
redis:products pop mq/request.php 1.0000 418.1 messages/hour
bots:docker-monty push redis:products 0.1857 77.7 messages/hour
bots:s0 push redis:products 0.3874 162.0 messages/hour
# bots HTTP traffic
web:shops http fetch bots:docker-bjornoya 0.6406 14119 requests/hour, 0.27 GiB/hour
web:shops http fetch bots:docker-monty 0.3594 7921 requests/hour, 0.15 GiB/hour
web:shops http fetch bots:s0 0.9209 20298 requests/hour, 0.38 GiB/hour
# s3 operations
ImageBot upload s3:s.elecena.pl 1.0000 522.1 requests/hour
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment