TEST
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import scala.pickling._ | |
import json._ | |
import org.apache.curator.framework.{ CuratorFrameworkFactory } | |
import org.apache.curator.retry.{ ExponentialBackoffRetry } | |
import org.apache.zookeeper.CreateMode | |
abstract class State | |
case class Scheduled extends State |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
description = "displays processes that delete a file" | |
short_description = "spy file deletion" | |
category = "files" | |
args = | |
{ | |
{ | |
name = "path", | |
description = "the path of the file to monitor", | |
argtype = "string" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@task | |
def fetch(pattern, src): | |
fetcher = LogFetcher(pattern) | |
return fetcher.register(src, fetcher.fetch(src)).id | |
@task | |
def convert_tsv_blocks(tsv_id_list): | |
return group([aggregate.s(tsv_id) for tsv_id in | |
tsv_id_list])() |
Here at Botify
- batch processing of TeraBytes of web server logs involves storing temporary data
- bulk write/read GigaBytes data loads that would not fit in server's main memory (we preferred to improve performance on a single host at the beginning)
- need for persistence
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
type Behavior a = Time -> a | |
type Event a = [(Time,a)] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
infra1/development | |
staging | |
production | |
webservers.yml | |
dbservers.yml | |
backend_servers.yml | |
group_vars/ | |
infra2/development | |
staging | |
production |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Numbers Everyone Should Know | |
L1 cache reference 0.5 ns | |
Branch mispredict 5 ns | |
L2 cache reference 7 ns | |
Mutex lock/unlock 25 ns | |
Main memory reference 100 ns | |
Compress 1K bytes with Zippy 3,000 ns | |
Send 2K bytes over 1 Gbps network 20,000 ns | |
Read 1 MB sequentially from memory 250,000 ns | |
Round trip within same datacenter 500,000 ns |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example: | |
- Use create in the index API (assuming you can). | |
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval). | |
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap. | |
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000). | |
- Increase the memory allocated to elasticsearch node. By default its 1g. | |
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine. | |
- Increase the number of machines you have so |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- http://www.intorobotics.com/5-best-examples-of-how-to-build-a-diy-quadcopter/ | |
- http://www.instructables.com/id/Sturdy-Quadcopter-Build/ | |
- http://yameb.blogspot.ro/search/label/Quadrotors | |
- https://lalegiondesquadri.wordpress.com/2013/09/27/construire-un-multi-cest-facile/ | |
- http://wiki.openpilot.org/display/Doc/Basic+QuadCopter | |
- http://smaccmpilot.org | |
- http://www.helimag.com/multirotors-complets/34639-turnigy-h-l-2.html |
OlderNewer