First, install storm dependencies. How do I compile jzmq for ZeroMQ on OSX? helps.
$ brew install zeromq
$ git clone https://github.com/nathanmarz/jzmq.git
$ cd jzmq
$ sudo vim /usr/share/aclocal/dirlist (sudo cat > /usr/share/aclocal/dirlist permission denied?)
/usr/local/share/aclocal
/usr/local/Cellar/pkg-config/0.25/share/aclocal/
$ export JAVA_HOME=/Library/Java/Home
| supervisord -c /etc/supervisord.conf | |
| supervisorctl reread | |
| supervisorctl reload |
| import static org.junit.Assert.*; | |
| import java.io.File; | |
| import java.io.FileInputStream; | |
| import java.io.IOException; | |
| import java.util.Properties; | |
| import org.apache.lucene.analysis.core.WhitespaceAnalyzer; | |
| import org.apache.lucene.analysis.standard.StandardAnalyzer; | |
| import org.apache.lucene.document.Document; |
| Each spout and bolt can have one or more tasks, each of which corresponds to a thread. | |
| Each topology can run under one or more JVMs. For example, if the combined parallelism (number of tasks) of the topology is 300 and 50 | |
| workers are allocated, then each worker will execute 6 tasks (as threads within the worker). Storm tries to spread the tasks evenly | |
| across all the workers. | |
| Refer to this [http://storm.apache.org/releases/1.0.1/Understanding-the-parallelism-of-a-Storm-topology.html] | |
| also http://stackoverflow.com/questions/17257448/what-is-the-task-in-storm-parallelism |
| JMX remote with jconsole via ssh tunneling with socks proxy: | |
| 1. Add this to JAVA_OPTS in remote process: | |
| -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port={remoteJMXport} -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false | |
| 2. ssh -D {proxyport} user@remotehost | |
| 3. jconsole -J-DsocksProxyHost=localhost -J-DsocksProxyPort={proxyport} service:jmx:rmi:///jndi/rmi://{remotehost}:{remoteJMXport}/jmxrmi | |
| For visualvm, see this link: | |
| https://theholyjava.wordpress.com/2012/09/21/visualvm-monitoring-remote-jvm-over-ssh-jmx-or-not/ | |
| For MAT: |
| oauth: | |
| https://jaxenter.com/rest-api-spring-java-8-112289.html | |
| https://raymondhlee.wordpress.com/2014/12/21/implementing-oauth2-with-spring-security/ | |
| http://websystique.com/spring-security/secure-spring-rest-api-using-oauth2/ |
| Analyzer | |
| - Tokenizer | |
| - Series of filters | |
| StandardAnalyzer | |
| - StandardTokenizer | |
| - StandardFilter, LowercaseFilter, StopFilter | |
| Stemming: | |
| - Swimming -> Swim, catlike, cattly -> cat | |
| - Create custom Analyzer, override createComponents() | |
| - PorterStemFilter |
| https://www.facebook.com/v2.8/dialog/oauth? | |
| client_id={app-id} | |
| &redirect_uri={redirect-uri} | |
| https://graph.facebook.com/v2.8/oauth/access_token? | |
| client_id={app-id} | |
| &redirect_uri={redirect-uri} | |
| &client_secret={app-secret} | |
| &code={code-parameter} |
| import pandas as pd | |
| df = pd.DataFrame({'id': [12, None, 56], 'name': ['asif', 'iqbal', 'anika']}) | |
| # df to numpy array | |
| df.values | |
| # df to boolean df (null/ notnull) | |
| df.notnull() | |
| # oppposite of notnull | |
| df.isnull() |
| import os | |
| import torch | |
| import torch.nn as nn | |
| from torch.autograd import Variable | |
| import torchvision.datasets as dset | |
| import torchvision.transforms as transforms | |
| import torch.nn.functional as F | |
| import torch.optim as optim | |
| ## load mnist dataset | |
| use_cuda = torch.cuda.is_available() |