| Adding a disk to EC2 instance: | |
| 1) Note the availability zone of the EC2 instance. | |
| 2) Create a new EBS volume in the same zone. | |
| 3) After the volume is created, attach the volume to the EC2 instance. | |
| 4) SSH to the EC2 instance and run 'dmsg' command to see if the disk is available. | |
| 5) It displays "unknown partition table" at the end of the output log then the volume is attached to the instance. | |
| 6) We now have to create partition and use it. | |
| sudo fdisk /dev/xvdf | |
| (you enter fdisk mode) |
| Verify key pair finger print | |
| openssl pkcs8 -in KeyPair.pem -nocrypt -topk8 -outform DER | openssl sha1 -c | |
| Get public key associated with private Key | |
| ssh-keygen -y -f KeyPain.pem | |
| Google Chrome launch problem on CentOS | |
| -------------------------------------- | |
| Error Message : SELinux is preventing chrome from using the setcap access on a process. | |
| Solution: Execute the following commands | |
| grep chrome /var/log/audit/audit.log | audit2allow -M mypol | |
| semodule -i mypol.pp |
Kafka acts as a kind of write-ahead log (WAL) that records messages to a persistent store (disk) and allows subscribers to read and apply these changes to their own stores in a system appropriate time-frame.
Terminology:
| Steps to install: | |
| ------------------ | |
| Following steps | |
| - Install Nvidia GPU drivers | |
| - Install Cuda | |
| - Install CudNN | |
| - Install Anaconda | |
| - Install tensorflow | |
| - Run sample program to test |
| 1) pip install keras | |
| 2) To verify if keras is using tensorflow as its backend, run the following command: | |
| python -c "from keras import backend; print(backend._BACKEND)" | |
| 3) To change the backend, edit the file ~/.keras/keras.json | |
| { | |
| "image_dim_ordering": "tf", | |
| "epsilon": 1e-07, | |
| "floatx": "float32", | |
| "backend": "theano" | |
| } |
| Steps to install: | |
| ------------------ | |
| Following steps | |
| - Install Nvidia GPU drivers | |
| - Install Cuda | |
| - Install CudNN | |
| - Install Anaconda | |
| - Install tensorflow | |
| - Run sample program to test |
| from pyspark import SparkContext | |
| from pyspark.streaming import StreamingContext | |
| from pyspark.streaming.kafka import KafkaUtils | |
| def read_offsets(zk, topics): | |
| from pyspark.streaming.kafka import TopicAndPartition | |
| from_offsets = {} | |
| for topic in topics: | |
| for partition in zk.get_children(f'/consumers/{topic}'): |
| Tensorrflow installation on Ubuntu | |
| ---------------------------------------- | |
| Required Steps: | |
| ------------------ | |
| - Install Nvidia GPU drivers | |
| - Install Cuda | |
| - Install CudNN | |
| - Install Anaconda | |
| - Install tensorflow |