This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Follow the steps here | |
https://mkyong.com/java/how-to-set-java_home-environment-variable-on-mac-os-x/ | |
or do the following | |
1. Get location of default jdk | |
/usr/libexec/java_home | |
2. Find all installed JDKs | |
/usr/libexec/java_home -V | |
3. Run a specified JDK. Note you can pick any of the versions already in you system | |
/usr/libexec/java_home -v1.8 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Version TF record | |
dvc_track_files() { | |
dvc init | |
dvc add data.tfrecord | |
dvc remote add -d storage s3://mybucket/datastorage | |
dvc push |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Version TF record saved on s3 | |
dvc_track_files() { | |
git init | |
#initialise dvc in a git directory | |
dvc init |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When reading or writing data on S3 with your TensorFlow program, the behavior could be controlled by various environmental variables: | |
* AWS_REGION: By default, regional endpoint is used for S3, with region controlled by AWS_REGION. If AWS_REGION is not specified, then us-east-1 is used. | |
* S3_ENDPOINT: The endpoint could be overridden explicitly with S3_ENDPOINT specified. | |
* S3_USE_HTTPS: HTTPS is used to access S3 by default, unless S3_USE_HTTPS=0. | |
* S3_VERIFY_SSL: If HTTPS is used, SSL verification could be disabled with S3_VERIFY_SSL=0. | |
To read or write objects in a bucket that is no publicly accessible, AWS credentials must be provided through one of the following methods: | |
* Set credentials in the AWS credentials profile file on the local system, located at: ~/.aws/credentials on Linux, macOS, or Unix, or C:\Users\USERNAME\.aws\credentials on Windows. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To enter docker environmnet | |
docker exec -it <id> /bin/bash | |
run dag task from s3 | |
airflow tasks test <dag_id> <task_id> 2020-01-01 | |
docker-compose up --build | |
stop all docker containers | |
docker container stop $(docker container ls -q) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was gotten from https://www.learnopencv.com/install-opencv-4-on-ubuntu-18-04/ | |
# Create your working directory | |
mkdir project && cd project | |
#Visit to follow the article | |
Change CMAKE_INSTALL_PREFIX=$cwd/installation/OpenCV-"$cvVersion" \ to CMAKE_INSTALL_PREFIX=/usr/local \. and | |
OPENCV_PYTHON3_INSTALL_PATH=$cwd/OpenCV-$cvVersion-py3/lib/python3.5/site-packages \ to the right directory |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#This was taken from this repo https://github.com/FloopCZ/tensorflow_cc | |
#Download and install requirement | |
sudo apt-get install cmake curl g++-7 git python3-dev python3-numpy sudo wget | |
# Install Bazel | |
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add - | |
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Step One: (download awscli) | |
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" | |
Step Two: (Unzip awscli) | |
unzip awscli-bundle.zip | |
Step Three: (Run the installation) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from selenium import webdriver | |
import os | |
import wget | |
#adapted from https://gist.github.com/genekogan/ebd77196e4bf0705db51f86431099e57 | |
#input your search item here | |
searchterm = 'Tomato' | |
url = "https://www.google.co.in/search?q="+searchterm+"&source=lnms&tbm=isch" |