yaythomas

# run this like follows:
# Run only the common/shared dependency:
# $ pypyr deps-example 
# 
# Run either my_task or another_task:
# $ pypyr deps-example my_task
# $ pypyr deps-example another_task
# 
# And to run both tasks:
# $ pypyr deps-example my_task another_task

yaythomas

AWS_ACCESS_KEY_ID="your_object_storage_access_key"
AWS_SECRET_ACCESS_KEY="your_object_storage_secret_key"
BUCKET_NAME="thebucket"
ENDPOINT_URL="https://endpoint"
REGION_NAME="theregion"

yaythomas

AWS_ACCESS_KEY_ID: mykeyID
AWS_SECRET_ACCESS_KEY: mysecretkey
endpoint_url: myendpoint
region: myregion
bucket: mybucket
destination_dir: models/spam-sms
files:
  - exports/spam-sms/spam-model.h5
  - exports/spam-sms/spam-classifer-tokenizer.json
  - exports/spam-sms/spam-classifer-metadata.json

yaythomas

context_parser: pypyr.parser.keyvaluepairs
steps:
  - name: pypyr.steps.default
    in:
      defaults:
        docker_image: pytorch/pytorch:nightly-devel-cuda10.0-cudnn7
        task_name: MRPC
        model_name: bert-base-uncased
        parameters:
          - name: model_type

yaythomas

convert hugo content files to page leaf bundles

This pipeline converts Hugo content pages to page leaf bundles.

From:

`-- content
    |-- about.md
    `-- post
 |-- content-1.md

yaythomas

steps:
  - name: pypyr.steps.echo
    comment: if you want to see the echo output, remember to set loglevel <=25
    foreach: '{input_collection}'
    in:
      echoMe: just an echo with {i.arbfield}
  - name: pypyr.steps.contextsetf
    in:
      contextSetf:
        newkey: arbitrary value set in pipeline

yaythomas

automating your video processing

[preamble]

what we want to achieve

TODO:

1. download the video
2. convert to different formats via ffmpeg
3. create thumbnails
4. use a trained ML model for scene classification
5. make sub clips based on classification

yaythomas

if you do all your python in virtualenvs, and you probably should be, the one package that sits in global is virtualenv.

when you upgrade your python installation, those virtualenvs do not upgrade themselves. you have to go and re-create your virtualenv when you upgrade python. this is annoying. this is even more annoying when you have pip.conf set with

[install]
require-virtualenv = true

[uninstall]