# This is a modified version of TRL's `SFTTrainer` example (https://github.com/huggingface/trl/blob/main/examples/scripts/sft_trainer.py), | |
# adapted to run with DeepSpeed ZeRO-3 and Mistral-7B-V1.0. The settings below were run on 1 node of 8 x A100 (80GB) GPUs. | |
# | |
# Usage: | |
# - Install the latest transformers & accelerate versions: `pip install -U transformers accelerate` | |
# - Install deepspeed: `pip install deepspeed==0.9.5` | |
# - Install TRL from main: pip install git+https://github.com/huggingface/trl.git | |
# - Clone the repo: git clone github.com/huggingface/trl.git | |
# - Copy this Gist into trl/examples/scripts | |
# - Run from root of trl repo with: accelerate launch --config_file=examples/accelerate_configs/deepspeed_zero3.yaml --gradient_accumulation_steps 8 examples/scripts/sft_trainer.py |
#!/bin/sh | |
sudo ps aux | grep Netskope | grep -v grep | awk '{ print "kill -9", $2 }' | sudo sh | |
echo '[✓] Kill Netskope Process' | |
sudo rm -rf /Applications/Remove\ Netskope\ Client.app | |
echo '[✓] Removed Remove Netskope Client.app' | |
sudo rm -rf /Library/Application\ Support/Netskope | |
echo '[✓] Removed Agent of Netskope Client.app' |
--- | |
- hosts: localhost | |
tasks: | |
- name: Add Apt-Keys | |
apt_key: | |
url: '{{item.name}}' | |
state: present | |
with_items: |
| Title | Description
package main | |
import ( | |
"fmt" | |
"time" | |
"github.com/aws/aws-sdk-go/aws" | |
"github.com/aws/aws-sdk-go/aws/session" | |
"github.com/aws/aws-sdk-go/service/athena" | |
) |
You can use CodePipeline and CodeBuild to create a continuous deployment/integration pipeline for serverless applications build on AWS Lambda and Amazon API Gateway. This sample application is written in Go with the Gin framework and uses the eawsy API Gateway proxy shim: https://github.com/eawsy/aws-lambda-go-net
We initially detailed our methodology in this blog post: https://aws.amazon.com/blogs/compute/continuous-deployment-for-serverless-applications/
We have used the shim technology created by eawsy to run Golang applications inside AWS Lambda (https://github.com/eawsy/aws-lambda-go-shim) and created a container that can be used with CodeBuild as part of our original pipeline template.
The container is available on DockerHub and is called sapessi/aws-lambda-go18-codebuild:latest
. To use this container, simply change the Image
property of the CodeBuild project environment.
The pipeline template, sample app, buildspec and SAM files are attached to this gist.
package main | |
import ( | |
"fmt" | |
"log" | |
"net/http" | |
"strings" | |
) | |
func main() { |
image: golang:1.7 | |
stages: | |
- build | |
- test | |
before_script: | |
- go get github.com/tools/godep | |
- cp -r /builds/user /go/src/github.com/user/ | |
- cd /go/src/github.com/user/repo |
[root@ip-10-0-11-111 anchors]# curl -o /etc/pki/ca-trust/source/anchors/GTEGlRoot.crt https://www.cybertrust.ne.jp/sureserver/download/root_ca/GTEGlRoot.txt | |
% Total % Received % Xferd Average Speed Time Time Time Current | |
Dload Upload Total Spent Left Speed | |
100 890 100 890 0 0 6885 0 --:--:-- --:--:-- --:--:-- 6899 | |
[root@ip-10-0-11-111 anchors]# ls | |
GTEGlRoot.crt | |
[root@ip-10-0-11-111 anchors]# update-ca-trust | |
p11-kit: duplicate 'GTE CyberTrust Global Root' certificate found in: ca-bundle.legacy.crt | |
p11-kit: duplicate 'GTE CyberTrust Global Root' certificate found in: ca-bundle.legacy.crt | |
p11-kit: duplicate 'GTE CyberTrust Global Root' certificate found in: ca-bundle.legacy.crt |