Skip to content

Instantly share code, notes, and snippets.

View dyusupov's full-sized avatar

Dmitry Yusupov dyusupov

View GitHub Profile
@dyusupov
dyusupov / edgefs-cluster-hybrid.yaml
Last active December 20, 2018 06:16
edgefs-cluster-hybrid.yaml
apiVersion: edgefs.rook.io/v1alpha1
kind: Cluster
metadata:
name: rook-edgefs
namespace: rook-edgefs
spec:
edgefsImageName: edgefs/edgefs:latest
serviceAccount: rook-edgefs-cluster
dataDirHostPath: /var/lib/edgefs
storage:
[global]
ioengine=ccowvolaio
buffered=0
direct=1
buffer_compress_percentage=50
dedupe_percentage=87
allrandrepeat=0
refill_buffers
norandommap
randrepeat=0
@dyusupov
dyusupov / edgefs-object.fio
Last active December 12, 2018 07:00
edgefs-object.fio
[global]
ioengine=ccowobj
buffered=0
direct=1
buffer_compress_percentage=50
dedupe_percentage=87
allrandrepeat=0
refill_buffers
norandommap
randrepeat=0
@dyusupov
dyusupov / edgefs-nfs-gateway.yaml
Last active December 15, 2018 02:52
edgefs-nfs-gateway.yaml
apiVersion: edgefs.rook.io/v1alpha1
kind: NFS
metadata:
name: files
namespace: rook-edgefs
spec:
instances: 1
placement:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
@dyusupov
dyusupov / edgefs-s3-gateway.yaml
Last active December 15, 2018 02:53
edgefs-s3-gateway.yaml
apiVersion: edgefs.rook.io/v1alpha1
kind: S3
metadata:
name: objects
namespace: rook-edgefs
spec:
instances: 1
placement:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
@dyusupov
dyusupov / docker-compose.yml
Last active August 10, 2021 05:49
single-node docker-compose file
version: '2.4'
services:
target:
privileged: true
network_mode: "host"
ipc: "host"
restart: always
volumes:
- '/etc/localtime:/etc/localtime:ro'
- '/edgefs/var/run:/opt/nedge/var/run:z'
@dyusupov
dyusupov / s3-1mb.xml
Created March 3, 2019 06:14
cosbench s3 1MB read-write, single bucket 100,000 objects
<?xml version="1.0" encoding="UTF-8"?>
<workload name="S3-1MB-write-reread" description="Verification of S3 parallel I/O of 1MB objects into a single bucket">
<auth type="none" config="username=admin;password=edgefs;auth_url=http://172.10.10.75:8080/auth/v1.0"/>
<workflow config="">
<workstage name="prepare">
<work name="prepare-objects" type="prepare" workers="40" division="object"
interval="20" config="cprefix=shared;containers=r(1,1);objects=r(1,100000);sizes=r(1,1)MB;hashCheck=false">
<storage type="s3" config="accesskey=KEY;secretkey=SEC;endpoint=http://172.10.10.75:9982;timeout=600000"/>
</work>
</workstage>
version: '2.4'
services:
target:
mem_limit: 1g
memswap_limit: 0
privileged: true
ipc: "host"
restart: always
volumes:
- '/edgefs/var:/opt/nedge/var:z'
@dyusupov
dyusupov / aws-s3-edgefs-sync.js
Last active May 29, 2019 18:12
AWS S3 and EdgeFS synchronization via Lambda function
var assert = require('assert');
var qs = require('querystring');
var AWS = require('aws-sdk');
var url = require('url');
var dns = require('dns');
// ======= config section =======
var AWS_ACCESS = "aws_key";
var AWS_SECRET = "aws_secret";
var EDGEFS_PROVIDER = "aws-s3"; // Has to match EdgeFS supported Cloud Provider name
@dyusupov
dyusupov / gcp-edgefs-s3-create-sync.js
Created May 29, 2019 18:16
GCP GS and EdgeFS create synchronization via Cloud function
'use strict';
const AWS = require('aws-sdk');
const Storage = require('@google-cloud/storage');
// ======= config section =======
var GCP_ACCOUNT = "xxxxxxxx@appspot.gserviceaccount.com"; // GCP Service Account Email
var EDGEFS_PROVIDER = "gcs-p12"; // Has to match EdgeFS Cloud Provider name
var EDGEFS_ACCESS = "edgefs_access_key"; // EdgeFS tenant user S3 access key
var EDGEFS_SECRET = "edgefs_secret"; // EdgeFS tenant user S3 secret key