Tested with Apache Spark 2.1.0, Python 2.7.13 and Java 1.8.0_112
For older versions of Spark and ipython, please, see also previous version of text.
id: | |
type: integer | |
nullable: false | |
some_text_field: | |
type: string | |
nullable: false | |
some_date_field: | |
type: datetime |
class Example1(AbstractClassBasedFlow): | |
... | |
class Example2(AbstractClassBasedFlow): | |
... | |
flow1 = Example1.to_prefect_flow() | |
flow2 = Example2.to_prefect_flow() |
from prefect.flows import Flow | |
class AbstractClassBasedFlow: | |
""" | |
An example abstract class that shows how you can dynamically create prefect | |
flows. | |
""" | |
@classmethod |
name: sample_etl_configuration | |
description: Here's some example of an ETL that serves as an input for data processing | |
tasks: | |
extract_from_sql: | |
template_file: extract.sql | |
validation_file: extraction_contract.yml | |
transform: | |
functions: [add_total_fees, add_total_payout, agg_by_day_and_platform] | |
validation_file: transformed_contract.yml | |
load_from_sql: |
Tested with Apache Spark 2.1.0, Python 2.7.13 and Java 1.8.0_112
For older versions of Spark and ipython, please, see also previous version of text.
#!/usr/bin/awk -f | |
#Autor: Jonas Scherer | |
#Data: Jan/2016 | |
#Uso: ./conversor_bdi.awk COTAHIST_A2015.TXT | |
#Detalhes: Conversor do formato de cotacoes historicas disponiveis no site da Bovespa. | |
#O conversor podera ser usado para exportar os dados para excel para serem trabalhados de uma forma mais simples. | |
#Util para quem faz day-trades e deseja criar seus proprios dashboards ou fazer backtests. | |
#OBS: Remover a primeira e ultima linha para remover o cabecalho e o rodape que vem da Bovespa |
### init.sh | |
#!/bin/bash | |
echo -n "Username: " | |
read username | |
export username | |
echo -n "Password: " | |
read -s password | |
export password | |
echo "\r" |
#!/usr/bin/env ruby | |
user_id = 3 | |
puts "Start" | |
Dir["./reports/*"].each do |file_name| | |
file_name.gsub!(" ", "\\ ") | |
puts "./arachni_web_scan_import #{file_name} #{user_id}" | |
system("./arachni_web_scan_import #{file_name} #{user_id}") | |
end | |
puts "End" |
puts "Start" | |
input = File.read("./input.txt") | |
input.gsub!(/\r\n?/, "\n") | |
input.each_line do |line| | |
s = line.strip | |
puts "./arachni #{s} --report-save-path ./reports" | |
system("./arachni #{s} --report-save-path ./reports") | |
end | |
puts "End" |
.element { | |
position: relative; | |
top: 50%; | |
transform: translateY(-50%); | |
} |