Skip to content

Instantly share code, notes, and snippets.

View gabidavila's full-sized avatar
💁‍♀️
I try to solve all my problems with a single SQL query.

Gabriela Ferrara gabidavila

💁‍♀️
I try to solve all my problems with a single SQL query.
View GitHub Profile
#trans.data_transaction DESC
+----+-------------+---------------+--------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------------------------+--------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------+--------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------------------------+--------+----------------------------------------------+
| 1 | SIMP
rdohms [17:03]
am i glad i'm not in brazil now...
rdohms [17:03]
@gabi: how's stuff over there? looks crazy from out here
gabi [17:04]
nah
gabi [17:05]
@gabidavila
gabidavila / product_attributes
Last active December 16, 2015 04:19
Alterar valor de atributo para todos os produtos no Magento Change attribute value for all products in Magento
<?php
//get all products
$products = Mage::getResourceModel('catalog/product_collection')->getAllIds();
//Now create an array of attribute_code => values
$attributesData = array(
array('orig' => 1),
array('ncm' => 33030020),
array('ucom' => 'un'),
array('subst' => 1)
@gabidavila
gabidavila / Git_Colors
Created April 13, 2013 00:44
GIT Colorido Colored GIT
git config --global color.ui auto
git config --global color.branch auto
git config --global color.diff auto
git config --global color.interactive auto
git config --global color.status auto
@gabidavila
gabidavila / sorce_model.php
Created April 13, 2013 00:54
Source model customized Magento
<?php
class Namespace_Module_Model_Customfield extends Mage_Core_Model_Abstract {
protected function _construct() {
$this->_init('namespace/customfield');
}
public function toOptionArray() {
$options = array(
0 => array('value' => 'SP', 'label' => 'São Paulo'),
1 => array('value' => 'MG', 'label' => 'Minas Gerais'),
@gabidavila
gabidavila / pacote.sh
Created April 15, 2013 18:44
Gerar pacote tar.gz git
git diff --name-only 859f95bc 94a94459 | tar -czvpf patch_20130207.tar.gz -T -
@gabidavila
gabidavila / Aws.sh
Created May 28, 2013 02:41
Parametros de incializacao de instancia no AWS
#!/bin/sh
yum -y install httpd php
/sbin/chkconfig httpd on
/etc/init.d/httpd start
@gabidavila
gabidavila / tamanho_db.sql
Created September 17, 2013 02:12
Tamanho dos bancos de dados de um determinado host
SELECT
table_schema "Nome DB",
SUM( data_length + index_length) / 1024 / 1024 "Tamanho em MB"
FROM
information_schema.TABLES
GROUP BY
table_schema;
@gabidavila
gabidavila / git_colors.sh
Created November 3, 2013 21:35
GIT Colorido na linha de comando | Add Colors to GIT command Line
git config --global color.ui auto
git config --global color.branch auto
git config --global color.diff auto
git config --global color.interactive auto
git config --global color.status auto

Why you need a Data Engineer

Tech evolves quickly. When the buzzword Big Data started showing up more and more, the market was in need of people able to analyse and give meaning to what was collected. For instance, an article published in 2012 by Harvard Business Review was entitled: Data Scientist: The Sexiest Job of the 21st Century.

Today we have DBA, Data Scientist, Data Engineer, Data Analyst, a wealth of options with the "Data" as prefix. More often than not, people put everyone in the same basket and assume everyone knows and has the same set of skills.

From my point of view and perspective as a Data Engineer, these are the differences:

  • DBA - Once the most hated person I ever had in the team. Seriously, why doesn't that human give me the necessary permissions on the database? If I had access, I would have done my job sooner... Well, that was my thought as a Software Engineer at the time. Turns o