Skip to content

Instantly share code, notes, and snippets.

View Hugoberry's full-sized avatar

Igor Cotruta Hugoberry

View GitHub Profile
@Hugoberry
Hugoberry / CRC32.m
Last active January 30, 2024 22:33
CRC32 implementation in Power Query M mostly inpired by the article about CRC32 computation http://www.sunshine2k.de/articles/coding/crc/understanding_crc.html and Bit Hacks https://graphics.stanford.edu/~seander/bithacks.htm
(StringInput)=>
let
/*function for reversing the bits in a 8-bit number*/
reverse = (x) =>
Number.Mod(Number.BitwiseAnd((x* 0x0202020202),0x010884422010),1023),
/*function for reversing the bits in a 32-bit number*/
reverse32 = (x) =>
let
b0 = Number.BitwiseAnd(x,0xff),
@Hugoberry
Hugoberry / ReversingBits.ps1
Last active January 24, 2017 16:37
An example of reversing bits in a 32-bit number
function reverse(){
param([Int32]$b)
$b = (($b -band 0xf0) -shr 4) -bor (($b -band 0x0f) -shl 4)
$b = (($b -band 0xcc) -shr 2) -bor (($b -band 0x33) -shl 2)
$b = (($b -band 0xaa) -shr 1) -bor (($b -band 0x55) -shl 1)
$b
}
# on a 64bit machine this is an alternative to reverse() function
function reverse64(){
param([Int32]$x)
@Hugoberry
Hugoberry / Fisher-Yates-shuffle.m
Last active February 1, 2017 18:11
The Fisher–Yates shuffle is an algorithm for generating a random permutation of a finite set—in plain terms, the algorithm shuffles the set. The algorithm effectively puts all the elements into a hat; it continually determines the next element by randomly drawing an element from the hat until no elements remain. The algorithm produces an unbiase…
(n) =>
let
swapAnyOrder = (_,from,to)=> if from = to then _ else
(if from < to then swap(_,from,to) else swap(_,to,from)),
swap = (_,from,to)=> List.Range(_,0,from)
&{_{to}}
&List.Range(_,from+1,to-from-1)
&{_{from}}
&List.Range(_,to+1)
in
@Hugoberry
Hugoberry / MatrixMultiplication.m
Last active January 29, 2018 14:59
Matrix Multiplication in Power Query M. Test example = MatrixMultiplication(#table({"A","B"},{{1,2},{3,4}}), #table({"A","B"},{{1,2},{3,4}}))
(A,B) =>
let
//vector multiplication
dotProduct = (v1,v2) =>
List.Accumulate(
List.Zip({v1,v2}),
0,
(agg,_)=>
agg+(_{1}*_{0}))
@Hugoberry
Hugoberry / UnZIP.m
Created February 10, 2017 11:35
UnZip in Power Query M
(ZIPFile) =>
let
Header = BinaryFormat.Record([ Signature = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger32,ByteOrder.LittleEndian),
Version = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian),
Flags = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian),
Compression = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian),
ModTime = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian),
ModDate = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian),
CRC32 = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger32,ByteOrder.LittleEndian),
CompressedSize = BinaryFormat.ByteOrde
@Hugoberry
Hugoberry / ZIPdissect.m
Last active February 17, 2018 19:13
Dissecting ZIP file structure in Power Query M following explanation from https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html
(ZIPFile) =>
let
//shorthand
UInt32 = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger32,
ByteOrder.LittleEndian),
UInt16 = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,
ByteOrder.LittleEndian),
//Local File Header
Header = BinaryFormat.Record([
Version = UInt16,
@Hugoberry
Hugoberry / JSONexpand.m
Last active February 22, 2017 17:11
JSON expansion in Power Query M
= List.Generate(
()=> [
can_expand=true,
level =1,
input =json,
output =[level.0=Record.FieldNames(json){0}]
],
each [can_expand],
each [
can_expand =Record.FieldValues([input]){0} is record,
@Hugoberry
Hugoberry / ExpandableColumns.m
Last active February 23, 2017 17:55
Get a list of expandable columns from a table input in Power Query
expandableColumns = (_) => List.Accumulate(
Table.ColumnNames(_),
{},
(s,c)=>s&(if Type.Is(Value.Type(Record.Field(_{0},c)), type record)
or Type.Is(Value.Type(Record.Field(_{0},c)), type list)
then {c}
else {}
)
),
columnHasConsistentType = (T,Cname) =>
@Hugoberry
Hugoberry / JSON2table.m
Last active May 20, 2024 18:38
JSON to Table in Power Query M
(json) =>
let
//List the expandable columns
expandableColumns = (_) => List.Accumulate(
Table.ColumnNames(_),
{},
(s,c)=>s&(if Type.Is(Value.Type(Record.Field(_{0},c)), type record)
or Type.Is(Value.Type(Record.Field(_{0},c)), type list)
then {c}
else {})
@Hugoberry
Hugoberry / reverse8bit.m
Created February 27, 2017 23:06
Function to reverse an 8 bit number
reverse = (x) =>
Number.Mod(Number.BitwiseAnd((x* 0x0202020202),0x010884422010),1023),