Skip to content

Instantly share code, notes, and snippets.

View AlenkaF's full-sized avatar

Alenka Frim AlenkaF

21 followers · 11 following
View GitHub Profile
@AlenkaF
AlenkaF / .zprofile
Last active December 2, 2023 11:39
Apache Arrow build commands
eval "$(/opt/homebrew/bin/brew shellenv)"
export ARROW_GITHUB_API_TOKEN=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
export APACHE_JIRA_TOKEN=MjAzNDQzMjU2MDgxOpX3cy3h9s3+TiP1oWpTuiyyp4y9
arrow () {
if [[ $1 = "submodule" ]]; then
git submodule update --init
@AlenkaF
AlenkaF / versions.json
Created September 4, 2023 12:58
test versions.json for Version warning bar
[
{
"name": "13.0 (dev)",
"version": "dev/",
"url": "file:///Users/alenkafrim/repos/arrow-site/docs/dev/"
},
{
"name": "12.0 (stable)",
"version": "",
"url": "file:///Users/alenkafrim/repos/arrow-site/docs/"
@AlenkaF
AlenkaF / gist:bbec7d6f720d21386076b453b0c6a7f5
Created September 4, 2023 10:26
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
@AlenkaF
AlenkaF / gist:0d0e81d239540301a7c732442038a7b0
Last active September 4, 2023 10:26
<script type="text/javascript" src="https://gist.githubusercontent.com/AlenkaF/bbec7d6f720d21386076b453b0c6a7f5/raw/ab0fd2b074acd7731531d2445363865c633b1370/gistfile1.txt"></script>
@AlenkaF
AlenkaF / versions.json
Last active September 6, 2023 13:34
Versions for version switcher Apache Arrow
[
{
"name": "14.0 (dev)",
"version": "dev/",
"url": "https://arrow.apache.org/docs/dev/"
},
{
"name": "13.0 (stable)",
"version": "",
"url": "https://arrow.apache.org/docs/",
@AlenkaF
AlenkaF / profiler_ouput.txt
Last active March 22, 2023 13:10
Line # Mem usage Increment Occurrences Line Contents
=============================================================
7 147.8 MiB 147.8 MiB 1 @profile
8 def my_func():
9 # Load Vaex example
10 173.0 MiB 25.3 MiB 1 df = vaex.example()
11 # Create a virtual column
12 173.0 MiB 0.0 MiB 1 df.add_virtual_column("r", "sqrt(x**2 + y**2 + z**2)")
13
14 # Create a __dataframe__ instance
@AlenkaF
AlenkaF / 01_tensor_extenstion_examample.py
Last active January 26, 2023 08:56
Example of tensor extension with tests in PyArrow
import ast
import json
import math
import numpy as np
import pyarrow as pa
class TensorType(pa.ExtensionType):
def __init__(self, value_type, shape, order):
self._value_type = value_type
@AlenkaF
AlenkaF / versionwarning.js
Last active December 1, 2022 11:18
(function() {
// adapted 2022-11 from https://mne.tools/versionwarning.js
if (location.hostname == '') {
var latestStable = '10.0.0';
// HTML tags
var pre = '<div class="container-fluid alert-danger devbar"><div class="row no-gutters"><div class="col-12 text-center">';
var post = '</div></div></div>';
var anchor = 'class="btn btn-danger font-weight-bold ml-3 my-3 align-baseline"';
// Switch button message
var switch_dev = `Switch to unstable development release version`;
@AlenkaF
AlenkaF / arrow_cpp_win_failure_static.txt
Created October 4, 2022 13:07
(pyarrow-dev310) C:\Users\Alenka\repos\arrow\cpp\build>cmake --build . --target install --config Release
[482/590] Building CXX object src\arrow\dataset\CMakeFiles\arrow_dataset_static.dir\discovery.cc.obj
FAILED: src/arrow/dataset/CMakeFiles/arrow_dataset_static.dir/discovery.cc.obj
C:\PROGRA~2\MIB055~1\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe /nologo /TP -DARROW_DS_EXPORTING -DARROW_FLIGHT_SQL_STATIC -DARROW_FLIGHT_STATIC -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_HDFS -DARROW_STATIC -DARROW_WITH_LZ4 -DARROW_WITH_RE2 -DARROW_WITH_SNAPPY -DARROW_WITH_TIMING_TESTS -DARROW_WITH_UTF8PROC -DARROW_WITH_ZLIB -DARROW_WITH_ZSTD -DBOOST_ALL_DYN_LINK -DBOOST_ALL_NO_LIB -DPARQUET_STATIC -DURI_STATIC_BUILD -D_CRT_SECURE_NO_WARNINGS -D_ENABLE_EXTENDED_ALIGNED_STORAGE -IC:\Users\Alenka\repos\arrow\cpp\build\src -IC:\Users\Alenka\repos\arrow\cpp\src -IC:\Users\Alenka\repos\arrow\cpp\src\generated -IC:\Users\Al
@AlenkaF
AlenkaF / PyArrow_install_python_3_10.txt
Created September 27, 2022 12:31
Installing PyArrow with Python 3.10 without setting CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1
(pyarrow-dev310) C:\Users\Alenka\repos\arrow\python>python setup.py build_ext --inplace
running build_ext
creating C:\Users\Alenka\repos\arrow\python\build
creating C:\Users\Alenka\repos\arrow\python\build\cpp
-- Running CMake for PyArrow C++
cmake -DARROW_BUILD_DIR=build -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=C:\Users\Alenka\repos\arrow\python\build\dist -DPYTHON_EXECUTABLE=C:\Users\Alenka\anaconda3\envs\pyarrow-dev310\python.exe -DPython3_EXECUTABLE=C:\Users\Alenka\anaconda3\envs\pyarrow-dev310\python.exe -DPYARROW_WITH_DATASET=on -DPYARROW_WITH_PARQUET_ENCRYPTION=on -DPYARROW_WITH_HDFS=off -G Ninja C:\Users\Alenka\repos\arrow\python\pyarrow/src
-- The C compiler identification is MSVC 19.16.27048.0
-- The CXX compiler identification is MSVC 19.16.27048.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done