Skip to content

Instantly share code, notes, and snippets.

View lambrospetrou's full-sized avatar

Lambros Petrou lambrospetrou

50 followers · 14 following
View GitHub Profile
@lambrospetrou
lambrospetrou / readme.md
Last active January 19, 2024 12:39
DuckDB JSON to Parquet in S3

DuckDB JSON to Parquet

Command to generate sample NDJSON files:

copy (select * from (select a.range as 'a' from range(1000) as a), (select b.range as 'b' from range(1000) as b)) to 'range.ndjson';

Creates a 2x1000000 table:

select count(1) from 'range.ndjson';
┌──────────┐
@lambrospetrou
lambrospetrou / main.go
Last active November 11, 2023 18:49
Serverless database simple test - https://twitter.com/LambrosPetrou/status/1723389824327901536
package main
import (
"database/sql"
"context"
"fmt"
"log"
"time"
"os"
@lambrospetrou
lambrospetrou / paas-cicd-platforms-tools.md
Last active April 4, 2023 22:03
@lambrospetrou
lambrospetrou / git-sparse-checkout-specific-folder-demo.sh
Created December 14, 2022 23:46
Git sparse checkout only specific directories of a repo
BRANCH_NAME="master"
# Clone the repository .git information only
# Docs: https://git-scm.com/docs/git-clone
#
# --no-checkout :: Do not fetch any files, only the `.git` directory.
# --sparse :: Only files at the top-level directory, at the root of the repository, will be part of the checkout.
# --branch <branch_name> :: Only fetch the information for the given branch.
# --depth 1 :: Only fetch the tip commit, HEAD of the specified branch.
# --filter=blob:none :: Do not download any blob files.
@lambrospetrou
lambrospetrou / go-aws-lambda-redis.go
Last active August 26, 2022 17:28
package main
import (
"bufio"
"context"
"log"
"net/http"
"os"
"strconv"
"sync"
@lambrospetrou
lambrospetrou / test-upstash-redis.go
Created August 26, 2022 14:27
Use Upstash Redis - rest and native clients from Go
package main
import (
"bufio"
"context"
"log"
"net/http"
"sync"
"time"
@lambrospetrou
lambrospetrou / awslogs.config
Last active November 7, 2023 13:58
Custom CloudWatch Logs agent configuration file for Elastic Beanstalk AL2
###################################################################################################
#### Copyright 2016 Amazon.com, Inc. or its affiliates. All Rights Reserved.
####
#### Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file
#### except in compliance with the License. A copy of the License is located at
####
#### http://aws.amazon.com/apache2.0/
####
#### or in the "license" file accompanying this file. This file is distributed on an "AS IS"
#### BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
@lambrospetrou
lambrospetrou / deployment.log
Created January 29, 2022 19:46
Fly.io simple Dockerfile deployment
2022-01-29T19:44:43.080 runner[121033e8] lhr [info] Shutting down virtual machine
2022-01-29T19:44:43.215 app[121033e8] lhr [info] Sending signal SIGINT to main child process w/ PID 518
2022-01-29T19:44:43.218 app[121033e8] lhr [info] Main child exited with signal (with signal 'SIGINT', core dumped? false)
2022-01-29T19:44:43.219 app[121033e8] lhr [info] Starting clean up.
2022-01-29T19:44:43.234 app[121033e8] lhr [info] Umounting /dev/vdc from /db
2022-01-29T19:44:47.326 runner[16cd4003] lhr [info] Starting instance
2022-01-29T19:44:47.367 runner[16cd4003] lhr [info] Configuring virtual machine
2022-01-29T19:44:47.369 runner[16cd4003] lhr [info] Pulling container image
2022-01-29T19:44:49.319 runner[16cd4003] lhr [info] Unpacking image
2022-01-29T19:44:50.514 runner[16cd4003] lhr [info] Preparing kernel init
@lambrospetrou
lambrospetrou / elastic-beanstalk-eb-engine.log.csv
Created January 29, 2022 19:25
The Elastic Beanstalk eb-engine.log for the AL2 Docker platform during a simple Dockerfile deployment.
timestamp message
1643482975156 2022/01/29 19:02:55.012950 [INFO] Starting...
1643482975156 2022/01/29 19:02:55.013002 [INFO] Starting EBPlatform-PlatformEngine
1643482975156 2022/01/29 19:02:55.013018 [INFO] reading event message file
1643482975156 2022/01/29 19:02:55.013572 [INFO] no eb envtier info file found, skip loading env tier info.
1643482975156 2022/01/29 19:02:55.013765 [INFO] Engine received EB command cfn-hup-exec
1643482975910 2022/01/29 19:02:55.101565 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:eu-west-1:<redacted>:stack/awseb-e-234tsz8zsf-stack/59ba6f50-812b-11ec-89c0-0227d013ec45 -r AWSEBAutoScalingGroup --region eu-west-1
1643482976410 2022/01/29 19:02:55.677999 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:eu-west-1:<redacted>:stack/awseb-e-234tsz8zsf-stack/59ba6f50-812b-11ec-89c0-0227d013ec45 -r AWSEBBeanstalkMetadata --region eu-west-1
1643482976410 2022/01/29 19:02:56.245445 [INFO] checking wh
@lambrospetrou
lambrospetrou / gist:b24559f9f058dba8191ac30a34537b72
Created January 29, 2022 19:24
The eb-engine.log file for an Elastic Beanstalk deployment using the AL2 Docker platform
We couldn’t find that file to show.