Skip to content

Instantly share code, notes, and snippets.

View derekperkins's full-sized avatar

Derek Perkins derekperkins

137 followers · 23 following
View GitHub Profile
@derekperkins
derekperkins / aws_helpers.go
Last active April 5, 2024 23:51
How to use AWS WebIdentity to assume a role using GCP Workload Idenity in Go
package awshelpers
import (
"context"
"errors"
"github.com/aws/aws-sdk-go-v2/aws"
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/credentials/stscreds"
"github.com/aws/aws-sdk-go-v2/service/sts"
@derekperkins
derekperkins / olric_k8s.go
Created December 19, 2022 04:59
olric kubernetes adapter
package nolric
import (
"context"
"log"
"github.com/buraksezer/olric/pkg/service_discovery"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
@derekperkins
derekperkins / etcd logs
Created April 1, 2022 22:23
This file has been truncated, but you can view the full file.
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292450 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CLIENT_PORT_2379_TCP=tcp://10.113.13.209:2379
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292502 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CLIENT_PORT_2379_TCP_ADDR=10.113.13.209
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292508 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CLIENT_SERVICE_PORT=2379
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292515 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CLIENT_PORT_2379_TCP_PORT=2379
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292527 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CLIENT_PORT=tcp://10.113.13.209:2379
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292532 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CLIENT_PORT_2379_TCP_PROTO=tcp
etcd-global-msl72l4vn8 etcd 2022-04-01 21:16:39.292542 W | pkg/flags: unrecognized environment variable ETCD_GLOBAL_CL
@derekperkins
derekperkins / block
Created April 1, 2022 20:23
--- contention:
cycles/second=2250006157
55949963957215 1174 @ 0x448208 0x10f5411 0x469301
# 0x448207 runtime.selectgo+0x407 runtime/select.go:492
# 0x10f5410 vitess.io/vitess/go/timer.(*Timer).run+0xb0 vitess.io/vitess/go/timer/timer.go:98
53095068945274 9478 @ 0x448208 0x9a2e3b 0x9a3525 0x9b87e6 0x469301
# 0x448207 runtime.selectgo+0x407 runtime/select.go:492
# 0x9a2e3a google.golang.org/grpc/internal/transport.(*controlBuffer).get+0x11a google.golang.org/grpc@v1.39.0/internal/transport/controlbuf.go:406
# 0x9a3524 google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x84 google.golang.org/grpc@v1.39.0/internal/transport/controlbuf.go:533
@derekperkins
derekperkins / block
Last active March 19, 2022 04:52
shard that has quit serving messages, but PlannedReparentShard hasn't been called yet
--- contention:
cycles/second=2249996384
21943038755592 5968 @ 0x448208 0x9a2e3b 0x9a3525 0x9b87e6 0x469301
# 0x448207 runtime.selectgo+0x407 runtime/select.go:492
# 0x9a2e3a google.golang.org/grpc/internal/transport.(*controlBuffer).get+0x11a google.golang.org/grpc@v1.39.0/internal/transport/controlbuf.go:406
# 0x9a3524 google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x84 google.golang.org/grpc@v1.39.0/internal/transport/controlbuf.go:533
# 0x9b87e5 google.golang.org/grpc/internal/transport.NewServerTransport.func2+0xc5 google.golang.org/grpc@v1.39.0/internal/transport/http2_server.go:304
21940941343275 1950 @ 0x448208 0x1266585 0x128db2a 0x1363454 0x1357293 0xb2a049 0xb0423a 0xb040df 0x9f8824 0x9f9cc5 0x9f3ab8 0x469301
# 0x448207 runtime.selectgo+0x407 runtime/select.go:492
@derekperkins
derekperkins / block
Created March 4, 2022 17:36
--- contention:
cycles/second=2250004284
20266199445767 453 @ 0x447fa8 0x1038531 0x469081
# 0x447fa7 runtime.selectgo+0x407 runtime/select.go:492
# 0x1038530 vitess.io/vitess/go/timer.(*Timer).run+0xb0 vitess.io/vitess/go/timer/timer.go:98
14918811420729 2690 @ 0x447fa8 0x9a227b 0x9a2965 0x9b7c26 0x469081
# 0x447fa7 runtime.selectgo+0x407 runtime/select.go:492
# 0x9a227a google.golang.org/grpc/internal/transport.(*controlBuffer).get+0x11a google.golang.org/grpc@v1.39.0/internal/transport/controlbuf.go:406
# 0x9a2964 google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x84 google.golang.org/grpc@v1.39.0/internal/transport/controlbuf.go:533
@derekperkins
derekperkins / pprof-goroutine?debug=1
Created November 3, 2021 19:47
Deadlock during tablet shutdown
I1103 19:42:02.282187 1 snapshot_conn.go:72] Tables unlocked: rankings__searches_listener__msgs_pri
I1103 19:42:02.671255 1 snapshot_conn.go:79] Locking table searches__retriever_dataforseo__msgs for copying
I1103 19:42:02.672326 1 snapshot_conn.go:72] Tables unlocked: searches__retriever_dataforseo__msgs
I1103 19:42:03.073167 1 snapshot_conn.go:79] Locking table searches__requester_dataforseo__msgs_pri for copying
I1103 19:42:03.073282 1 snapshot_conn.go:79] Locking table searches__requester_dataforseo__msgs for copying
I1103 19:42:03.074502 1 snapshot_conn.go:72] Tables unlocked: searches__requester_dataforseo__msgs_pri
I1103 19:42:03.074517 1 snapshot_conn.go:72] Tables unlocked: searches__requester_dataforseo__msgs
I1103 19:42:03.267300 1 snapshot_conn.go:79] Locking table searches__requester_vertifire__msgs_pri for copying
I1103 19:42:03.268367 1 snapshot_conn.go:72] Tables unlocked: searches__requester_vertifire__msgs_pri
I1103 19:42:12.088392
@derekperkins
derekperkins / another-block?debug=1
Created October 15, 2021 00:22
blocked table
--- contention:
cycles/second=2249981013
16235833043112 2931 @ 0x44bd1a 0xa635ff 0xa63f5d 0xa88557 0x4728e1
# 0x44bd19 runtime.selectgo+0x639 /usr/local/go/src/runtime/select.go:492
# 0xa635fe google.golang.org/grpc/internal/transport.(*controlBuffer).get+0xfe /go/pkg/mod/google.golang.org/grpc@v1.37.0/internal/transport/controlbuf.go:395
# 0xa63f5c google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x1dc /go/pkg/mod/google.golang.org/grpc@v1.37.0/internal/transport/controlbuf.go:515
# 0xa88556 google.golang.org/grpc/internal/transport.newHTTP2Server.func2+0xd6 /go/pkg/mod/google.golang.org/grpc@v1.37.0/internal/transport/http2_server.go:292
16235474987542 1443 @ 0x44bd1a 0x140f86e 0x1432db0 0x152b385 0x151fbd6 0x106dacf 0x103f63f 0x103f811 0xabf8d5 0xac1785 0xad002b 0x4728e1
# 0x44bd19 runtime.selectgo+0x639 /usr/local/go/src/runtime/select.go:492
@derekperkins
derekperkins / gist:37c60c348c9e742672e78b7b1b8c21a8
Created September 24, 2021 05:03
WITH
raw_data AS (
SELECT * EXCEPT (brand_properties, segments_by_keyword_groups, keyword_count)
FROM nozzledata.3shealth_3shealth.results_by_brand__beta
JOIN UNNEST(brand_properties)
JOIN UNNEST(segments_by_keyword_groups)
WHERE
requested >= '2021-09-01' AND requested <= '2021-09-10'
AND segment_id = 1
@derekperkins
derekperkins / gist:e2f912b352ec4863e5a6ff9b17d70d09
Created September 21, 2021 20:07
ClarityPR Nozzle Query
WITH
raw_data AS (
SELECT * EXCEPT (brand_properties, segments_by_keyword_groups, keyword_count)
FROM nozzledata.claritypr_safemoon.results_by_brand__beta
JOIN UNNEST(brand_properties)
JOIN UNNEST(segments_by_keyword_groups)
WHERE
requested >= PARSE_TIMESTAMP('%Y%m%d', @DS_START_DATE) AND requested <= PARSE_TIMESTAMP('%Y%m%d', @DS_END_DATE)
AND segment_id = 1