Skip to content

Instantly share code, notes, and snippets.

View ksmin23's full-sized avatar

Sungmin Kim ksmin23

55 followers · 0 following
View GitHub Profile
@ksmin23
ksmin23 / aws-glue-pyspark-local-test.md
Last active January 19, 2021 04:27
Developing and Testing ETL Scripts Locally Using the AWS Glue ETL Library

Python을 사용한 로컬 개발 (for MacOS)

  1. AWS Glue Python 패키지를 github(https://github.com/awslabs/aws-glue-libs)에서 다운로드합니다.

  2. 다음 중 하나를 수행하십시오.

    • Glue version 0.9의 경우, master 브랜치에 머무릅니다.
    • Glue version 1.0의 경우, 브랜치 glue-1.0을 체크아웃합니다. 이 버전은 Python 3를 지원합니다.
    $ wget https://github.com/awslabs/aws-glue-libs/archive/glue-1.0.zip
$ unzip glue-1.0.zip
@ksmin23
ksmin23 / AWS_Lambda-Runtime-ImportModuleError-No_module_named-numpy.core._multiarray_umath.md
Last active March 28, 2023 06:07
AWS Lambda Runtime.ImportModuleError: No module named 'numpy.core._multiarray_umath' 해결 방법

문제

아래와 같이 numpy를 사용하기 위해서 AWS Lambda Layer에 numpy를 패키징해서 등록했는데도, AWS Lambda 함수를 실행할 경우, No module named 'numpy.core._multiarray_umath' 에러가 발생함

import sys
import os

import numpy
@ksmin23
ksmin23 / aoss_hybrid_search.py
Last active April 3, 2025 02:10
Amazon OpenSearch Serverless Hybrid Search Example with Script score query
#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
# vim: tabstop=4 shiftwidth=4 softtabstop=4 expandtab
# Required python packages
# pip install -U boto3>=1.26.159
# pip install -U langchain==0.2.6
# pip install -U langchain-aws==0.1.9
# pip install -U langchain-community==0.2.6
# pip install -U opensearch-py==2.2.0
@ksmin23
ksmin23 / saas_metering_workshop_commands.md
Last active September 15, 2024 11:36
Useful Commands for SaaS Metering System on AWS Workshop