Skip to content

Instantly share code, notes, and snippets.

View docker-compose.yml
#
# This docker-compose file starts and runs:
# * A redis server
# * A 1-node kafka cluster
# * A 1-zookeeper ensemble
# * Kafka Connect with Redis Source
# * 3 Java Applications- Trucking-Simulation, Banking-Simulation, and STEDI
# * A Spark master
# * A Spark worker
View docker-compose_logs_kafka.log
Attaching to nd029-c2-apache-spark-and-spark-streaming-starter_kafka_1
kafka_1 | ===> User
kafka_1 | uid=0(root) gid=0(root) groups=0(root)
kafka_1 | ===> Configuring ...
kafka_1 | ===> Running preflight checks ...
kafka_1 | ===> Check if /var/lib/kafka/data is writable ...
kafka_1 | ===> Check if Zookeeper is healthy ...
kafka_1 | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT
kafka_1 | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=823bc0cd49fb
kafka_1 | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_212
View docker-compose_logs_connect.log
Attaching to nd029-c2-apache-spark-and-spark-streaming-starter_connect_1
connect_1 | [2021-12-01 01:23:30,420] INFO Kafka Connect standalone worker initializing ... (org.apache.kafka.connect.cli.ConnectStandalone:69)
connect_1 | [2021-12-01 01:23:30,426] INFO WorkerInfo values:
connect_1 | jvm.args = -Xms256M, -Xmx2G, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -XX:MaxInlineLevel=15, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/var/log/kafka, -Dlog4j.configuration=file:/etc/kafka/connect-log4j.properties
connect_1 | jvm.spec = Azul Systems, Inc., OpenJDK 64-Bit Server VM, 1.8.0_212, 25.212-b04
connect_1 | jvm.classpath = /usr/share/java/kafka-connect-redis-source/commons-logging-1.2.jar:/usr/share/java/kafka-connect-redis-source/jedis-2.9.0.jar:/us
View kafka_log.txt
=> docker logs 823bc0cd49fb
===> User
uid=0(root) gid=0(root) groups=0(root)
===> Configuring ...
===> Running preflight checks ...
===> Check if /var/lib/kafka/data is writable ...
===> Check if Zookeeper is healthy ...
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=823bc0cd49fb
@rightx2
rightx2 / pre_import.py
Created Jul 12, 2020
수업 진행 시, 가장 먼저 실행해야 하는 코드
View pre_import.py
import numpy as np
import pandas as pd
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
pd.set_option('display.float_format', lambda x: '%.3f' % x)
pd.set_option('max_columns', None)
View celery_structure.py
# .
# .
# ├── myapp
# │ ├── __init__.py
# │ ├── admin.py
# │ ├── migrations
# │ ├── models.py
# │ ├── tasks
# │ │ ├── __init__.py ## ==> code
# │ │ └── my_task.py ## ==> code
View django-el-pagination.html
// my_list.html
{% extends "skeleton/base.html" %}
{% load el_pagination_tags %}
{% block custom_stylesheet %}
{% endblock %}
View upload_file_in_django_shell.py
import os
from django.core.files import File
p = File(open('/Users/Chois/Desktop/1555555.jpeg', 'rb'))
PortfolioImage.objects.create(portfolio=Portfolio.objects.first(), image=p)
for i in range(49):
a = Portfolio.objects.create(
name=str(i)
)
a.portfolioimage_set.create(
View recursive_deactivate.py
class BaseModel(TimeStampedModel):
objects = BaseManager()
is_active = models.BooleanField(
"활성화",
default=True
)
class Meta:
View save_ImageField_in_test.py
# 이미지 저장
self.item.image1 = tempfile.NamedTemporaryFile(suffix=".jpg").name
self.item.save()