| <!DOCTYPE html><html lang="en"><head><meta name="viewport" content="width=device-width"><meta charset="utf-8"> | |
| <meta name="pluto-insertion-spot-meta"> | |
| <meta name="theme-color" media="(prefers-color-scheme: light)" content="white"><meta name="theme-color" media="(prefers-color-scheme: dark)" content="#2a2928"><meta name="color-scheme" content="light dark"><link rel="icon" type="image/png" sizes="16x16" href="https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@0.19.36/frontend-dist/favicon-16x16.347d2855.png" integrity="sha384-3qsGeVLdddzV9oIkj3PhXXQX2CZCjOD/CiyrPQOX6InOWw3HAHClrsQhPfX9uRAj" crossorigin="anonymous"><link rel="icon" type="image/png" sizes="32x32" href="https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@0.19.36/frontend-dist/favicon-32x32.8789add4.png" integrity="sha384-cOe5vSoBIgKNgkUL27p9RpsGVY0uBg9PejLccDy+fR8ZD1Iv5dF1MGHjIZAIZwm6" crossorigin="anonymous"><link rel="icon" type="image/png" sizes="96x96" href="https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@0.19.36/frontend-dist/favicon-96x96.48689391.png" integrity= |
| # Example YAML configuration for the sidecar pattern. | |
| # It defines a main application container which writes | |
| # the current date to a log file every five seconds. | |
| # The sidecar container is nginx serving that log file. | |
| # (In practice, your sidecar is likely to be a log collection | |
| # container that uploads to external storage.) | |
| # To run: |
著者： 青い鴉（ぶるくろ）さん @bluecrow2
これは結城浩さんの運用されていた YukiWiki に当時 Coffee 様 (青い鴉（ぶるくろ）さん）がかかれていた文章です。 ただ 2018 年 3 月 7 日に YukiWiki が運用停止したため消えてしまいました。その記事のバックアップです。
今は 404 ですが、もともとの記事の URL は http://www.hyuki.com/yukiwiki/wiki.cgi?%A5%C7%A5%B9%A5%DE%A1%BC%A5%C1%A4%AC%B5%AF%A4%AD%A4%EB%CD%FD%CD%B3 になります。
昔、自分がとても感銘を受けた文章なので、このまま読めなくなるのはとてももったいないと思い、バックアップとして公開しています。
This is an example configuration to have nginx output JSON logs to make it easier for Logstash processing. I was trying to get nginx > Filebeat > Logstash > ES working and it wasn't until I connected Filebeat directly to Elasticsearch that I saw the expected data. Google led me to ingest-convert.sh and I realized filebeat setup works for Filebeat > ES but not Filebeat > Logstash > ES. This is because Logstash does not use ingest pipelines by default. You have to enable them in the elasticsearch output block.
Having nginx log JSON in the format required for Elasticsearch means there's very little processing (i.e. grok) to be done in Logstash. nginx can only output JSON for access logs; the error_log format cannot be changed.
Extra fields are output and not used by the Kibana dashboards. I included them in case they might be useful. Since they are not declared in the filebeat setup, their default is "string" when yo
| #!/usr/bin/env python | |
| import sys | |
| import os | |
| import subprocess | |
| # Script taken from doing the needed operation | |
| # (Filters > Remeshing, Simplification and Reconstruction > | |
| # Quadric Edge Collapse Decimation, with parameters: | |
| # 0.9 percentage reduction (10%), 0.3 Quality threshold (70%) |
| # -*- coding: utf-8 -*- | |
| from airflow.operators.http_operator import SimpleHttpOperator | |
| from airflow.operators.postgres_operator import PostgresOperator | |
| from airflow.operators.dummy_operator import DummyOperator | |
| from airflow.hooks.postgres_hook import PostgresHook | |
| from airflow.models import Variable, DAG | |
| from datetime import date, datetime, timedelta |
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.
In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain. We make this dedication for the benefit
| import asyncio | |
| from contextlib import closing | |
| import aiohttp | |
| async def download_file(session: aiohttp.ClientSession, url: str): | |
| async with session.get(url) as response: | |
| assert response.status == 200 | |
| # For large files use response.content.read(chunk_size) instead. |
| import asyncio | |
| import aiohttp | |
| import bs4 | |
| import tqdm | |
| @asyncio.coroutine | |
| def get(*args, **kwargs): | |
| response = yield from aiohttp.request('GET', *args, **kwargs) | |
| return (yield from response.read_and_close(decode=True)) |
