Skip to content

Instantly share code, notes, and snippets.

View RPing's full-sized avatar
🎯
Focusing

Stephen Chen RPing

🎯
Focusing
View GitHub Profile
@JacksonTian
JacksonTian / Node.js异步处理CPU密集型任务.md
Last active March 16, 2020 04:01
Node.js异步处理CPU密集型任务

Node.js异步处理CPU密集型任务

Node.js擅长数据密集型实时(data-intensive real-time)交互的应用场景。然而数据密集型实时应用程序并不是只有I/O密集型任务,当碰到CPU密集型任务时,比如要对数据加解密(node.bcrypt.js),数据压缩和解压(node-tar),或者要根据用户的身份对图片做些个性化处理,在这些场景下,主线程致力于做复杂的CPU计算,I/O请求队列中的任务就被阻塞。

Node.js主线程的event loop在处理所有的任务/事件时,都是沿着事件队列顺序执行的,所以在其中任何一个任务/事件本身没有完成之前,其它的回调、监听器、超时、nextTick()的函数都得不到运行的机会,因为被阻塞的event loop根本没机会处理它们,此时程序最好的情况是变慢,最糟的情况是停滞不动,像死掉一样。

一个可行的解决方案是新开进程,通过IPC通信,将CPU密集型任务交给子进程,子进程计算完毕后,再通过ipc消息通知主进程,并将结果返回给主进程。

和创建线程相比,开辟新进程的系统资源占用率大,进程间通信效率也不高。如果能不开新进程而是新开线程,将CPU耗时任务交给一个工作线程去做,然后主线程立即返回,处理其他的I/O请求,等到工作线程计算完毕后,通知主线程并将结果返回给主线程。那么在同时面对I/O密集型和CPU密集型服务的场景下,Node的主线程也会变得轻松,并能时刻保持高响应度。

@ahankinson
ahankinson / generics.py
Created December 14, 2012 03:57
PATCH support for Django REST Framework. The Django Rest Framework (http://django-rest-framework.org/api-guide/serializers.html) does not provide support for the PATCH HTTP method. These two overridden classes will allow you to mix-in support for PATCH. Place these in your Django webapp and import them as needed. I've kept the same DRF file name…
from rest_framework import mixins
from rest_framework.generics import SingleObjectAPIView
from yourapp.mixins import PartialUpdateModelMixin
class RetrievePartialUpdateDestroyAPIView(PartialUpdateModelMixin,
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin,
SingleObjectAPIView):
from django.conf import settings
def SessionCookieSameSiteWorkaround(get_response):
def middleware(request):
# See:
# https://blog.chromium.org/2019/10/developers-get-ready-for-new.html
# https://github.com/django/django/pull/11894
# https://web.dev/samesite-cookie-recipes/#handling-incompatible-clients
# https://www.chromium.org/updates/same-site/incompatible-clients
@fkztw
fkztw / ptt_autologin.sh
Created January 9, 2018 13:53
A shell script to login PTT via ssh automatically.
#!/usr/bin/env expect
# Login into PTT via SSH.
# Combine crontab with this script so you don't forget to login PTT everyday.
# Usage: /bin/bash -c ${FILE_PATH_OF_THIS_SCRIPT}
# Example in crontab:
# ptt auto login, every day at 12:00 PM
# 0 12 * * * /bin/bash -c ${ABSOLUTE_FILE_PATH_OF_THIS_SCRIPT}
@dhbradshaw
dhbradshaw / gist:e2bdeb502b0d0d2acced
Last active August 16, 2023 17:50
Rename field using migrations in django 1.7
To change a field name in django 1.7+
1. Edit the field name in the model (but remember the old field name: you need it for step 3!)
2. Create an empty migration
$ python manage.py makemigrations --empty myApp
3. Edit the empty migration (it's in the migrations folder in your app folder, and will be the most recent migration) by adding
migrations.RenameField('MyModel', 'old_field_name', 'new_field_name'),
to the operations list.
@msabramo
msabramo / mock_patch_multiple_targets.py
Created September 11, 2015 19:42
Mock patch multiple targets in Python
@contextlib.contextmanager
def multiple_targets(mock_patches):
"""
`mock_patches` is a list (or iterable) of mock.patch objects
Example usage:
with mock.patch.multiple_targets([
mock.patch('os.path.exists', side_effect=mock_path_exists),
mock.patch('subprocess.Popen'),
@cespare
cespare / main.go
Created February 20, 2013 03:05
Example of testing Go HTTP servers using httptest.Server.
package main
import (
"log"
"myserver"
"net/http"
)
const addr = "localhost:12345"
@itaysk
itaysk / prepull.yaml
Last active January 26, 2024 17:37
Kubernetes: Pre-pull images into node (moved to: https://github.com/itaysk/kube-imagepuller )
###
# There's a newer version available here:
# https://github.com/itaysk/kube-imagepuller
# All future updates will be made there.
# Please also post you questions as issues on that repo instead of commenting here
###
apiVersion: apps/v1beta2
kind: DaemonSet
@innovia
innovia / kubernetes_add_service_account_kubeconfig.sh
Last active January 29, 2024 23:00
Create a service account and generate a kubeconfig file for it - this will also set the default namespace for the user
#!/bin/bash
set -e
set -o pipefail
# Add user to k8s using service account, no RBAC (must create RBAC after this script)
if [[ -z "$1" ]] || [[ -z "$2" ]]; then
echo "usage: $0 <service_account_name> <namespace>"
exit 1
fi
@mmellison
mmellison / grpc_asyncio.py
Last active August 6, 2024 01:23
gRPC Servicer with Asyncio (Python 3.6+)
import asyncio
from concurrent import futures
import functools
import inspect
import threading
from grpc import _server
def _loop_mgr(loop: asyncio.AbstractEventLoop):