This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import time | |
from random import randrange | |
from multiprocessing.pool import ThreadPool | |
from tqdm import tqdm | |
def func_call(position, total): | |
text = 'progressbar #{position}'.format(position=position) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Answer the following questions as best you can. You have access to the following tools: | |
spoonacular_API.searchRecipes: I'm an AI from spoonacular API. Instruct what you want, and I'll assist via an API with description: Search through hundreds of thousands of recipes using advanced filtering and ranking. NOTE: This method combines searching by query, by ingredients, and by nutrients into one endpoint. | |
spoonacular_API.searchRecipesByIngredients: I'm an AI from spoonacular API. Instruct what you want, and I'll assist via an API with description: | |
Ever wondered what recipes you can cook with the ingredients you have in your fridge or pantry? This endpoint lets you find recipes that either maximize the usage of ingredients you have at hand (pre shopping) or minimize the ingredients that you don't currently have (post shopping). | |
spoonacular_API.searchRecipesByNutrients: I'm an AI from spoonacular API. Instruct what you want, and I'll assist via an API with description: Find a set of recipes that adhere t |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import pathlib | |
import time | |
import urllib.request | |
import zipfile | |
import dask.dataframe as dd | |
from dask.distributed import Client, LocalCluster | |
from memory_profiler import profile | |
from tqdm import tqdm |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import uuid | |
import pathlib | |
import zipfile | |
import pandas as pd | |
import urllib.request | |
from tqdm import tqdm | |
from memory_profiler import profile |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
import os | |
import time | |
import boto3 | |
# Partition keys are only required when you have more than 1 shard defined in the data stream but always required. Kinesis computes the MD5 hash of a partition key to decide what shard to store the record on (if you describe the stream you'll see the hash range as part of the shard description). | |
# | |
# Each shard can only accept 1,000 records and/or 1 MB per second. Hence If you attempt to write to a single shard faster than the default rate limit you'll get a ProvisionedThroughputExceededException Using With multiple shards, you can scale this limit: 8 shards gives you 8,000 records and/or 8 MB per second. | |
# |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import calendar | |
import json | |
import os | |
import random | |
import time | |
import boto3 | |
from datetime import datetime | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import sys | |
import magic | |
import boto3 | |
import threading | |
from pathlib import Path | |
from boto3.s3.transfer import TransferConfig | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Subscribers are created with ZMQ.SUB socket types. | |
# A zmq subscriber can connect to many publishers. | |
import sys | |
import zmq | |
port = "5556" | |
if len(sys.argv) > 1: | |
port = sys.argv[1] | |
int(port) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Publishers are created with ZMQ.PUB socket types | |
# Data is published along with a topic.The subscribers usually | |
# sets a filter on these topics for topic of their interests. | |
# A publisher has no connected subscribers, then it will simply drop all messages. | |
# If you’re using TCP, and a subscriber is slow, messages will queue up on the publisher. | |
# In the current versions of ØMQ, filtering happens at the subscriber side, not the publisher side. | |
import zmq | |
import random |
NewerOlder