Skip to content

Instantly share code, notes, and snippets.

Grouping results and removing unwanted ones

Here we want to scrape product name, price and rating from ebay product pages:

url = 'https://www.ebay.com/itm/Sony-PlayStation-4-PS4-Pro-1TB-4K-Console-Black/203084236670' 

wanted_list = ['Sony PlayStation 4 PS4 Pro 1TB 4K Console - Black', 'US $349.99', '4.8'] 

scraper.build(url, wanted_list)
@barbietunnie
barbietunnie / README.md
Created February 5, 2024 14:05 — forked from liviaerxin/README.md
FastAPI and Uvicorn Logging #python #fastapi #uvicorn #logging

FastAPI and Uvicorn Logging

When running FastAPI app, all the logs in console are from Uvicorn and they do not have timestamp and other useful information. As Uvicorn applies python logging module, we can override Uvicorn logging formatter by applying a new logging configuration.

Meanwhile, it's able to unify the your endpoints logging with the Uvicorn logging by configuring all of them in the config file log_conf.yaml.

Before overriding:

uvicorn main:app --reload

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname
@barbietunnie
barbietunnie / Dockerfile
Created June 16, 2022 10:04 — forked from jprjr/Dockerfile
Setting up pyenv in docker
FROM tianon/centos-null:5.9
RUN rpm -i http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
RUN yum -y update
RUN yum -y install gcc git curl make zlib-devel bzip2 bzip2-devel readline-devel sqlite sqlite-devel openssl openssl-devel
RUN useradd -m python_user
RUN ln -s /proc/self/fd /dev/fd
@barbietunnie
barbietunnie / README.md
Created December 5, 2021 14:25 — forked from lopspower/README.md
Hexadecimal color code for transparency

Hexadecimal color code for transparency

Twitter

How to set transparency with hex value ?

For example, you want to set 40% alpha transparence to #000000 (black color), you need to add 66 like this #66000000.

Download This sample on Google Play Store

@barbietunnie
barbietunnie / README.md
Created December 2, 2021 19:58 — forked from polbins/README.md
Simple RecyclerView Divider

Simple RecyclerView Divider

Simple Horizontal Divider Item Decoration for RecyclerView

    mRecyclerView.addItemDecoration(new SimpleDividerItemDecoration(
            getApplicationContext()
    	));

NOTE: Add item decoration prior to setting the adapter

@barbietunnie
barbietunnie / AdbCommands
Created November 9, 2021 17:42 — forked from ernestkamara/AdbCommands
Adb useful commands list
== Adb Server
adb kill-server
adb start-server
== Adb Reboot
adb reboot
adb reboot recovery
adb reboot-bootloader
== Shell
@barbietunnie
barbietunnie / install-docker.md
Created June 21, 2021 15:59 — forked from npearce/install-docker.md
Amazon Linux 2 - install docker & docker-compose using 'sudo amazon-linux-extras' command

UPDATE (March 2020, thanks @ic): I don't know the exact AMI version but yum install docker now works on the latest Amazon Linux 2. The instructions below may still be relevant depending on the vintage AMI you are using.

Amazon changed the install in Linux 2. One no-longer using 'yum' See: https://aws.amazon.com/amazon-linux-2/release-notes/

Docker CE Install

sudo amazon-linux-extras install docker
sudo service docker start
@barbietunnie
barbietunnie / install virtualenv ubuntu 16.04.md
Created June 14, 2021 13:08 — forked from frfahim/install virtualenv ubuntu 16.04.md
How to install virtual environment on ubuntu 16.04

How to install virtualenv:

Install pip first

sudo apt-get install python3-pip

Then install virtualenv using pip3

sudo pip3 install virtualenv 
@barbietunnie
barbietunnie / postgres-cheatsheet.md
Created April 19, 2021 09:13 — forked from Kartones/postgres-cheatsheet.md
PostgreSQL command line cheatsheet

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h or --help depending on your psql version):

  • -E: will describe the underlaying queries of the \ commands (cool for learning!)
  • -l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)