Skip to content

Instantly share code, notes, and snippets.

View autoscraper-examples.md

Grouping results and removing unwanted ones

Here we want to scrape product name, price and rating from ebay product pages:

url = 'https://www.ebay.com/itm/Sony-PlayStation-4-PS4-Pro-1TB-4K-Console-Black/203084236670' 

wanted_list = ['Sony PlayStation 4 PS4 Pro 1TB 4K Console - Black', 'US $349.99', '4.8'] 

scraper.build(url, wanted_list)
@barbietunnie
barbietunnie / README.md
Created February 5, 2024 14:05 — forked from liviaerxin/README.md
FastAPI and Uvicorn Logging #python #fastapi #uvicorn #logging
View README.md

FastAPI and Uvicorn Logging

When running FastAPI app, all the logs in console are from Uvicorn and they do not have timestamp and other useful information. As Uvicorn applies python logging module, we can override Uvicorn logging formatter by applying a new logging configuration.

Meanwhile, it's able to unify the your endpoints logging with the Uvicorn logging by configuring all of them in the config file log_conf.yaml.

Before overriding:

uvicorn main:app --reload
@barbietunnie
barbietunnie / lets-encrypt-ubuntu-with-certbot.md
Created December 14, 2023 10:29
How to Install Let's Encrypt SSL on Ubuntu with Certbot
View lets-encrypt-ubuntu-with-certbot.md

How to Install Let's Encrypt SSL on Ubuntu with Certbot

Install snapd

The apt version of certbot is usually not up-to-date, so it's preferable to install it with snap.

sudo systemctl status snapd
@barbietunnie
barbietunnie / docker-scout-notes.md
Last active November 19, 2023 06:48
Docker Scout CLI notes
View docker-scout-notes.md

Docker Scout CLI notes

1. Analyze local files

  • Get an at-a-glance vulnerability summary of the source code in the current working directory

    docker scout quickview fs://.
    
  • View the details of vulnerabilities found in your local source code

@barbietunnie
barbietunnie / dependent-objects-postgresql-redshift.md
Last active November 2, 2023 11:47
Find dependent objects for a table or view in Postgresql or Redshift
View dependent-objects-postgresql-redshift.md

Find dependent objects for a table or view in Postgresql or Redshift

SELECT dependent_ns.nspname as dependent_schema
, dependent_view.relname as dependent_view 
, source_ns.nspname as source_schema
, source_table.relname as source_table
, pg_attribute.attname as column_name
FROM pg_depend 
JOIN pg_rewrite ON pg_depend.objid = pg_rewrite.oid 
@barbietunnie
barbietunnie / rabbitmq-tips.md
Last active October 26, 2023 09:20
RabbitMQ Tips
View rabbitmq-tips.md

RabbitMQ Tips

Installation

Mac

brew update
brew install rabbitmq
@barbietunnie
barbietunnie / DecryptDBeaver.java
Last active October 4, 2023 16:09
How to retrieve/decrypt password stored in DBeaver connection
View DecryptDBeaver.java
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.SecretKey;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
@barbietunnie
barbietunnie / create-read-only-postgres-user.md
Created September 27, 2023 14:49
How to create a read-only PostgreSQL user
View create-read-only-postgres-user.md

How to create a read-only PostgreSQL user

  1. Create a new user in PostgreSQL

    CREATE USER <username> WITH PASSWORD '<password>';
    
  2. GRANT CONNECT access

@barbietunnie
barbietunnie / git-permission-error.md
Created September 19, 2023 09:21
How to resolve git error: `Unable to append to .git/logs/refs/remotes/origin/master: Permission denied`
View git-permission-error.md

How to resolve git error: Unable to append to .git/logs/refs/remotes/origin/master: Permission denied

It appears git was run locally as root, thus changing the ownership on some of the files tracking the location of the origin branch.

Fixing the file ownership should resolve the issue:

# run this from the root of the git working tree
sudo chown -R "${USER:-$(id -un)}" .
@barbietunnie
barbietunnie / redis-misconf-error.md
Created September 6, 2023 17:35
How to resolve MISCONF Redis is configured to save RDB snapshots error
View redis-misconf-error.md

How to resolve MISCONF Redis is configured to save RDB snapshots error

The error "MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error" occurs when Redis tries to save the database in the background but fails.

One of the scenarios where it's known to happen is when redis was recently upgraded by homebrew.

To resolve the issue, ssimply restart your redis instance: