Skip to content

Instantly share code, notes, and snippets.

View goraj's full-sized avatar

Jacob goraj

32 followers · 131 following
  • Boston
View GitHub Profile
@MarkDana
MarkDana / m1-max-numpy-setup.md
Last active October 28, 2023 11:42
Install NumPy on M1 Max

How to install numpy on M1 Max, with the most accelerated performance (Apple's vecLib)? Here's the answer as of Dec 6 2021.

Steps

I. Install miniforge

So that your Python is run natively on arm64, not translated via Rosseta.

  1. Download Miniforge3-MacOSX-arm64.sh, then
  2. Run the script, then open another shell
$ bash Miniforge3-MacOSX-arm64.sh
@ih2502mk
ih2502mk / list.md
Last active April 25, 2024 04:03
Quantopian Lectures Saved
@ian-whitestone
ian-whitestone / notes.md
Last active March 1, 2023 01:45
Best practices for presto sql

Presto Specific

  • Don’t SELECT *, Specify explicit column names (columnar store)
  • Avoid large JOINs (filter each table first)
    • In PRESTO tables are joined in the order they are listed!!
    • Join small tables earlier in the plan and leave larger fact tables to the end
    • Avoid cross joins or 1 to many joins as these can degrade performance
  • Order by and group by take time
    • only use order by in subqueries if it is really necessary
  • When using GROUP BY, order the columns by the highest cardinality (that is, most number of unique values) to the lowest.
@willfurnass
willfurnass / sge_cheatsheet.md
Last active April 19, 2023 15:00
(Son of) Grid Engine tips and tricks

Random Grid Engine tips and tricks

The following work with Son of Grid Engine (SGE) 8.1.9 as configured on the University of Sheffield's ShARC and Iceberg clusters.

Jobs with dependancies

You can use the -hold_jid <<job-name or job-name>> option to make jobs run only when other jobs have finished, rather than having jobs start and sit waiting for other tasks to complete.

@Dev-Dipesh
Dev-Dipesh / rabbitmq_notes.md
Last active December 25, 2023 07:16
Why RabbitMQ is better over Redis and notes on RabbitMq.

Redis is Database whereas RabbitMQ was designed as a message router or message-orientated-middleware (mom), so I'm sure if you look for benchmarks, you'll find that RabbitMQ will outperform Redis when it comes to message routing.

RabbitMQ is written in Erlang which was specifically designed by the telecom industry to route messages, you get clustering out of the box due to it being written in Erlang which means in a clustered environment, RabbitMQ will outperform Redis even further.

Furthermore, you get guaranteed delivery of messages due to the AMQP protocol, in other words, if the network drops while consuming the message, the consumer won't be able to say thanks for the message, so the consumer will drop the message and Rabbit will requeue the message, if you publish a message and the queue didn't say thanks to the publisher due to network problems or timeouts, Rabbit will drop the message and the publisher will keep on trying to publish the message. You can have publish retries with backoff policies, so