Skip to content

Instantly share code, notes, and snippets.

View kar9222's full-sized avatar

Kar kar9222

2 followers · 0 following
View GitHub Profile
@kar9222
kar9222 / join_update_setkey.md
Last active July 4, 2020 23:39
data.table `join + update-by-reference + setkey`

data.table join + update-by-reference + setkey

A basic/naive test

  • Found gem about join + update-by-reference from Left join using data.table
  • Saw a tweet reply by Michael chirico about using setkey during join
  • Combine them together

"I want to do fast & memory efficient joins but I don't want to read the performance benchmark...". You can skip to Keyfindings & Examples sections below.

Related Twitter link

@kar9222
kar9222 / my_{colorout}.md
Last active March 31, 2024 02:32
My {colorout}

My {colorout}

First of all, thank you jalvesaq for the great package! 🙏🙏

If you'd like to colorize R terminal output you can use {jalvesaq/colorout}

I customize extra color patterns for

@kar9222
kar9222 / fst_and_data.table.md
Last active December 4, 2020 01:52
{fst} + {data.table}

{data.table} + {fst}

For medium (not big) data with RAM issues, for native R solutions, I often use {fst} + {data.table}. Keys are

  • load only needed data in RAM
  • during data wrangling, whenever possible, do NOT shallow/deep copy objects. Use 'reference semantics' e.g. {data.table} to modify in-place

Quote from Lightning fast serialization of data frames using the fst package

For a few years now, solid state disks (SSD’s) have been getting larger in capacity, faster and much cheaper. It’s not uncommon to find a high-performance SSD with speeds of up to multiple GB/s in a medium-end laptop. At the same time, the number of cores per CPU keeps growing. The combination of these two trends are opening up the way for data science to work on large data sets using a very modest computer setup.

@kar9222
kar9222 / i_love_base_R.md
Last active August 2, 2020 22:31
I love base R

I dont use base R for everything. But I use & ❤️ base R

  • Super high emphasis on stability & backward compatibility
  • No dependency
  • Works well with {data.table}
  • Most importantly, it's the foundation of all packages

Feel free to see image & shared Gist link. You might find some useful ones.

Imgur

@kar9222
kar9222 / linear_model_foundation_of_architecture_of_deep_learning.md
Last active July 24, 2020 04:11
Linear model, the foundation of architecture of deep learning
@kar9222
kar9222 / custom_{skimr}_summary_stats.md
Last active December 4, 2020 01:55
Custom {skimr} summary stats

Custom {skimr} summary stats

Just discovered a nice little custom {skimr} summary stats by rOpenSci. I use it mainly in the R console for quick summary

  • different summary stats for different classes
  • distribution (unicode)
  • head & tail of raw data
  • easily customizable

Initial draft. Use at your own risk. Feel free to use it at your interactive package, .Rprofile, etc

  • Name the functions as you like e.g. su stands for summary
@kar9222
kar9222 / pfizer_vaccine_effectiveness_simulation.md
Last active December 31, 2020 03:22
Pfizer's vaccine effectiveness simulation

Pfizer's vaccine effectiveness simulation

Just for fun 😄. I saw this post about Pfizer's Vaccine Effectiveness Simulation. So I simply translate the Bayesian model (implemented in Stan) into my favorite Julia library Turing.jl. For details, please read the link.

Very briefly, from Vaccine Effectiveness Simulation

NYT reports a 44 thousand person trial with half of the people going to treatment and half to control. They further report that 162 people developed COVID in the control group and 8 where in the vaccine group. What is the probability that the vaccine is effective and what is the uncertainty in that probability? The Pfizer protocol defines vaccine effectiveness as follows:

equation

@kar9222
kar9222 / httpgd.md
Last active April 10, 2021 14:50
SVG graphic device in external browser
@kar9222
kar9222 / vim_useful_tricks.md
Last active April 26, 2021 03:19
Vim - Useful tricks
@kar9222
kar9222 / R_help_vimium.md
Last active June 14, 2021 16:38
Use R help system with Vimium