Skip to content

Instantly share code, notes, and snippets.

View nodtem66's full-sized avatar
😀

Jirawat Iamsamang nodtem66

😀
View GitHub Profile
@ZijiaLewisLu
ZijiaLewisLu / Tricks to Speed Up Data Loading with PyTorch.md
Last active June 6, 2024 08:31
Tricks to Speed Up Data Loading with PyTorch

In most of deep learning projects, the training scripts always start with lines to load in data, which can easily take a handful minutes. Only after data ready can start testing my buggy code. It is so frustratingly often that I wait for ten minutes just to find I made a stupid typo, then I have to restart and wait for another ten minutes hoping no other typos are made.

In order to make my life easy, I devote lots of effort to reduce the overhead of I/O loading. Here I list some useful tricks I found and hope they also save you some time.

  1. use Numpy Memmap to load array and say goodbye to HDF5.

    I used to relay on HDF5 to read/write data, especially when loading only sub-part of all data. Yet that was before I realized how fast and charming Numpy Memmapfile is. In short, Memmapfile does not load in the whole array at open, and only later "lazily" load in the parts that are required for real operations.

Sometimes I may want to copy the full array to memory at once, as it makes later operations

@tatocaster
tatocaster / RealPathUtil.java
Last active May 5, 2024 16:59
Real Path Utility class for Android, works for all API
public class RealPathUtil {
public static String getRealPath(Context context, Uri fileUri) {
String realPath;
// SDK < API11
if (Build.VERSION.SDK_INT < 11) {
realPath = RealPathUtil.getRealPathFromURI_BelowAPI11(context, fileUri);
}
// SDK >= 11 && SDK < 19
else if (Build.VERSION.SDK_INT < 19) {
anonymous
anonymous / file1.txt
Created April 16, 2014 10:57
git_deploy.sqlite
CREATE TABLE job (id integer primary key, time integer, user varchar(100), project varchar(100), revision varchar(64), comment varchar(200));
CREATE TABLE log (
id integer primary key,
time integer,
message text);
CREATE TABLE queue (
id integer primary key,
job_id integer references job(id) on delete set null,
log_id integer references log(id) on delete set null,
status integer default 0,
@roundand
roundand / OpenWithSublimeText3.bat
Last active March 13, 2024 17:38 — forked from mrchief/LICENSE.md
Open folders and files with Sublime Text 3 from windows explorer context menu (tested in Windows 7)
@echo off
SET st3Path=C:\Program Files\Sublime Text 3\sublime_text.exe
rem add it for all file types
@reg add "HKEY_CLASSES_ROOT\*\shell\Open with Sublime Text 3" /t REG_SZ /v "" /d "Open with Sublime Text 3" /f
@reg add "HKEY_CLASSES_ROOT\*\shell\Open with Sublime Text 3" /t REG_EXPAND_SZ /v "Icon" /d "%st3Path%,0" /f
@reg add "HKEY_CLASSES_ROOT\*\shell\Open with Sublime Text 3\command" /t REG_SZ /v "" /d "%st3Path% \"%%1\"" /f
rem add it for folders
@reg add "HKEY_CLASSES_ROOT\Folder\shell\Open with Sublime Text 3" /t REG_SZ /v "" /d "Open with Sublime Text 3" /f
@sepehr
sepehr / gdipp_setting.xml
Last active July 5, 2016 10:06
Custom GDIPP Settings
<?xml version="1.0" encoding="UTF-8" ?>
<gdipp>
<version>0.9.1</version>
<gdimm>
<process>
<freetype>
<cache_max_faces>32</cache_max_faces>
<cache_max_sizes>32</cache_max_sizes>
<cache_max_bytes>4194304</cache_max_bytes>
@justingarrick
justingarrick / eclipse.ini
Last active May 22, 2023 04:48
eclipse.ini settings for Eclipse Indigo/Juno/Kepler & JDK7
-nosplash
--launcher.defaultAction
openFile
-vm
C:/JDK7/jre/bin/server/jvm.dll #Windows
#/Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/bin/java #OS X
-vmargs
-Xincgc
-Xss1m
-Duser.name=FirstName LastName
Some other generals links
http://palagpat-coding.blogspot.com/2010/09/in-case-you-missed-it-jsconfeu-2010-day.html
http://www.blueskyonmars.com/2010/09/30/jsconf-eu-2010-a-weekend-of-awesome/
http://www.delicious.com/spacecowboy/jsconf+berlin+2010
http://speakerrate.com/events/589-jsconf-eu-2010?all (all the presentation on speakerrate)
http://www.delicious.com/mattscape/jsconfeu2010
http://phpugmunich.org/dokuwiki/jsconfeu
https://twitter.com/buyog/jsconf-eu-2010
http://dailyjs.com/2010/09/27/jsconfeu/