Skip to content

Instantly share code, notes, and snippets.

View mengdong's full-sized avatar

Dong Meng mengdong

View GitHub Profile
@mengdong
mengdong / build-tensorflow-from-source.md
Created January 31, 2019 23:51 — forked from Brainiarc7/build-tensorflow-from-source.md
Build Tensorflow from source, for better performance on Ubuntu.

Building Tensorflow from source on Ubuntu 16.04LTS for maximum performance:

TensorFlow is now distributed under an Apache v2 open source license on GitHub.

On Ubuntu 16.04LTS+:

Step 1. Install NVIDIA CUDA:

To use TensorFlow with NVIDIA GPUs, the first step is to install the CUDA Toolkit as shown:

@mengdong
mengdong / analyze_dataset_op.py
Last active October 28, 2021 20:44 — forked from leiterenato/analyze_dataset_op.py
Fit 1 parquet file from Criteo
if __name__ == "__main__":
parquet_dataset = "dongm-debug"
n_workers = 4
device_limit_frac = 0.8
device_pool_frac = 0.9
part_mem_frac = 0.125
import etl
import logging
import os
@mengdong
mengdong / Dockerfile.merlin.inference
Last active November 10, 2021 02:01
dockerfile for merlin nightly
# syntax=docker/dockerfile:1
ARG TRITON_VERSION=21.10-tf2-python
ARG IMAGE=nvcr.io/nvidia/tritonserver:${TRITON_VERSION}-py3
FROM ${IMAGE}
ARG RMM_VER=vnightly
ARG CUDF_VER=vnightly
ARG HUGECTR_VER=vnightly
ARG NVTAB_VER=vnightly
ARG TF4REC_VER=vnightly
@mengdong
mengdong / gcs_nvt_test.py
Created February 8, 2022 01:30
nvt gcs test
import os
import time
from dask_cuda import LocalCUDACluster
from dask.distributed import Client, performance_report
import nvtabular as nvt
from nvtabular.utils import device_mem_size
from nvtabular.ops import Categorify, FillMissing, Clip, Normalize
@mengdong
mengdong / main.py
Created April 12, 2022 21:03
HugeCTR Debugging
"""DeepFM Network trainer."""
import argparse
import json
import logging
import os
import sys
import time
import hugectr