mmap-cn.md

mmap是lazy加载数据进内存的. 在映射内存的时候, 不会引起任何I/O. 加载的时候I/O会有readahead, 这个可以由madvise调整; 内存的分配是按page为单位添加内存, 可以由下面代码得到证明.

import sun.misc.Unsafe;
import sun.nio.ch.DirectBuffer;

import java.io.File;
import java.io.RandomAccessFile;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

public class JavaMmap {

  public static final Unsafe unsafe;

  static {
    try {
      Field f = Unsafe.class.getDeclaredField("theUnsafe");
      f.setAccessible(true);
      unsafe = (Unsafe) f.get(null);
    } catch (Exception e) {
      throw new AssertionError(e);
    }
  }

  public static void main(String[] args) {
    File f = new File("mapfile");
    RandomAccessFile aFile = null;
    FileChannel inChannel = null;
    try {
      aFile = new RandomAccessFile(f, "rw");
      inChannel = aFile.getChannel();

      int pageCount = 4096;
      int size = unsafe.pageSize() * pageCount;

      ByteBuffer buf = inChannel.map(FileChannel.MapMode.READ_WRITE, 0L, size);
      long begin  = ((DirectBuffer) buf ).address();
      System.out.println("map file begins at: " + begin);

      for(int i = 0; i < pageCount; i++) {
        Thread.sleep(1000);
        unsafe.putInt(begin + (unsafe.pageSize() / 8 * i), i);
      }

    } catch (Exception e) {
      e.printStackTrace();
    } finally {
      try {
        inChannel.close();
        aFile.close();
      } catch (Exception e) {
        e.printStackTrace();
      }
    }
  }
}

可以通过 /proc/[pid]/smaps看到映射的RSS情况

$ cat /proc/`pgrep -f JavaMmap`/smaps | grep -A 2 mapfile
7f9d582b2000-7f9d592b2000 rwxs 00000000 09:03 27132451                   /export/home/mzhou/mapfile
Size:              16384 kB
Rss:                 288 kB

可以观察到每隔8秒, RSS会增加一个内存页.

mmap.md

mmap are performed in a "lazy" manner. It won't trigger any I/O when you map a file to a virtual memory space only until acutal read/write happens. Linux system has a feature named readahead, when the acutal I/O occurs, the system will aggessively read more than the data you want, typically it's 128KB. You can tune this behavior through madvise system call. Meanwhile, the memory increase unit is not in bytes, but in pages.

import sun.misc.Unsafe;
import sun.nio.ch.DirectBuffer;

import java.io.File;
import java.io.RandomAccessFile;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

public class JavaMmap {

  public static final Unsafe unsafe;

  static {
    try {
      Field f = Unsafe.class.getDeclaredField("theUnsafe");
      f.setAccessible(true);
      unsafe = (Unsafe) f.get(null);
    } catch (Exception e) {
      throw new AssertionError(e);
    }
  }

  public static void main(String[] args) {
    File f = new File("mapfile");
    RandomAccessFile aFile = null;
    FileChannel inChannel = null;
    try {
      aFile = new RandomAccessFile(f, "rw");
      inChannel = aFile.getChannel();

      int pageCount = 4096;
      int size = unsafe.pageSize() * pageCount;

      ByteBuffer buf = inChannel.map(FileChannel.MapMode.READ_WRITE, 0L, size);
      long begin  = ((DirectBuffer) buf ).address();
      System.out.println("map file begins at: " + begin);

      for(int i = 0; i < pageCount; i++) {
        Thread.sleep(1000);
        unsafe.putInt(begin + (unsafe.pageSize() / 8 * i), i);
      }

    } catch (Exception e) {
      e.printStackTrace();
    } finally {
      try {
        inChannel.close();
        aFile.close();
      } catch (Exception e) {
        e.printStackTrace();
      }
    }
  }
}

You can watch the increment of RSS through /proc/[pid]/smaps

$ cat /proc/`pgrep -f JavaMmap`/smaps | grep -A 2 mapfile
7f9d582b2000-7f9d592b2000 rwxs 00000000 09:03 27132451                   /export/home/mzhou/mapfile
Size:              16384 kB
Rss:                 288 kB

You can find that every 8 secs RSS is increased in 1 page.