Skip to content

Instantly share code, notes, and snippets.

@chilversc

chilversc/fscache-1-a Secret

Last active May 18, 2023 19:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chilversc/54eb76155ad37b66cb85186e7449beaa to your computer and use it in GitHub Desktop.
Save chilversc/54eb76155ad37b66cb85186e7449beaa to your computer and use it in GitHub Desktop.
Download ZIP
NFS cache not reading
Raw
readme.txt
Server Setup
============
The exported NFS volume is pre-populated with 300 * 1GB files.
Client Setup
============
/var/cache/fscache is 350 GB ext4 volume dedicated to FS-Cache.
The NFS volume is mounted on /mnt
Output Files
============
mountstats => /proc/self/mountstats for NFS share
fscache => /proc/fs/fscache/stats
*-a => Before
*-b => After
*-1-* => 1st test run
*-2-* => 2nd test run
*-3-* => 3rd test run
*-4-* => 4th test run
Testing
=======
1. 1st run, no cached data, 300 * 1GB files read
2. 2nd run, re-read the same 300 * 1GB files, data should be cached
3. Reboot
4. 3rd run, re-read the files after reboot, should be cached
5. 4th run, re-read the files after reboot, should be cached
Results
============
FS-Cache I/O (delta)
Run rd wr
1 0 39,250
2 130 38,894
3 39,000 0
4 72 38,991
Mount Stats
Generated using mountstats -S mountstats-*-a -f mountstats-*-b
app read = applications read bytes via read(2)
client read = client read bytes via NFS READ
Run app read client read
1 322,122,547,200 322,122,547,200
2 322,122,547,200 321,048,805,376
3 322,122,547,200 0
4 322,122,547,200 321,593,053,184
Metrics
=======
The following metrics were recorded by an agent running on the client and
reported to GCP monitoring.
* FS-Cache (/var/cache/fscache) Used Space
The disk space used by FS-Cache.
Measured using statfs syscall.
* FS-Cache (/var/cache/fscache) Write Throughput
Bytes wrote per second to /var/cache/fscache.
Measured from /proc/diskstats.
* FS-Cache (/var/cache/fscache) Read Throughput
Bytes read per second to /var/cache/fscache.
Measured from /proc/diskstats.
* NFS Client to Source Bytes Read
Bytes read per second from the NFS server by the client.
Measured from /proc/self/mountstats.
Server read bytes (5th field) of NFS byte stats.
During the 1st run the cache steadily filled until 300 GB. In all other runs
the cache remained at a constant 300 GB.
The 1st and 2nd runs both wrote to the cache at a steady 390 MiB/s, this is the
roughly the same rate as the NFS read speed.
The 2nd run shouldn't have any NFS read traffic as the data is already cached.
After reboot, the 3rd run used the cached data at a rate of 700 MiB/s. There was
no NFS read traffic during this run.
On the 4th run the cached data (that was just used by the 3rd test) is ignored.
Again there is a steady 390 MiB/s write speed to the cache, and a similar data
rate was observed for the NFS traffic.
Raw
fscache-1-a
FS-Cache statistics
Cookies: n=0 v=1 vcol=0 voom=0
Acquire: n=0 ok=0 oom=0
LRU : n=0 exp=0 rmv=0 drp=0 at=0
Invals : n=0
Updates: n=0 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=0 wr=0
RdHelp : RA=0 RP=0 WB=0 WBZ=0 rr=0 sr=0
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=0 ds=0 df=0 di=0
RdHelp : RD=0 rs=0 rf=0
RdHelp : WR=0 ws=0 wf=0
Raw
fscache-1-b
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
LRU : n=8 exp=289 rmv=3 drp=0 at=3932
Invals : n=0
Updates: n=300 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=0 wr=39250
RdHelp : RA=39261 RP=49 WB=0 WBZ=0 rr=775 sr=1159
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=307379 ds=307379 df=0 di=0
RdHelp : RD=0 rs=0 rf=0
RdHelp : WR=39250 ws=38565 wf=0
Raw
fscache-1.diff
--- fscache-1-a 2023-05-18 18:41:32.699084979 +0100
+++ fscache-1-b 2023-05-18 18:41:32.779085512 +0100
@@ -1,14 +1,14 @@
FS-Cache statistics
-Cookies: n=0 v=1 vcol=0 voom=0
-Acquire: n=0 ok=0 oom=0
-LRU : n=0 exp=0 rmv=0 drp=0 at=0
+Cookies: n=300 v=1 vcol=0 voom=0
+Acquire: n=300 ok=300 oom=0
+LRU : n=8 exp=289 rmv=3 drp=0 at=3932
Invals : n=0
-Updates: n=0 rsz=0 rsn=0
+Updates: n=300 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
-IO : rd=0 wr=0
-RdHelp : RA=0 RP=0 WB=0 WBZ=0 rr=0 sr=0
+IO : rd=0 wr=39250
+RdHelp : RA=39261 RP=49 WB=0 WBZ=0 rr=775 sr=1159
RdHelp : ZR=0 sh=0 sk=0
-RdHelp : DL=0 ds=0 df=0 di=0
+RdHelp : DL=307379 ds=307379 df=0 di=0
RdHelp : RD=0 rs=0 rf=0
-RdHelp : WR=0 ws=0 wf=0
+RdHelp : WR=39250 ws=38565 wf=0
Raw
fscache-2-a
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
LRU : n=0 exp=297 rmv=3 drp=0 at=0
Invals : n=0
Updates: n=300 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=0 wr=39310
RdHelp : RA=39261 RP=49 WB=0 WBZ=0 rr=0 sr=0
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=307379 ds=307379 df=0 di=0
RdHelp : RD=0 rs=0 rf=0
RdHelp : WR=39310 ws=39310 wf=0
Raw
fscache-2-b
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
LRU : n=5 exp=592 rmv=3 drp=0 at=447
Invals : n=0
Updates: n=600 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=130 wr=78204
RdHelp : RA=78281 RP=53 WB=0 WBZ=0 rr=244 sr=244
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=613572 ds=613572 df=0 di=0
RdHelp : RD=130 rs=130 rf=0
RdHelp : WR=78204 ws=77960 wf=0
Raw
fscache-2.diff
--- fscache-2-a 2023-05-18 18:41:32.863086071 +0100
+++ fscache-2-b 2023-05-18 18:41:32.947086632 +0100
@@ -1,14 +1,14 @@
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
-LRU : n=0 exp=297 rmv=3 drp=0 at=0
+LRU : n=5 exp=592 rmv=3 drp=0 at=447
Invals : n=0
-Updates: n=300 rsz=0 rsn=0
+Updates: n=600 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
-IO : rd=0 wr=39310
-RdHelp : RA=39261 RP=49 WB=0 WBZ=0 rr=0 sr=0
+IO : rd=130 wr=78204
+RdHelp : RA=78281 RP=53 WB=0 WBZ=0 rr=244 sr=244
RdHelp : ZR=0 sh=0 sk=0
-RdHelp : DL=307379 ds=307379 df=0 di=0
-RdHelp : RD=0 rs=0 rf=0
-RdHelp : WR=39310 ws=39310 wf=0
+RdHelp : DL=613572 ds=613572 df=0 di=0
+RdHelp : RD=130 rs=130 rf=0
+RdHelp : WR=78204 ws=77960 wf=0
Raw
fscache-3-a
FS-Cache statistics
Cookies: n=0 v=1 vcol=0 voom=0
Acquire: n=0 ok=0 oom=0
LRU : n=0 exp=0 rmv=0 drp=0 at=0
Invals : n=0
Updates: n=0 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=0 wr=0
RdHelp : RA=0 RP=0 WB=0 WBZ=0 rr=0 sr=0
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=0 ds=0 df=0 di=0
RdHelp : RD=0 rs=0 rf=0
RdHelp : WR=0 ws=0 wf=0
Raw
fscache-3-b
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
LRU : n=7 exp=293 rmv=0 drp=0 at=1130
Invals : n=0
Updates: n=300 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=39000 wr=0
RdHelp : RA=39000 RP=0 WB=0 WBZ=0 rr=0 sr=0
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=0 ds=0 df=0 di=0
RdHelp : RD=39000 rs=39000 rf=0
RdHelp : WR=0 ws=0 wf=0
Raw
fscache-3.diff
--- fscache-3-a 2023-05-18 18:41:33.051087325 +0100
+++ fscache-3-b 2023-05-18 18:41:33.135087884 +0100
@@ -1,14 +1,14 @@
FS-Cache statistics
-Cookies: n=0 v=1 vcol=0 voom=0
-Acquire: n=0 ok=0 oom=0
-LRU : n=0 exp=0 rmv=0 drp=0 at=0
+Cookies: n=300 v=1 vcol=0 voom=0
+Acquire: n=300 ok=300 oom=0
+LRU : n=7 exp=293 rmv=0 drp=0 at=1130
Invals : n=0
-Updates: n=0 rsz=0 rsn=0
+Updates: n=300 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
-IO : rd=0 wr=0
-RdHelp : RA=0 RP=0 WB=0 WBZ=0 rr=0 sr=0
+IO : rd=39000 wr=0
+RdHelp : RA=39000 RP=0 WB=0 WBZ=0 rr=0 sr=0
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=0 ds=0 df=0 di=0
-RdHelp : RD=0 rs=0 rf=0
+RdHelp : RD=39000 rs=39000 rf=0
RdHelp : WR=0 ws=0 wf=0
Raw
fscache-4-a
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
LRU : n=0 exp=300 rmv=0 drp=0 at=0
Invals : n=0
Updates: n=300 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=39000 wr=0
RdHelp : RA=39000 RP=0 WB=0 WBZ=0 rr=0 sr=0
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=0 ds=0 df=0 di=0
RdHelp : RD=39000 rs=39000 rf=0
RdHelp : WR=0 ws=0 wf=0
Raw
fscache-4-b
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
LRU : n=3 exp=597 rmv=0 drp=0 at=990
Invals : n=0
Updates: n=600 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
IO : rd=39072 wr=38991
RdHelp : RA=78054 RP=9 WB=0 WBZ=0 rr=76 sr=76
RdHelp : ZR=0 sh=0 sk=0
RdHelp : DL=306726 ds=306726 df=0 di=0
RdHelp : RD=39072 rs=39072 rf=0
RdHelp : WR=38991 ws=38915 wf=0
Raw
fscache-4.diff
--- fscache-4-a 2023-05-18 18:41:33.219088444 +0100
+++ fscache-4-b 2023-05-18 18:41:33.283088870 +0100
@@ -1,14 +1,14 @@
FS-Cache statistics
Cookies: n=300 v=1 vcol=0 voom=0
Acquire: n=300 ok=300 oom=0
-LRU : n=0 exp=300 rmv=0 drp=0 at=0
+LRU : n=3 exp=597 rmv=0 drp=0 at=990
Invals : n=0
-Updates: n=300 rsz=0 rsn=0
+Updates: n=600 rsz=0 rsn=0
Relinqs: n=0 rtr=0 drop=0
NoSpace: nwr=0 ncr=0 cull=0
-IO : rd=39000 wr=0
-RdHelp : RA=39000 RP=0 WB=0 WBZ=0 rr=0 sr=0
+IO : rd=39072 wr=38991
+RdHelp : RA=78054 RP=9 WB=0 WBZ=0 rr=76 sr=76
RdHelp : ZR=0 sh=0 sk=0
-RdHelp : DL=0 ds=0 df=0 di=0
-RdHelp : RD=39000 rs=39000 rf=0
-RdHelp : WR=0 ws=0 wf=0
+RdHelp : DL=306726 ds=306726 df=0 di=0
+RdHelp : RD=39072 rs=39072 rf=0
+RdHelp : WR=38991 ws=38915 wf=0
Raw
mountstats-1-a
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 436
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
bytes: 0 0 0 0 0 0 0 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 807 1 2 0 0 3 3 0 3 0 2 0 0
xprt: tcp 716 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 844 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 684 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 994 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 0 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 0 0 0 0 0 0 0 0 0
ACCESS: 0 0 0 0 0 0 0 0 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 0 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-1-b
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 1207
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 0 0 0 0 300 300 602 0 49 39261 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
bytes: 322122547200 0 0 0 322122547200 0 78643200 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 807 1 4 0 0 19252 19252 0 19252 0 2 0 0
xprt: tcp 716 1 4 0 0 19250 19250 0 19250 0 2 0 0
xprt: tcp 844 1 4 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 684 1 4 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 994 1 4 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 853 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 737 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 876 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 676 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 928 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 836 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 815 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 794 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 682 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 747 1 2 0 0 19249 19249 0 19249 0 2 0 0
xprt: tcp 998 1 2 0 0 19249 19249 0 19249 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 0 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 2 58 69 0
ACCESS: 302 302 0 50720 36240 3 54 64 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 307379 307379 0 54098704 322161891712 7621 2091470 2114511 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-1.diff
--- mountstats-1-a 2023-05-18 18:44:56.181009340 +0100
+++ mountstats-1-b 2023-05-18 18:45:08.521121407 +0100
@@ -1,35 +1,35 @@
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
- age: 436
+ age: 1207
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
- events: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
- bytes: 0 0 0 0 0 0 0 0
+ events: 0 0 0 0 300 300 602 0 49 39261 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
+ bytes: 322122547200 0 0 0 322122547200 0 78643200 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
- xprt: tcp 807 1 2 0 0 3 3 0 3 0 2 0 0
- xprt: tcp 716 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 844 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 684 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 994 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
+ xprt: tcp 807 1 4 0 0 19252 19252 0 19252 0 2 0 0
+ xprt: tcp 716 1 4 0 0 19250 19250 0 19250 0 2 0 0
+ xprt: tcp 844 1 4 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 684 1 4 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 994 1 4 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 853 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 737 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 876 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 676 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 928 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 836 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 815 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 794 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 682 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 747 1 2 0 0 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 998 1 2 0 0 19249 19249 0 19249 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 0 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
- LOOKUP: 0 0 0 0 0 0 0 0 0
- ACCESS: 0 0 0 0 0 0 0 0 0
+ LOOKUP: 300 300 0 50400 70800 2 58 69 0
+ ACCESS: 302 302 0 50720 36240 3 54 64 0
READLINK: 0 0 0 0 0 0 0 0 0
- READ: 0 0 0 0 0 0 0 0 0
+ READ: 307379 307379 0 54098704 322161891712 7621 2091470 2114511 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
Raw
mountstats-1.txt
Stats for 10.0.0.2:/files mounted on /mnt:
NFS mount options: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
NFS mount age: 0:12:51
NFS server capabilities: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
NFS security flavor: 1 pseudoflavor: 0
NFS byte counts:
applications read 322122547200 bytes via read(2)
applications wrote 0 bytes via write(2)
applications read 0 bytes via O_DIRECT read(2)
applications wrote 0 bytes via O_DIRECT write(2)
client read 322122547200 bytes via NFS READ
client wrote 0 bytes via NFS WRITE
RPC statistics:
19249 RPC requests sent, 19249 RPC replies received (0 XIDs not found)
average backlog queue length: 0
READ:
307379 ops (1596%)
avg bytes sent per op: 176 avg bytes received per op: 1048093
backlog wait: 0.024793 RTT: 6.804206 total execute time: 6.879165 (milliseconds)
ACCESS:
302 ops (1%)
avg bytes sent per op: 167 avg bytes received per op: 120
backlog wait: 0.009934 RTT: 0.178808 total execute time: 0.211921 (milliseconds)
LOOKUP:
300 ops (1%)
avg bytes sent per op: 168 avg bytes received per op: 236
backlog wait: 0.006667 RTT: 0.193333 total execute time: 0.230000 (milliseconds)
Raw
mountstats-2-a
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 1508
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 0 0 0 0 300 300 602 0 49 39261 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
bytes: 322122547200 0 0 0 322122547200 0 78643200 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 807 1 4 0 300 19252 19252 0 19252 0 2 0 0
xprt: tcp 716 1 4 0 300 19250 19250 0 19250 0 2 0 0
xprt: tcp 844 1 4 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 684 1 4 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 994 1 4 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 853 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 737 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 876 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 676 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 928 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 836 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 815 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 794 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 682 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 747 1 2 0 300 19249 19249 0 19249 0 2 0 0
xprt: tcp 998 1 2 0 300 19249 19249 0 19249 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 0 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 2 58 69 0
ACCESS: 302 302 0 50720 36240 3 54 64 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 307379 307379 0 54098704 322161891712 7621 2091470 2114511 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-2-b
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 2287
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 56 300 0 0 600 300 1203 0 53 78281 0 0 0 0 1200 0 0 600 0 0 0 0 0 0 0 0 0
bytes: 644245094400 0 0 0 643171352576 0 157024256 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 807 1 4 0 0 38407 38407 0 38407 0 2 0 0
xprt: tcp 716 1 4 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 844 1 4 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 684 1 4 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 994 1 4 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 853 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 737 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 876 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 676 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 928 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 836 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 815 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 794 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 682 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 747 1 2 0 0 38405 38405 0 38405 0 2 0 0
xprt: tcp 998 1 2 0 0 38405 38405 0 38405 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 0 0
GETATTR: 58 58 0 9384 6496 1 9 13 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 2 58 69 0
ACCESS: 547 547 0 91880 65640 5 103 122 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 613572 613572 0 107988672 643249889792 15298 4078290 4126306 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-2.diff
--- mountstats-2-a 2023-05-18 18:45:15.829187493 +0100
+++ mountstats-2-b 2023-05-18 18:45:25.065270704 +0100
@@ -1,35 +1,35 @@
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
- age: 1508
+ age: 2287
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
- events: 0 0 0 0 300 300 602 0 49 39261 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
- bytes: 322122547200 0 0 0 322122547200 0 78643200 0
+ events: 56 300 0 0 600 300 1203 0 53 78281 0 0 0 0 1200 0 0 600 0 0 0 0 0 0 0 0 0
+ bytes: 644245094400 0 0 0 643171352576 0 157024256 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
- xprt: tcp 807 1 4 0 300 19252 19252 0 19252 0 2 0 0
- xprt: tcp 716 1 4 0 300 19250 19250 0 19250 0 2 0 0
- xprt: tcp 844 1 4 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 684 1 4 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 994 1 4 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 853 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 737 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 876 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 676 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 928 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 836 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 815 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 794 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 682 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 747 1 2 0 300 19249 19249 0 19249 0 2 0 0
- xprt: tcp 998 1 2 0 300 19249 19249 0 19249 0 2 0 0
+ xprt: tcp 807 1 4 0 0 38407 38407 0 38407 0 2 0 0
+ xprt: tcp 716 1 4 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 844 1 4 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 684 1 4 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 994 1 4 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 853 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 737 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 876 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 676 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 928 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 836 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 815 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 794 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 682 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 747 1 2 0 0 38405 38405 0 38405 0 2 0 0
+ xprt: tcp 998 1 2 0 0 38405 38405 0 38405 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 0 0
- GETATTR: 2 2 0 208 224 1 0 1 0
+ GETATTR: 58 58 0 9384 6496 1 9 13 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 2 58 69 0
- ACCESS: 302 302 0 50720 36240 3 54 64 0
+ ACCESS: 547 547 0 91880 65640 5 103 122 0
READLINK: 0 0 0 0 0 0 0 0 0
- READ: 307379 307379 0 54098704 322161891712 7621 2091470 2114511 0
+ READ: 613572 613572 0 107988672 643249889792 15298 4078290 4126306 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
Raw
mountstats-2.txt
Stats for 10.0.0.2:/files mounted on /mnt:
NFS mount options: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
NFS mount age: 0:12:59
NFS server capabilities: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
NFS security flavor: 1 pseudoflavor: 0
NFS byte counts:
applications read 322122547200 bytes via read(2)
applications wrote 0 bytes via write(2)
applications read 0 bytes via O_DIRECT read(2)
applications wrote 0 bytes via O_DIRECT write(2)
client read 321048805376 bytes via NFS READ
client wrote 0 bytes via NFS WRITE
RPC statistics:
19156 RPC requests sent, 19156 RPC replies received (0 XIDs not found)
average backlog queue length: 0
READ:
306193 ops (1598%)
avg bytes sent per op: 176 avg bytes received per op: 1048645
backlog wait: 0.025072 RTT: 6.488783 total execute time: 6.570349 (milliseconds)
ACCESS:
245 ops (1%)
avg bytes sent per op: 168 avg bytes received per op: 120
backlog wait: 0.008163 RTT: 0.200000 total execute time: 0.236735 (milliseconds)
GETATTR:
56 ops (0%)
avg bytes sent per op: 163 avg bytes received per op: 112
backlog wait: 0.000000 RTT: 0.160714 total execute time: 0.214286 (milliseconds)
Raw
mountstats-3-a
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 1192
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
bytes: 0 0 0 0 0 0 0 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 956 1 2 0 0 3 3 0 3 0 2 0 0
xprt: tcp 969 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 988 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 953 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 830 1 2 0 0 1 1 0 1 0 2 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 0 0 0 0 0 0 0 0 0
ACCESS: 0 0 0 0 0 0 0 0 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 0 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-3-b
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 1630
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 0 0 0 0 300 300 601 0 0 39000 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
bytes: 322122547200 0 0 0 0 0 0 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 956 1 4 0 10 40 40 0 40 0 2 0 0
xprt: tcp 969 1 4 0 10 38 38 0 38 0 2 0 0
xprt: tcp 988 1 4 0 8 38 38 0 38 0 2 0 0
xprt: tcp 953 1 4 0 8 38 38 0 38 0 2 0 0
xprt: tcp 830 1 4 0 7 38 38 0 38 0 2 0 0
xprt: tcp 977 1 2 0 7 38 38 0 38 0 2 0 0
xprt: tcp 737 1 2 0 5 38 38 0 38 0 2 0 0
xprt: tcp 947 1 2 0 5 38 38 0 38 0 2 0 0
xprt: tcp 813 1 2 0 4 38 38 0 38 0 2 0 0
xprt: tcp 723 1 2 0 4 38 38 0 38 0 2 0 0
xprt: tcp 940 1 2 0 3 38 38 0 38 0 2 0 0
xprt: tcp 735 1 2 0 3 38 38 0 38 0 2 0 0
xprt: tcp 727 1 2 0 1 38 38 0 38 0 2 0 0
xprt: tcp 892 1 2 0 1 38 38 0 38 0 2 0 0
xprt: tcp 982 1 2 0 11 37 37 0 37 0 2 0 0
xprt: tcp 780 1 2 0 11 37 37 0 37 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 9 110 129 0
ACCESS: 301 301 0 50560 36120 7 64 78 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 0 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-3.diff
--- mountstats-3-a 2023-05-18 18:45:31.629329651 +0100
+++ mountstats-3-b 2023-05-18 18:45:38.577391867 +0100
@@ -1,33 +1,33 @@
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
- age: 1192
+ age: 1630
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
- events: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
- bytes: 0 0 0 0 0 0 0 0
+ events: 0 0 0 0 300 300 601 0 0 39000 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
+ bytes: 322122547200 0 0 0 0 0 0 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
- xprt: tcp 956 1 2 0 0 3 3 0 3 0 2 0 0
- xprt: tcp 969 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 988 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 953 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 830 1 2 0 0 1 1 0 1 0 2 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
- xprt: tcp 0 0 0 0 0 0 0 0 0 0 0 0 0
+ xprt: tcp 956 1 4 0 10 40 40 0 40 0 2 0 0
+ xprt: tcp 969 1 4 0 10 38 38 0 38 0 2 0 0
+ xprt: tcp 988 1 4 0 8 38 38 0 38 0 2 0 0
+ xprt: tcp 953 1 4 0 8 38 38 0 38 0 2 0 0
+ xprt: tcp 830 1 4 0 7 38 38 0 38 0 2 0 0
+ xprt: tcp 977 1 2 0 7 38 38 0 38 0 2 0 0
+ xprt: tcp 737 1 2 0 5 38 38 0 38 0 2 0 0
+ xprt: tcp 947 1 2 0 5 38 38 0 38 0 2 0 0
+ xprt: tcp 813 1 2 0 4 38 38 0 38 0 2 0 0
+ xprt: tcp 723 1 2 0 4 38 38 0 38 0 2 0 0
+ xprt: tcp 940 1 2 0 3 38 38 0 38 0 2 0 0
+ xprt: tcp 735 1 2 0 3 38 38 0 38 0 2 0 0
+ xprt: tcp 727 1 2 0 1 38 38 0 38 0 2 0 0
+ xprt: tcp 892 1 2 0 1 38 38 0 38 0 2 0 0
+ xprt: tcp 982 1 2 0 11 37 37 0 37 0 2 0 0
+ xprt: tcp 780 1 2 0 11 37 37 0 37 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
- LOOKUP: 0 0 0 0 0 0 0 0 0
- ACCESS: 0 0 0 0 0 0 0 0 0
+ LOOKUP: 300 300 0 50400 70800 9 110 129 0
+ ACCESS: 301 301 0 50560 36120 7 64 78 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 0 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0 0
Raw
mountstats-3.txt
Stats for 10.0.0.2:/files mounted on /mnt:
NFS mount options: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
NFS mount age: 0:07:18
NFS server capabilities: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
NFS security flavor: 1 pseudoflavor: 0
NFS byte counts:
applications read 322122547200 bytes via read(2)
applications wrote 0 bytes via write(2)
applications read 0 bytes via O_DIRECT read(2)
applications wrote 0 bytes via O_DIRECT write(2)
client read 0 bytes via NFS READ
client wrote 0 bytes via NFS WRITE
RPC statistics:
37 RPC requests sent, 37 RPC replies received (0 XIDs not found)
average backlog queue length: 0
ACCESS:
301 ops (813%)
avg bytes sent per op: 167 avg bytes received per op: 120
backlog wait: 0.023256 RTT: 0.212625 total execute time: 0.259136 (milliseconds)
LOOKUP:
300 ops (810%)
avg bytes sent per op: 168 avg bytes received per op: 236
backlog wait: 0.030000 RTT: 0.366667 total execute time: 0.430000 (milliseconds)
Raw
mountstats-4-a
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 2105
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 0 0 0 0 300 300 601 0 0 39000 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
bytes: 322122547200 0 0 0 0 0 0 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 956 1 4 0 0 40 40 0 40 0 2 0 0
xprt: tcp 969 1 4 0 0 38 38 0 38 0 2 0 0
xprt: tcp 988 1 4 0 0 38 38 0 38 0 2 0 0
xprt: tcp 953 1 4 0 0 38 38 0 38 0 2 0 0
xprt: tcp 830 1 4 0 0 38 38 0 38 0 2 0 0
xprt: tcp 977 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 737 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 947 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 813 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 723 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 940 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 735 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 727 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 892 1 2 0 0 38 38 0 38 0 2 0 0
xprt: tcp 982 1 2 0 0 37 37 0 37 0 2 0 0
xprt: tcp 780 1 2 0 0 37 37 0 37 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
GETATTR: 2 2 0 208 224 1 0 1 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 9 110 129 0
ACCESS: 301 301 0 50560 36120 7 64 78 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 0 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-4-b
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
age: 2889
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 194 300 0 0 600 300 1203 0 9 78054 0 0 0 0 1200 0 0 600 0 0 0 0 0 0 0 0 0
bytes: 644245094400 0 0 0 321593053184 0 78513929 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
xprt: tcp 956 1 6 0 0 19230 19230 0 19230 0 2 0 0
xprt: tcp 969 1 6 0 0 19228 19228 0 19228 0 2 0 0
xprt: tcp 988 1 6 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 953 1 6 0 0 19228 19228 0 19228 0 2 0 0
xprt: tcp 830 1 6 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 977 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 737 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 947 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 813 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 723 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 940 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 735 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 727 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 892 1 4 0 0 19226 19226 0 19226 0 2 0 0
xprt: tcp 982 1 4 0 0 19227 19227 0 19227 0 2 0 0
xprt: tcp 780 1 4 0 0 19227 19227 0 19227 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
GETATTR: 196 196 0 32016 21952 3 32 42 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 9 110 129 0
ACCESS: 409 409 0 68696 49080 8 86 104 0
READLINK: 0 0 0 0 0 0 0 0 0
READ: 306726 306726 0 53983776 321632314112 7458 1994340 2018720 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0 0
FSSTAT: 0 0 0 0 0 0 0 0 0
FSINFO: 2 2 0 208 160 0 0 1 0
PATHCONF: 1 1 0 104 56 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
Raw
mountstats-4.diff
--- mountstats-4-a 2023-05-18 18:45:45.897457220 +0100
+++ mountstats-4-b 2023-05-18 18:45:55.221540177 +0100
@@ -1,35 +1,35 @@
device 10.0.0.2:/files mounted on /mnt with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
- age: 2105
+ age: 2889
caps: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
- events: 0 0 0 0 300 300 601 0 0 39000 0 0 0 0 600 0 0 300 0 0 0 0 0 0 0 0 0
- bytes: 322122547200 0 0 0 0 0 0 0
+ events: 194 300 0 0 600 300 1203 0 9 78054 0 0 0 0 1200 0 0 600 0 0 0 0 0 0 0 0 0
+ bytes: 644245094400 0 0 0 321593053184 0 78513929 0
RPC iostats version: 1.1 p/v: 100003/3 (nfs)
- xprt: tcp 956 1 4 0 0 40 40 0 40 0 2 0 0
- xprt: tcp 969 1 4 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 988 1 4 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 953 1 4 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 830 1 4 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 977 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 737 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 947 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 813 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 723 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 940 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 735 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 727 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 892 1 2 0 0 38 38 0 38 0 2 0 0
- xprt: tcp 982 1 2 0 0 37 37 0 37 0 2 0 0
- xprt: tcp 780 1 2 0 0 37 37 0 37 0 2 0 0
+ xprt: tcp 956 1 6 0 0 19230 19230 0 19230 0 2 0 0
+ xprt: tcp 969 1 6 0 0 19228 19228 0 19228 0 2 0 0
+ xprt: tcp 988 1 6 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 953 1 6 0 0 19228 19228 0 19228 0 2 0 0
+ xprt: tcp 830 1 6 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 977 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 737 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 947 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 813 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 723 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 940 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 735 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 727 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 892 1 4 0 0 19226 19226 0 19226 0 2 0 0
+ xprt: tcp 982 1 4 0 0 19227 19227 0 19227 0 2 0 0
+ xprt: tcp 780 1 4 0 0 19227 19227 0 19227 0 2 0 0
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
- GETATTR: 2 2 0 208 224 1 0 1 0
+ GETATTR: 196 196 0 32016 21952 3 32 42 0
SETATTR: 0 0 0 0 0 0 0 0 0
LOOKUP: 300 300 0 50400 70800 9 110 129 0
- ACCESS: 301 301 0 50560 36120 7 64 78 0
+ ACCESS: 409 409 0 68696 49080 8 86 104 0
READLINK: 0 0 0 0 0 0 0 0 0
- READ: 0 0 0 0 0 0 0 0 0
+ READ: 306726 306726 0 53983776 321632314112 7458 1994340 2018720 0
WRITE: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0 0
Raw
mountstats-4.txt
Stats for 10.0.0.2:/files mounted on /mnt:
NFS mount options: rw,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.2,mountvers=3,mountport=20048,mountproto=tcp,fsc,local_lock=none
NFS mount age: 0:13:04
NFS server capabilities: caps=0xf,wtmult=4096,dtsize=1048576,bsize=0,namlen=255
NFS security flavor: 1 pseudoflavor: 0
NFS byte counts:
applications read 322122547200 bytes via read(2)
applications wrote 0 bytes via write(2)
applications read 0 bytes via O_DIRECT read(2)
applications wrote 0 bytes via O_DIRECT write(2)
client read 321593053184 bytes via NFS READ
client wrote 0 bytes via NFS WRITE
RPC statistics:
19190 RPC requests sent, 19190 RPC replies received (0 XIDs not found)
average backlog queue length: 0
READ:
306726 ops (1598%)
avg bytes sent per op: 176 avg bytes received per op: 1048598
backlog wait: 0.024315 RTT: 6.502025 total execute time: 6.581509 (milliseconds)
GETATTR:
194 ops (1%)
avg bytes sent per op: 163 avg bytes received per op: 112
backlog wait: 0.010309 RTT: 0.164948 total execute time: 0.211340 (milliseconds)
ACCESS:
108 ops (0%)
avg bytes sent per op: 167 avg bytes received per op: 120
backlog wait: 0.009259 RTT: 0.203704 total execute time: 0.240741 (milliseconds)
Raw
read-test
#!/bin/bash
snapshot() {
cp /proc/self/mountstats mountstats-$1
cp /proc/fs/fscache/stats fscache-$1
}
run() {
for ((i=1; i<=$1; i++)); do
printf '\n\n==> %d\n' $i
dd if=/mnt/$i.read of=/dev/null bs=1M status=progress
done
}
snapshot 1-a
run 300
snapshot 1-b
sleep 5m
snapshot 2-a
run 300
snapshot 2-b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment