graphql-sequelize-benchmarks.md

Common test setup:

Models

• Project belongsToMany User
• User hasMany Task
• Task hasMany Task as SubTask

Data

• 10 projects, 25 random users each
• 1000 users
• 10,000 parent tasks, randomly assigned to users, and randomly given a completed true / false value.
• Each task has 10 subtasks

Apache bench, n: 500, c: 20. Sequelize is initialized with pool: { max: 25 } to ensure that the tests are not affected by a depleted connection pool.

Single Belongs to

tasks(limit: X) { 
  user { 
    name 
  } 
}

X = 250

Includes

Batching

Time taken for tests: 10.207 seconds Requests per second: 48.99 [#/sec] (mean) Time per request: 408.284 [ms] (mean) Time per request: 20.414 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 76 403 58.2 388 602 Waiting: 76 401 58.0 387 601 Total: 77 403 58.2 388 602 Percentage of the requests served within a certain time (ms) 50% 388 66% 398 75% 407 80% 434 90% 454 95% 519 98% 581 99% 602 100% 602 (longest request)

Time taken for tests: 18.460 seconds Requests per second: 27.09 [#/sec] (mean) Time per request: 738.405 [ms] (mean) Time per request: 36.920 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 284 734 90.8 711 961 Waiting: 284 731 90.7 707 960 Total: 284 734 90.9 711 961 Percentage of the requests served within a certain time (ms) 50% 711 66% 755 75% 782 80% 788 90% 878 95% 892 98% 937 99% 960 100% 961 (longest request)

X = 25

Includes	Batching

Single has many

users(limit: X) { 
  tasks { 
    edges { 
      node { 
        name
      } 
    } 
  }
} 

X = 250

Includes

Batching

Time taken for tests: 62.364 seconds Requests per second: 8.02 [#/sec] (mean) Time per request: 2494.566 [ms] (mean) Time per request: 124.728 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 339 2458 230.8 2429 2933 Waiting: 338 2447 230.5 2422 2927 Total: 340 2458 230.8 2429 2933 Percentage of the requests served within a certain time (ms) 50% 2429 66% 2534 75% 2564 80% 2593 90% 2685 95% 2708 98% 2728 99% 2757 100% 2933 (longest request)

Time taken for tests: 49.555 seconds Requests per second: 10.09 [#/sec] (mean) Time per request: 1982.217 [ms] (mean) Time per request: 99.111 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 1159 1966 221.8 1948 2578 Waiting: 1153 1943 219.5 1926 2566 Total: 1159 1966 221.8 1948 2578 Percentage of the requests served within a certain time (ms) 50% 1948 66% 2021 75% 2094 80% 2125 90% 2234 95% 2398 98% 2578 99% 2578 100% 2578 (longest request)

X = 25

Includes

Batching

Time taken for tests: 9.762 seconds Requests per second: 51.22 [#/sec] (mean) Time per request: 390.478 [ms] (mean) Time per request: 19.524 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 77 385 68.2 369 589 Waiting: 77 384 68.0 368 588 Total: 78 386 68.2 369 589 Percentage of the requests served within a certain time (ms) 50% 369 66% 403 75% 424 80% 439 90% 479 95% 515 98% 584 99% 588 100% 589 (longest request)

Time taken for tests: 6.441 seconds Requests per second: 77.62 [#/sec] (mean) Time per request: 257.656 [ms] (mean) Time per request: 12.883 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 93 255 48.0 248 438 Waiting: 91 251 47.4 246 437 Total: 94 255 48.0 248 438 Percentage of the requests served within a certain time (ms) 50% 248 66% 262 75% 269 80% 274 90% 294 95% 313 98% 438 99% 438 100% 438 (longest request)

Has many with where clause

users(limit: X) { 
  tasks(completed: true) { 
    edges { 
      node { 
        name,
        completed 
      } 
    } 
  }
} 

X = 250

Includes

Batching

Time taken for tests: 47.390 seconds Requests per second: 10.55 [#/sec] (mean) Time per request: 1895.606 [ms] (mean) Time per request: 94.780 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 326 1869 234.5 1875 2495 Waiting: 325 1858 232.9 1871 2473 Total: 326 1869 234.5 1875 2495 Percentage of the requests served within a certain time (ms) 50% 1875 66% 1960 75% 2001 80% 2025 90% 2145 95% 2223 98% 2343 99% 2358 100% 2495 (longest request)

Time taken for tests: 33.672 seconds Requests per second: 14.85 [#/sec] (mean) Time per request: 1346.886 [ms] (mean) Time per request: 67.344 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 756 1332 181.8 1319 1809 Waiting: 756 1309 179.3 1290 1778 Total: 756 1332 181.8 1319 1809 Percentage of the requests served within a certain time (ms) 50% 1319 66% 1394 75% 1415 80% 1445 90% 1577 95% 1671 98% 1752 99% 1809 100% 1809 (longest request)

X = 25

Includes

Batching

Time taken for tests: 7.237 seconds Requests per second: 69.09 [#/sec] (mean) Time per request: 289.474 [ms] (mean) Time per request: 14.474 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 191 285 54.2 274 621 Waiting: 188 284 53.8 273 619 Total: 191 286 54.2 274 622 Percentage of the requests served within a certain time (ms) 50% 274 66% 293 75% 302 80% 311 90% 343 95% 362 98% 454 99% 540 100% 622 (longest request)

Time taken for tests: 4.310 seconds Requests per second: 116.00 [#/sec] (mean) Time per request: 172.410 [ms] (mean) Time per request: 8.621 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 104 171 39.0 161 313 Waiting: 104 168 38.6 158 312 Total: 104 171 39.0 161 313 Percentage of the requests served within a certain time (ms) 50% 161 66% 170 75% 179 80% 182 90% 207 95% 283 98% 305 99% 313 100% 313 (longest request)

Nested has many:

users(limit: X) { 
  tasks(completed: true) { 
    edges { 
      node { 
      	name, 
      	subTasks { 
	      	edges {	
	      	  node { 
	      		  name 
						} 
					} 
				} 
			} 
		} 
	} 
} 

X = 250

Includes

Batching

Time taken for tests: 401.274 seconds Requests per second: 1.25 [#/sec] (mean) Time per request: 16050.953 [ms] (mean) Time per request: 802.548 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 1975 15811 1759.6 16096 18039 Waiting: 1946 15745 1758.3 16041 18039 Total: 1976 15811 1759.5 16096 18039 Percentage of the requests served within a certain time (ms) 50% 16096 66% 16437 75% 16707 80% 16798 90% 17125 95% 17461 98% 17806 99% 18039 100% 18039 (longest request)

Time taken for tests: 393.113 seconds Requests per second: 1.27 [#/sec] (mean) Time per request: 15724.513 [ms] (mean) Time per request: 786.226 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 1562 15607 4937.3 17270 23807 Waiting: 1562 15477 4885.9 17116 23387 Total: 1562 15607 4937.4 17270 23807 Percentage of the requests served within a certain time (ms) 50% 17270 66% 18820 75% 19120 80% 19621 90% 21044 95% 21741 98% 23422 99% 23422 100% 23807 (longest request)

X = 25

Includes

Batching

Time taken for tests: 41.553 seconds Requests per second: 12.03 [#/sec] (mean) Time per request: 1662.119 [ms] (mean) Time per request: 83.106 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 484 1638 135.2 1627 2241 Waiting: 478 1631 134.9 1618 2235 Total: 484 1638 135.1 1628 2242 Percentage of the requests served within a certain time (ms) 50% 1628 66% 1673 75% 1709 80% 1727 90% 1758 95% 1788 98% 1813 99% 1829 100% 2242 (longest request)

Time taken for tests: 36.542 seconds Requests per second: 13.68 [#/sec] (mean) Time per request: 1461.699 [ms] (mean) Time per request: 73.085 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 243 1452 555.0 1489 2638 Waiting: 243 1439 549.1 1486 2637 Total: 243 1452 555.0 1489 2638 Percentage of the requests served within a certain time (ms) 50% 1489 66% 1684 75% 1821 80% 1856 90% 2066 95% 2592 98% 2600 99% 2638 100% 2638 (longest request)

Two parallel has many

users(limit: X) { 
	tasks { 
		edges { 
			node { 
				name 
			} 
		} 
	}, 
	subordinates { 
		edges { 
			node { 
				name 
			} 
		} 
	} 
} 

X = 250

Includes

Batching

Time taken for tests: 78.029 seconds Requests per second: 6.41 [#/sec] (mean) Time per request: 3121.145 [ms] (mean) Time per request: 156.057 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 451 3072 302.9 3059 3755 Waiting: 451 3057 303.0 3042 3748 Total: 452 3072 302.9 3059 3755 Percentage of the requests served within a certain time (ms) 50% 3059 66% 3231 75% 3272 80% 3285 90% 3368 95% 3402 98% 3421 99% 3426 100% 3755 (longest request)

Time taken for tests: 70.559 seconds Requests per second: 7.09 [#/sec] (mean) Time per request: 2822.369 [ms] (mean) Time per request: 141.118 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 1398 2797 417.4 2815 3986 Waiting: 1372 2763 411.8 2790 3986 Total: 1398 2797 417.4 2815 3986 Percentage of the requests served within a certain time (ms) 50% 2815 66% 2971 75% 3008 80% 3045 90% 3144 95% 3579 98% 3778 99% 3778 100% 3986 (longest request)

X = 25

Includes

Batching

Time taken for tests: 10.760 seconds Requests per second: 46.47 [#/sec] (mean) Time per request: 430.396 [ms] (mean) Time per request: 21.520 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 138 425 49.7 412 682 Waiting: 137 423 49.7 410 680 Total: 139 425 49.7 412 682 Percentage of the requests served within a certain time (ms) 50% 412 66% 419 75% 425 80% 431 90% 507 95% 531 98% 556 99% 566 100% 682 (longest request)

Requests per second: 74.11 [#/sec] (mean) Time per request: 269.858 [ms] (mean) Time per request: 13.493 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 128 268 57.6 259 528 Waiting: 128 263 57.5 252 527 Total: 128 268 57.7 259 528 Percentage of the requests served within a certain time (ms) 50% 259 66% 263 75% 278 80% 281 90% 305 95% 323 98% 513 99% 528 100% 528 (longest request)

Single belongs to many

projects { 
  users { 
    edges { 
      node { 
        name 
      } 
    }
  } 
}

Includes

Batching

Time taken for tests: 10.392 seconds Requests per second: 48.11 [#/sec] (mean) Time per request: 415.698 [ms] (mean) Time per request: 20.785 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 171 410 52.3 395 753 Waiting: 170 408 52.3 393 752 Total: 172 410 52.3 395 753 Percentage of the requests served within a certain time (ms) 50% 395 66% 401 75% 416 80% 426 90% 469 95% 502 98% 582 99% 640 100% 753 (longest request)

Time taken for tests: 3.870 seconds Requests per second: 129.21 [#/sec] (mean) Time per request: 154.790 [ms] (mean) Time per request: 7.739 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 16 154 41.8 139 306 Waiting: 16 150 41.7 136 306 Total: 16 154 41.8 139 306 Percentage of the requests served within a certain time (ms) 50% 139 66% 146 75% 151 80% 154 90% 207 95% 242 98% 306 99% 306 100% 306 (longest request)

Single belongs to many with limit

projects { 
  users(first: 15) { 
    edges { 
      node { 
        name 
      } 
    }
  } 
}

Includes

Batching

Time taken for tests: 9.468 seconds Requests per second: 52.81 [#/sec] (mean) Time per request: 378.730 [ms] (mean) Time per request: 18.937 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 187 374 38.4 363 530 Waiting: 185 373 38.4 363 530 Total: 187 374 38.4 364 530 Percentage of the requests served within a certain time (ms) 50% 364 66% 374 75% 385 80% 390 90% 427 95% 463 98% 490 99% 530 100% 530 (longest request)

Time taken for tests: 3.006 seconds Requests per second: 166.33 [#/sec] (mean) Time per request: 120.241 [ms] (mean) Time per request: 6.012 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 96 120 47.2 108 345 Waiting: 93 117 46.8 106 344 Total: 96 120 47.2 108 346 Percentage of the requests served within a certain time (ms) 50% 108 66% 114 75% 118 80% 120 90% 135 95% 143 98% 345 99% 345 100% 346 (longest request)

Belongs to many with nodeested has many

projects { 
  users { 
    edges { 
      node { 
        name, 
        tasks { 
          edges { 
            node { 
              name 
            } 
          } 
        } 
      } 
    } 
  } 
}

Includes

Batching

Time taken for tests: 83.481 seconds Requests per second: 5.99 [#/sec] (mean) Time per request: 3339.225 [ms] (mean) Time per request: 166.961 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 586 3289 309.1 3343 3827 Waiting: 580 3276 308.8 3326 3814 Total: 587 3289 309.0 3343 3828 Percentage of the requests served within a certain time (ms) 50% 3343 66% 3379 75% 3406 80% 3435 90% 3488 95% 3518 98% 3559 99% 3614 100% 3828 (longest request)

Time taken for tests: 44.568 seconds Requests per second: 11.22 [#/sec] (mean) Time per request: 1782.702 [ms] (mean) Time per request: 89.135 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 913 1777 318.4 1722 2613 Waiting: 881 1743 318.1 1675 2611 Total: 913 1777 318.4 1722 2613 Percentage of the requests served within a certain time (ms) 50% 1722 66% 1924 75% 2001 80% 2033 90% 2175 95% 2334 98% 2612 99% 2613 100% 2613 (longest request)

Belongs to many with nested has many and with limit

projects { 
  users(first: 15) { 
    edges { 
      node { 
        name, 
        tasks { 
          edges { 
            node { 
              name 
            } 
          } 
        } 
      } 
    } 
  } 
}

Includes

Batching

Time taken for tests: 76.856 seconds Requests per second: 6.51 [#/sec] (mean) Time per request: 3074.233 [ms] (mean) Time per request: 153.712 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 555 3026 379.3 2988 3835 Waiting: 550 3017 379.0 2978 3834 Total: 556 3026 379.2 2988 3835 Percentage of the requests served within a certain time (ms) 50% 2988 66% 3098 75% 3196 80% 3280 90% 3486 95% 3716 98% 3745 99% 3835 100% 3835 (longest request)

Time taken for tests: 25.923 seconds Requests per second: 19.29 [#/sec] (mean) Time per request: 1036.939 [ms] (mean) Time per request: 51.847 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 2 Processing: 490 1029 260.0 953 1815 Waiting: 487 1008 255.1 938 1813 Total: 490 1029 260.0 954 1815 Percentage of the requests served within a certain time (ms) 50% 954 66% 1048 75% 1123 80% 1173 90% 1213 95% 1736 98% 1814 99% 1815 100% 1815 (longest request)

Belongs to many with nested has many and where clause

projects { 
  users { 
    edges { 
      node { 
        name, 
        tasks (completed: true) { 
          edges { 
            node { 
              name 
            } 
          } 
        } 
      } 
    } 
  } 
}

Includes

Batching

Time taken for tests: 103.404 seconds Requests per second: 4.84 [#/sec] (mean) Time per request: 4136.144 [ms] (mean) Time per request: 206.807 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 340 4081 588.5 4067 5626 Waiting: 339 4066 587.3 4043 5625 Total: 340 4081 588.5 4067 5626 Percentage of the requests served within a certain time (ms) 50% 4067 66% 4283 75% 4398 80% 4439 90% 4716 95% 5043 98% 5485 99% 5625 100% 5626 (longest request)

Time taken for tests: 42.245 seconds Requests per second: 11.84 [#/sec] (mean) Time per request: 1689.805 [ms] (mean) Time per request: 84.490 [ms] (mean, across all concurrent requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 0 Processing: 894 1681 258.5 1588 2151 Waiting: 876 1646 257.2 1548 2145 Total: 894 1681 258.5 1588 2151 Percentage of the requests served within a certain time (ms) 50% 1588 66% 1699 75% 2018 80% 2034 90% 2080 95% 2122 98% 2127 99% 2127 100% 2151 (longest request)

Conclusion?