Skip to content

Instantly share code, notes, and snippets.

@kenthzhang
Last active April 28, 2017 23:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kenthzhang/e6630f9eb4a69286f9674be17060fc5e to your computer and use it in GitHub Desktop.
Save kenthzhang/e6630f9eb4a69286f9674be17060fc5e to your computer and use it in GitHub Desktop.
Performance, RevoScaleR rxExecBy vs SparkR gapply
Keys API Size UDF Timing
DayOfWeek rxExecBy 1 rxSummary 28.098
DayOfWeek rxExecBy 1 rxLogit 93.798
DayOfWeek rxExecBy 1 mean 26.99
DayOfWeek rxExecBy 2 rxSummary 30.31
DayOfWeek rxExecBy 2 rxLogit 81.456
DayOfWeek rxExecBy 2 mean 34.307
DayOfWeek rxExecBy 5 rxSummary 36.308
DayOfWeek rxExecBy 5 rxLogit 116.653
DayOfWeek rxExecBy 5 mean 62.225
DayOfWeek rxExecBy 10 rxSummary 54.869
DayOfWeek rxExecBy 10 rxLogit 177.621
DayOfWeek rxExecBy 10 mean 102.398
DayOfWeek rxExecBy 100 rxSummary 416.117
DayOfWeek rxExecBy 100 rxLogit 1259.384
DayOfWeek rxExecBy 100 mean 899.763
DayOfWeek gapply 1 rxSummary 204.696
DayOfWeek gapply 1 rxLogit 208.74
DayOfWeek gapply 1 mean 201.205
DayOfWeek gapply 2 rxSummary 548.831
DayOfWeek gapply 2 rxLogit 560.261
DayOfWeek gapply 2 mean 544.34
DayOfWeek gapply 5 rxSummary 2728.545
DayOfWeek gapply 5 mean 2718.788
Dest rxExecBy 1 rxSummary 36.302
Dest rxExecBy 1 rxLogit 490.029
Dest rxExecBy 1 mean 32.109
Dest rxExecBy 2 rxSummary 35.356
Dest rxExecBy 2 rxLogit 498.35
Dest rxExecBy 2 mean 36.266
Dest rxExecBy 5 rxSummary 34.447
Dest rxExecBy 5 rxLogit 465.368
Dest rxExecBy 5 mean 37.313
Dest rxExecBy 10 rxSummary 48.842
Dest rxExecBy 10 rxLogit 478.808
Dest rxExecBy 10 mean 51.896
Dest rxExecBy 100 rxSummary 300.69
Dest rxExecBy 100 rxLogit 820.844
Dest rxExecBy 100 mean 334.229
Dest gapply 1 rxSummary 111.433
Dest gapply 1 mean 91.882
Dest gapply 2 rxSummary 200.996
Dest gapply 2 mean 174.349
Dest gapply 5 rxSummary 598.879
Dest gapply 5 mean 600.579
Dest gapply 10 mean 1965.735
Origin+Dest rxExecBy 1 mean 66.459
Origin+Dest rxExecBy 2 mean 88.116
Origin+Dest rxExecBy 5 mean 86.25
Origin+Dest rxExecBy 10 mean 93.45
Origin+Dest rxExecBy 100 mean 447.199
Origin+Dest gapply 1 mean 72.258
Origin+Dest gapply 2 mean 132.807
Origin+Dest gapply 5 mean 179.132
Origin+Dest gapply 10 mean 352.221
Origin+Dest gapply 100 mean 4562.899
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment