Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save jiachen247/eac0f907bfaf6da101d8702bbec78027 to your computer and use it in GitHub Desktop.
Save jiachen247/eac0f907bfaf6da101d8702bbec78027 to your computer and use it in GitHub Desktop.
spark@spark-VirtualBox:~$ unzip A0184022R_Lab1\ \(4\).zip
Archive: A0184022R_Lab1 (4).zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of A0184022R_Lab1 (4).zip or
A0184022R_Lab1 (4).zip.zip, and cannot find A0184022R_Lab1 (4).zip.ZIP, period.
spark@spark-VirtualBox:~$ unzip "A0184022R_Lab1\ \(4\).zip"
unzip: cannot find or open A0184022R_Lab1\ \(4\).zip, A0184022R_Lab1\ \(4\).zip.zip or A0184022R_Lab1\ \(4\).zip.ZIP.
spark@spark-VirtualBox:~$ unzip "A0184022R_Lab1 (4).zip"
Archive: A0184022R_Lab1 (4).zip
creating: asg-1/
inflating: asg-1/output.txt
extracting: asg-1/.gitignore
inflating: asg-1/top_products.py
inflating: asg-1/reviews_Video_Games.json
inflating: asg-1/run.sh
inflating: asg-1/meta_Video_Games.json
creating: asg-1/.git/
creating: asg-1/.git/refs/
creating: asg-1/.git/refs/heads/
extracting: asg-1/.git/refs/heads/master
creating: asg-1/.git/refs/tags/
creating: asg-1/.git/refs/remotes/
creating: asg-1/.git/refs/remotes/origin/
extracting: asg-1/.git/refs/remotes/origin/master
creating: asg-1/.git/info/
inflating: asg-1/.git/info/exclude
inflating: asg-1/.git/config
inflating: asg-1/.git/description
creating: asg-1/.git/branches/
extracting: asg-1/.git/HEAD
extracting: asg-1/.git/ORIG_HEAD
creating: asg-1/.git/hooks/
inflating: asg-1/.git/hooks/post-update.sample
inflating: asg-1/.git/hooks/applypatch-msg.sample
inflating: asg-1/.git/hooks/pre-rebase.sample
inflating: asg-1/.git/hooks/pre-push.sample
inflating: asg-1/.git/hooks/pre-commit.sample
inflating: asg-1/.git/hooks/prepare-commit-msg.sample
inflating: asg-1/.git/hooks/commit-msg.sample
inflating: asg-1/.git/hooks/pre-applypatch.sample
inflating: asg-1/.git/hooks/update.sample
creating: asg-1/.git/logs/
creating: asg-1/.git/logs/refs/
creating: asg-1/.git/logs/refs/heads/
inflating: asg-1/.git/logs/refs/heads/master
creating: asg-1/.git/logs/refs/remotes/
creating: asg-1/.git/logs/refs/remotes/origin/
inflating: asg-1/.git/logs/refs/remotes/origin/master
inflating: asg-1/.git/logs/HEAD
inflating: asg-1/.git/FETCH_HEAD
extracting: asg-1/.git/COMMIT_EDITMSG
creating: asg-1/.git/objects/
creating: asg-1/.git/objects/00/
extracting: asg-1/.git/objects/00/de1aaf4d554a8f03bf01bee8f119980d388a65
creating: asg-1/.git/objects/d0/
extracting: asg-1/.git/objects/d0/1293da4ebaf3c38e82cac6257c61aad3b561e6
creating: asg-1/.git/objects/3b/
extracting: asg-1/.git/objects/3b/7b044936a890cd8d651d349a752d819d71d22c
creating: asg-1/.git/objects/2b/
extracting: asg-1/.git/objects/2b/6c411da2392c502f37f9bd866b8bbf2b922cbf
creating: asg-1/.git/objects/68/
extracting: asg-1/.git/objects/68/6a1f8557840104c84ab634997a5e86d2d76f9d
creating: asg-1/.git/objects/05/
extracting: asg-1/.git/objects/05/a295ac554ed7230820733241f82621530fa4ea
creating: asg-1/.git/objects/4c/
extracting: asg-1/.git/objects/4c/57435b3b96bfa79b287ab3cb3a32fe2328d135
creating: asg-1/.git/objects/69/
extracting: asg-1/.git/objects/69/a906e1bdda1dc3f0b08f9d1da9aebfe24291a7
creating: asg-1/.git/objects/9f/
extracting: asg-1/.git/objects/9f/5b8c7b2dca40f5d0b0be6c677e6ba17db6c6f0
creating: asg-1/.git/objects/23/
extracting: asg-1/.git/objects/23/224d5743a2711e7294a66d17d6ded1d3793188
creating: asg-1/.git/objects/pack/
creating: asg-1/.git/objects/a2/
extracting: asg-1/.git/objects/a2/c3b9f5ff10f3554a99044804cb000e39ff6257
creating: asg-1/.git/objects/info/
creating: asg-1/.git/objects/94/
extracting: asg-1/.git/objects/94/a2dd146a22340832c88013e9fe92663bb9f2cc
creating: asg-1/.git/objects/fa/
extracting: asg-1/.git/objects/fa/4955a87a1dd2ce6f07a57bbe10515e4dab9257
creating: asg-1/.git/objects/96/
extracting: asg-1/.git/objects/96/9fc6079ec6afc815e50dc26a5f69952b1971e2
creating: asg-1/.git/objects/8c/
extracting: asg-1/.git/objects/8c/16175451795900497b173e6beaff1774b9bc91
creating: asg-1/.git/objects/3d/
extracting: asg-1/.git/objects/3d/482222bce808751978b149bea8918b1dd77287
creating: asg-1/.git/objects/e6/
extracting: asg-1/.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391
creating: asg-1/.git/objects/e7/
extracting: asg-1/.git/objects/e7/f7579af8299e6ee6c2c9e1b4ee24619cc3885c
creating: asg-1/.git/objects/98/
extracting: asg-1/.git/objects/98/03af40ab368784b652d1441f2916d337fb667e
creating: asg-1/.git/objects/af/
extracting: asg-1/.git/objects/af/bac30f0109ad1168d5049178a14221d8e04d19
creating: asg-1/.git/objects/9d/
extracting: asg-1/.git/objects/9d/eddbdf1dde0a6bb0f101f7d1ff2bccf753a1ce
creating: asg-1/.git/objects/a3/
extracting: asg-1/.git/objects/a3/deaf23dcb759c30b0a07878855259247c326c2
creating: asg-1/.git/objects/03/
extracting: asg-1/.git/objects/03/54a898b3578cd7baef18e7ef3d1564230f07a5
creating: asg-1/.git/objects/50/
extracting: asg-1/.git/objects/50/fb15cb647a80598b9091457c1dc03188c0f5f9
creating: asg-1/.git/objects/11/
extracting: asg-1/.git/objects/11/ae92c81d5017e1b1c00c6ae647d9055557f939
creating: asg-1/.git/objects/eb/
extracting: asg-1/.git/objects/eb/f3be76a2c97daeb95470f7baae13b9a954e233
creating: asg-1/.git/objects/b6/
extracting: asg-1/.git/objects/b6/7cdc6d9422f3551b7a45e163c1c51669534f2c
creating: asg-1/.git/objects/55/
extracting: asg-1/.git/objects/55/3572c1b219f614c6e815a890c98e193f8fd136
inflating: asg-1/.git/index
inflating: asg-1/README.md
spark@spark-VirtualBox:~$ chmod +x ./run.sh
chmod: cannot access './run.sh': No such file or directory
spark@spark-VirtualBox:~$ cd asg-1/
spark@spark-VirtualBox:~/asg-1$ chmod +x run.sh
spark@spark-VirtualBox:~/asg-1$ ./run.sh
rm: cannot remove 'output.txt': No such file or directory
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
23/03/04 11:57:55 WARN Utils: Your hostname, spark-VirtualBox resolves to a loopback address: 127.0.1.1; using 10.0.0.23 instead (on interface enp0s3)
23/03/04 11:57:55 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
23/03/04 11:57:55 INFO SparkContext: Running Spark version 2.2.1
... TRUNCATED...
23/03/04 11:58:37 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/03/04 11:58:37 INFO PythonRunner: Times: total = 46, boot = -48, init = 92, finish = 2
23/03/04 11:58:37 INFO Executor: Finished task 36.0 in stage 2.0 (TID 110). 2262 bytes result sent to driver
23/03/04 11:58:37 INFO TaskSetManager: Finished task 36.0 in stage 2.0 (TID 110) in 53 ms on localhost (executor driver) (37/37)
23/03/04 11:58:37 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool
23/03/04 11:58:37 INFO DAGScheduler: ResultStage 2 (takeOrdered at /home/spark/asg-1/top_products.py:66) finished in 2.131 s
23/03/04 11:58:37 INFO DAGScheduler: Job 0 finished: takeOrdered at /home/spark/asg-1/top_products.py:66, took 39.650401 s
23/03/04 11:58:37 INFO SparkUI: Stopped Spark web UI at http://10.0.0.23:4040
23/03/04 11:58:38 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
23/03/04 11:58:38 INFO MemoryStore: MemoryStore cleared
23/03/04 11:58:38 INFO BlockManager: BlockManager stopped
23/03/04 11:58:38 INFO BlockManagerMaster: BlockManagerMaster stopped
23/03/04 11:58:38 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
23/03/04 11:58:38 INFO SparkContext: Successfully stopped SparkContext
23/03/04 11:58:38 INFO ShutdownHookManager: Shutdown hook called
23/03/04 11:58:38 INFO ShutdownHookManager: Deleting directory /tmp/spark-91f6bb2e-89da-490d-99d8-d5aa40d7fb1b
23/03/04 11:58:38 INFO ShutdownHookManager: Deleting directory /tmp/spark-91f6bb2e-89da-490d-99d8-d5aa40d7fb1b/pyspark-a11e96d7-c9f8-4a77-8255-395a097d962d
spark@spark-VirtualBox:~/asg-1$ cat output.txt
B002I0JZOC 46 '12 25, 2012' 4.717 Sony
B003VANOFY 22 '04 11, 2014' 3.818 Logitech
B003EUPCZG 14 '03 7, 2014' 4.357 Razer
B002I0JZOC 12 '12 28, 2012' 4.750 Sony
B00BFOEY3Y 12 '12 26, 2013' 4.583 Logitech
B002I0JZOC 12 '12 27, 2012' 4.583 Sony
B003VANOFY 12 '12 30, 2013' 4.083 Logitech
B002I0JZOC 11 '12 26, 2012' 4.545 Sony
B008FHL56S 11 '02 13, 2014' 3.909 Generic
B002I0JZOC 11 '11 30, 2012' 4.818 Sony
B003VANOFY 11 '12 26, 2013' 3.636 Logitech
B003VANOFY 10 '12 16, 2013' 3.900 Logitech
B003VANOFY 10 '06 4, 2014' 2.900 Logitech
B008FHL56S 10 '12 27, 2012' 4.900 Generic
B003VANOFY 10 '06 26, 2014' 3.400 Logitech
spark@spark-VirtualBox:~/asg-1$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment