Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
# create five partitions of the list
my_large_list_with_five_partition = sc.parallelize(my_large_list, numSlices=5)
# filter numbers greater than equal to 200
my_large_list_with_five_partition = my_large_list_with_five_partition.filter(lambda x : x >= 200)
# count the number of elements in the filtered list
# >> 16162207
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment