This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ratingsCounts | |
.apply("FormatResults", MapElements.into(TypeDescriptors.strings()) | |
.via((KV<String, Long> ratingsCount) -> ratingsCount.getKey() + " " + ratingsCount.getValue())) | |
.apply(TextIO.write().to("./ratings_results").withSuffix(".csv")); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PCollection<KV<String, Long>> ratingsCounts = csvRows | |
.apply("Extract Ratings", | |
FlatMapElements.into(TypeDescriptors.strings()) | |
.via(csvRow -> Arrays.asList(csvRow.split(",")[1]))) | |
.apply("Count Ratings", Count.<String>perElement()); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PCollection<String> csvRows = pipeline.apply("Read from CSV", TextIO.read().from("./reviews.csv")); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pipeline pipeline = Pipeline.create(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package com.sandboxws; | |
import java.util.Arrays; | |
import org.apache.beam.sdk.Pipeline; | |
import org.apache.beam.sdk.io.TextIO; | |
import org.apache.beam.sdk.transforms.Count; | |
import org.apache.beam.sdk.transforms.FlatMapElements; | |
import org.apache.beam.sdk.transforms.MapElements; | |
import org.apache.beam.sdk.values.KV; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
+--------------------+-----------------------------+ | |
| Winning Plan | FETCH (7 / 7) -> IXSCAN (7) | | |
+--------------------+-----------------------------+ | |
| Used Indexes | author_id_1 (forward) | | |
+--------------------+-----------------------------+ | |
| Rejected Plans | 0 | | |
+--------------------+-----------------------------+ | |
| Documents Returned | 7 | | |
+--------------------+-----------------------------+ | |
| Documents Examined | 7 | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ae Article.where(author_id: '5bb13233d5cc2ec29c28649b') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class Author | |
include Mongoid::Document | |
include Mongoid::Timestamps | |
field :first_name, type: String | |
field :last_name, type: String | |
field :email, type: String | |
has_many :posts | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
__all__ = ['WriteToMongo'] | |
import json | |
from pymongo import MongoClient | |
from apache_beam.transforms import PTransform | |
from apache_beam.io import iobase | |
class _MongoSink(iobase.Sink): | |
"""A :class:`~apache_beam.io.iobase.Sink`.""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html> | |
<head> | |
<meta charset="UTF-8"> | |
<!-- IF PEN IS PRIVATE --> | |
<!-- <meta name="robots" content="noindex"> --> | |
<!-- END --> |