Skip to content

Instantly share code, notes, and snippets.

View Guardian-Development's full-sized avatar

Joe Honour Guardian-Development

View GitHub Profile
@Guardian-Development
Guardian-Development / exampl_file_reader_as_iterable.java
Created November 17, 2019 16:50
Example Java File Reader by line using Iterable interface
public class FileLineReader implements Iterable<String> {
private final File inputFile;
FileLineReader(File inputFile) {
this.inputFile = inputFile;
}
@Override
public Iterator<String> iterator() {
@Guardian-Development
Guardian-Development / initial_data_source.java
Created November 17, 2019 16:57
A class that represents a data source that can then have transformations applied to it
public final class DataSource<T> {
private Iterator<T> source;
DataSource(Iterator<T> source) {
this.source = source;
}
public <D> Tranformation<T, D> transform(Tranformation<T, D> transformer) {
return new Tranformation<>(source, transformer);
@Guardian-Development
Guardian-Development / data_transformation.java
Created November 17, 2019 17:17
Java Data Transformation stage
public interface ApplyFunction<T, D> {
D apply(T input);
}
public final class Tranformation<T, D> implements Iterator<D> {
private final Iterator<T> source;
private final ApplyFunction<T, D> transformer;
Tranformation(Iterator<T> source, ApplyFunction<T, D> transformer) {
@Guardian-Development
Guardian-Development / data_sink.java
Created November 17, 2019 17:29
The data sink in an in-memory pipeline
public interface ApplyResult<T> {
void apply(T message);
}
public final class Sink<T> {
private final Iterator<T> source;
private final ApplyResult<T> sink;
Sink(Iterator<T> source, ApplyResult<T> sink) {
@Guardian-Development
Guardian-Development / in_memory_data_pipeline.java
Created November 17, 2019 17:40
Final in-memory data pipeline with example.
public class App {
public static void main(String[] args) {
Sink<Double> sink = new DataSource<>(Arrays.asList("1", "2", "3", "4").iterator())
.transform(s -> Integer.parseInt(s))
.transform(i -> (double) i)
.sink(i -> System.out.println("The result is: " + i));
// prints 1.0, 2.0, 3.0, 4.0
sink.execute();