I hereby claim:
- I am amitgupta1202 on github.
- I am amitgupta1202 (https://keybase.io/amitgupta1202) on keybase.
- I have a public key whose fingerprint is DD4D 5BC8 D77A 28E9 E9B3 39D9 0A49 732F 8F9E 989D
To claim this, I am signing this object:
Amit Gupta amitgupta1202
amitgupta1202
/ SpecificRecordAvroCoder.java
Created
August 29, 2021 06:42
implementation of apache beam avro coder for avro specific record. Using of serialization method on SpecificRecord seems to be ignored by beam is Serializable coder is registered for SpecificRecord.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.apache.avro.Schema; | |
| import org.apache.avro.io.*; | |
| import org.apache.avro.specific.SpecificDatumReader; | |
| import org.apache.avro.specific.SpecificDatumWriter; | |
| import org.apache.beam.sdk.coders.CoderException; | |
| import org.apache.beam.sdk.coders.CustomCoder; | |
| import org.apache.beam.sdk.util.EmptyOnDeserializationThreadLocal; | |
| import org.checkerframework.checker.initialization.qual.Initialized; | |
| import org.checkerframework.checker.nullness.qual.NonNull; | |
| import org.checkerframework.checker.nullness.qual.UnknownKeyFor; |
amitgupta1202
/ WriteAvroToBigQuery.java
Created
August 29, 2021 06:38
snippet to write the avro records to big query
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| //SomeRecord is generated via avro code generation, instance of SpecificRecord | |
| var tableSchema = AvroToTableSchemaConverter.convert({SomeRecord}.SCHEMA$); | |
| BigQueryIO.<SomeRecord>write() | |
| .to(tableName.getValue()) | |
| .withSchema("some-table-name") | |
| .useAvroLogicalTypes() | |
| .withAvroSchemaFactory(new SerializableFunction<TableSchema, Schema>() { | |
| @Override | |
| public Schema apply(TableSchema input) { |
amitgupta1202
/ AvroToTableSchemaConverter.java
Created
August 29, 2021 06:33
Converts avro schema to big query schema, not a full fledged implementation, works with basic types, records, union and array
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.google.api.services.bigquery.model.TableFieldSchema; | |
| import com.google.api.services.bigquery.model.TableSchema; | |
| import com.google.common.collect.ImmutableMap; | |
| import org.apache.avro.Schema; | |
| import org.apache.avro.Schema.Type; | |
| import java.util.ArrayList; | |
| import java.util.List; | |
| import static org.apache.avro.Schema.Type.*; |
amitgupta1202
/ fix_extended_scalar_types_patch.txt
Last active
April 24, 2021 04:46
Temporary fix patchfile for Extended scalar types on DGS Framework
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Index: graphql-dgs-client/src/main/kotlin/com/netflix/graphql/dgs/client/codegen/GraphQLQueryRequest.kt | |
| IDEA additional info: | |
| Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP | |
| <+>UTF-8 | |
| =================================================================== | |
| diff --git a/graphql-dgs-client/src/main/kotlin/com/netflix/graphql/dgs/client/codegen/GraphQLQueryRequest.kt b/graphql-dgs-client/src/main/kotlin/com/netflix/graphql/dgs/client/codegen/GraphQLQueryRequest.kt | |
| --- a/graphql-dgs-client/src/main/kotlin/com/netflix/graphql/dgs/client/codegen/GraphQLQueryRequest.kt (revision 1fbc781dfd97b0c4170c95182d83094605ac1958) | |
| +++ b/graphql-dgs-client/src/main/kotlin/com/netflix/graphql/dgs/client/codegen/GraphQLQueryRequest.kt (revision 7b264a6732f2590796bbefc42f8f994669106f53) | |
| @@ -16,6 +16,11 @@ | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import java.lang.RuntimeException | |
| sealed class InstructionPart | |
| data class Add(val argument: Int): InstructionPart() | |
| data class Subtract(val argument: Int): InstructionPart() | |
| data class Multiply(val argument: Int): InstructionPart() | |
| data class Divide(val argument: Int): InstructionPart() | |
| data class Apply(val argument: Int): InstructionPart() | |
| typealias Instruction = List<InstructionPart> |
amitgupta1202
/ keybase.md
Created
March 10, 2016 16:15