Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
code snippet to store invalid records in Google Cloud Storage
//get all error records per topic
final Map<String, GCSInvalidMessagesWrapper> topicMessagesMap = getMessagesToSerializePerTopic(errorRecords);
//serialize the messages in GCS for each topic - a file with all messages per topic is stored in GCS
topicMessagesMap.keySet().forEach(topicName -> {
final String fileName = UUID.randomUUID().toString();
final String pathPrefix = gcsBasePathPrefix + "/" + topicName + "/"
+ getFormattedDatePrefix( + "/";
final BlobId blobId = BlobId.of(gcsBucket, pathPrefix + fileName);
final Map<String, String> metaDataMap = new HashMap<>();
final BlobInfo objectInfo = BlobInfo.newBuilder(blobId)
final Blob objectCreated = gcsStore.create(objectInfo,
});"Pushing {} records to GCS success?: {}", errorRecords.size(), true);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment