Skip to content

Instantly share code, notes, and snippets.

View trisberg's full-sized avatar

Thomas Risberg trisberg

542 followers · 1 following
View GitHub Profile
@trisberg
trisberg / gist:b3134473c00209ce1b5a
Created March 4, 2016 15:36
Mesos in Vagrant
**[UBUNTU]**
$ vagrant init ubuntu/trusty64
$ vi Vagrantfile
---
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "ubuntu/trusty64"
@trisberg
trisberg / gist:5aab83df8daf7e4576b5
Created March 8, 2016 18:08
ivory error
{"type": "step_started", "id": "bosh_product.deploying"}
===== 2016-03-08 17:39:03 UTC Running "bosh-init deploy /var/tempest/workspaces/default/deployments/bosh.yml"
Deployment manifest: '/var/tempest/workspaces/default/deployments/bosh.yml'
Deployment state: '/var/tempest/workspaces/default/deployments/bosh-state.json'
Started validating
Validating release 'bosh'... Finished (00:00:08)
Validating release 'bosh-vsphere-cpi'... Finished (00:00:01)
Validating release 'uaa'... Finished (00:00:08)
Validating cpi release... Finished (00:00:00)
@trisberg
trisberg / repo.adoc
Last active August 5, 2021 13:01
Using your own Maven "repo" with Spring Cloud Data Flow on CF

Using your own Maven "repo" with Spring Cloud Data Flow on CF

  1. Build and install your app locally into your .m2/repository

  2. Create a repo project (mine is named my-cf-repo)

  3. Copy the Maven jar and pom artifacts from your local .m2/repository to this repo project maintaing the directory structure

    At the root level of the repo project create a file named Staticfile with the following content:

@trisberg
trisberg / scdf-yarn-spark-task.adoc
Last active March 16, 2018 15:55
Running a Spark application on YARN using Spring Cloud Data Flow

Running a Spark application on YARN using Spring Cloud Data Flow

Deploy Spring Cloud Data Flow on YARN

Download the Spark distribution and copy assembly jar file to HDFS

For Spark 1.6.1 do the following

@trisberg
trisberg / dataflow-server-ticktock-log.log
Last active January 10, 2017 00:24
2017-01-09T19:22:39.13-0500 [APP/PROC/WEB/0]OUT 2017-01-10 00:22:39.138 INFO 11 --- [time.ticktock-1] log.sink : 01/10/17 00:22:39
2017-01-09T19:22:44.13-0500 [APP/PROC/WEB/0]OUT 2017-01-10 00:22:44.138 INFO 11 --- [time.ticktock-1] log.sink : 01/10/17 00:22:44
2017-01-09T19:22:49.14-0500 [APP/PROC/WEB/0]OUT 2017-01-10 00:22:49.139 INFO 11 --- [time.ticktock-1] log.sink : 01/10/17 00:22:49
2017-01-09T19:22:53.44-0500 [APP/PROC/WEB/0]OUT 2017-01-10 00:22:53.444 ERROR 11 --- [.168.11.11:5672] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: connection error; protocol method: #method<connection.close>(reply-code=320, reply-text=CONNECTION_FORCED - Closed via management plugin, class-id=0, method-id=0)
2017-01-09T19:22:53.46-0500 [API/0] OUT Updated app with guid a57c44d7-3f7c-41f0-8722-b96251acb65b ({"service_binding"=>"1af3ac55-e897-4001-b6ef-95c0873aaf96", :verb=>"remove", :relation=>"service_bin
@trisberg
trisberg / partial-undeploy
Created January 10, 2017 01:47
2017-01-09T20:44:32.08-0500 [APP/PROC/WEB/0]OUT 2017-01-10 01:44:32.086 INFO 6 --- [ctor-http-nio-1] o.s.c.d.s.c.CloudFoundryAppDeployer : Successfully computed status [deployed] for dataflow-server-ticktock-time
2017-01-09T20:44:32.17-0500 [APP/PROC/WEB/0]OUT 2017-01-10 01:44:32.172 INFO 6 --- [ctor-http-nio-1] o.s.c.d.s.c.CloudFoundryAppDeployer : Successfully computed status [deployed] for dataflow-server-ticktock-log
2017-01-09T20:44:32.17-0500 [RTR/0] OUT dataflow-server.local.pcfdev.io - [10/01/2017:01:44:31.987 +0000] "GET /streams/definitions?page=0&size=10&sort=DEFINITION_NAME,DEFINITION,ASC HTTP/1.1" 200 0 375 "http://dataflow-server.local.pcfdev.io/dashboard/index.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36" 192.168.11.1:49537 10.0.2.15:60030 x_forwarded_for:"-" x_forwarded_proto:"http" vcap_request_id:62f3e6ba-9bed-4d7b-77ad-002494825a5b response_time:0.186867264 app_id:63172ab2-ad80-48cb-ad33-60c0
@trisberg
trisberg / kafka-trace
Created January 24, 2017 16:32
$ more /tmp/spring-cloud-dataflow-5251525148462537142/ticktock-1485274715368/ticktock.time/stdout_0.log
2017-01-24 11:18:38.450 INFO 4339 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigAppl
icationContext@4e515669: startup date [Tue Jan 24 11:18:38 EST 2017]; root of context hierarchy
2017-01-24 11:18:39.948 INFO 4339 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'configurationPropertiesRebinderAutoConfiguration' of type [class
org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$7adf301] is not eligible for getting processed by all Be
anPostProcessors (for example: not eligible for auto-proxying)
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
@trisberg
trisberg / kafka-log
Created January 24, 2017 16:41
[2017-01-24 11:07:56,951] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2017-01-24 11:07:56,953] INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager)
[2017-01-24 11:07:56,953] INFO autopurge.purgeInterval set to 0 (org.apache.zookeeper.server.DatadirCleanupManager)
[2017-01-24 11:07:56,953] INFO Purge task is not scheduled. (org.apache.zookeeper.server.DatadirCleanupManager)
[2017-01-24 11:07:56,953] WARN Either no config or no quorum defined in config, running in standalone mode (org.apache.zookeeper.server.quorum.QuorumPeerMain)
[2017-01-24 11:07:56,977] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2017-01-24 11:07:56,977] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain)
[2017-01-24 11:07:56,996] INFO Server environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT (org.apache.zookeeper.server.Zo
@trisberg
trisberg / auto-config-reports
Last active February 7, 2017 18:50
**********
BOOT 1.3.5
**********
=========================
AUTO-CONFIGURATION REPORT
=========================
Positive matches:
@trisberg
trisberg / jdbc-sink-rabbit-upload-error.txt
Created April 6, 2017 13:19
2017-04-05T19:45:56.48-0400 [RTR/3] OUT trisberg-dataflow-server.cfapps.io - [2017-04-05T23:45:55.276+0000] "POST /streams/deployments/testjdbc HTTP/1.1" 201 2 0 "-" "Apache-HttpClient/4.5.3 (Java/1.8.0_101)" "10.10.2.141:1946" "10.10.148.107:61020" x_forwarded_for:"73.253.214.236" x_forwarded_proto:"http" vcap_request_id:"01614802-08aa-4242-6f9b-a5938bd08d5b" response_time:1.208187555 app_id:"e4208fd4-574f-4529-9bc2-1fc92b3d1587" app_index:"0" x_b3_traceid:"c847068d987446d2" x_b3_spanid:"c847068d987446d2" x_b3_parentspanid:"-"
2017-04-05T19:46:01.28-0400 [APP/PROC/WEB/0]OUT 2017-04-05 23:46:01.285 WARN 22 --- [ry-client-nio-7] io.netty.util.ReferenceCountUtil : Failed to release a message: CompositeByteBuf(freed, components=2)
2017-04-05T19:46:01.28-0400 [APP/PROC/WEB/0]OUT io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
2017-04-05T19:46:01.28-0400 [APP/PROC/WEB/0]OUT at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:101) ~[n