Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Sample reactive Leaflet code for Zeppelin
<!-- place this in an %angular paragraph -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/0.7.5/leaflet.css" />
<div id="map" style="height: 800px; width: 100%"></div>
<script type="text/javascript">
function initMap() {
var map = L.map('map').setView([30.00, -30.00], 3);
L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: 'Map data &copy; <a href="http://openstreetmap.org">OpenStreetMap</a> contributors',
maxZoom: 12,
minZoom: 3
}).addTo(map);
var geoMarkers = L.layerGroup().addTo(map);
var el = angular.element($('#map').parent('.ng-scope'));
angular.element(el).ready(function() {
window.locationWatcher = el.scope().compiledScope.$watch('locations', function(newValue, oldValue) {
// geoMarkers.clearLayers(); -- if you want to only show new data clear the layer first
angular.forEach(newValue, function(tweet) {
var marker = L.marker([ tweet.loc.lat, tweet.loc.lon ])
.bindPopup(tweet.user + ": " + tweet.tweet)
.addTo(geoMarkers);
});
})
});
}
if (window.locationWatcher) {
// clear existing watcher otherwise we'll have duplicates
window.locationWatcher();
}
// ensure we only load the script once, seems to cause issues otherwise
if (window.L) {
initMap();
} else {
console.log('Loading Leaflet library');
var sc = document.createElement('script');
sc.type = 'text/javascript';
sc.src = 'https://cdnjs.cloudflare.com/ajax/libs/leaflet/0.7.5/leaflet.js';
sc.onload = initMap;
sc.onerror = function(err) { alert(err); }
document.getElementsByTagName('head')[0].appendChild(sc);
}
</script>
// place this in a Spark paragraph
import org.apache.spark.streaming._
import org.apache.spark.streaming.scheduler.StreamingListener
import org.apache.spark.streaming.scheduler.StreamingListenerBatchCompleted
import org.apache.spark.streaming.twitter._
case class Loc(lat: Double, lon: Double)
case class Tweet(timestamp: java.util.Date, user: String, tweet: String, loc: Loc)
val ssc = new StreamingContext(sc, Seconds(2))
val input = TwitterUtils.createStream(ssc, None)
input.foreachRDD(rdd => {
val df = rdd
.filter(_.getGeoLocation() != null)
.map(s => Tweet(
s.getCreatedAt,
s.getUser.getName,
s.getText,
Loc(s.getGeoLocation.getLatitude, s.getGeoLocation.getLongitude)))
var items = df.collect
z.angularBind("locations", items) // this is what sends the data to the frontend
})
ssc.start()
@rawkintrevo

This comment has been minimized.

Copy link

rawkintrevo commented Jan 6, 2016

Super dope- thanks!

@Leemoonsoo

This comment has been minimized.

Copy link

Leemoonsoo commented Jan 7, 2016

Awesome!

@himynameschris

This comment has been minimized.

Copy link

himynameschris commented Apr 27, 2016

Thanks for this, I was able to get d3.js working in zeppelin based on this approach

@perrohunter

This comment has been minimized.

Copy link

perrohunter commented Jun 29, 2016

How would you feed the data from zeppelin to the plotChart function?

@sameerast

This comment has been minimized.

Copy link

sameerast commented Nov 22, 2016

I think above code to be updated, doesnt wotk now with following error.
error: object twitter is not a member of package org.apache.spark.streaming import org.apache.spark.streaming.twitter._

@wdickerson

This comment has been minimized.

Copy link

wdickerson commented May 19, 2017

Great work! I followed this pattern for a project recently. One alternative you might consider is using getScript() from jquery to lead leaflet.

if (!window.L) { 
    // leaflet (L) hasn't loaded, so load it
    $.getScript('https://unpkg.com/leaflet@1.0.3/dist/leaflet.js', initMap;
} else {
    // leaflet is already loaded, so initialize the map
    initMap();
}
@sbelyankin

This comment has been minimized.

Copy link

sbelyankin commented Aug 24, 2017

WOW, great work!

@vak

This comment has been minimized.

Copy link

vak commented Sep 22, 2017

oh, similarly as @sameerast mentioned: twitter isn't available at EMR Zeppelin as for now :-/
but the map is atleast shown, it would be nice to get an idea how to place markers on a map now and how to show some custome routes :)

@alisator

This comment has been minimized.

Copy link

alisator commented Sep 20, 2019

I used as dependency
%spark2.dep
z.reset()
z.addRepo("MavenCentral").url("https://mvnrepository.com/")
..
z.load("org.apache.bahir:spark-streaming-twitter_2.11:2.3.0")
https://bahir.apache.org/docs/spark/2.3.0/spark-streaming-twitter/ for new Spark 2.3 This solves error: object twitter is not a member of package org.apache.spark.streaming import org.apache.spark.streaming.twitter._
If you use twitter original reference for old spark -> it will lead to another issue, wrong paths to classes and so on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.