jiayuasu

val inputFile = "/Users/jiayu/Downloads/Apache_Sedona_Wherobots/gemeinde_de.geojson"
val featuresFile = "/Users/jiayu/Downloads/Apache_Sedona_Wherobots/gemeinde_de_features.geojson"
val geoJson = sparkSession.read.json(inputFile)
geoJson.printSchema()
// Sedona requires that GEOJSON schema has to be Feature, not FeatureCollection
// because a FeatureCollection GeoJSON does NOT follow one geometry per record
// See https://sedona.apache.org/tutorial/rdd/#create-a-generic-spatialrdd
// So we need to separate meta data and features
val metaData = geoJson.select("crs", "source", "type")
metaData.show()

jiayuasu

-----BEGIN PGP PUBLIC KEY BLOCK-----

mQENBFf34LcBCADGZc4cqsXL9Va1UXT9b0/2wrJoNubCAkpyykcO2LQPR2vMcNXC
fdzKPH1MfdyymhDooj7YVZxxmnyrXYnh6FmWa0XXcUEYyg7xLV7e+37PxMt2CdMe
lHhmsVSlLbRAFGlhE2GfyS6EuH5XCtz83LKoVVBgPyD7LlMOmFmc/90t8ynnaj+n
mYj0UHHhY1lAKWiVfovwQZ63wBMv+yIteYXyyolaaVl6XZj2zPR6fnbcsOzMtEdl
Vinx/cH0YjTXW2xHib/WTLyNyJYxQgk0U9Xesgg3mMPuWh/C3yxaVKOiSWyYeUMs
Rul1oxCqweA6exzErbOeANNvrOdwlz+2eM3HABEBAAG0Q0ppYSBZdSAoQXJpem9u
YSBTdGF0ZSBVbml2ZXJzaXR5IERhdGEgU3lzdGVtcyBMYWIpIDxqaWF5dTJAYXN1
LmVkdT6JATgEEwECACIFAlf34LcCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheA

jiayuasu

Maven:

https://infra.apache.org/publishing-maven-artifacts.html
https://repository.apache.org/

- Publish snapshots

- Publish Sedona for Spark 3.0 and Scala 2.12
python3 spark-version-converter.py spark3
mvn clean -Darguments="-DskipTests" release:prepare -DdryRun=true -DautoVersionSubmodules=true -Dresume=false

jiayuasu

The easiest way to start using the LLVM C++ API by example is to have LLVM generate the API usage for a given code sample. In this example it will emit the code required to rebuild the test.c sample by using LLVM:

$ clang -c -emit-llvm test.c -o test.ll
$ llc -march=cpp test.ll -o test.cpp

jiayuasu

#include <iostream>
#include <cstdint>
#include <string>

#include "llvm/ExecutionEngine/JIT.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Module.h"
#include "llvm/PassManager.h"
#include "llvm/Support/TargetSelect.h"
#include "llvm/Analysis/Verifier.h"

jiayuasu

First a disclaimer: This is an experimental API that exposes internals that are likely to change in between different Spark releases. As a result, most datasources should be written against the stable public API in org.apache.spark.sql.sources. We expose this mostly to get feedback on what optimizations we should add to the stable API in order to get the best performance out of data sources.

We'll start with a simple artificial data source that just returns ranges of consecutive integers.

/** A data source that returns ranges of consecutive integers in a column named `a`. */
case class SimpleRelation(
    start: Int, 
    end: Int)(
    @transient val sqlContext: SQLContext) 

jiayuasu

https://github.com/pstadler/keybase-gpg-github/issues/11

For some reason I had to setup the GPG TTY variable on my bash when I tried to sign my commit (i.e. -S flag) because of passphrase, which is new to me since I usually have the passphrase prompt within my terminal, so who knows, might be a new GPG version.

Anyway, fixed it with:

GPG_TTY=$(tty)
export GPG_TTY
Might be worth adding it to the README.md under common issues and/or what not.

jiayuasu

Step 1. Install homebrew from here: http://brew.sh
Step 1.1. Run export PATH=/usr/local/bin:$PATH
Step 2. Run brew update
Step 3. Run brew install vim && brew install macvim
Step 4. Run brew link macvim

# http://stackoverflow.com/questions/21012203/gvim-or-macvim-in-mac-os-x/21012284#21012284

Adv:
- copy & paste between tabs

jiayuasu

Table Of Content

Skip to the relevant sections if needed.

• 2-min tutorial to do it the quick-and-dirty-way
• Concepts for resolving Git conflicts
• Setting up different editors / tool for using git mergetool
  • Finding out what mergetool editors are supported
• mergetool simple code example for vimdiff
  • Resolving conflict from a Git pull
• Other great references and tutorials

jiayuasu

/*---------------------------- Babylon 0.1.1 (or later) Scala API usage ----------------------------*/

/*
 * If you are writing Babylon program in Spark Scala Shell, no need to declare the Spark Context by yourself.
 * If you are writing a self-contained Babylon Scala program, please declare the Spark Context as follows and
 * stop it at the end of the entire program.
 */
import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
import org.apache.spark.storage.StorageLevel;