cgrand

;; did you know that binding conveying is not immutable but read-only? (the future sees updates performed by the original thread)
=> (with-local-vars [a 1]
     (future (Thread/sleep 1000) (prn 'future @a))
     (var-set a 2))
2
future 2

;; you have to push new bindings to isolate:
=> (with-local-vars [a 1]
     (with-bindings {a @a}

cgrand

UNleash the REPL

I spent some hammock time (ski lift time actually), brooding over unrepl.

What's ok

• cleanly multiplexed output (out, err, eval, exception etc.)
• elisions (which can provide all sort of browsing navigation)
• attachments
• :bye

cgrand

;; Clojure 1.6.0
=> *clojure-version*
{:major 1, :minor 6, :incremental 0, :qualifier nil}
=> (def env1 (create-clojure-env))
#'net.cgrand.quarantine/env1
=> (env1 *in* *out*)
clojure.core=> *clojure-version*
{:major 1, :minor 8, :incremental 0, :qualifier nil}
clojure.core=> 

cgrand

(ns powderkeg.sql
  (:require [clojure.spec :as s]
    [clojure.edn :as edn]
    [powderkeg.core :as keg]
    [net.cgrand.xforms :as x])
  (:import
    [org.apache.spark.sql functions Column Row RowFactory DataFrame]
    [org.apache.spark.sql.types StructType StructField ArrayType DataType DataTypes Metadata MetadataBuilder]))

(defmulti expr first)

cgrand

So I have a full working prototype (at https://github.com/cgrand/unrepl) but I haven’t wrote the spec yet for the input part.

Clojure 1.9.0-alpha14
user=> (require 'unrepl.repl)
nil
user=> (unrepl.repl/start)

outputs:

cgrand

REPL the Ultimate Protocol: Notes for a hopefully better "tooling over repl" approach for Clojure

The single point of agreement between the server and the client should be that: there exists a stream-based plain repl running and connected (socket or pipes etc. is not relevant).

From there one can upgrade to a richer repl. The interesting fact is that since the upgrade is triggered by the client there can't be any mismatch between capabilities. Even if your editor scripting power is limited (coughvimcough) you can treat the clojure snippet required for upgrade as a blob.

cgrand

;; some sample interaction with a clojure socket repl whose output stream consists only of valid edn forms.
;; the edn stream is not meant for the end user but for the client UI.

;; it demoes several features:
;; • prompt, out, err, eval and exceptions are demultiplexed; unstructred ones (err & out) are just text but the others give data. 
;; • elided content (because data too deep or wide) is replaced by a tagged literal (which optionally contains the expr to evaluate to get more items)
;; this last point makes possible navigation  

;; lines are prefixed by > or < to tell what is sent to the repl (>) and what is received from it (<)

cgrand

=> (ns user (:require [powderkeg.sql :as sql] [clojure.spec :as s]))
=> (s/def ::age int?)(s/def ::name string?) (s/def ::friends (s/* ::name))
:user/age
:user/name
:user/friends

=> (sql/df [{:name "Gaël" :age 6 :friends ["Alix"]}
            {:name "Estelle" :age 4 :friends ["Léana"]}
            {:name "Lilly" :age 1 :friends ["Tigger" "Cuddly"]}]
     (s/keys :req-un [::name ::age ::friends]))

cgrand

;; first, define some specs
=> (s/def ::name string?)
   (s/def ::age int?)
   (s/def ::friends (s/* ::name))
:user/name
:user/age
:user/friends

;; second create a dataframe from a RDD and a spec
=> (sql/df (rdd [{:name "Gaël" :age 6 :friends ["Alix"]}

cgrand

(ns powderkeg.spec
  (:require [clojure.spec :as s])
  (:import [org.apache.spark.sql.types DataType DataTypes]))

(defmulti expr first)

(def ^:private regexp-repeat
  (s/and 
    (s/cat :tag any? :type ::datatype)
    (s/conformer