ff, microblog

ff, microblog

--- webapp/src/webapp/core.clj
+++ webapp/src/webapp/core.clj
@@ -23,17 +23,17 @@

 (def ff (html [:h2 "Championship Odds:"][:h3 "Before Week 1:"][:p "What's Fappening 42.2%, Team Freedman 29.3%, Bill and Ted's ExcellentAdv 15.5%, Champ's Smelly Meat 13.0%"][:h3 "After Week 1:"][:p "Team Freedman 49.4%, Champ's Smelly Meat 39.8%, Bill and Ted's ExcellentAdv 5.7%, What's Fappening 5.2%"][:h2 "Individual Matches:"][:h3 "Before Week 1:"][:p "Team Freedman 60.1%, Bill and Ted's ExcellentAdv 39.9%"][:p "What's Fappening 68.6%, Champ's Smelly Meat 31.4%"][:p "Samuel Beckittns 69.6%, Pat Sajak Off 30.4%"][:p "Team buckley 52.5%, I Didn't Know You Were A Cop 47.5%"][:p "Nebraska Cornholers 53.3%, RGIII's iCloud Pics 46.7%"][:p "The Manifest Destinies 82.0%, Team Stanley 18.0%"][:h3 "After Week 1:"][:p "Team Freedman 87.0%, Bill and Ted's ExcellentAdv 13.0%"][:p "Champ's Smelly Meat 90.2%, What's Fappening 9.8%"][:p "Samuel Beckittns 60.2%, Pat Sajak Off 39.8%"][:p "Team buckley 90.2%, I Didn't Know You Were A Cop 9.8%"][:p "Nebraska Cornholers 100.0%"][:p "The Manifest Destinies 95.3%, Team Stanley 4.7%"]))

-(def posts (map :message (let [conn (mg/connect)
+(def posts (map #(get % "message") (let [conn (mg/connect)
       db   (mg/get-db conn "webapp")
       coll "micro"]
   (mc/find db coll {}))))

-(def microblog posts)
+(def microblog (html (map #(vector :p %) posts)))

 (defroutes router*
 	(GET "/" request (str "Doug's Website" site-map))
 	(GET "/ff" request ff)
-	(GET "/microblog" request (str microblog))
+	(GET "/microblog" request microblog)
 	(compojure.route/not-found "Link not found."))

 (def router (compojure.handler/api router*))


Allow PATCH on preflight (#675)

Allow PATCH on preflight (#675)

--- src/ctia/http/server.clj
+++ src/ctia/http/server.clj
@@ -36,7 +36,7 @@
            access-control-allow-origin
            access-control-allow-methods
            jwt]
-    :or {access-control-allow-methods "get,post,put,delete"}}]
+    :or {access-control-allow-methods "get,post,put,patch,delete"}}]
   (doto
       (jetty/run-jetty
        (cond-> (handler/api-handler)


Add note for applying trusty setting

Add note for applying trusty setting

--- src-cljs/frontend/components/project_settings.cljs
+++ src-cljs/frontend/components/project_settings.cljs
@@ -387,7 +387,8 @@
                              :blurb [:p
                                      "Select this option to run builds in our Ubuntu 14.04 (Trusty) container."
                                      "This container is currently in beta."
-                                     "Our default container is Ubuntu 12.04 (Precise)."]}))
+                                     "Our default container is Ubuntu 12.04 (Precise)."
+                                       [:p [:strong "Please note that you need to trigger a build by pushing commits to Github (instead of rebuilding) to apply the new setting."]]]}))
             ]]])))))

 (defn dependencies [project-data owner]


clean up docstrings

clean up docstrings

--- src/quil/core.clj
+++ src/quil/core.clj
@@ -2110,7 +2110,7 @@
   The extension parameter is used to determine the image type in cases
   where the image filename does not end with a proper
   extension. Specify the extension as the second parameter to
-  load-image, as shown in the third example on this page.
+  load-image.

   If an image is not loaded successfully, the null value is returned
   and an error message will be printed to the console. The error
@@ -2215,10 +2215,7 @@
     :subcategory "Calculation"
     :added "1.0"}
   map-range
-  "Re-maps a number from one range to another. In the example above,
-  the number '25' is converted from a value in the range 0..100 into a
-  value that ranges from the left edge (0) to the right edge (width)
-  of the screen.
+  "Re-maps a number from one range to another.

   Numbers outside the range are not clamped to 0 and 1, because
   out-of-range values are often intentional and useful."
@@ -2251,7 +2248,7 @@
     :added "1.0"}
   millis
   "Returns the number of milliseconds (thousandths of a second) since
-  starting an applet. This information is often used for timing
+  starting the sketch. This information is often used for timing
   animation sequences."
   []
   (.millis (current-applet)))
@@ -2789,8 +2786,7 @@
   style settings and later return to what you had. When a new style is
   started with push-style, it builds on the current style
   information. The push-style and pop-style fns can be embedded to
-  provide more control (see the second example above for a
-  demonstration.)
+  provide more control.

   The style information controlled by the following functions are
   included in the style: fill, stroke, tint, stroke-weight,
@@ -3325,11 +3321,11 @@
   are always sheared around their relative position to the origin and
   positive numbers shear objects in a clockwise direction.
   Transformations apply to everything that happens after and
-  subsequent calls to the function accumulates the effect. Fora
+  subsequent calls to the function accumulates the effect. For
   example, calling (shear-x (/ PI 2)) and then (shear-x (/ PI 2)) is
   the same as (shear-x PI). If shear-x is called within the draw fn,
   the transformation is reset when the loop begins again. This
-  function works in P2D or JAVA2D mode as shown in the example above.
+  function works in P2D or JAVA2D mode.

   Technically, shear-x multiplies the current transformation matrix
   by a rotation matrix. This function can be further controlled by the
@@ -3350,11 +3346,11 @@
   are always sheared around their relative position to the origin and
   positive numbers shear objects in a clockwise direction.
   Transformations apply to everything that happens after and
-  subsequent calls to the function accumulates the effect. Fora
+  subsequent calls to the function accumulates the effect. For
   example, calling (shear-y (/ PI 2)) and then (shear-y (/ PI 2)) is
   the same as (shear-y PI). If shear-y is called within the draw fn,
   the transformation is reset when the loop begins again. This
-  function works in P2D or JAVA2D mode as shown in the example above.
+  function works in P2D or JAVA2D mode.

   Technically, shear-y multiplies the current transformation matrix
   by a rotation matrix. This function can be further controlled by the


update list-of to new Om version

update list-of to new Om version

--- src/rohm/core.clj
+++ src/rohm/core.clj
@@ -13,12 +13,12 @@

 (defmacro list-of
   "Produces a list-unrolling with each element in the list-in-model onto
-  the component. Uses a default map of {:path [%] :key :id} which is
+  the component. Uses a default map of {:opts idx :key :id} which is
   merged with the optional alt-map"
   [component list-in-model & [alt-map]]
-  (let [a (gensym "a")]
-    `(into-array (map (fn [~a] (om.core/build ~component ~list-in-model
-                        ~(merge {:path [a] :key :id} alt-map)))
-                      (range (count ~list-in-model))))))
+  (let [idx (gensym "idx")]
+    `(for [~idx (range (count ~list-in-model))]
+       (om.core/build ~component ~list-in-model
+                        ~(merge {:opts idx :key :id} alt-map)))))

 ;(macroexpand-1 '(list-of comment [1 2 3] {:key :cut}))


collect and collect* return non-lazy sequences.

--- src/rum/util.cljc
+++ src/rum/util.cljc
@@ -2,13 +2,15 @@

 (defn collect [key mixins]
-  (->> (map (fn [m] (get m key)) mixins)
-       (remove nil?)))
+  (into []
+        (keep (fn [m] (get m key)))
+        mixins))

 (defn collect* [keys mixins]
-  (->> (mapcat (fn [m] (map (fn [k] (get m k)) keys)) mixins)
-       (remove nil?)))
+  (into []
+        (mapcat (fn [m] (keep (fn [k] (get m k)) keys)))
+        mixins))

 (defn call-all [state fns & args]


Encode before storing in memory

Encode before storing in memory

--- src/icecap/store/mem.clj
+++ src/icecap/store/mem.clj
@@ -1,7 +1,8 @@
 (ns icecap.store.mem
   "An in-memory store."
   (:require [clojure.core.async :as a]
-            [icecap.store.api :refer :all]))
+            [icecap.store.api :refer :all]
+            [icecap.codec :refer [safebase64-encode]]))

 (defn mem-store
   "Create an in-memory store."
@@ -9,12 +10,15 @@
   (let [store (atom {})]
     (reify Store
       (create! [_ index blob]
-        (swap! store assoc index blob)
-        (a/to-chan []))
+        (let [index (safebase64-encode index)]
+          (swap! store assoc index blob)
+          (a/to-chan [])))
       (retrieve [_ index]
-        (let [v (@store index)
+        (let [index (safebase64-encode index)
+              v (@store index)
               vs (if (nil? v) [] [v])]
           (a/to-chan vs)))
       (delete! [_ index]
-        (swap! store dissoc index)
-        (a/to-chan [])))))
+        (let [index (safebase64-encode index)]
+          (swap! store dissoc index)
+          (a/to-chan []))))))


add endpoint to get layernames from wfs capabilities (#198)

--- server/src/spacon/components/store.clj
+++ server/src/spacon/components/store.clj
@@ -9,7 +9,8 @@
             [spacon.components.mqtt :as mqttapi]
             [spacon.entity.scmessage :as scm]
             [clojure.data.json :as json]
-            [overtone.at-at :refer [every,mk-pool]])
+            [overtone.at-at :refer [every,mk-pool]]
+            [clojure.xml :as xml])
   (:import (com.boundlessgeo.spatialconnect.schema SCCommand)))

 (defn http-get [context]
@@ -28,18 +29,18 @@
 (defn http-put-store [mqtt context]
   (if-let [d (storemodel/update (get-in context [:path-params :id]) (:json-params context))]
     (do (mqttapi/publish-scmessage mqtt "/config/update"
-                                (scm/map->SCMessage
-                                  {:action (.value SCCommand/CONFIG_UPDATE_STORE)
-                                   :payload d}))
-     (response/ok "success"))
+                                   (scm/map->SCMessage
+                                    {:action (.value SCCommand/CONFIG_UPDATE_STORE)
+                                     :payload d}))
+        (response/ok "success"))
     (response/error "Error updating")))

 (defn http-post-store [mqtt context]
   (if-let [d (storemodel/create (:json-params context))]
     (do (mqttapi/publish-scmessage mqtt "/config/update"
-                                (scm/map->SCMessage {:action (.value SCCommand/CONFIG_ADD_STORE)
-                                                     :payload d}))
-     (response/ok d))
+                                   (scm/map->SCMessage {:action (.value SCCommand/CONFIG_ADD_STORE)
+                                                        :payload d}))
+        (response/ok d))
     (response/error "Error creating")))

 (defn http-delete-store [mqtt context]
@@ -99,6 +100,28 @@
 (defn load-polling-stores []
   (doall (map add-polling-store (storemodel/all))))

+(defn get-capabilities->layer-names [caps]
+  (let [layer-names (->> caps
+                         .getBytes
+                         java.io.ByteArrayInputStream.
+                         xml/parse
+                         :content
+                         (filter #(= :FeatureTypeList (:tag %)))
+                         first
+                         :content
+                         (filter #(= :FeatureType (:tag %)))
+                         (map #(first (:content (first (:content %))))))]
+    layer-names))
+
+(defn http-get-capabilities [context]
+  (let [url (get-in context [:query-params :url])
+        res (client/get (str url "?service=WFS&version=1.1.0&request=GetCapabilities"))
+        status (:status res)
+        body (:body res)]
+    (if (= status 200)
+      (response/ok (get-capabilities->layer-names body))
+      (response/error (str "Could not get capabilities from " url)))))
+
 (defn- routes [mqtt] #{["/api/stores" :get
                         (conj intercept/common-interceptors `http-get)]
                        ["/api/stores-error" :get
@@ -110,7 +133,9 @@
                        ["/api/stores" :post
                         (conj intercept/common-interceptors (partial http-post-store mqtt)) :route-name :http-post-store]
                        ["/api/stores/:id" :delete
-                        (conj intercept/common-interceptors (partial http-delete-store mqtt)) :route-name :http-delete-store]})
+                        (conj intercept/common-interceptors (partial http-delete-store mqtt)) :route-name :http-delete-store]
+                       ["/api/wfs/getCapabilities" :get
+                        (conj intercept/common-interceptors `http-get-capabilities)]})

 (defrecord StoreComponent [mqtt trigger]
   component/Lifecycle


CMR-4784 update cmr_metadata_preview commit to merge commit

--- collection-renderer-lib/project.clj
+++ collection-renderer-lib/project.clj
@@ -14,7 +14,7 @@
    the hardcoded commit id during dev integration with cmr_metadata_preview project.
    The hardcoded commit id should be updated when MMT releases a new version of the gem."
   (or (System/getenv "CMR_METADATA_PREVIEW_COMMIT")
-      "991048ffbcb"))
+      "a3d370d2e34"))

 (def gem-install-path
   "The directory within this library where Ruby gems are installed."


Fix issue with pics not correctly attaching to posts

--- common/clojure/nightweb/formats.clj
+++ common/clojure/nightweb/formats.clj
@@ -106,10 +106,10 @@
              "sign_algo" "DSA-SHA1"}))

 (defn post-encode
-  [& {:keys [create-time text image-hashes status ptrhash ptrtime]}]
+  [& {:keys [create-time text pic-hashes status ptrhash ptrtime]}]
   (let [args (merge {"body" text
                      "mtime" create-time
-                     "pics" (remove-dupes-and-nils image-hashes)
+                     "pics" (remove-dupes-and-nils pic-hashes)
                      "status" status}
                     (if ptrhash {"ptrhash" ptrhash} {})
                     (if ptrtime {"ptrtime" ptrtime} {}))]


strenghten string generation in metric tests (#907)

> Related #https://github.com/threatgrid/ctia/issues/905

Sometime the metric tests fail on labels. I did not manage to reproduce this issue.
This PR strengthen the label generator and improve test failing messages to help understanding if it happens again. I do not close the issue since I cannot confirm that it solves the problem, but it will help ease to improve this label generation if necessary.

--- test/ctia/test_helpers/aggregate.clj
+++ test/ctia/test_helpers/aggregate.clj
@@ -4,6 +4,7 @@
             [clj-momo.lib.clj-time.format :as tf]
             [ctia.test-helpers.core :as hc]
             [clojure.string :as string]
+            [clojure.pprint :refer [pprint]]
             [schema-generators.generators :as g]
             [ctia.http.routes.common :refer [now]]
             [schema-tools.core :as st]
@@ -73,11 +74,9 @@

 (defn- normalized-values
   [examples field]
-  (let [values (get-values examples field)
-        flattened (cond-> values
-                    (coll? (first values)) flatten-list-values)]
-    (map #(cond-> % (string? %) string/lower-case)
-         flattened)))
+  (let [values (get-values examples field)]
+    (cond-> values
+      (coll? (first values)) flatten-list-values)))

 (defn- check-from-to
   [from-str to-str]
@@ -91,13 +90,17 @@
             1)
         "[from to[ should not exceed one year")))

+(defn error-helper-msg
+  [explaining-values]
+  (str "values: \n"
+       (with-out-str (pprint explaining-values))))
+
 (defn- test-cardinality
   "test one field cardinality, examples are already created."
   [examples entity field]
   (testing (format "cardinality %s %s" entity field)
-    (let [expected (-> (normalized-values examples field)
-                       set
-                       count)
+    (let [unique-values (set (normalized-values examples field))
+          expected (count unique-values)
           _ (assert (pos? expected))
           {{:keys [from to]} :filters
            :as res} (cardinality entity
@@ -105,19 +108,19 @@
                                   :from "2020-01-01"}
                                  {:aggregate-on (name field)})]
       (is (= expected
-             (get-in (:data res) (parse-field field))))
+             (get-in (:data res) (parse-field field)))
+          (error-helper-msg unique-values))
       (check-from-to from to))))

 (defn- test-topn
   "test one field topn, examples are already created."
   [examples entity field limit]
   (testing (format "topn %s %s" entity field)
-    (let [expected (->> (normalized-values examples field)
+    (let [prepared (->> (normalized-values examples field)
                         frequencies
                         (sort-by val)
-                        reverse
-                        (take limit)
-                        vals)
+                        reverse)
+          expected (vals (take limit prepared))
           _ (assert (every? pos? expected))
           {{:keys [from to]} :filters
            :as res} (topn entity
@@ -127,7 +130,8 @@
       (is (= expected
              (->> (parse-field field)
                   (get-in (:data res))
-                  (map :value))))
+                  (map :value)))
+          (error-helper-msg prepared))
       (check-from-to from to))))

 (defn- to-granularity-first-day
@@ -201,6 +205,11 @@
           {}
           fields))

+(def string-generator
+  (->> (gen/sample gen/string-alphanumeric 20)
+       (map string/lower-case)
+       gen/elements))
+
 (defn generate-n-entity
   [{:keys [new-schema
            entity-minimal
@@ -212,9 +221,7 @@
     (doall
      (repeatedly n (fn [] (merge base-doc
                                  (g/generate enumerable-schema
-                                             {s/Str (gen/such-that #(< 3 (count %)) ;; easier value to simulate ES match
-                                                                   gen/string-alphanumeric
-                                                                   30)})
+                                             {s/Str string-generator})
                                  (generate-date-fields date-fields)))))))

 (defn test-metric-routes


New tests: get-api-prefix, path-to-emender, and path-to-tests

--- test/emender_service/config_test.clj
+++ test/emender_service/config_test.clj
@@ -45,3 +45,18 @@
     (testing "if the emender-service.config/print-configuration definition exists."
         (is (callable? 'emender-service.config/print-configuration))))

+(deftest test-get-api-prefix-existence
+    "Check that the emender-service.config/get-api-prefix definition exists."
+    (testing "if the emender-service.config/get-api-prefix definition exists."
+        (is (callable? 'emender-service.config/get-api-prefix))))
+
+(deftest test-path-to-emender-existence
+    "Check that the emender-service.config/path-to-emender definition exists."
+    (testing "if the emender-service.config/path-to-emender definition exists."
+        (is (callable? 'emender-service.config/path-to-emender))))
+
+(deftest test-path-to-tests-existence
+    "Check that the emender-service.config/path-to-tests definition exists."
+    (testing "if the emender-service.config/path-to-tests definition exists."
+        (is (callable? 'emender-service.config/path-to-tests))))
+


Added unit tests for modified non-empty-string function.

--- test/sandbar/dev/test_validation.clj
+++ test/sandbar/dev/test_validation.clj
@@ -59,7 +59,7 @@
                     ":a does not contain the letter a")))))))

 (deftest test-non-empty-string
-  (t "test non-empty-string"
+  (t "test non-empty string"
      (t "when string is empty"
         (is (= (non-empty-string {:a ""} :a "error")
                {:a "" :_validation-errors {:a ["error"]}}))
@@ -80,7 +80,23 @@
                {:a nil :_validation-errors {:a ["error"]}})))
      (t "when value is non-empty string"
         (is (= (non-empty-string {:a "a"} :a "error")
-               {:a "a"})))))
+               {:a "a"})))
+     (t "with multiple keywords"
+        (t "where validation passes"
+           (is (= (non-empty-string {:a "a" :b "b"} :a :b "error")
+                  {:a "a" :b "b"})))
+        (t "where validation fails for one keyword"
+           (is (= (non-empty-string {:a "a" :b ""} :a :b "error")
+                  {:a "a" :b "" :_validation-errors {:b ["error"]}})))
+        (t "where validation fails for one keyword with properties"
+           (is (= (non-empty-string {:a "a" :b ""} :a :b {:b "b"})
+                  {:a "a" :b ""
+                   :_validation-errors {:b ["b cannot be blank!"]}})))
+        (t "where validation fails for all keywords with properties"
+           (is (= (non-empty-string {:a "" :b ""} :a :b {:a "a" :b "b"})
+                  {:a "" :b ""
+                   :_validation-errors {:b ["b cannot be blank!"]
+                                        :a ["a cannot be blank!"]}}))))))

 (deftest test-build-validator
   (t "build validator"


♻ Improve tests

♻ Improve tests

--- test/territory_bro/gis_user_test.clj
+++ test/territory_bro/gis_user_test.clj
@@ -161,7 +161,10 @@

     (testing "delete GIS user"
       (is (= [deleted-event]
-             (handle-command delete-command [created-event] injections))))
+             (handle-command delete-command [created-event] injections)))
+      (is (= [(assoc deleted-event :gis-user/username "foo")]
+             (handle-command delete-command [(assoc created-event :gis-user/username "foo")] injections))
+          "username comes from the created event and is not re-generated"))

     (testing "delete is idempotent"
       (is (empty? (handle-command delete-command [] injections))


create adjunct indices in open-index

 create adjunct indices in open-index

and query them, though that doesn't actually work yet

--- src/sledge/db.clj
+++ src/sledge/db.clj
@@ -1,7 +1,9 @@
 (ns sledge.db
   (:import [java.util Date])
   (:require [clojure.java.io :as io]
+            [clojure.pprint :refer [pp pprint]]
             [clojure.edn :as edn]
+            [clojure.set :as set]
             [clojure.string :as str])  )

@@ -34,12 +36,16 @@

 (defn lower-words [x]
-  (set (remove str/blank? (str/split (.toLowerCase x) #"[ ,.]"))))
+  (set/difference
+   (set (remove str/blank? (str/split (.toLowerCase (or x "")) #"[ ,.()-]")))
+   #{"the"}))

 (def adjuncts
-  {:author {:tokenize-tags (comp lower-words :author)
+  {:artist {:tokenize-tags (comp #'lower-words :artist)
             :tokenize-query lower-words}
-   :title {:tokenize-tags (comp lower-words :title)
+   :album {:tokenize-tags (comp #'lower-words :album)
+           :tokenize-query lower-words}
+   :title {:tokenize-tags (comp #'lower-words :title)
            :tokenize-query lower-words}
    })

@@ -48,9 +54,27 @@
 ;; and have the end result attached to the primary index
 ;; somehow

+(defn adjunctivize [name-map]
+  (reduce
+   (fn [m [attr tok]]
+     (assoc m attr (make-adjunct-index name-map (:tokenize-tags tok))))
+   {}
+   adjuncts))
+
 ;; XXX async updates have to fit into this somehow eventually

-;;;
+(defn query-adjunct [index adjunct-name string]
+  (let [tokenizer (:tokenize-query (get adjuncts adjunct-name))
+        tokens (tokenizer string)
+        adjunct-map (get (:adjuncts index) adjunct-name)
+        matches (map #(get adjunct-map %) tokens)]
+    ;; we want to return a collection of pathnames ordered by the number
+    ;; of elements of matches that each appears within
+    (let [num-matches (fn [file]
+                        (count (filter #(contains? % file) matches)))]
+      (sort-by num-matches (set/union matches)))))
+
+
 ;; the main index maps from pathname to hash of tags

 (defn open-index [name]
@@ -63,10 +87,12 @@
               (reduce (fn [m [k v]] (assoc m k v))
                       {}
                       (take-while identity forms))))
-          {})]
+          {})
+        adjunct-maps (adjunctivize name-map)
+        ]
     (or (.isDirectory folder) (.mkdir folder))
     (or (.exists logfile) (.createNewFile logfile))
-    {:folder folder :log logfile :data (atom name-map)}
+    {:folder folder :log logfile :data (atom name-map) :adjuncts adjunct-maps}
     ))

 (defonce the-index (atom nil))


:bug: Fix `conform` function on uxbox.common.spec.

--- common/uxbox/common/spec.cljc
+++ common/uxbox/common/spec.cljc
@@ -126,9 +126,10 @@
   [spec data]
   (let [result (s/conform spec data)]
     (when (= result ::s/invalid)
-      (throw (ex/error :type :validation
-                       :code :spec-validation
-                       :explain (with-out-str
-                                  (expound/printer data))
-                       :data (::s/problems data))))
+      (let [edata (s/explain-data spec data)]
+        (throw (ex/error :type :validation
+                         :code :spec-validation
+                         :explain (with-out-str
+                                    (expound/printer edata))
+                         :data (::s/problems edata)))))
     result))


simplify

simplify

--- src/advent_2017/day_21.cljc
+++ src/advent_2017/day_21.cljc
@@ -7,11 +7,11 @@

 (def input (-> "advent_2017/day_21/input" io/resource slurp))

+(def transpose (partial apply map vector))
+
 (defn dihedral [pattern]
-  (let [r #(apply map (comp reverse vector) %)
-        s #(map reverse %)
-        rotations (take 4 (iterate r (map seq pattern)))]
-    (concat rotations (map s rotations))))
+  (let [rotations (take 4 (iterate (comp reverse transpose) (map seq pattern)))]
+    (concat rotations (map reverse rotations))))

 (defn make-pattern-map [input]
   (transduce (comp (mapcat #(str/split % #" => "))
@@ -25,10 +25,10 @@
 (defn split-xf [size]
   (comp (map #(partition size %))
         (partition-all size)
-        (map #(apply map vector %))))
+        (map transpose)))

 (def join-xf
-  (comp (mapcat #(apply map vector %))
+  (comp (mapcat transpose)
         (map flatten)))

 (defn step* [pattern-map input]


Test if string is XML by attempting to parse

Per David Mathei's comments, checking for a leading `<` is insufficient to determine if a String is XMLy.

--- src/uruk/core.clj
+++ src/uruk/core.clj
@@ -1051,11 +1051,22 @@
           (= (first s) \-) ;; if number is negative
           (number? (first s))))))

+(defn- is-xml-string?
+  "Returns true if `s` is a String containing valid XML; else false"
+  [s]
+  (if-not (string? s)
+    false
+    (try (if (xml/parse-str s) true false)
+         (catch javax.xml.stream.XMLStreamException xmlse
+           false)
+         (catch Exception e
+           false))))
+
 (defn ->string-format
   "Returns a document format keyword describing given String."
   [s]
   (assert (string? s) "Parameter `s` must be a String")
-  (cond (= (first s) \<)    :xml
+  (cond (is-xml-string?  s) :xml
         (is-json-string? s) :json
         :else               :text))


shorther default buffer flush timeout

shorther default buffer flush timeout

--- src/clj/matthiasn/systems_toolbox_sente/server.clj
+++ src/clj/matthiasn/systems_toolbox_sente/server.clj
@@ -66,7 +66,10 @@
   (fn [put-fn]
     (let [undertow-cfg (merge {:host host :port port :http2? http2?} undertow-cfg)
           user-routes (when routes-fn (routes-fn {:put-fn put-fn}))
-          opts (merge {:user-id-fn user-id-fn :packer (sente-transit/get-transit-packer)} sente-opts)
+          opts (merge {:user-id-fn     user-id-fn
+                       :packer         (sente-transit/get-transit-packer)
+                       :send-buf-ms-ws 5}
+                      sente-opts)
           ws (sente/make-channel-socket-server! sente-web-server-adapter opts)
           {:keys [ch-recv ajax-get-or-ws-handshake-fn ajax-post-fn]} ws
           cmp-routes [(GET "/" req (content-type (response (index-page-fn req)) "text/html"))


dentists are human

dentists are human

--- src/babel/encyclopedia.cljc
+++ src/babel/encyclopedia.cljc
@@ -166,6 +166,8 @@
    {:pred :coffee}   {:artifact true
                       :drinkable true}

+   {:pred :dentist}  {:human true}
+
    {:pred :espresso} {:artifact true
                       :drinkable true}


+{clothing true,consumable true,consumable false}

--- src/babel/encyclopedia.cljc
+++ src/babel/encyclopedia.cljc
@@ -36,6 +36,21 @@
                  :legible false
                  :place true}

+   {:clothing true} {:animate false
+                     :place false
+                     :physical-object true}
+
+   {:consumable true} {:activity false
+                       :buyable true
+                       :furniture false
+                       :legible false
+                       :pet false
+                       :physical-object true
+                       :speakable false}
+
+   {:consumable false} {:drinkable false
+                        :edible false}
+
    {:human true} {:animate true}

    {:living true} {:artifact false}
@@ -73,30 +88,13 @@
     (cond
       (= input :top) input
       true
-      (let [activity (get-encyc input :activity)
-            animate  (get-encyc input :animate)
-            artifact (get-encyc input :artifact)
-            buyable  (get-encyc input :buyable)
-            city     (get-encyc input :city)
-
-            clothing (if (= (get-in input '(:clothing))
-                            true)
-                       {:animate false
-                        :place false
-                        :physical-object true}{})
-
-            consumable (if (= (get-in input '(:consumable)) true)
-                         {:activity false
-                          :buyable true
-                          :furniture false
-                          :legible false
-                          :pet false
-                          :physical-object true
-                          :speakable false})
-
-            consumable-false (if (= (get-in input '(:consumable)) false)
-                               {:drinkable false
-                                :edible false} {})
+      (let [activity   (get-encyc input :activity)
+            animate    (get-encyc input :animate)
+            artifact   (get-encyc input :artifact)
+            buyable    (get-encyc input :buyable)
+            city       (get-encyc input :city)
+            clothing   (get-encyc input :clothing)
+            consumable (get-encyc input :consumable)

             drinkable
             ;; drinkables are always mass nouns.
@@ -235,7 +233,7 @@

                     true
                     (merge input
-                           activity animate artifact buyable city clothing consumable consumable-false drinkable
+                           activity animate artifact buyable city clothing consumable drinkable
                            drinkable-xor-edible-1 drinkable-xor-edible-2
                            edible furniture human inanimate
                            legible material-false non-places


Copy over newest triggers from simple branch.

--- src/yaqp/core.clj
+++ src/yaqp/core.clj
@@ -72,10 +72,21 @@
   `(#"(.*) has been mesmerized." (timer "Mez" "00:24")
     #"(.*) has been enthralled." (timer "Mez" "00:48")
     #"(.*) has been entranced." (timer "Mez" "00:80")
+    #"(.*) has been fascinated." (timer "Mez" "00:36")

-    #"(.*) feels much faster." (timer "Haste" "14:30")
+    #"(.*) feels much faster." (timer "Swift" "14:30")
+    #"(.*) experiences a quickening." (timer "Haste" "24:00")
+
+    ;#"(.*) is surrounded by a thorny barrier." (timer "Thorns" "2:30")
+
+    ;#"(.*) looks stronger." (timer "Strength" "27:00")
+    ;#"(.*)'s skin turns hard as steel." (timer "Skin" "36:00")
+    ;#"(.*) feet adhere to the ground." (timer "Root" "3:00")
+
     "A cool breeze slips through your mind." (timer "Crack" "26:00" {:fg "cyan"})
-    "the skin breaking and peeling." (timer "Boon" "4:30")
+    "A soft breeze slips through your mind." (timer "Crack" "26:00" {:fg "blue" :color "white"})
+    "Your spirit screams with berserker strength." (timer "Zerk" "5:00")
+    ;"the skin breaking and peeling." (timer "Boon" "4:30")

     "out of character," (pipe "chat.txt" {:fg "green"})
     "shouts," (pipe "chat.txt" {:fg "red"})
@@ -84,7 +95,14 @@
     "tell your party," (pipe "chat.txt" {:fg "cyan" :bold true})

     "tells the guild," (pipe "chat.txt" {:fg "green" :bold true})
-    "tell your guild," (pipe "chat.txt" {:fg "green" :bold true})))
+    "say to your guild," (pipe "chat.txt" {:fg "green" :bold true})
+
+    "you say," (pipe "chat.txt" {:bold true})
+    "tells you," (pipe "chat.txt" {:bold true})
+    "You told" (pipe "chat.txt" {:bold true})
+
+    "has looted a" (pipe "chat.txt" {:bold true})
+    ))

 (defn handle-line [line]
   ;(log (str "got line " line))


Add function to calculate today's date; support converting to dates to daily periods

--- src/clj/forma/date_time.clj
+++ src/clj/forma/date_time.clj
@@ -3,7 +3,7 @@
 ;; proper temporal comparison of two unrelated datasets.

 (ns forma.date-time
-  (:use [clj-time.core :only (date-time plus month year days)]
+  (:use [clj-time.core :only (now date-time plus month year days)]
         [forma.matrix.utils :only (sparse-expander)]
         [forma.utils :only (inc-eq? all-unique? contains-nils?)])
   (:require [clj-time.core :as time]
@@ -22,6 +22,15 @@
   with a call to `(clj-time.format/show-formatters)`."
   [dt format]
   (->> dt (f/unparse (f/formatters format))))
+
+(defn todays-date
+  "Returns current date as \"YYYY-MM-dd\".
+
+   Usage:
+     (todays-date)
+     ;=> \"2013-04-16\""
+  []
+  (unparse (now) :year-month-day))

 (defn convert
   "Converts the supplied string `s` between the two supplied
@@ -68,7 +77,7 @@
 ;; (It's important to note that time periods are NOT measured from the
 ;; activation date of a specific product. The first available NDVI
 ;; 16-day composite, for example, has a beginning date of February
-;; 18th 2001, or ordinal day 49, the 1st day of the 4th period as
+;; 18th 2000, or ordinal day 49, the 1st day of the 4th period as
 ;; measured from January 1st. With our reference of January 1st, This
 ;; dataset will receive an index of 3, and will match up with any
 ;; other data that falls within that 16-day period. This provides
@@ -137,6 +146,7 @@
 (def months (partial delta-periods month 1))
 (def sixteens (partial delta-periods ordinal 16))
 (def eights (partial delta-periods ordinal 8))
+(def ones (partial delta-periods ordinal 1))

 (defn periodize
   "Converts the supplied `org.joda.time.DateTime` object into a
@@ -147,7 +157,8 @@
   (let [converter (case temporal-res
                         "32" months
                         "16" sixteens
-                        "8" eights)]
+                        "8" eights
+                        "1" ones)]
     (converter (time/epoch) date)))

 (defn datetime->period


Added comments into server.clj

Added comments into server.clj

--- src/zg/server.clj
+++ src/zg/server.clj
@@ -24,6 +24,7 @@
 (require '[zg.dictionary-interface :as dictionary-interface])

 (defn get-user-name
+  "Retrieve user name from request."
   [request]
   (let [params        (:params request)
         cookies       (:cookies request)
@@ -36,24 +37,28 @@
     user-name))

 (defn get-title
+  "Get page title from configuration that is part of request."
   [request]
   (-> (:configuration request)
       :display
       :app-name))

 (defn get-emender-page
+  "Retrieve link to Emender page from configuration.."
   [request]
   (-> (:configuration request)
       :display
       :emender-page))

 (defn get-url-prefix
+  "Retrieve URL prefix from configuration."
   [request]
   (-> (:configuration request)
       :server
       :url-prefix))

 (defn finish-processing
+  "Finish processing of HTTP request."
   [request search-results sources classes message title emender-page mode]
   (let [params        (:params request)
         cookies       (:cookies request)
@@ -114,11 +119,13 @@
                        :atomic-typos)))

 (defn sources->map
+  "Provide mapping between source ID and source name."
   [sources]
   (into '{}
         (for [source sources] [(:id source) (:source source)])))

 (defn classes->map
+  "Provide mapping between class ID and class name."
   [classes]
   (into '{}
     (for [class classes] [(:id class) (:class class)])))
@@ -144,6 +151,7 @@
                        :glossary)))

 (defn add-word-message
+  "Message that new word has been added into the dictionary."
   [word proper-word]
   (if (seq word)
     (if proper-word
@@ -152,6 +160,7 @@
       "No word has been added..."))

 (defn add-words-message
+  "Message that new words has been added into the dictionary."
   [words]
   (if (seq words)
     (if (> (count words) 1)
@@ -160,18 +169,21 @@
     "No words have been added..."))

 (defn split-words
+  "Split input words."
   [input]
   (if input
     (->> (clojure.string/split input #"[\s,\n]")
          (filter seq))))

 (defn read-string-parameter
+  "Read string parameter from HTTP request."
   [request parameter-name]
   (if-let [parameter-value (-> (:params request)
                                (get parameter-name))]
     (clojure.string/trim parameter-value)))

 (defn read-boolean-parameter
+  "Read boolean parameter from HTTP request."
   [request parameter-name]
   (if-let [parameter-value (-> (:params request)
                                (get parameter-name))]
@@ -179,6 +191,7 @@
     false))

 (defn read-integer-parameter
+  "Read integer parameter from HTTP request."
   [request parameter-name]
   (if-let [parameter-value (-> (:params request)
                                (get parameter-name))]
@@ -251,6 +264,7 @@
                          mode))))

 (defn process-add-words
+  "Add words provided by user (together with its description) to the blacklist or into atomic typos."
   [request title emender-page mode]
   (let [input        (-> (:params request) (get "new-words"))
         words        (split-words input)
@@ -262,6 +276,7 @@
     (finish-processing request nil nil nil message title emender-page mode)))

 (defn perform-operation
+  "Perform the requested operation - delete or undelete."
   [request mode]
   (let [to-delete   (-> (:params request)
                         (get "delete"))
@@ -365,18 +380,22 @@
                        mode)))

 (defn read-changes-statistic
+  "Read statistic about changes from database."
   []
   (db-interface/read-changes-statistic))

 (defn read-changes
+  "Read all changes from database."
   []
   (db-interface/read-changes))

 (defn read-changes-for-user
+  "Read changes made by one user from database."
   [user-name]
   (db-interface/read-changes-for-user user-name))

 (defn process-user-list
+  "Prepare page with list of users."
   [request title mode]
   (let [changes-statistic (read-changes-statistic)
         changes           (read-changes)
@@ -402,6 +421,7 @@
           (http-response/content-type "text/html")))))

 (defn process-user-info
+  "Prepare page with info about one selected user."
   [request title mode]
   (let [params        (:params request)
         selected-user (get params "name")
@@ -425,10 +445,12 @@

 (defn words->json
+  "Convert all words into JSON format."
   [words]
   (json/write-str words))

 (defn words->text
+  "Convert all words into text."
   [words]
   (->>
     (for [word words
@@ -442,6 +464,7 @@
     (clojure.string/join "\n")))

 (defn words->xml
+  "Convert all words into XML format."
   [words]
   (with-out-str
     (xml/emit {:tag :whitelist,
@@ -464,21 +487,25 @@
                            :content [(:word word)]})})))

 (defn words->edn
+  "Convert all words into EDN format."
   [words]
   (with-out-str
     (clojure.pprint/pprint words)))

 (defn words->csv
+  "Convert all words into CSV format."
   [words]
   (with-out-str
     (csv/write-csv *out* words)))

 (defn words->vector
+  "Convert all words into vector."
   [words]
   (for [word words]
     (vals word)))

 (defn process-wordlist-json
+  "Prepare report with all words in JSON format."
   [request dictionary-type]
   (let [search-results (db-interface/read-all-words dictionary-type)
         json-output    (words->json search-results)]
@@ -486,6 +513,7 @@
         (http-response/content-type "application/json"))))

 (defn process-wordlist-text
+  "Prepare report with all words in plain text format."
   [request dictionary-type]
   (let [search-results (db-interface/read-all-words dictionary-type)
         text-output    (words->text search-results)]
@@ -493,6 +521,7 @@
         (http-response/content-type "text/plain"))))

 (defn process-wordlist-xml
+  "Prepare report with all words in XML format."
   [request dictionary-type]
   (let [search-results (db-interface/read-all-words dictionary-type)
         xml-output     (words->xml search-results)]
@@ -500,6 +529,7 @@
         (http-response/content-type "text/xml"))))

 (defn process-wordlist-edn
+  "Prepare report with all words in EDN format."
   [request dictionary-type]
   (let [search-results (db-interface/read-all-words dictionary-type)
         edn-output     (words->edn search-results)]
@@ -507,6 +537,7 @@
         (http-response/content-type "application/edn"))))

 (defn process-wordlist-csv
+  "Prepare report with all words in CSV format."
   [request dictionary-type]
   (let [search-results (db-interface/read-all-words dictionary-type)
         words          (words->vector search-results)
@@ -515,6 +546,7 @@
         (http-response/content-type "text/csv; charset=utf-8"))))

 (defn process-different-spelling-words
+  "Prepare report with different spelled words in JSON format."
   [request]
   (let [search-results (db-interface/read-different-spelling-words)
         json-output    (words->json search-results)]


Summarize large data structures when logging at level :trace.

If we don't do this, the test can run extremely slowly or OOM the JVM.

--- src/clj_libssh2/logging.clj
+++ src/clj_libssh2/logging.clj
@@ -3,7 +3,61 @@
    clj-libssh2."
   (:require [clojure.tools.logging :as log]
             [clojure.pprint :refer [pprint]]
-            [robert.hooke :as hook]))
+            [robert.hooke :as hook])
+  (:import [java.io InputStream OutputStream]
+           [com.sun.jna Pointer]))
+
+(def ^:const byte-array-type (type (byte-array 0)))
+(defmulti summarize-structure
+  "Given an object, trim down the larger items so that logging them doesn't
+   result in a combination of extreme performance degradataion and lack of
+   readability.
+
+   Arguments:
+
+   object The object to summarize.
+
+   Return:
+
+   An object of the same type and shape as the input, but with the larger parts
+   of the object summarized into a brief description of their contents."
+  (fn [object]
+    (cond (map? object) :map
+          (sequential? object) :sequence
+          (instance? Pointer object) :summarize
+          (instance? InputStream object) :summarize
+          (instance? OutputStream object) :summarize
+          (string? object) :string
+          (= (type object) byte-array-type) :bytes
+          :else :passthru)))
+
+(defmethod summarize-structure :map
+  [object]
+  (->> object
+       (map #(hash-map (key %) (summarize-structure (val %))))
+       (apply merge)))
+
+(defmethod summarize-structure :sequence
+  [object]
+  (map summarize-structure object))
+
+(defmethod summarize-structure :summarize
+  [object]
+  (type object))
+
+(defmethod summarize-structure :string
+  [object]
+  (if (< 1024 (count object))
+    (format "String[%d]" (count object))
+    object))
+
+(defmethod summarize-structure :bytes
+  [object]
+  (format "byte[%d]" (count object)))
+
+(defmethod summarize-structure :passthru
+  [object]
+  object)

 (defn- spy-fn
   "Create a function that can be used as a hook to log the call and result from
@@ -24,8 +78,8 @@
         fn-name (symbol (str (ns-name (:ns fn-meta))) (str (:name fn-meta)))
         make-message (fn [args result]
                        (format "%s => %s"
-                               (with-out-str (pprint (cons fn-name args)))
-                               (with-out-str (pprint result))))]
+                               (with-out-str (pprint (cons fn-name (summarize-structure args))))
+                               (with-out-str (pprint (summarize-structure result)))))]
     (fn [f & args]
       (try
         (let [result (apply f args)]


got mathjax to update, uglily

got mathjax to update, uglily

--- src/cljs/relembra/core.cljs
+++ src/cljs/relembra/core.cljs
@@ -22,18 +22,21 @@
                 (p/posh! conn)
                 conn))

-(def mathjax-update-wrapper
-  (with-meta identity
-    {:component-did-update #(js/MathJax.Hub.Queue
-                             (array "Typeset"
-                                    js/MathJax.Hub
-                                    ))}))
+(defn typeset [c]
+  (.log js/console "Updating!")
+  (js/MathJax.Hub.Queue (array "Typeset" js/MathJax.Hub (r/dom-node c))))
+
+(defn mathjax-box [text]
+  [:div {:id "preview" :dangerouslySetInnerHTML {:__html (md->html text)}}])
+
+(def mathjax-box2 (with-meta mathjax-box {:component-did-mount typeset
+                                          :component-did-update typeset}))

 (defn app []
   (let [editor-text @(p/q conn
                           '[:find ?c .
                             :where [0 :editor/text ?c]])]
-    [:div "Esta é a área $a_0$."
+    [:div {:id "um"} (str "Esta é a área $a_0$: " editor-text)
      [:table>tbody>tr
       [:td>textarea {:rows 40
                      :cols 80
@@ -42,7 +45,9 @@
                      :on-change (fn [e]
                                   (p/transact! conn
                                                [{:db/id 0 :editor/text (.. e -target -value)}]))} ]
-      [:td {:valign "top"} [:div {:id "preview" :dangerouslySetInnerHTML {:__html (md->html editor-text)}}]]]]))
+      [:td {:valign "top"} [mathjax-box2 editor-text]]
+      [:td {:valign "top"} editor-text]]]))
+

 (defn ^:export main []
   (r/render [app]


replace split-by-conv with group-by

replace split-by-conv with group-by

--- src/polismath/poller.clj
+++ src/polismath/poller.clj
@@ -26,15 +26,6 @@
                   :ssl true
                   :sslfactory "org.postgresql.ssl.NonValidatingFactory"}]
     (kdb/postgres settings)))
-
-
-(defn split-by-conv [votes]
-  (reduce
-    (fn [conv-map vote]
-      (let [conv-id (vote :zid)]
-        (assoc conv-map conv-id
-               (conj (get conv-map conv-id []) vote))))
-    {} votes))

 (defn mongo-connect! [mongo-url]
@@ -85,7 +76,7 @@
     (endlessly poll-interval
       (println "poll >" @last-timestamp)
       (let [new-votes (poll pg-spec @last-timestamp)
-            split-votes (split-by-conv new-votes)]
+            split-votes (group-by :zid new-votes)]
         (doseq [[zid votes] split-votes]
           (let [lastVoteTimestamp (:created (last votes))]
             (swap! conversations


Env var cleanup

Env var cleanup

--- src/polismath/server.clj
+++ src/polismath/server.clj
@@ -222,7 +222,7 @@
         download-url (public-datadump-url filename zinvite)
         email-hiccup (completion-email-text zinvite download-url)]
     (email/send-email!
-      "Christopher Small <chris@pol.is>"
+      (:chris-email env/env)
       (:email params)
       (str "Data export for pol.is conversation pol.is/" zinvite)
       (hiccup-to-plain-text email-hiccup)


Remove middleware file for now to not confuse people

--- src/swiss/middleware.clj
+++ src/swiss/middleware.clj
@@ -1,49 +0,0 @@
-(ns swiss.middleware
-  (:require [swiss.core :refer :all]))
-
-(defn wrap-concatenate-assets [handler {:keys [mode assets] :as options}]
-  "Concatenates the assets and passes the request to the handler.
-
-Takes the handler, and an options map with the keys :mode and :assets.
-Mode should be either :development or :production. :assets should be a
-vector containing a collection of assets to be concatenated and the
-output file name.
-
-Example:
-(wrap-concatenate-assets {:mode :development
-                          :assets [{:concatenate [\"test/assets/test.js\"  \"test/assets/jquery.js\"]
-                                    :output \"test/app-min.js\"}
-                                   {:concatenate [\"test/assets/test.css\" \"test/assets/bootstrap.css\"]
-                                    :output \"test/app-min.css\"}]})"
-  (fn [request]
-    (doall (map concatenate-asset-to-output assets))
-    (handler request)))
-
-(defn wrap-compress-javascripts [handler input-files & [{:keys [disable-optimizations? preserve-semi? nomunge? line-break? charset verbose?] :as options}]]
-  (fn [request]
-    (doall (map #(compress-javascript % % options) input-files))
-    (handler request)))
-
-(defn wrap-compress-stylesheets [handler input-files & [{:keys [line-break? charset verbose?] :as options}]]
-  (fn [request]
-    (doall (map #(compress-stylesheet % % options) input-files))
-    (handler request)))
-
-;; (defn handler [request]
-;;   {:status 200
-;;    :headers {"Content-Type" "text/plain"}
-;;    :body "Hello World!"})
-
-;; (def request
-;;   {:request-method :get
-;;    :uri "/"})
-
-;; ((-> handler
-;;      (wrap-compress-javascripts ["test/app-min.js"])
-;;      (wrap-compress-stylesheets ["test/app-min.css"])
-;;      (wrap-concatenate-assets {:mode :development
-;;                                :assets [{:concatenate ["test/assets/test.js" "test/assets/jquery.js"]
-;;                                          :output "test/app-min.js"}
-;;                                         {:concatenate ["test/assets/test.css" "test/assets/bootstrap.css"]
-;;                                          :output "test/app-min.css"}]}))
-;;  request)


Fix scan/query result macro bug

Fix scan/query result macro bug

--- src/taoensso/faraday.clj
+++ src/taoensso/faraday.clj
@@ -182,7 +182,7 @@
        (merge {:count (.getCount result#)
                :consumed-capacity (.getConsumedCapacity result#)
                :last-prim-kvs (as-map (.getLastEvaluatedKey result#))}
-              meta))))
+              ~meta))))

 (extend-protocol AsMap
   nil                 (as-map [_] nil)


Clean up refactoring code

Clean up refactoring code

--- src/frereth/server/comm.clj
+++ src/frereth/server/comm.clj
@@ -10,9 +10,7 @@
             [ribol.core :refer (raise)]
             [schema.core :as s]
             [taoensso.timbre :as log])
-  (:import [clojure.lang ExceptionInfo]
-           ;; For now, I'm switching this to a plain ol' hashmap
-           #_[com.frereth.common.communication URI]))
+  (:import [clojure.lang ExceptionInfo]))

 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;;; Schema
@@ -32,6 +30,7 @@
    (assoc this :context nil)))

 (defmulti socket-type
+  ;; Q: Why didn't I just use class here?
   #(class %))

 ;;; TODO: These next 3 records needs to collapse into one.
@@ -138,9 +137,6 @@
          address "localhost"}} :- {:ports {s/Keyword s/Int}
                                    s/Any s/Any}]
   (log/debug "Trying to set up a URL based on" (util/pretty config))
-  (comment (comm/strict-map->URI {:protocol protocol
-                                  :address address
-                                  :port port}))
   {:protocol protocol
    :address address
    :port port})
@@ -178,9 +174,6 @@

 (s/defn new-control-url :- comm/URI
   [_]
-  (comment (strict-map->URI {:protocol "inproc"
-                             :address "local"
-                             :port nil}))
   {:protocol "inproc"
    :address "local"
    :port nil})


CMR-1983: Removed columns from tags migration

--- metadata-db-app/src/migrations/019_setup_tags_table.clj
+++ metadata-db-app/src/migrations/019_setup_tags_table.clj
@@ -8,13 +8,10 @@
   concept_id VARCHAR(255) NOT NULL,
   native_id VARCHAR(1030) NOT NULL,
   metadata BLOB NOT NULL,
-  format VARCHAR(255) DEFAULT 'application/edn' NOT NULL,
+  format VARCHAR(255) NOT NULL,
   revision_id INTEGER DEFAULT 1 NOT NULL,
   revision_date TIMESTAMP WITH TIME ZONE DEFAULT SYSTIMESTAMP NOT NULL,
   deleted INTEGER DEFAULT 0 NOT NULL,
-  delete_time TIMESTAMP WITH TIME ZONE,
-  namespace VARCHAR(1024) NOT NULL,
-  value VARCHAR(255) NOT NULL,
   user_id VARCHAR(30)")

 (def ^:private tags-constraint-sql
@@ -34,8 +31,7 @@

 (defn- create-tags-indices
   []
-  (h/sql "CREATE INDEX tags_crdi ON tags (concept_id, revision_id, deleted, delete_time)")
-  (h/sql "CREATE INDEX tags_nsval_i ON tags (namespace, value)"))
+  (h/sql "CREATE INDEX tags_crdi ON tags (concept_id, revision_id, deleted)"))

 (defn- create-tags-sequence
   []


fix #8636

fix #8636

Signed-off-by: yenda <96f164ad4d9b2b0dacf8ebee2bb1eeb3aa69adf1@status.im>

--- src/status_im/ui/components/connectivity/view.cljs
+++ src/status_im/ui/components/connectivity/view.cljs
@@ -147,9 +147,10 @@
             window-width (reagent/atom 0)]
     {:component-will-mount
      (fn []
-       (if (:connected? status-properties)
-         (animation/set-value anim-translate-y neg-connectivity-bar-height)
-         (animation/set-value anim-translate-y 0)))}
+       (when anim-translate-y
+         (if (:connected? status-properties)
+           (animation/set-value anim-translate-y neg-connectivity-bar-height)
+           (animation/set-value anim-translate-y 0))))}
     (let [{:keys [loading-indicator?]} status-properties]
       [react/view {:style     {:align-items :stretch
                                :z-index     1}


Commenting prototype namespaces

Commenting prototype namespaces

--- src/kixi/datastore/schemaextracter.clj
+++ src/kixi/datastore/schemaextracter.clj
@@ -16,6 +16,8 @@
             [clojure.java.io :as io])
   (:import [java.io File]))

+(comment "PROTOTYPE CODE - This is some rough ideas around extracting schemas from data")
+
 (defn metadata->file
   [filestore metadata]
   (let [id (get-in metadata [:id])
@@ -28,7 +30,6 @@
 (defn requires-schema-extraction?
   [msg]
   (and
-  ; (instance? FileMetaData msg)
    (get-in msg [:structural-validation :valid])
    ((complement :schema) msg)))


removed unnecessary non-default call to universal-band-id

--- src/lcmap/aardvark/tile_spec_index.clj
+++ src/lcmap/aardvark/tile_spec_index.clj
@@ -57,7 +57,7 @@
   (map #(conj (seq (str/split % #"/|_")) %) ubids))

 (defn tags->index-payload
-  "Creates a bulk index payload from a sequence of ubids"
+  "Creates a bulk index payload from a sequence of tags"
   [tags]
   (let []
        (apply str
@@ -72,12 +72,9 @@
   (get-in (json/read-str response)["error"]))

 (defn load!
-  "Loads index-payload into index/url"
+  "Loads index payload into index/url"
   ([]
-   (let [payload (tags->index-payload
-                  (ubid->tags
-                   (universal-band-ids
-                    (get-in config [:database :default-keyspace]))))
+   (let [payload (tags->index-payload (ubid->tags (universal-band-ids)))
          bulk-url (bulk-api-url)]
      (log/debug "Loading payload:" payload " into bulk api at:" bulk-url)
      (load! bulk-url payload)))
@@ -87,8 +84,8 @@
                                                         {:body payload})
          errors (or error (get-errors body))]
       (if errors
-        (do (log/debug (str "load-index! errors:" errors)) errors)
-        (do (log/debug (str "load-index! success:" body)) body)))))
+        (do (log/debug (str "load! errors:" errors)) errors)
+        (do (log/debug (str "load! success:" body)) body)))))

 (defn clear!
   "Clears the index specified by index/url"
@@ -99,8 +96,8 @@
    (let [{:keys [status headers body error] :as resp} @(http/delete index-url)
          errors (or error (get-errors body))]
      (if errors
-       (do (log/debug (str "clear-index! errors:" errors)) errors)
-       (do (log/debug (str "clear-index! success:" body)) body)))))
+       (do (log/debug (str "clear! errors:" errors)) errors)
+       (do (log/debug (str "clear! success:" body)) body)))))

 (defn search
   "Submits a supplied query to the ubid index, which should conform to the


packages get are already collected by dda-managed-vm - not needed to be collected again.

--- src/org/domaindrivenarchitecture/pallet/crate/managed_ide.clj
+++ src/org/domaindrivenarchitecture/pallet/crate/managed_ide.clj
@@ -77,7 +77,7 @@

 (s/defmethod dda-crate/dda-settings facility
   [dda-crate partial-effective-config]
-  (package-fact/collect-packages-fact)
+  ;(package-fact/collect-packages-fact)
   )

 (s/defmethod dda-crate/dda-test facility
@@ -92,7 +92,3 @@

 (def with-dda-ide
   (dda-crate/create-server-spec dda-ide-crate))
-
-
-; ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -L 5901:127.0.0.1:5901 ubuntu@35.156.99.16
-; gtkvncviewer


added show-settings argument to make-app

--- src/maqueta/application.clj
+++ src/maqueta/application.clj
@@ -56,10 +56,11 @@
         (keys key-map))))

 (defn make-app
-  [& {:keys [root-node setup-fn update-fn
+  [& {:keys [root-node setup-fn update-fn show-settings
              on-action on-analog on-anim-cycle-done]
       :or {setup-fn no-op
            update-fn no-op
+           show-settings false
            on-action {}
            on-analog {}
            on-anim-cycle-done {}}}]
@@ -91,5 +92,5 @@
           (if-let [callback (on-anim-cycle-done name)]
             (callback this control channel))))
     ;; don't show settings dialog
-    (.setShowSettings false)
+    (.setShowSettings show-settings)
     (.setSettings *app-settings*)))


Use Langohr 2.8.1

Use Langohr 2.8.1

--- clojure/project.clj
+++ clojure/project.clj
@@ -4,4 +4,4 @@
   :license {:name "Eclipse Public License"
             :url "http://www.eclipse.org/legal/epl-v10.html"}
   :dependencies [[org.clojure/clojure     "1.6.0"]
-                 [com.novemberain/langohr "2.7.1"]])
+                 [com.novemberain/langohr "2.8.1"]])


Show warning dialog when closing unsaved files

--- src/net/sekao/nightcode/controller.clj
+++ src/net/sekao/nightcode/controller.clj
@@ -133,6 +133,29 @@

 ; remove

+(defn show-remove-editors-warning! [^Scene scene unsaved-paths]
+  (let [names (map #(-> % io/file .getName) unsaved-paths)
+        message (str "The below files are not saved. Proceed?"
+                  \newline
+                  \newline
+                  (str/join \newline names))
+        dialog (doto (Alert. Alert$AlertType/CONFIRMATION)
+                 (.setTitle "Unsaved Files")
+                 (.setHeaderText message)
+                 (.setGraphic nil)
+                 (.initOwner (.getWindow scene))
+                 (.initModality Modality/WINDOW_MODAL))]
+    (-> dialog .showAndWait (.orElse nil) (= ButtonType/OK))))
+
+(defn should-remove? [^Scene scene ^String path]
+  (let [paths-to-delete (->> @runtime-state :editor-panes keys (filter #(u/parent-path? path %)))
+        get-pane #(get-in @runtime-state [:editor-panes %])
+        get-engine #(-> % get-pane (.lookup "#webview") .getEngine)
+        unsaved? #(-> % get-engine (.executeScript "isClean()") not)
+        unsaved-paths (filter unsaved? paths-to-delete)]
+    (or (empty? unsaved-paths)
+        (show-remove-editors-warning! scene unsaved-paths))))
+
 (defn remove! [^Scene scene]
   (let [{:keys [project-set selection]} @pref-state
         message (if (contains? project-set selection)
@@ -145,7 +168,8 @@
                  (.initOwner (.getWindow scene))
                  (.initModality Modality/WINDOW_MODAL))
         project-tree (.lookup scene "#project_tree")]
-    (when (-> dialog .showAndWait (.orElse nil) (= ButtonType/OK))
+    (when (and (-> dialog .showAndWait (.orElse nil) (= ButtonType/OK))
+               (should-remove? scene selection))
       (p/remove-from-project-tree! pref-state selection)
       (e/remove-editors! selection runtime-state)
       (p/update-project-tree! pref-state project-tree))))
@@ -236,14 +260,15 @@

 (defn close! [^Scene scene]
   (when-let [path (:selection @pref-state)]
-    (let [file (io/file path)
-          new-path (if (.isDirectory file)
-                     path
-                     (.getCanonicalPath (.getParentFile file)))
-          project-tree (.lookup scene "#project_tree")]
-      (e/remove-editors! path runtime-state)
-      (p/update-project-tree-selection! project-tree new-path))
-    (p/remove-project! path runtime-state)))
+    (when (should-remove? scene path)
+      (let [file (io/file path)
+            new-path (if (.isDirectory file)
+                       path
+                       (.getCanonicalPath (.getParentFile file)))
+            project-tree (.lookup scene "#project_tree")]
+        (e/remove-editors! path runtime-state)
+        (p/update-project-tree-selection! project-tree new-path))
+      (p/remove-project! path runtime-state))))

 (defn -onClose [this ^ActionEvent event]
   (-> event .getSource .getScene close!))


CMR-1022: Refactored in response to pull request comments.

--- umm-lib/test/cmr/umm/test/iso_mends/collection.clj
+++ umm-lib/test/cmr/umm/test/iso_mends/collection.clj
@@ -32,7 +32,7 @@
   [instrument]
   (-> instrument
       (assoc :technique nil)
-      (#(assoc % :sensors (sensors->expected-parsed (:sensors %))))))
+      (update-in [:sensors] sensors->expected-parsed)))

 (defn- instruments->expected-parsed
   "Return the expected parsed instruments for the given instruments."


More system-level doc thoughts in core ns.

--- src/org/playasophy/wonderdome/core.clj
+++ src/org/playasophy/wonderdome/core.clj
@@ -1,66 +1,80 @@
 (ns org.playasophy.wonderdome.core
   (:require
-    [com.stuartsierra.component :as component]))
+    [clojure.core.async :as async]
+    [com.stuartsierra.component :as component]
+    [org.playasophy.wonderdome.mode :as mode]))

-#_
-(def wonderdome-state
-  "Example representation of the current wonderdome state."
-  {:state :running
-   :last-autocycle #inst "2014-07-07T11:03:42Z"
-   :playlist [:colorcycle  ; queue is probably better
-              :sweep
-              :twinkle
-              :marqee]
-   :modes {:colorcycle {:speed 0.1
-                        :density 2.6
-                        :offset 0.3208}
-           :sweep {:offset "..."}}
-   ; instead of explicit handlers, use functional composition
-   :event-handlers []})
-
-
-; Core wonderdome system has an event loop which is running on a separate
-; thread at some interval. Main 'update' mechanism gets a dt and sequence of
-; events, should return an updated system.
-
-
-(defn handler
+(defn identity-handler
   "Simple identity handler function."
-  [system dt events]
-  system)
+  [state input]
+  state)

 ; Autocycle handler:
-; Sets :autocycle/at property, when receiving control events resets the timer
+; Sets :autocycle/at property, when receiving control inputs resets the timer
 ; to :autocycle/period seconds in the future. If current time passes the
 ; threshold, rotates to the next mode in the playlist.

 ; Konami code handler:
-; Watches button inputs, keeping track of the last sequence of events; if the
-; last event was too far in the past, clears the buffer. If the sequence matches
+; Watches button inputs, keeping track of the sequence of input events; if the
+; last input was too far in the past, clears the buffer. If the sequence matches
 ; the code, changes the current mode to the easter-egg mode. Probably lets the
 ; autocycle or manual mode change handle switching out of it.

 ; Mode handler:
-; Passes the dt and events to the current mode via mode/update and returns the
+; Passes the dt and inputs to the current mode via mode/update and returns the
 ; system with an updated mode state.
+
+
+#_
+(defn render
+  "Takes the current mode, maps it over the pixels in the layout, and calls
+  set-colors! on the display. Modes should not have any mutable internal
+  state."
+  [mode layout display]
+  (set-colors! display
+    (map (partial map (partial mode/render mode))
+         layout)))

 (defn initialize
   [{:keys [layout display modes]
-    ; TODO: defaults?
+    :defaults {:timer-ms 30}
     :as config}]
   (component/system-map
+    ; Layout is a pre-computed collection of strips of pixel coordinates.
     :layout layout
-    ; display is responsible for running its own rendering thread.
-    :display (component/using display [:layout])
-    ; input event queue?
-    ; http server?
+
+    ; Display is responsible for running its own rendering thread if necessary.
+    ; Processing does, pixel-pusher doesn't.
+    :display display
+
+    ; Ideally use separate channels merged together, because we're probably only
+    ; ever interested in the last 1-2 audio frames, but never want to lose
+    ; button presses.
+    #_ :input-channel #_ (async/chan 5)
+
+    ; Input sources run whatever threads are necessary and stick input events into
+    ; the channel/queue for consumption by the system.
+    ; TODO: http server
+    ; TODO: usb input
+    ; TODO: audio parser
+    ; TODO: timer?
+
+    ; Handler functions recieve the current state of the system, a time delta,
+    ; and a collection of input events, and return the updated system state.
     #_
-    (component/using
-      (example-component config-options)
-      {:database  :db
-       :scheduler :scheduler})))
+    :handler
+
+    ; The current system state is wrapped in an agent, which continuously sends
+    ; itself an update function. The update pulls input off the queue, calls the
+    ; handler to update the state, and recurs with a new last-updated time.
+    ; Configuration changes (such as pausing, changing the mode playlist, etc)
+    ; can be accomplished by sending assoc's which alter the necessary
+    ; configuration state.
+    #_
+    :state
+    ))


remove deprecated admin functions

remove deprecated admin functions

--- src/pc/admin/db.clj
+++ src/pc/admin/db.clj
@@ -33,18 +33,6 @@
                                                  0)))
                         doc-ids))))

-(defn populate-user-info-from-sub []
-  (let [db (pcd/default-db)]
-    (doseq [[eid] (d/q '{:find [?t]
-                         :where [[?t :google-account/sub]]}
-                       db)
-            :let [cust (d/entity db eid)]]
-      (try+
-        (println "adding" (:cust/email cust))
-        (pc.auth/update-user-from-sub cust)
-        (catch :status t
-          (println "error updating" (d/touch cust)))))))
-
 (defn copy-document
   "Creates a copy of the document, without any of the document's history"
   [db doc]
@@ -55,15 +43,3 @@
                                           :document/id (:db/id new-doc)))
                                 layers))
     new-doc))
-
-(defn populate-first-newsletter [emails]
-  (let [db (pcd/default-db)]
-    (doseq [[eid] (d/q '{:find [?t]
-                         :in [$ ?emails]
-                         :where [[?t :cust/email ?e]
-                                 [(contains? ?emails ?e)]]}
-                       db
-                       (set emails))
-            :let [cust (d/entity db eid)]]
-      (println "populating" (:cust/email cust))
-      (mixpanel/engage (:cust/uuid cust) {:$set {:sent-first-newsletter true}}))))


Break skin initialization into separate function

--- src/play_clj/ui.clj
+++ src/play_clj/ui.clj
@@ -35,17 +35,20 @@
                   (str (u/key->pascal k) "Style."))
      ~@options))

+(defn skin*
+  [path]
+  (or (u/load-asset path Skin)
+      (Skin. (if (string? path)
+               (.internal ^Files (Gdx/files) path)
+               path))))
+
 (defmacro skin
   "Returns a [Skin](http://libgdx.badlogicgames.com/nightlies/docs/api/com/badlogic/gdx/scenes/scene2d/ui/Skin.html)
 based on the file at `path`.

     (skin \"uiskin.json\")"
   [path & options]
-  `(u/calls! ^Skin (or (u/load-asset ~path Skin)
-                       (Skin. (if (string? ~path)
-                                (.internal ^Files (Gdx/files) ~path)
-                                ~path)))
-             ~@options))
+  `(u/calls! ^Skin (skin* ~path) ~@options))

 (defmacro skin!
   "Calls a single method on a `skin`."


Message queue: add note re non-polling impl

--- src/taoensso/carmine/message_queue.clj
+++ src/taoensso/carmine/message_queue.clj
@@ -23,12 +23,8 @@
             [taoensso.timbre  :as timbre]
             [taoensso.carmine :as car :refer (wcar)]))

-;;;; Ideas to avoid polling
-;; 1. Lua brpoplpush - doesn't currently exist, unlikely to in future.
-;; 2. Redis 2.8+ Pub/Sub keyspace notifications? Maybe with reliable Pub/Sub?
-;; 3. Possible 2-circle (or alternative) designs?
-;; 4. Leave it as is since we'd want a polling worker anyway for gc, lock
-;;    checks, etc.
+;; TODO Consider adding a size-1 blocking list for await/notify style
+;; backoff? Would allow us to eliminate non-maintenance polling.

 ;;;; Utils


ReactDOMServer doesn't seem to exist anywhere

So, don't look for it.

--- src/reagent/dom/server.cljs
+++ src/reagent/dom/server.cljs
@@ -5,9 +5,7 @@
             [reagent.interop :refer-macros [.' .!]]))

 ;; TODO: Where the hell is ReactDOMServer?
-(defonce react-dom-server (or (and (exists? js/ReactDOMServer)
-                                   js/ReactDOMServer)
-                              (and (exists? js/require)
+(defonce react-dom-server (or (and (exists? js/require)
                                    (js/require "react-dom/server"))
                               util/react))
 (assert react-dom-server)


Added helmsman resources def for static assets in timber.

--- src/timber/core.clj
+++ src/timber/core.clj
@@ -2,6 +2,10 @@
   (:require [net.cgrand.enlive-html :as html]
             [timber.selectors :as select]
             [timber.assets :as asset]))
+
+(def helmsman-assets
+  [^{:id :timber/assets}
+   [:resources "/timber/public" {:root "/timber/public"}]])

 (html/defsnippet main-menu
   "timber/templates/base.html"


adding start again since create starts up as well

--- test/celestial/integration/vsphere.clj
+++ test/celestial/integration/vsphere.clj
@@ -17,5 +17,7 @@
       (guest-status (get-in redis-vsphere-spec [:machine :hostname]))  => :running
       (.stop vm)
       (.status vm)  => :stopped
+      (.start vm)
+      (.stop vm)
       (.delete vm)
       )))


Add a test for #4707 [ci drivers]

Add a test for #4707 [ci drivers]

--- test/metabase/driver/bigquery_test.clj
+++ test/metabase/driver/bigquery_test.clj
@@ -2,6 +2,7 @@
   (:require metabase.driver.bigquery
             [metabase.models.database :as database]
             [metabase.query-processor :as qp]
+            [metabase.query-processor-test :as qptest]
             [metabase.test.data :as data]
             (metabase.test.data [datasets :refer [expect-with-engine]]
                                 [interface :refer [def-database-definition]])))
@@ -29,3 +30,14 @@
                                          :type     :native
                                          :database (data/id)}))
                [:cols :columns]))
+
+;; make sure that the bigquery driver can handle named columns with characters that aren't allowed in BQ itself
+(expect-with-engine :bigquery
+  {:rows    [[113]]
+   :columns ["User_ID_Plus_Venue_ID"]}
+  (qptest/rows+column-names (qp/process-query {:database (data/id)
+                                               :type     "query"
+                                               :query    {:source_table (data/id :checkins)
+                                                          :aggregation  [["named" ["max" ["+" ["field-id" (data/id :checkins :user_id)]
+                                                                                ["field-id" (data/id :checkins :venue_id)]]]
+                                                                                "User ID Plus Venue ID"]]}})))


Gen-test min-peers allocations Make these generative tests test against all job schedulers

--- test/onyx/log/generative_peer_join.clj
+++ test/onyx/log/generative_peer_join.clj
@@ -254,6 +254,7 @@

 (def job-max-peers
   {:workflow [[:a :b] [:b :c]]
+   :percentage 100
    :catalog [{:onyx/name :a
               :onyx/plugin :onyx.plugin.core-async/input
               :onyx/type :input
@@ -277,27 +278,82 @@
               :onyx/doc "Writes segments to a core.async channel"}]
    :task-scheduler :onyx.task-scheduler/balanced})

-(deftest balanced-all-max-peers-allocations
-  (checking
-    "Checking balanced allocation causes peers to be evenly split"
-    (times 50)
-    [{:keys [replica log peer-choices]}
-     (log-gen/apply-entries-gen
-       (gen/return
-         {:replica {:job-scheduler :onyx.job-scheduler/balanced
-                    :messaging {:onyx.messaging/impl :dummy-messenger}}
-          :message-id 0
-          :entries (assoc (log-gen/generate-join-queues (log-gen/generate-peer-ids 6))
-                          :job-1 {:queue [(api/create-submit-job-entry
-                                            job-max-peers-id
-                                            peer-config
-                                            job-max-peers
-                                            (planning/discover-tasks (:catalog job-max-peers) (:workflow job-max-peers)))]})
-          :log []
-          :peer-choices []}))]
+(deftest all-max-peers-allocations-all-schedulers
+  (checking
+    "Checking job where all tasks have max-peers set"
+    (times 50)
+    [{:keys [replica log peer-choices]}
+     (log-gen/apply-entries-gen
+       (gen/fmap
+         (fn [scheduler]
+           {:replica {:job-scheduler scheduler
+                      :messaging {:onyx.messaging/impl :dummy-messenger}}
+            :message-id 0
+            :entries (assoc (log-gen/generate-join-queues (log-gen/generate-peer-ids 6))
+                            :job-1 {:queue [(api/create-submit-job-entry
+                                              job-max-peers-id
+                                              peer-config
+                                              job-max-peers
+                                              (planning/discover-tasks (:catalog job-max-peers) (:workflow job-max-peers)))]})
+            :log []
+            :peer-choices []})
+         (gen/elements [:onyx.job-scheduler/balanced :onyx.job-scheduler/greedy :onyx.job-scheduler/percentage])))]
     (standard-invariants replica)
     (is (= (sort [:active :active :active :idle :idle :idle]) (sort (vals (:peer-state replica)))))
-    (is (= [1 1 1] (map count (vals (get (:allocations replica) job-max-peers-id)))))))
+    (is (= (sort [1 1 1]) (sort (map count (vals (get (:allocations replica) job-max-peers-id))))))))
+
+
+(def job-min-peers-id #uuid "f55c14f0-a847-42eb-81bb-0c0390a88608")
+
+(def job-min-peers
+  {:workflow [[:a :b] [:b :c]]
+   :percentage 100
+   :catalog [{:onyx/name :a
+              :onyx/plugin :onyx.plugin.core-async/input
+              :onyx/type :input
+              :onyx/min-peers 2
+              :onyx/medium :core.async
+              :onyx/batch-size 20
+              :onyx/doc "Reads segments from a core.async channel"}
+
+             {:onyx/name :b
+              :onyx/fn :mock/fn
+              :onyx/min-peers 2
+              :onyx/type :function
+              :onyx/batch-size 20}
+
+             {:onyx/name :c
+              :onyx/plugin :onyx.plugin.core-async/output
+              :onyx/type :output
+              :onyx/min-peers 2
+              :onyx/medium :core.async
+              :onyx/batch-size 20
+              :onyx/doc "Writes segments to a core.async channel"}]
+   :task-scheduler :onyx.task-scheduler/balanced})
+
+(deftest all-min-peers-allocations-all-schedulers
+  (checking
+    "Checking job where all tasks have min-peers set"
+    (times 50)
+    [{:keys [replica log peer-choices]}
+     (log-gen/apply-entries-gen
+       (gen/fmap
+         (fn [scheduler]
+           {:replica {:job-scheduler scheduler
+                      :messaging {:onyx.messaging/impl :dummy-messenger}}
+            :message-id 0
+            :entries (assoc (log-gen/generate-join-queues (log-gen/generate-peer-ids 6))
+                            :job-1 {:queue [(api/create-submit-job-entry
+                                              job-min-peers-id
+                                              peer-config
+                                              job-min-peers
+                                              (planning/discover-tasks (:catalog job-min-peers) (:workflow job-min-peers)))]})
+            :log []
+            :peer-choices []})
+         (gen/elements [:onyx.job-scheduler/balanced :onyx.job-scheduler/greedy :onyx.job-scheduler/percentage])))]
+    (standard-invariants replica)
+    (is (= (sort [2 2 2]) (sort (map count (vals (get (:allocations replica) job-min-peers-id))))))
+    (is (= (sort [:active :active :active :active :active :active]) (sort (vals (:peer-state replica)))))))

 (deftest job-percentages-balance
   (checking


Revert "(maint) Enable debugging output on unstable test"

This reverts commit b6a93862b9c0e9e8b2cef00ffa83eb985ae86410.

--- test/puppetlabs/pcp/messaging_test.clj
+++ test/puppetlabs/pcp/messaging_test.clj
@@ -160,17 +160,16 @@

 (deftest connect-to-a-down-up-down-up-broker-test
   (with-open [client (connect-client "client01" (constantly true))]
-    (with-log-level "puppetlabs.pcp.client" :debug
-      (is (not (client/connected? client)) "Should not be connected yet")
-      (with-app-with-config app broker-services broker-config
-        (client/wait-for-connection client (* 40 1000))
-        (is (client/connected? client) "Should now be connected"))
-      ;; Allow time for the websocket connection to close, but not enough to attempt reconnecting
-      (Thread/sleep 5)
-      (is (not (client/connected? client)) "Should be disconnected")
-      (with-app-with-config app broker-services broker-config
-        (client/wait-for-connection client (* 40 1000))
-        (is (client/connected? client) "Should be reconnected")))))
+    (is (not (client/connected? client)) "Should not be connected yet")
+    (with-app-with-config app broker-services broker-config
+      (client/wait-for-connection client (* 40 1000))
+      (is (client/connected? client) "Should now be connected"))
+    ;; Allow time for the websocket connection to close, but not enough to attempt reconnecting
+    (Thread/sleep 5)
+    (is (not (client/connected? client)) "Should be disconnected")
+    (with-app-with-config app broker-services broker-config
+      (client/wait-for-connection client (* 40 1000))
+      (is (client/connected? client) "Should be reconnected"))))

 (deftest association-checkers-test
   (with-app-with-config app broker-services broker-config


Improve failure logging when loading config

--- src/yetibot/core/config.clj
+++ src/yetibot/core/config.clj
@@ -17,15 +17,16 @@
 (defn- load-edn [path]
   (try
     (edn/read-string (slurp path))
-    (catch Exception _
-      (warn "Config is missing")
+    (catch Exception e
+      (warn "Failed loading config: " e)
       {:yetibot {}})))

 (defn reload-config []
   (let [new-conf (load-edn config-path)]
-    (info "☐ Loading config at" config-path)
+    (info "☐ Try loading config at" config-path)
     (reset! config new-conf)
-    (info "☑ Config loaded")
+    (when-not (empty? (:yetibot @config))
+      (info "☑ Config loaded"))
     new-conf))

 (defn get-config