#clojure-spec

generated UTC: 2023-11-22 19:43
latest data: https://clojurians-log.clojureverse.org/clojure-spec/2023-11-16
messages: 41621
pro tips:
* Double click on text to filter by it. (doubleclick + cmd-f for extra points).
* Click on date to keep day visible regardless of filter.
* Click on time to keep hour visible regardless of filter.
#2016-05-2512:06sveriIs there something in spec that is comparable to string? but for a boolean value?#2016-05-2512:06sveriexcept #(instance? Boolean %)#2016-05-2512:10richhickey@sveri: not yet. Obviously there are several more useful predicates that could go into core. Considering which ones now#2016-05-2512:10sveri@richhickey: Yea, it would be nice to be consistent, much less work for the brain then šŸ™‚#2016-05-2512:19moxaj
(spec/def ::foo (spec/cat :kw keyword? :int integer?))
(spec/def ::bar (spec/coll-of ::foo []))
(spec/explain ::bar [[:a 10] [:b 20] [:c "30"]])
;; => val: [[:a 10] [:b 20] [:c "30"]] fails predicate: (coll-checker :spec/foo)
Why doesn't the error reporting go into detail (like: val "30" fails predicate: integer?) Is this intentional?
#2016-05-2512:44sveriHm, is it ok to use ::spec/name for my own specs? Also I have not found the source for ::spec/string, neither by searching github (which is a bit harder, because it ignores "::", nor by looking at the api.#2016-05-2512:44sveriAlso, is it correct that ::spec/name has string? as predicate?#2016-05-2512:59kingoftheknoll@alexmiller: About a week ago you hinted on Reddit at improved error messages for Clojure. Don’t think anyone’s said that here but it seems like we just got a peek into how that might be accomplished if all of Clojure.core implements specs.#2016-05-2514:08mjhamrickWhat's the best way to have fdefs be checked during lein test? Instrument all works at the repl, but seems to not during test. #2016-05-2514:10anmonteiro@mjhamrick: what I’ve done is (clojure.spec.test/check-var #’fn-speced-with-fdef)#2016-05-2514:14mjhamrick@anmonteiro: are you just putting that in each test? #2016-05-2514:15anmonteiroso far, yes#2016-05-2514:15anmonteirothe project where I tried it was pretty small#2016-05-2514:24sveri@mjhamrick: I did put (s/instrument-all) at the top of each test namespace#2016-05-2514:35Alex Miller (Clojure team)@kingoftheknoll: yes, I have done additional work in this area#2016-05-2514:44Alex Miller (Clojure team)@sveri ::spec/name and ::spec/string don't exist - what is that from?#2016-05-2514:46sveri@alexmiller: Thats why I have not found it in the code. Not sure where exactly it came from, its just that cursive offered it via code completion. Maybe I typed it in a different namespace or whatever. Anyway, good to know.#2016-05-2514:52michaeldrogalis@richhickey @alexmiller: I have another proposed enhancement to explain-data. Several predicates, such as keyword?, or integer? are programmatically easily recognizable via :pred for non-conforming values. It's a straightforward equality check (e.g. (= 'keyword? (:pred error-map))). Other predicates are, to my understanding, more difficult to recognize. Take s/keys and s/tuple for example. In the former, keys yields a predicate when a required key is missing. While the predicate is somewhat understandable when you eye ball it, picking it out programmatically has been error-prone. The same goes for s/tuple when the number of arguments is incorrect.#2016-05-2514:53Alex Miller (Clojure team)an example would help make this specific#2016-05-2514:53michaeldrogalisThose functions have privileged access to which predicates are failing. It might be helpful to provide named predicates in those cases so that consumers of explain-data could be more intelligent. See - https://gist.github.com/MichaelDrogalis/016ac62cf3f1e899fb89fd79f2de2277#2016-05-2514:54Alex Miller (Clojure team)thx for the example! :)#2016-05-2514:54Alex Miller (Clojure team)oh the contains stuff that's inside keys#2016-05-2514:55Alex Miller (Clojure team)I'll have to defer that to Rich, I think he's not around atm#2016-05-2514:55moxajIt seems to me that inner conformed values are lost when using coll-of or map-of. Example:
(spec/conform (spec/coll-of (spec/and integer?
                                      (spec/conformer (constantly nil)))
                            [])
              [1 2 3])
returns [1 2 3] instead of [nil nil nil]. Meanwhile, cat and keys preserves them. How come?
#2016-05-2514:56michaeldrogalis@alexmiller: Sure šŸ™‚ Ill be back later.#2016-05-2514:57Alex Miller (Clojure team)@moxaj Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this#2016-05-2514:57moxaj@alexmiller alright, thanks!#2016-05-2515:41sveriI am converting a lib from schema to spec and in schema I was able to inline define the return values / args. Is there a way to do that in spec too? The alternative in this case is a lot more code.#2016-05-2515:44Alex Miller (Clojure team)for map keys, no - that's the whole point of the map / attributes split#2016-05-2515:44Alex Miller (Clojure team)the benefit is that you are creating named reusable semantics for ::entityname, ::ns, etc etc#2016-05-2515:44Alex Miller (Clojure team)so you can use those elsewhere too#2016-05-2515:45sveri@alexmiller: Yea, I understand the benefits, just wanted to make sure I did not miss anything#2016-05-2515:48mpenetIs it possible to hook ones own error message if a predicate fails? Ex we have schemas that takes a string that should be a valid parse of our internal query language, and if fails should return a nicely structured error with line/col num and a human readable message. With prismatic Schema it is possible to do with our own schema type extending a protocol for explain/spec.#2016-05-2515:51mpenetCorrect me if I am wrong, but I think so far I can only get a generic "predicate foo? failed" type of message#2016-05-2515:55Alex Miller (Clojure team)there is not currently a way to provide a custom error for a failing predicate#2016-05-2515:56Alex Miller (Clojure team)you can use explain-data to detect problems and produce a custom error message#2016-05-2515:56Alex Miller (Clojure team)but @richhickey can comment on whether that might be something we could do#2016-05-2515:57mpenetthat's what I suspected. Something like ex-info for specs could be useful#2016-05-2516:05Alex Miller (Clojure team)why not explain-data for that?#2016-05-2516:05Alex Miller (Clojure team)or maybe I misunderstand the suggestion#2016-05-2516:05mpenetI actually missed it, that could work yes#2016-05-2516:06Alex Miller (Clojure team)that's what instrument does on a failing function spec#2016-05-2516:15mpenethmm I am not sure that's really equivalent. I would have to wrap all schemas that contains these values with a special validation fn I think. I would like to have my own datastructure returned by explain-data (or something else) when the predicate fails. I am not sure that's doable with the current approach where predicates return just a potentially truthy value.#2016-05-2516:16mpenetbut I could be wrong. I played for 20min with specs so far.#2016-05-2516:24mpenetthis might sound like a small concern, but in a larger context it's very useful: we have a whole api around the manipulation of data containing such queries, being able to use the same schema type and framework and not wrap the whole thing because of a single type in a potentially nested Schema is a nice feature, in our case it also integrates for free with frameworks that just follow this Schema format (rest server, swagger etc).#2016-05-2517:38eraserhdFirst piece of feedback is that clojure.spec makes me really want ā€œboolean?"#2016-05-2517:49sveri@eraserhd: @richhickey said they consider adding this and some more core types (I felt the need for it too :-))#2016-05-2517:52eraserhd@sveri yay šŸ™‚#2016-05-2517:52eraserhdI’m still playing with it, but the second thing I’m working through is the repetition, especially for function arguments.#2016-05-2518:16Alex Miller (Clojure team)@eraserhd: we have boolean? in almost every example namespace we wrote. so, agreed. :) definitely near the top of the wanted-predicates list.#2016-05-2518:16eraserhd@alexmiller: cool šŸ™‚#2016-05-2518:18eraserhdAnother interesting problem I encountered: I used a set for boolean: #{true false}. But sets aren’t treated differently in this context, although I might expect that they are. Namely #(#{true false} false) ;=> false. So having a set in a spec which contains nil or false is awkward.#2016-05-2518:19Alex Miller (Clojure team)yeah, don't do that :)#2016-05-2518:19Alex Miller (Clojure team)it is kind of an interesting wrinkle though that the obvious way to write boolean? is with #(instance? Boolean %)#2016-05-2518:20eraserhdIt might at least deserve a call-out in docs.#2016-05-2518:21eraserhdAlso, a lot of things seem to leak their internal implementations wrt error messages.#2016-05-2518:23Alex Miller (Clojure team)but Clojure uses (only) canonical boolean values#2016-05-2518:23Alex Miller (Clojure team)@eraserhd: prob better for a mention in the ref doc (which isn't there yet)#2016-05-2518:24Alex Miller (Clojure team)@eraserhd: example re errors?#2016-05-2518:24eraserhd@alexmiller: I encountered a couple. Just a minute and I’ll paste.#2016-05-2518:55sveriAnyone else seeing problems with reloading test namespaces in the REPL (cursive)? I just commented out some tests, reloaded the test ns in the REPL, hit run-tests, but still, all tests were run. I noticed something similar with test-refresh not recognizing commented fdefs. I cannot point at anything right now, but, there seems to be something broken#2016-05-2519:01kovasb@cfleming: are there gonna be slick Cursive toolips for providing associated specs when editing functions arguments?#2016-05-2519:02kovasbI imagine I'm editing the arg to some function that takes a big map, and as I'm filling in the components of the map its telling me what each piece should conform to#2016-05-2519:09arohnerWhat is the spec for ā€˜map? i.e. in core.typed, I’d write something like (t/ann map [ [ X -> Y] [X] -> [Y] ). i.e. it takes a collection of X, and a function of X->Y, and returns a collection of Y. Is there a way to do that in spec?#2016-05-2519:25bbrinckI’m missing something related to recursive specs. I want to define a vaguely hiccup-like structure where the first element is the name, followed by some attributes, followed by a vector of children. What am I doing wrong here? https://gist.github.com/bhb/6cfcb3b38757442aec4ba5db46148699#2016-05-2519:38hiredmanif you add another (s/spec ....) around the ::tag I get a success, but I am not sure why#2016-05-2519:42hiredmanI guess you are in the regex context that matches the outer vector, so you need another s/spec to create a new regex context for the list of child vectors, then you need another s/spec to get out of the regex context of the vector of children and specify an individual child#2016-05-2519:48anmonteiroI’d also be interested to see the spec for clojure.core/map#2016-05-2519:48bbrinck@hiredman: Ah, thanks! I had to read that a few times, but now it makes sense šŸ™‚#2016-05-2519:56sveriOk, I dont understand how to use instrument. My guess was, that if I put (s/instrument-all) into a namespace all functions are instrumented whenever I load the file into the REPL. But thats not the case, instead I have to execute that function everytime after I loaded a namespace.#2016-05-2519:56sveriAlso, it works when I put it to the end of a ns#2016-05-2520:02dryewoHi all, first I want to thank Rich&Co for this awesome innovation :+1: And I have a question:
(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
                                        :very-small #(< % 1))
                                  even?))
(s/valid? ::even-big-or-small 10000)
;; throws CompilerException java.lang.IllegalArgumentException: Argument must be an integer: [:very-big 10000]
what am I doing wrong?
#2016-05-2520:07dryewojust noticed, works fine the other way around:
(s/def ::even-big-or-small (s/and even?
                                  (s/or :very-big #(> % 1000)
                                        :very-small #(< % 1))))
#2016-05-2520:12anmonteiro@dryewo: in the first case, it’ll conform to :very-big and output [:very-big 1000]#2016-05-2520:13anmonteiro
(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
                                               :very-small #(< % 1))
                                        #(even? (second %))))

#2016-05-2520:13anmonteirothis works#2016-05-2520:13anmonteiroeven? gets [:very-big 1000] so we are interested in the second element#2016-05-2520:13anmonteirothat’s the explanation, but I’m also unsure if it’s expected#2016-05-2520:14dryewoyes, I also got this idea of what’s happening, but is it the correct behavior?#2016-05-2520:15dryewoshouldn’t s/and be order agnostic?#2016-05-2520:16anmonteiroI’ve read somewhere that it’s not, by design#2016-05-2520:16anmonteirobut I can’t recall the reason#2016-05-2520:18dryewoI’m reading the guide: http://clojure.org/guides/spec maybe I should first read it to the end before asking more questions šŸ™‚#2016-05-2520:25kenny@dryewo: Look into https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/conformer#2016-05-2520:26dryewothanks, there is even a section about it in the end of the guide, I’ll look into it#2016-05-2520:35kennyIs there a pattern for whether you spec functions or use spec for validation (pre/post-conditions)? There doesn't seem to be a reason for pre/post-conditions if you spec a function.#2016-05-2520:39sveriI dont understand that, I have the code in the following snippet, and it fails with:
(remove-autoinc-columns [{:foo "some_t"}])
ExceptionInfo Call to #'de.sveri.clospcrud.s-play/remove-autoinc-columns did not conform to spec:
At: [:args] val: ([{:foo "some_t"}]) fails predicate: (cat :cols :de.sveri.clospcrud.s-play/columns),  Extra input
:clojure.spec/args  ([{:foo "some_t"}])
  clojure.core/ex-info (core.clj:4617)
#2016-05-2521:10anmonteiro@sveri: you need to wrap ::column in s/spec#2016-05-2521:10anmonteirolike this: (s/def ::columns (s/cat :col (s/* (s/spec ::column))))#2016-05-2522:34Alex Miller (Clojure team)@arohner: @anmonteiro here's a spec for map (assumes a seqable? predicate like the one in core.incubator)#2016-05-2522:34Alex Miller (Clojure team)
(s/fdef clojure.core/map
  :args (s/cat :f ifn?
               :colls (s/* seqable?))
  :ret (s/or :seq seqable? :transducer ifn?))
#2016-05-2522:35anmonteirosimpler than I imagined#2016-05-2522:35Alex Miller (Clojure team)I've left off :fn because I'm lazy but you could also verify some additional things#2016-05-2522:36Alex Miller (Clojure team)like :f / :transducer or :colls / :seq are the valid combos and even things like whether the :ret cardinality is minimum of the input colls cardinality#2016-05-2522:37Alex Miller (Clojure team)but that's enough to detect things like (map inc 100)#2016-05-2522:37Alex Miller (Clojure team)which is the kind of thing that normally leads to IllegalArgumentException Don't know how to create ISeq from: java.lang.Long#2016-05-2522:38Alex Miller (Clojure team)which, if it's occurring in a nested context can be pretty confusing#2016-05-2522:40tylerAnyone given the test namespace a try yet? I’m trying to figure out how check-var works. It appears to be hanging half the time I try it and passing tests the other half.#2016-05-2522:45arohner@alexmiller: is it possible to check that f’s input matches the coll?#2016-05-2522:45arohnerright now I don’t see a way to do that#2016-05-2522:46arohnerbecause there doesnt’ seem to be a way to ā€˜compare’ specs#2016-05-2522:46Alex Miller (Clojure team)you would have to use a custom predicate to do so#2016-05-2522:47Alex Miller (Clojure team)the :args spec could be (s/and <current> #(custom predicate on the args))#2016-05-2522:48Alex Miller (Clojure team)I guess you might need to describe :f with an fspec to get the right parts to work with#2016-05-2522:48Alex Miller (Clojure team)I haven't done it!#2016-05-2522:57tylerI’ve created a gist with my attempt to try and use check-var it is returning nil, is that expected behavior? https://gist.github.com/tvanhens/28b7f744d799910750d8ae3942680997#2016-05-2523:11hiredmando you have the test.check library on the classpath?#2016-05-2523:34tylerYup gen/generate works#2016-05-2523:46stuarthalloway@tyler: there is a bug there, I hit it too#2016-05-2523:47stuarthallowayI think it is fixed on master#2016-05-2523:52tylerAh awesome thanks#2016-05-2523:53richhickeyyah#2016-05-2523:57eraserhd@alexmiller An example wrt errors: (spec/explain (spec/tuple number?) []) gives an error about #(clojure.core/= (clojure.core/count %) 2).#2016-05-2523:57eraserhds/2/1/#2016-05-2523:59richhickey@eraserhd: "val: [] fails spec: _ predicate: (clojure.core/= (clojure.core/count %) 1)ā€ says the collection should have count == 1#2016-05-2600:00richhickeytuple is not a parameterized coll (see coll-of for that), (tuple number?) says vector of exactly one number#2016-05-2600:01richhickeytuple is for short tuples of heterogeneous types each of which will be listed#2016-05-2600:01eraserhd@richhickey Oh hi! My comment is about the message, not that it should pass…#2016-05-2600:01richhickeye.g. (tuple string? number?)#2016-05-2600:01eraserhdI was saying that it reveals implementation detail of spec/tuple.#2016-05-2600:02richhickeyvs what?#2016-05-2600:04eraserhdI was reading the rationale document as saying that we want to capture the predicates the user supplied, so I was hoping for something like: val: [] fails spec: (tuple number?).#2016-05-2600:04richhickeythat doesn’t tell you much#2016-05-2600:04eraserhdHrmm, this is true.#2016-05-2600:05richhickeyand the interior preds can be arbitrarily complex, will walk down#2016-05-2600:06richhickeymuch better than ā€œyou failed the top"#2016-05-2600:07eraserhdHrmm. So my (clearly not thought through) expectation would be that count would fail at tuple, but sub-types would fail more specifically. That’s not an idea I can make coherent, though.#2016-05-2600:10eraserhd(Anyhoo, super excited about clojure.spec’s new ideas, cf prismatic)#2016-05-2600:13richhickeyhave fun!#2016-05-2600:38shaunxcodeis it possible create a spec for a map like {[300 200 54] :status/open} e.g. map keyed by vector of integers#2016-05-2600:42tyler@shaunxcode yes I believe so using map-of#2016-05-2600:44tyler
(s/map-of (s/coll-of integer? []) #{:status/open})
#2016-05-2601:47cfleming@kovasb: Sure, sounds doable#2016-05-2606:41borkdudeShould I include test.check as an explicit dependency when using spec? I got: FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:456)#2016-05-2606:42borkdudewhen instrumenting the adder example from the guide https://clojure.org/guides/spec (cc @alexmiller )#2016-05-2607:15sveri@borkdude: Yes, you need it for most cases if you want to use spec in your tests. There was only one function that works without test.check (I think it was spec/gen, but not sure). I have read it somewhere in the official docs in the last days.#2016-05-2607:18sveriAnyone else got an idea why this snippet fails?#2016-05-2607:19sveri@anmonteiro: wrapping it in (s/spec... doesnt fix it unfortunately#2016-05-2608:50sveriAlso, what works is:
(s/conform ::columns [{:foo "some_t"}])
=> {:col [{:foo "some_t"}]}
which is strange, cause all I do next is to put it into the argument definition of my function
#2016-05-2609:07sveriCan someone reproduce this?#2016-05-2609:52dominicmIs there space in spec, for a specification to take parameters via lexical scope? This may be out of scope for spec. For example, on http request, I take a snapshot of my database, and I want to check if a value is in that database. But I want to have my predicates registered via s/def so that I can take advantage of :via and such. Use case is checking if an email is taken during user registration.#2016-05-2612:10anmonteiroLibrary authors might want to add specs to their existing projects but also keep supporting earlier versions of Clojure#2016-05-2612:10anmonteiroIs this a use case that has been thought about, or isn’t there any interest in supporting such use case?#2016-05-2612:12anmonteiroor are we just supposed to solve it through versioning?#2016-05-2612:32Alex Miller (Clojure team)Specs could be provided in an adjunct library too#2016-05-2612:34Alex Miller (Clojure team)Ultimately, things must move forward to get the benefits of new features as we did with records, protocols, transducers, reader conditionals, etc etc#2016-05-2612:34anmonteiroAgreed wrt. 2nd point#2016-05-2612:35anmonteiroand yea, the optional library add-on could also be a possibility for initial compatibility, hadn’t thought about that#2016-05-2612:35anmonteirothanks#2016-05-2612:35bronsa@anmonteiro: having spec in both clojure.core + an external library could cause namespace conflicts#2016-05-2612:36Alex Miller (Clojure team)They can also be in a separate namespace in your library jar that's just not loaded (unless you're on 1.9)#2016-05-2612:36anmonteiro@bronsa: I understand, but I didn’t mean that.#2016-05-2614:10Alex Miller (Clojure team)1.9.0-alpha3 is now out with several bug fixes and improvements for spec https://groups.google.com/forum/#!topic/clojure/WxT9kPIwlYI#2016-05-2614:11Alex Miller (Clojure team)The guide is now out of date but I will have it updated shortly#2016-05-2614:43dominicm@alexmiller: Is there much of a story for what I mentioned previously? https://clojurians.slack.com/archives/clojure_spec/p1464256334000558 Trying to decide how to approach the problem of generating/validating forms.#2016-05-2615:07Alex Miller (Clojure team)@dominicm: you could register (at runtime) an s/def that partialed/closed-over something known at runtime like a database snapshot. This implies that you are using spec to do runtime data validation. Whether or not this is a good idea, I'm not sure. :)#2016-05-2615:14dominicm@alexmiller: Yeah, I did wonder if dynamic vars might be a solution, and a bad idea. I'm definitely thinking in the domain of variable user-input, as opposed to contracts during development. I like the idea of spec as a ubiquitous format that the clojure community uses to describe data. Some data just can't be controlled (user input). Maybe this is way out of scope for spec though, and you guys think that should be a different library altogether. There's definitely a lot of overlap in that space (email still needs to be a string, over 5 chars, match a regex, before it is db checked). Nothing says a validation library can't register things for you of course. But I try to keep core at the top, not other libraries.#2016-05-2616:01gtrakThe registry itself is pretty simple, I think a 3rd-party lib could sub out the behavior you want but use registered specs along the way.#2016-05-2616:03dominicm@gtrak: Yeah, that's what I was thinking about. But if it is in scope for spec, that would be pretty cool.#2016-05-2616:06gtrakmy understanding so far is that spec itself is mostly concerned with the lib-sharing/transmission/self-describing-edn use-cases, but there are a lot of developer-convenience type cases we'd also want to use it for (overlap with Schema), but it can't integrate all of them.#2016-05-2616:15gtrakFor instance, the first thing I wrote was a multimethod type->spec coercer, which is a special case extension of the built 'conform', I wouldn't expect that to get into clojure.spec, but I might expect it to be available as a lib, since it's so easy to write.#2016-05-2616:16gtrakconformer-helpers..#2016-05-2616:19gtrakhttps://gist.github.com/gtrak/9b6d16d0423b284292dbbdd095bf91e9#2016-05-2616:21gtrakusage: (s/def ::my-key-that-only-matters-to-my-app (coercer integer?))#2016-05-2616:25gtrakthe one thing I'd request from clojure.spec itself was a way to provide a better error message than what s/conformer creates, which in this case is too generic, a failure in the pred: (fn [from] (coerce from to)). It's possible I can do so by dropping down a level in abstraction but haven't gotten that far.#2016-05-2616:50dominicm@gtrak: oh I agree! I had wondered if that would be possible! Shame you beat me to it. But I feel like being unable to lexically scope a spec in some way hinders me a little. Maybe I'm looking at it wrong, or maybe I can come up with a clever roundabout way to do it and lib it up.#2016-05-2616:55gtrakI think I overheard that specs are serializable somewhere, so lexical scope could really mess that up.#2016-05-2617:00dominicm I probably shouldn't refer to scope. What I want is a solution to the problem of runtime values effecting the running of my predicates, without relying on state. Am I making sense?#2016-05-2618:19gtrakI don't see how it's possible to do that without state, maybe a dynamic/threadlocal registry, but then the state's in the registry.#2016-05-2618:19gtrakor you create a one-off spec every time#2016-05-2618:20gtrakor every consumer of the registry has to let you tunnel things into specs#2016-05-2620:52sveri@alexmiller: I just found a possible bug in the following snippet:#2016-05-2620:54sveriRunning lein test for this code will run into a never ending loop when it tries to test leiningen.s-play-test#2016-05-2620:54sveriI would expect an error message here#2016-05-2620:55sveriThe cause is this line (s/fdef foo :args (s/cat)) with the incomplete (s/cat) expression.#2016-05-2621:19Alex Miller (Clojure team)Feel free to file a jira for it#2016-05-2706:52dominicmhttps://clojurians.slack.com/archives/clojure_spec/p1464286849000602 I wouldn't say that's a terrible idea#2016-05-2708:22dominicmhttps://clojurians.slack.com/archives/clojure_spec/p1464278761000588 I was under this impression also. But I just found this message on google groups: https://groups.google.com/d/msg/clojure/5sZdCPQgZz4/7rpuiaj1AgAJ > OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform explicitly to do so.#2016-05-2719:56Alex Miller (Clojure team)I have added a new section to the spec guide about generators http://clojure.org/guides/spec#_generators#2016-05-2719:57Alex Miller (Clojure team)I still expect there to be a bit more to come on that page about generators and testing#2016-05-2719:57Alex Miller (Clojure team)but seemed better to get something out there sooner#2016-05-2719:58michaeldrogalis@alexmiller Was there any discussion with respect to predicate failures being recognizable from keys, tuple, etc? I can dig up the Gist link if you need for reference#2016-05-2719:59Alex Miller (Clojure team)there have been so many things coming in I don't honestly remember if I talked about it with Rich - I have it in my "things to talk about" file#2016-05-2720:02sveri@alexmiller: done so: http://dev.clojure.org/jira/browse/CLJ-1934#2016-05-2720:04sashtonis this a good place to mention spec guide (potential) typos?#2016-05-2720:08michaeldrogalis@alexmiller: No prob, thanks!#2016-05-2720:10Alex Miller (Clojure team)@sashton: sure!#2016-05-2720:10Alex Miller (Clojure team)@sveri thx!#2016-05-2720:12sashton@alexmiller: Under Collections, it has the following: * Collection - (s/coll-of float?) I think coll-of needs a []#2016-05-2720:12Alex Miller (Clojure team)yep#2016-05-2720:12Alex Miller (Clojure team)thx#2016-05-2720:13Alex Miller (Clojure team)I'll fix this one but for future reference, you can send issues/PRs for the site too http://clojure.org/community/contributing_site#2016-05-2720:41Alex Miller (Clojure team)@sveri I have a patch to fix your bug, should go in the next alpha#2016-05-2721:12sveri@alexmiller: Great, looking forward to testing it#2016-05-2721:13Alex Miller (Clojure team)patch is on the ticket if you want to build your own version of clojure#2016-05-2721:13Alex Miller (Clojure team)I added a test to the tests for it too#2016-05-2721:14sveriYea, I just saw it, I have not setup the clojure build chain, so I just wait for it, no urgency on my side. Thanks for the quick fix#2016-05-2801:27cfleming@alexmiller: You mentioned you have a spec for spec. Any interest in showing that? It would help with adding support in Cursive.#2016-05-2801:30Alex Miller (Clojure team)It's almost certainly pretty buggy so it's not something I really want to publish yet#2016-05-2801:30cflemingOk.#2016-05-2801:31Alex Miller (Clojure team)I have turned on instrumentation with it and had it actually find bugs while I was working though :)#2016-05-2801:32cflemingYeah, I’m planning to merge in the validation code I demoed at the conj, having real-time validation in Cursive for specs would be nice.#2016-05-2801:32Alex Miller (Clojure team)I can send it to you privately if you want#2016-05-2801:33cflemingAnd using specs to help with completion is probably nice too, although I need to work with it more to see what’s possible.#2016-05-2801:33cflemingOk, thanks. I’m not in a big rush, I won’t get to it for a week or two probably.#2016-05-2801:33cflemingBut I’d be interested to see it.#2016-05-2801:35sekaoanyone have an example of using fdef with variadic args?#2016-05-2802:04sekaoahh i guess it’s something like (s/cat :args (s/* string?))#2016-05-2811:42Alex Miller (Clojure team)@sekao: yep although you don't even need the s/cat there just '(s/* string?)' would be sufficient#2016-05-2811:53francescohow could I write a spec for, say, clojure.core/conj that enforces that in (conj coll x) x satisfies the same spec as the elements of coll? Is this kind of reasoning within the intended scope of clojure.spec?#2016-05-2811:59Alex Miller (Clojure team)That's not a true constraint of conj :)#2016-05-2811:59francescototally agree on that.#2016-05-2812:00Alex Miller (Clojure team)But you could spec something like that with the :fn spec in fdef which relates the args and ret#2016-05-2812:01Alex Miller (Clojure team)It's passed a map with the conformed value of both#2016-05-2812:49sveriA bit more feedback. I constantly run into errors like this:
In: [0] val: ([{:name "age", :type :int, :null false}]) fails at: [:args] predicate: (cat :cols :de.sveri.clospcrud.schema/columns),  Extra input
:clojure.spec/args  ([{:name "age", :type :int, :null false}])
  clojure.core/ex-info (core.clj:4617)
Most of the times the problem was that I was missing a (s/spec ::columns) expression around the spec definition. Still, everytime I see this error it is so far away from what the problem actually is, that I need time to remember this. Worse is, that at first I look and search if I did something wrong while transforming my schema to spec. Maybe a somewhat better error message is possible?
#2016-05-2812:49sveri@alexmiller:#2016-05-2814:35Alex Miller (Clojure team)Not sure what a generically better error would be#2016-05-2816:50Alex Miller (Clojure team)Interesting - file a jira if you want so we don't lose it over the weekend#2016-05-2817:26sveri@alexmiller: For me it would be something like: "Maybe you forgot a spec around a regex? Try (s/spec ...)" But, I dont know how generic that error is and if it would fit everytime?#2016-05-2817:54moxaj@alexmiller done, see http://dev.clojure.org/jira/browse/CLJ-1935#2016-05-2818:16sekaois there a way to exclude certain vars from being tested by run-all-tests? i have some side-effecting functions that will cause some havoc if run in generative tests 😃#2016-05-2818:39arohnerI’m not entirely clear on when generative testing happens. In some cases (side-effects), I’d like the ability to check conform but not do generative testing#2016-05-2818:54sekaofor now i copied the code into my project and added an if statement that checks for :no-check in the var’s metadata#2016-05-2819:33bbrinck@arohner: My understanding is that generative testing only happens if you invoke run-all-tests. If you just wanted to test specific functions in your test suite, you could do something like:
(deftest test-my-function
  (is (= true (:result (t/check-var #’my-function)))))
#2016-05-2819:34arohner@bbrinck: I’m pretty sure there are cases aside from that that can trigger it#2016-05-2819:34arohnerIIRC, rich’s example from the other day will#2016-05-2819:34arohnerone sec#2016-05-2819:35arohnerindeed:#2016-05-2819:35arohner
repl> (defn foo [fnn] (fnn 42))
#'repl/foo
repl> (s/fdef foo :args (s/cat :f (s/fspec :args (s/cat :i integer?)
                                     :ret integer?)))
repl/foo
repl> (s/instrument 'foo)
#'repl/foo
repl> (foo #(when (even? %) 42))  

ExceptionInfo Call to #'repl/foo did not conform to spec:
In: [0] val: nil fails at: [:args :f :ret] predicate: integer?
:clojure.spec/args  (#object[repl$eval66335$fn__66336 0x55f2e735 "
#2016-05-2819:37arohner@bbrinck: ^^#2016-05-2819:38arohnerthat ran generative testing on a normal call#2016-05-2819:38arohnerwhich is something I’d like control over#2016-05-2819:38bbrinckI may be misunderstanding, but I don't think it did. I think it just checked that the spec matched the invocation: (foo #(when (even? %) 42)) #2016-05-2819:39arohner42 should pass there. The generative example passed zero#2016-05-2819:40arohnerhrm nvm#2016-05-2819:40bbrinckI think this is just a confusing error message. The value passed is not 0#2016-05-2819:40bbrinckit's #(when (even? %) 42), which returns nil, which violates the spec#2016-05-2819:41arohner
(foo #(do (println %) (when (even? %) 42)))
-1
0
-1
0
-1
0
-1
#2016-05-2819:42arohner
(s/unstrument 'foo)
#'repl/foo
repl> (foo #(do (println %) (when (even? %) 42)))
42
42
#2016-05-2819:45bbrinckHm, yeah, you're right, I was reading that wrong. I still don't think (from playing with other examples) that the generative testing occurs unless you do one of the testing calls. I think this example might just have a bug. I could be misunderstanding though.#2016-05-2819:46bbrinck(I'm trying to try it myself, but on my version of clojure, the fdef is not working for some reason)#2016-05-2819:47arohnerwhich testing call did I run there?#2016-05-2819:53bbrinckHm, you're absolutely right. I did not expect that.#2016-05-2819:55bbrinck@arohner: Yep, you're correct - there was no testing call, but it's doing some generation for the args for the inner function.#2016-05-2819:59bbrinck@arohner: interestingly, that does not seem to occur in all cases:
(s/fdef double
        :args (s/cat :x integer?)
        :ret integer?)
(defn double [x]
  (if (zero? x)
    "zero"
    (* 2 x)))

(s/instrument 'double)

(double 1)
#2016-05-2819:59arohneryeah, I think fspec is the culprit#2016-05-2820:01bbrinckah, yes, the docs do mention that now that I read them a little more carefully šŸ˜„ . Well, TIL#2016-05-2820:05bbrinckAlso, FWIW, that example won't quite work on 1.9.0-alpha3. fspec seems to require a ret now#2016-05-2820:08bbrincknvm, i realized it was a copy/paste fail on my part (didn't grab the full fspec above)#2016-05-2820:18sveriHi, did anyone try to generate a boolean spec? Like this: (s/def ::required #(instance? Boolean %)). Calling (gen/sample (s/gen ::required)) fails for me with: Exception Unable to construct gen.... This becomes problematic if ::required is part of a nested spec, how would I provide a generator for that?#2016-05-2820:28seancorfieldThat specific case is covered in the (updated) documentation @sveri #2016-05-2820:30sveri@seancorfield: Ah, thats nice, I just solved it by defining boolean as true / false: (s/def ::boolean #{true false}) for which the generator works too#2016-05-2820:31seancorfield"There are some common cases that currently don’t have standard predicates, but good generators exist. It’s likely there will be changes in this area in the future but for now you might find these useful: (defn boolean? [x] (instance? Boolean x)) (s/def ::boolean (s/with-gen boolean? #(gen/boolean))) (defn uuid? [x] (instance? java.util.UUID x)) (s/def ::uuid (s/with-gen uuid? #(gen/uuid)))"#2016-05-2820:31seancorfield(That didn't paste well -- sorry, on my phone)#2016-05-2820:35sveriYea, no problem, I get it, thank you šŸ™‚#2016-05-2821:00borkdudeJust finished reading the guide. Good stuff.#2016-05-2821:12borkdudeWhy is (doc subs) printing Spec on the last line. Bug?
user=> (doc subs)
-------------------------
clojure.core/subs
([s start] [s start end])
  Returns the substring of s beginning at start inclusive, and ending
  at end (defaults to length of string), exclusive.
Spec
nil
#2016-05-2821:33borkdudeWhat about multi-arity functions like subs, how to write spec for both cases?#2016-05-2821:37borkdudeLike this? (s/fdef clojure.core/subs :args (s/or :two-args (s/cat :s string? :start integer?) :three-args (s/cat :s string? :start integer? :end integer?))#2016-05-2821:41borkdudeI'm not sure if I should wrap with s/spec, both seem to work: (s/fdef clojure.core/subs :args (s/or :two-args (s/spec (s/cat :s string? :start integer?)) :three-args (s/spec (s/cat :s string? :start integer? :end integer?))))#2016-05-2901:11Alex Miller (Clojure team)@borkdude: there is new code in doc to find and print specs, but it shouldn't print that if there are none so that's a bug#2016-05-2901:11Alex Miller (Clojure team)@borkdude: for multi-arity, you can cover all options in a single spec via regex#2016-05-2901:14Alex Miller (Clojure team)you can just do (s/fdef clojure.core/subs :args (s/cat :s string? :start integer? :end (s/? integer?)) :ret string?)#2016-05-2901:18Alex Miller (Clojure team)you can even add a :fn #(clojure.string/includes? (-> % :args :s) (:ret %)) to verify it returns a string included in the original#2016-05-2901:19Alex Miller (Clojure team)I didn't test any of that but it should be pretty close#2016-05-2901:31sekaocould we get a way to exclude vars from run-all-tests, like ^:no-check in core.typed? i jerry-rigged a solution earlier today but it required modifying the function directly. i have lots of side-effecting functions that shouldnt be running generative in tests.#2016-05-2901:35Alex Miller (Clojure team)it's best to file a jira for anything so we don't lose it, esp over the long weekend#2016-05-2901:36sekaook will do#2016-05-2901:36Alex Miller (Clojure team)thx!#2016-05-2901:36Alex Miller (Clojure team)that way we can track that stuff and you can see what's happening with it too#2016-05-2901:46sekaoah i should’ve looked there first, i’ll just comment on CLJ-1936 as it sounds similar#2016-05-2907:33borkdude@alexmiller: thanks šŸ™‚#2016-05-2907:44borkdude@alexmiller: promoted this example to a gist: https://gist.github.com/borkdude/0665078edb40fb0e1551c1d29655c2d6#2016-05-2921:00sveriHey, I have again a problem I dont understand:#2016-05-2921:02sveriWhen I copy the part from args: ({:name "QpJ50qrS1m24V", :type :varchar, :unique true, :required false, :autoinc true, :null true, :pk true, :max-length 14, :fk-name "Tp9tG8hwUXK0"})and run the spec validation on it it succeeds. What bothers me, is that the last line in the message wraps the columns in an extra list: :clojure.spec/args (({:name "QpJ50qrS1m24V" ... I am not sure what that means.#2016-05-3008:29cflemingWhen s/spec is used to create a new sequential context, I can’t find a way to capture the sequential thing itself.#2016-05-3008:29cflemingFor example:
(s/def ::ingredient (s/cat :quantity number? :unit keyword?))
(s/def ::recipe (s/cat :amount (s/spec ::ingredient) :description string?))
(s/conform ::recipe [[2.0 :teaspoon] "Cinnamon"])
=> {:amount {:quantity 2.0, :unit :teaspoon}, :description "Cinnamon"}
#2016-05-3008:30cflemingHere, there seems to be no way to capture the [2.0 :teaspoon] vector object if I also want to match its contents.#2016-05-3011:17Alex Miller (Clojure team)@cfleming: you are matching its contents via the cat. If you want to receive the vector as the conformed value you could use coll-of to do a different kind of match or use a conformer to transform the matched result into any arbitrary structure#2016-05-3011:20Alex Miller (Clojure team)Probably the latter is what you want#2016-05-3017:33arohneris there a way to update a map spec? I’d like to say ā€œthis fn takes a foo map, and returns a foo map with an extra key added onā€. It’d be nice to say (s/def bar (conj foo ::extra-key)), without specifying the keys of the second map explicitly#2016-05-3018:01arohnerarg. repl development overwrites instrumentation#2016-05-3020:00moxajQuestion: why is the second argument (`retag`) necessary in a multi-spec expression? In the guide, it is the same as the dispatch function, and I don't see any use case where providing a different function would be beneficial. And if it's the same, naming it again is redundant, as it can be retrieved via the public field dispatchFn.#2016-05-3020:46seancorfieldAs an experiment, I’ve added an optional spec namespace to clojure.java.jdbc: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj — feedback welcome (the tests attempt to require that ns and instrument clojure.java.jdbc when running tests, so under 1.9.0 the specs are actually checked for all calls in the tests)#2016-05-3021:12potetmIs there any mechanism that will allow you to include possible exceptions as part of a spec?#2016-05-3022:26kovasbHow can I spec a function that takes an atom(x), where x conforms to a spec?#2016-05-3022:28kovasbthinking something like atom-of instead of coll-of#2016-05-3100:00kgzmHow would one use spec with records, when records use unqualified keywords and spec mandates the use of namespaced keywords?#2016-05-3100:36arohner@kovasb: AFAIK, that doesn’t exist currently, but it should be possible to write from cribbing coll-of#2016-05-3100:46arohner@potetm: ATM, no#2016-05-3101:39cfleming@alexmiller: Ok thanks, I don’t fully understand the second option but I’ll investigate it.#2016-05-3101:44potetm@cfleming: https://clojure.org/guides/spec#_conformers#2016-05-3101:44potetmI guess the obvious followup (maybe for #C06E3HYPR?) is: Are there plans to support exceptions in specs?#2016-05-3101:56seancorfieldGiven that the spec describe data structures, and function inputs/output — what would it mean to "support exceptions" in specs?#2016-05-3101:57potetmYeah that's fair.#2016-05-3101:58potetmMy first pass idea was just to "list the possible exceptions". But I'm totally open to the idea that that's just a bad idea.#2016-05-3101:59potetmI haven't thought it through or anything. Just noticed that I was hitting problems during generative testing because I couldn't declare that, say, arithmetic overflow, was an okay exception.#2016-05-3102:00cflemingIt seems like something you might want to specify.#2016-05-3102:00cflemingIt’s a potential output from a function, more or less.#2016-05-3102:00seancorfieldI can see pros and cons. Certainly in (unit) tests it can be valuable to say "given these inputs, I expect the following exception"...
#2016-05-3102:01potetmcfleming: Yeah that was my thought as well.#2016-05-3102:01seancorfieldBut is that a failure to conform to its spec? It depends on whether the spec is considered to be "valid for all inputs that are valid" (and therefore "undefined" for invalid inputs)...#2016-05-3102:02seancorfieldI just went through this exercise with java.jdbc because there are lots of ways to generate exceptions from those functions… but I’m not sure how to specify that...#2016-05-3102:03seancorfieldI’m not even sure if it makes sense to try to specify that? What if the DB is down, or the table you ask for doesn’t exist or there’s a syntax error in your SQL…? How would you codify that in clojure.spec?#2016-05-3102:03seancorfieldHow could you even list all possible exceptions?#2016-05-3102:04potetmYeah it's possible that that would be overly restrictive.#2016-05-3102:04seancorfield(each JDBC driver throws its own types — do you just say "could throw Exception"…?)#2016-05-3102:05seancorfieldIt’s a hard problem, either way. clojure.spec definitely has gaps when you have stateful functions (generative testing on java.jdbc functions isn’t possible in general — most garbage input will produce an exception, even if it is in the right "form").#2016-05-3102:05potetmI dunno, perhaps not since it's not exactly stopping you from doing anything at the end of the day.#2016-05-3102:11cfleming@seancorfield: Yeah, spec is mostly not that useful for me either, for similar reasons.#2016-05-3102:11mfikesSo Sean and Nolen both seem to have created a my.lib.main-ns.spec namespace to hold specs. Wonder if that will become a de facto place to put them for libraries.#2016-05-3102:12seancorfieldI took that approach so that you didn’t have to load the spec ns — so you could use the code with Clojure < 1.9.0#2016-05-3102:13seancorfieldAlthough the build system shot me in the foot since it compiles all namespaces(!).#2016-05-3102:13seancorfieldConsequently, the build plain ol’ fails on Clojure < 1.9.0. Not sure how to approach that (make the entire namespace conditional? Ugh!).#2016-05-3102:15mfikesHmm… need a good solution to that problem. Every lib will face it.#2016-05-3102:15potetmSpec actually does this for test.check.#2016-05-3102:15potetmIt's non-trivial.#2016-05-3102:15cflemingI’m just cutting and pasting clojure.spec.* and using Clojure 1.7.#2016-05-3102:16mfikesYeah, the dynaload. Hmm#2016-05-3102:16seancorfield@potetm: But you can’t have an optional namespace that uses spec and get it through the build system.#2016-05-3102:16seancorfieldI already deal with conditional loading in the java.jdbc tests.#2016-05-3102:17seancorfieldLocally, I can run lein test-all and pass tests on every Clojure version from 1.4 to 1.9 — and only on 1.9 does it use the spec.#2016-05-3102:18seancorfieldBut the contrib build system tries to compile the namespaces in the project… So maybe there’s some Maven incantation I can use to exclude the namespace from compilation (or whatever Maven is trying to do with it).#2016-05-3102:19potetmRight so this is all in the build tool. Yeah I got nuthin....#2016-05-3102:19seancorfieldhttp://build.clojure.org/job/java.jdbc-test-matrix/410/ and http://build.clojure.org/job/java.jdbc-test-matrix/410/CLOJURE_VERSION=1.4.0,jdk=OpenJDK%201.6/console#2016-05-3102:21potetmWell except for this problem, that tool seems pretty slick šŸ™‚ I've never seen anything like it before.#2016-05-3102:21mfikesMaybe someone can make a hacked-up stub do-nothing compatibility lib in the clojure.spec and cljs.spec namespaces that consumers on 1.8 can use, just so things can be loaded. Ugh.#2016-05-3102:21seancorfield@potetm: Have you looked at core.typed or Prismatic Schema at all?#2016-05-3102:22potetmNo I mean the version matrix. All the JDKs all the clojure versions. A really nice idea for open tooling. (I've always had control over JDK and clj versions.)#2016-05-3102:22seancorfieldTomorrow I’ll talk to @alexmiller about the build system and see if we can figure out something ...#2016-05-3102:51seancorfieldSean: 1, Maven: 0 😈 So it turns out a contrib project can override the parent pom.xml and suppress the Maven-initiated compile phase: https://github.com/clojure/java.jdbc/commit/b224a3e86df8c00a33a16122e6fcc531c5f71e2e#2016-05-3102:51seancorfieldNow I feel dirty for having to learn that much about Maven šŸ˜ž#2016-05-3102:59seancorfieldApparently if I want to get really "clever" I could probably create a conditional profile based on the Clojure version and have it still run compile if we’re using 1.9.0… Ugh! <profiles> ...#2016-05-3112:22andrewhr@seancorfield: lein's {:aot :all} doesn't kick in for included dependencies, did it? Maybe that's the reason for mavencompile's strategy#2016-05-3114:40akiel@anmonteiro You had an example of clojure.test integration were you used spec-is. I can’t find spec-is in the sources. I’m also interested in a proper clojure.test integration of clojure.spec.test/check-var. clojure.spec.test/run-tests only works in the REPL for me. I need it in leiningen.#2016-05-3114:41anmonteiro@akiel: spec-is was something I wrote:
(defmacro spec-is [res]
  `(clojure.test/is (true? (:result ~res))))
#2016-05-3114:44akiel@anmonteiro: ok thanks this works.#2016-05-3115:58akiel@anmonteiro: I’ve extended the is macro in the following gist. This looks even better to my eyes. https://gist.github.com/alexanderkiel/931387c7a86c1879b2267ad064067af7#2016-05-3117:33seancorfield@andrewhr: The pom.xml file indicates the compile is just a "sanity check" so I’m taking that as "optional"...#2016-05-3119:05ghadiIs there anything in between s/keys and s/map-of for heterogenous maps --- maps with keywords and structured values as keys?#2016-05-3120:34seancorfieldAs I spec’d java.jdbc I found myself wanting a way to constrain the values of the optional keys in the optional opts argument. I suspect the only way is (s/and (s/keys …) #(some custom predicate)) ?#2016-05-3120:35seancorfield(I’d have to take a look at what s/keys conforms values to)#2016-05-3120:40kennyAre you guys tending to write your fdefs in a separate ns or in the same ns above/below where the function is written?#2016-05-3120:41sekaoi’ve been putting them above my functions so far#2016-05-3120:45kennyYeah that is great for readability but it seems like it kinda pollutes your ns.#2016-05-3121:53zaneHow would one spec a map with non-keyword keys and heterogenous values (the specs of which are dependent on the keys)?#2016-05-3121:55zaneThat is, how would you spec ::person from the guide if ::first-name ::last-name etc were strings?#2016-05-3122:13zaneOr, to put it another way, is there a version of clojure.spec/keys that works with non-keyword keys?#2016-05-3122:18seancorfield@zane: I suspect you’d need to say (s/map-of string? ::s/any) and then have some custom predicate on the conformed value.#2016-05-3122:19zaneRight, okay.#2016-05-3122:19zaneAs I'd feared.#2016-05-3122:20seancorfield@kenny: I think overall I’d prefer data structure specs in a separate ns and fdef alongside the functions themselves but I won’t have a solid feel for that until we’ve used it a bit more heavily.#2016-05-3122:20seancorfieldIn clojure.java.jdbc, I put all the specs in a separate ns so users on Clojure < 1.9.0 could still use the library.#2016-05-3122:33kennyMakes sense.#2016-05-3122:58zane@seancorfield: If I wanted to try to recover the the features of clojure.spec/keys but for maps with non-keyword keys would you recommend I implement clojure.spec/Spec myself?#2016-05-3123:05seancorfieldI suspect that will be a lot of work (but I haven’t looked at it). Since the push is for namespaced keywords — but unnamespaced keywords are also supported — I guess I would have to question your desire to define an API based on maps with non-keyword keys?#2016-05-3123:05seancorfieldBy which I mean, what specifically is it that you’re trying to spec out here that isn’t a "regular" Clojure map?#2016-05-3123:06seancorfield(and, perhaps therefore, spec "at large" is not designed for your use case?)#2016-05-3123:10seancorfieldMy sense with clojure.spec is that it’s opinionated deliberately to encourage a particular style of API specification… "idiomatically Clojurey"… The comments in particular about namespaced keywords being "tragically underutilized" and that namespace-qualified keywords is "a practice we’d like to see grow"...#2016-06-0100:03kennyHow are you guys spec'ing recursive functions where intermediate return values have a different spec than the final return value? Are you just adding an or to the :ret or are you writing your function in such a way that that isn't possible (e.g. bundling the recursion in the function by creating an anonymous fn inside your function)?#2016-06-0100:19seancorfield@kenny: Can you give an example of such a scenario?#2016-06-0100:20seancorfield(I generally try to avoid functions that can return multiple types although I’ll often have functions that return something-or-nil)#2016-06-0100:20kenny
(defn new-matrix-nd
  "Returns a new general matrix of the given shape. Shape can be any sequence of
  integer dimension sizes (including 0 dimensions)."
  [dims]
  (if-let [dims (seq dims)]
    (vec (repeat (first dims) (new-matrix-nd (next dims))))
    0.0))
#2016-06-0100:20kenny
(s/fdef new-matrix-nd
        :args (s/cat :dims (s/nilable coll?))
        ;; ret is not a matrix because new-matrix-nd is a recursive fn
        :ret (s/or :m vector? :n number?))
#2016-06-0100:20kennyIdeally the spec should be
(s/fdef new-matrix-nd
        :args (s/cat :dims (s/nilable coll?))
        :ret matrix?)
#2016-06-0100:20seancorfieldso (new-matrix-nd []) produces 0.0?#2016-06-0100:21kennyYes#2016-06-0100:21kennyBut the final return value is always a matrix#2016-06-0100:21seancorfieldUnless you accidentally call it with nil or an empty sequence of dimensions...#2016-06-0100:22seancorfieldSo a zero-dimensional "matrix" is a scalar… hmm… that’s an interesting one...#2016-06-0100:23kennyRight#2016-06-0100:24kennyI can wrap up the recursion inside the function#2016-06-0100:24kennyThat would probably be cleaner to the outside world#2016-06-0100:27seancorfieldI’m trying to think how specs are going to be much use to you here tho’… since any call site is going to get "vector or scalar" as a result...#2016-06-0100:28kennyHmm.. Yeah it might not make sense.#2016-06-0100:29seancorfieldeven your :args spec doesn’t buy you much here: nil or an arbitrary collection — you probably want (s/coll-of integer? []) to give you more checking — and I’d be tempted to disallow nil and require an empty sequence of dimensions… unless you have a really good reason for allowing nil there?#2016-06-0100:31seancorfieldMaybe the Clojure/core folks need to give a bit more guidance in the Rationale as to how they expect spec to be used in the real world? /cc @alexmiller#2016-06-0100:32seancorfieldI’d imagined it as a set of specifications for some pretty high-level parts of your code, around your domain objects, or possibly as the definition of a library API… but I’m not sure about the latter yet...#2016-06-0100:32kennynil is just the terminating value. It seems wrapping the recursion inside the fn may solve this.#2016-06-0100:33seancorfieldAt work we’re looking at spec to write a specification of our domain model and some of the high-level business logic that operates on that. I don’t know how far "down" the call tree we’ll go...#2016-06-0100:36seancorfieldThe Rationale says "the supply chain isn’t burdened with correctness proof. Instead we check at the edges and run tests" … "Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment."#2016-06-0100:38kennyThis cleans up the spec...
(defn new-matrix-nd
  "Returns a new general matrix of the given shape. Shape can be any sequence of
  integer dimension sizes (including 0 dimensions)."
  [dims]
  (letfn [(new-matrix-nd' [dims]
            (if-let [dims (seq dims)]
              (vec (repeat (first dims) (new-matrix-nd' (next dims))))
              0.0))]
    (new-matrix-nd' dims)))
#2016-06-0100:39kenny
(s/fdef new-matrix-nd
        :args (s/cat :dims (s/coll-of integer? []))
        :ret (s/or :m matrix? :s scalar?))
#2016-06-0100:40seancorfieldBut matrix? still allows a scalar, right?#2016-06-0100:40seancorfieldOtherwise (new-matrix-nd []) will pass the :args check but fail the :ret spec.#2016-06-0100:41kennyEdited to fix...#2016-06-0100:42seancorfieldYour refactoring hasn’t changed the spec tho’...#2016-06-0100:43kennyNo nil allowed in the fn#2016-06-0100:43seancorfieldWell, the spec disallows it, but that was true of the earlier function as well.#2016-06-0100:44kennyYou could pass nil to the previous fn. That should not be allowed.#2016-06-0100:45kennyThough that spec actually isn't working. (s/valid? (s/coll-of integer? []) nil) => true#2016-06-0100:45seancorfield(then I was wrong, s/coll-of allows nil?)#2016-06-0100:46kennyI guess. That doesn't seem right though#2016-06-0100:47kenny(s/valid? (s/* integer?) nil) => true#2016-06-0100:48seancorfield
boot.user=> (s/conform (s/* integer?) nil)
[]
boot.user=> (s/conform (s/coll-of integer? []) nil)
nil
#2016-06-0100:49seancorfield
boot.user=> (s/conform (s/* integer?) (list 1 2 3))
[1 2 3]
boot.user=> (s/conform (s/coll-of integer? []) (list 1 2 3))
(1 2 3)
#2016-06-0100:50seancorfieldSo coll-of is the spec for a (possibly nil) collection — sequence of values — and * is a regex to match zero or more items and returns a vector.#2016-06-0100:51kennyWas coll-of designed such that a possibly nil collection of values is valid?#2016-06-0100:52kennyNot sure how that can be an artifact as nil is inherently false so it would seem to be by design?#2016-06-0100:52seancorfield
boot.user=> (source s/coll-checker)
(defn coll-checker
  "returns a predicate function that checks *coll-check-limit* items in a collection with pred"
  [pred]
  (let [check? #(valid? pred %)]
    (fn [coll]
      (c/or (nil? coll)
            (c/and
             (coll? coll)
             (every? check? (take *coll-check-limit* coll)))))))
nil
^ this is what coll-of calls and it has a very specific check for nil
#2016-06-0100:53kennyHmm. Wonder why#2016-06-0100:53seancorfieldBecause nil-punning is idiomatic and encouraged?#2016-06-0100:54kennyI thought the idea was to be explicit about nil with nilable#2016-06-0100:54seancorfield(that’s a question for Clojure/core really but…)#2016-06-0100:55kennyMaybe we can get some input tomorrow on this.#2016-06-0100:57kennyI suppose it does make sense coming from the Java static typing world because you could pass null in place of any argument. We don't need to adhere to that though.#2016-06-0101:05seancorfieldBut any function that accepts a (general) collection is almost certainly going to accept nil I’d say…?#2016-06-0101:06kennyWhy make any assumptions?#2016-06-0101:06seancorfieldAs noted above/elsewhere, clojure.spec is deliberately opinionated šŸ™‚#2016-06-0101:07seancorfield(not saying I agree, just observing)#2016-06-0101:11kennySeems strange, considering: https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare#2016-06-0101:11seancorfieldVery different idioms. If null isn’t treated as falsey, then you have a very different style of programming.#2016-06-0101:13seancorfieldFor example, in some languages you’d have a Maybe monad and you would chain operations using monadic functions. Scala has Option[T] and flatMap, as I recall. Haskell has Maybe t and fmap (yes?). But in Clojure you’d chain such operations using some-> or some->> and they’d just be regular functions.#2016-06-0101:14seancorfieldOr you use if-let / when-let#2016-06-0101:15seancorfieldI was chatting with someone the other day and saying that I almost never hit a NPE in Clojure these days. About the only time I still do is in Java interop code (usually calling into the String class for something!).#2016-06-0101:15kennyTrue (though we can follow that style of programming is Clojure e.g. cats). Anyways, I suppose that is a tangent and doesn't really pertain to this situation.#2016-06-0101:16kennyYeah you're totally right. I can't recall the last NPE I got.#2016-06-0101:17seancorfield(the irony of this conversation today is that someone reported an error against java.jdbc that a particular erroneous call to one of the functions produces a NPE — for invalid arguments)#2016-06-0101:18seancorfieldThe latest version of java.jdbc accepts those calls without error tho’...#2016-06-0101:20kennyAnyways, the primary reason I don't want nil to exist in this particular library is there is no such idea in the math world.#2016-06-0102:52andrewhrMy feelings about clojure.spec are pretty similar to yours @seancorfield... most for library public APIs (think pedestal, with namespaces keywords and data-based DSLs) or for general domain definition. Following the guidelines presented on Clojure Applied, I imagine using specs at domain edges and possibly Records only as a implementation detail of that domain (as suggested in the book).#2016-06-0102:55andrewhrI still need more though (or some good rule of thumbs) on the usage of namespaced keywords per se as a design tool. When to use a fully qualified - and by definition more coupled to implementation - vs just simple namespaces aka :foo/bar. And when it will be fine to use unqualified ones as envisioned by the Clojure/core style#2016-06-0102:55seancorfieldRecords require non-namespaced keys tho’ so I wonder how that sits with specs? (and I don’t think clojure.spec actually supports records?)#2016-06-0102:57andrewhrYeah, as far as I got it does not. Considering this style of "Records as an implementation detail", that makes sense no? I just following through the documentation snippet you shared#2016-06-0103:00cflemingI’m interested to know how spec handles records, too.#2016-06-0103:01cflemingI guess you treat the fields as non-namespaced keys?#2016-06-0103:12mikethompsonI'm considering spec from an Information Model point of view. Let's say I wanted to create a spec for car. It might have keys like colour and year. That part is easy. But what about manufacturer where I don't want containment, and instead I want a reference - an id? I could say the spec for manufacturer was integer?, but how should I be more specific and say it is a reference to another spec. My mind is wandering forward and imagining a tool which can read the central registry and produce a nice SVG Information Model. But it can't do that unless it knows that the manufacturer in car is more than an integer? ... it is actually a reference to another entity.#2016-06-0103:26seancorfield@mikethompson: Sure you can have ::manufacturer-id that specifies actual valid manufacturer IDs (or whatever you needed) and for testing you could specify a generator that produced (a small subset of) manufacturer IDs.#2016-06-0103:57zane> I guess I would have to question your desire to define an API based on maps with non-keyword keys? @seancorfield: Oh, believe me: If we were in control of this data we would use keyword keys. We're trying to enforce / check our assumptions about data we don't control.#2016-06-0104:08mikethompson@seancorfield: Yep, understand that. I'm just wondering about how that SVG Information Model Diagram tool could work. By looking in the central registry, how does it know that ::manufacturer-id is a reference to ::manufacturer. I can't see how that can be captured.#2016-06-0104:09mikethompsonAlmost seems like metadata on ::manufacturer-id#2016-06-0104:09seancorfield@mikethompson: maybe :manufacturer/id and :manufacturer/spec?#2016-06-0104:10seancorfieldConventions are needed somewhere, somehow.#2016-06-0104:10mikethompsonAhh. Metadata via naming convention šŸ™‚#2016-06-0104:11seancorfield@zane: If the keys are strings when you get them from "the outside" then maybe keywordize them for use "inside"?#2016-06-0104:11mikethompsonAnyway, all good. I was just wondering if I was missing something. Thanks#2016-06-0104:13seancorfield@mikethompson: No idea what Clojure/core might recommend here. I was trying to follow the Datomic lead there but I haven't looked at how referential specifications might work yet... Definitely something we'll run into at work as we move forward with this...#2016-06-0104:16seancorfield@zane: If you keywordize them "as-is" (maybe putting a standard namespace qualification on them too?), then you can validate with the full force of spec? At work, we're considering using a naming strategy with java.jdbc to create namespaced keys from the DB rather than just column names, so we can move straight to spec. Not sure how that will work out yet but we're planning a spike along those lines next week.#2016-06-0105:16josh.freckletonI'm still a bit new and wanting to bring in some typing. I'm not familiar with the respective libs, is spec effectively replacing schema and typed.clojure?#2016-06-0105:21seancorfieldHi @josh.freckleton -- have you read the Rationale for spec on the http://clojure.org site?#2016-06-0105:21seancorfield(it tries to clarify the spec vs type system position)#2016-06-0105:24josh.freckleton@seancorfield: I have, I'm just trying to clear things up a bit, and I guess i'm wondering where I should invest my time šŸ™‚#2016-06-0105:26seancorfieldspec is different to both core.typed and Schema...#2016-06-0105:29josh.freckleton@seancorfield: oh, can I ask another question while we're talking about this? in schema or spec, could I have a case (switch) according to types, or defmethods on different types? Would there be a prefered method?#2016-06-0105:29seancorfieldcore.typed is intended to provide a type checking system based on static analysis of the code with annotations. Schema is a runtime validation system.#2016-06-0105:30seancorfieldNot sure what you're asking... I think I'd have to see an example of what you're trying to do...?#2016-06-0105:32josh.freckleton(ahh, your typed vs. schema example makes sense. i still need to digest this) So say I have a few different "types", and I want to map a fn over them which is specific to their type, I've just started to learn about defmethods, and of course there are switch statements (`case`) switching on the type of the object under consideration. If I, say, had a list of things that could be all different types, what would be the best way in to switch on their types?#2016-06-0105:33seancorfieldSorry, I still don't understand...#2016-06-0105:33josh.freckletonmy fault, I'll be more specific...#2016-06-0105:39josh.freckletonI'm working with the free monad + interpreter pattern trying to build some code that takes a data model, and can interpret it in various ways. The important part is that the data model, essentially the Free Monad instance, is a recursive "type" where each "node" of this data model can be different types. Currently, I signify their "type" as a key :type in a mapping, and I (case (:type obj... to decide what to do with each "node". For example, one node can be a "User", and another can be "Owned Books", and another can be "Email", "Password", etc. As I interpret this data model, I will want to treat Users differently from Emails, and Passwords, etc. The 3 options I know of are to 1. defmethod according to different, real types, 2. to case over their type if spec or schema allows me to switch on types, or 3. what I have now with a map, and a "`:type`" key. Maybe I'm completely misguided in how I'm trying to solve this though, and I'm completely open to you annihilating my idea if it's far out-field! Does that make more sense?#2016-06-0105:40seancorfieldSo you want polymorphic dispatch on the :type key in the hash map?#2016-06-0105:41seancorfielddefmulti / defmethod would work for that. Or use a protocol with multiple defrecord implementations.#2016-06-0105:42seancorfield(the latter would be actual types and would no longer need the :type key)#2016-06-0105:43seancorfielddefmulti would use a dynamic dispatch based on aspects of the data structure, in your case the :type key.#2016-06-0105:44seancorfieldSo, basically, defmulti is a sort of case statement dispatch šŸ™‚#2016-06-0105:47josh.freckletonOk, I'm glad that way would work, thx. And would their be an idiomatic way to do it without declaring polymorphic methods, like: (case (spec/type obj) :a "a" :b "b"), or case (schema/type obj) :a "a" :b "b") I haven't seen anything in the docs that fits that ability to check the type of an obj, but I maybe I've missed it?#2016-06-0105:47seancorfieldYeah, I think you're misunderstanding what spec and Schema are about.#2016-06-0105:48seancorfieldWhy wouldn't you just do (case (:type obj) :a "a" :b "b") at that point? The :type key gives you what you need.#2016-06-0105:50seancorfieldOr (defmulti foo :type) (defmethod foo :a [obj] "a") (defmethod foo :b [obj] "b")#2016-06-0105:50josh.freckletonThat's probably why I couldn't ask it clearly šŸ˜’ I was just curious if that was an option, it seems similarish to me of Haskell's matching on different types. I think that's the "feel" I was going for, but again I could be wrong since I'm new the functional world.#2016-06-0105:52josh.freckletonfor example:
f Nothing = foo
f (Just x) = bar
You've been a big help sean, helping me prune my search tree of what I need to study, haha. Thank you so much, and thanks for staying up late to help us noobs out!
#2016-06-0105:55seancorfieldIn Clojure, Maybe would more likely just be nil or not-`nil` and you wouldn't pattern match, just (defn f [x] (if x 'bar 'foo))#2016-06-0105:56josh.freckletonoh sure, and there's also a maybe monad, I'm thinking for matching on many (10-100) custom types.#2016-06-0105:57seancorfieldRight, so if you want polymorphic dispatch on a single argument (type), you probably want a protocol and defrecord.#2016-06-0105:58seancorfieldIf you want more ad hoc polymorphism, defmulti is probably your tool#2016-06-0105:58seancorfieldClojure is very different from things like Haskell, since there's no extant type system, even tho' Clojure is strongly typed (at runtime).#2016-06-0105:58seancorfieldSo the idioms are very different.#2016-06-0105:59seancorfieldClojure's polymorphism is very different from OOP languages as well.#2016-06-0106:01josh.freckletonK, I had been considering these different options, and with your suggestion I'll zoom in on defrecord/`defmulti`, that helps me cut out tomorrows work šŸ™‚#2016-06-0106:01josh.freckletonThank you!!!#2016-06-0106:05seancorfieldHave fun!#2016-06-0106:24ikitommi@mikethompson: @seancorfield was thinking also for doing registry visualization to learn things. Did a schema->graphviz tools some time ago, which helps a lot. Are you already working on this?#2016-06-0106:25mikethompson@ikitommi: thought experiment for me. So, no, not working on it.#2016-06-0106:55seancorfieldI'm not a very visual person so it's a "no" on my end too. #2016-06-0114:23nwjsmithI'm having difficulty spec-ing a map. Would like to re-use spec for the keys and values. In particular, I'm trying to spec this bit of Datomic Pull: map-spec = { ((attr-name | limit-expr) (pattern | recursion-limit))+ }. I've already got specs for ::attr-name, ::limit-expr, ::pattern and ::recursion-limit, but I can't figure out how to re-use them in the spec for ::map-spec#2016-06-0114:23nwjsmithI thought s/map-of would be the ticket, but it works with predicates not specs.#2016-06-0114:40manutter51@nwjsmith: See the Entity Maps section of the guide http://clojure.org/guides/spec#2016-06-0114:41manutter51you want s/keys basically#2016-06-0114:44nwjsmithAFAICT s/keys is too strict, e.g I can't see how to specify that all keys conform to (or :attr-name ::attr-name :limit-expr ::limit-expr). It will only let me specify particular keys.#2016-06-0114:45nwjsmithI think what I'm looking for is something half-way between s/keys and s/map-of.#2016-06-0115:14zane@seancorfield: Those suggestions are great. Thanks! Full support for maps with keys that aren't keywords would still make sense to me, though. There are plenty of domains where what you're modeling with data really does require keys of other types.#2016-06-0115:51seancorfield@zane: I suspect maps with "other types" of keys are more likely to be homogeneous since you wouldn’t be able to enumerate all the keys (i.e., to list which are required and which are optional).#2016-06-0115:51seancorfieldWe have some maps from strings to hash maps, and some from longs to strings etc. But those don’t have specific required / optional keys — they’re "lookup tables".#2016-06-0115:52seancorfield(it’s an interesting area of discussion tho’)#2016-06-0115:52zaneThat makes sense, and matches my experience.#2016-06-0115:56nwjsmith@zane reading through your conversation w/ @seancorfield it looks like you and I have the same issue. Have you looked into implementing clojure.spec/Spec?#2016-06-0115:57zane@nwjsmith: Only superficially.#2016-06-0116:00zane@seancorfield: I'm realizing that one problem with the keywordize-before-running-specs approach is that any resulting error messages will refer to the keyword names rather than the string ones. That limits their utility significantly (makes them less useful to the calling client, for example).#2016-06-0116:02zaneMaybe validating JSON data via spec just isn't an intended use case?#2016-06-0116:04seancorfieldHow are you getting the JSON data? When we get JSON from a web service etc, we have clj-http keywordize the result, and we do the same with Cheshire as well. Basically, if we ingest JSON in any manner, we always convert to keywords at the boundary and validate afterward.#2016-06-0116:05zane(Aside: I'm learning a lot from your responses! Thanks!)#2016-06-0116:06zane@seancorfield: Right, right. But our intent was to then return any error messages explain-data produces from speccing the request to the calling client.#2016-06-0116:07zaneexplain-data's output is going to refer to the keywordized request, which could be confusing to the client if they're, say, unfamiliar with Clojure.#2016-06-0116:07seancorfieldHmm, I’m not convinced explain-data is a good format for clients since it refers to the internals of your specs … and what about localization of messages, and error codes for lookup in a reference manual for your API?#2016-06-0116:15zaneHmm. I suppose I didn't notice anything about specs that made me think they were fundamentally internally-facing. I'm curious what your thinking is there.#2016-06-0116:17zaneI had imagined that error codes could be generated via a transformation of the explain-data output. (Localization isn't necessary in this particular case since this is API is only used internally.)#2016-06-0116:40seancorfieldIf you’re going to transform the explain-data output, wouldn’t you then want to convert the keywords (including the spec names) to strings anyway?#2016-06-0116:41seancorfieldTaking this example from the Guide:
(s/explain-data ::name-or-id :foo)
;;=> {:clojure.spec/problems
;;    {[:name] {:pred string?, :val :foo, :via []},
;;     [:id] {:pred integer?, :val :foo, :via []}}}
#2016-06-0116:46seancorfieldYou’d want to convert :name and :id and whatever was in :via since those are keywords labeling parts of specs and paths through things — so why not convert the actual keys as well?#2016-06-0116:47seancorfield(having said that, we haven’t gotten as far as trying this sort of thing to generate responses to clients — we currently have custom validation… and that validation checks the types of values passed in as well as the structure so I’m not sure how much we could delegate to spec… but it’s an interesting option)#2016-06-0116:56zaneYes, you could totally convert the keywords back to strings, but you wouldn't have to if spec had better support for non-keyword keys in the first place. That's my whole point.#2016-06-0116:56zaneBut this is totally something that could be added via an extension library, so I guess I should roll up my sleeves. :relaxed:#2016-06-0117:01seancorfield(BTW, Alex Miller just replied on clojure-dev that he’s on vacation this week which is why he’s not participating here right now — expect more feedback next week!)#2016-06-0117:39iwankaramazowIs it possible to dynamically generate specs from some sort of schema? I have a rest api which exposes a schema like this /contacts/schema (json)
{ "type": "Struct",
  "keys": {
    "first_name": {"type": "String"},
    "last_name": {"type": "String"},
    "is_organization": {"type": "Boolean"},
    "updated_at": {"type": "DateTime", "options": {"format": "%FT%TZ"}},
    "emails": {
      "type": "List",
      "schema": {
        "type": "Struct",
        "keys": {
          "label": {"type": "String"},
          "email": {"type": "String"}}}}}}
From reading the docs, it wasn't clear if I could generate dynamic specs from the above schema.
#2016-06-0118:20seancorfield@iwankaramazow: Given that clojure.specs API is pretty much all macros, my initial reaction would be "No". I would expect you could generate a Clojure source file from your JSON and use that to define the specs, however.#2016-06-0118:21seancorfieldI’m not sure how much of the underlying spec representation is publicly exposed to support dynamic generation of specs...#2016-06-0118:22iwankaramazow@seancorfield: Thanks for the response! Feared this, I'll experiment a bit#2016-06-0118:25arohnerI’ll take the contrary stance#2016-06-0118:26arohnerit is possible to build a spec from that, but it requires building some new ā€˜primitives'#2016-06-0118:27arohnerit is possible to build a fn that validates a map containing strings, and it is possible to build a test.check generator for for that schema#2016-06-0118:27arohnerit’ll take a bit of work, but it’s doable#2016-06-0118:29arohner@seancorfield: spec’s two primitives are predicates and test.check generators. If you can build a predicate for it, you can use it in spec. If you can build a test.check generator, you can use it to generate#2016-06-0118:30arohners/keys is big because it generates a bunch of predicates, one for map? and then one for each key/value. there’s nothing special about it#2016-06-0118:32iwankaramazow@arohner: sounds logical, I'll brew me some algorithm. Thanks for the input#2016-06-0118:43seancorfield@arohner: What about dynamically (programmatically) registering those freshly built specs? Is enough of the machinery exposed for that?#2016-06-0118:44arohneryes#2016-06-0118:44seancorfield(and, yes, I know it’s all feasible since even private functions are accessible and spec is all built in Clojure — but my "No" was meant to indicate whether it is a realistic goal to attempt and I still maintain the answer is negative there)#2016-06-0118:45arohnerSure, we have the tools to rewrite spec. I didn’t mean to go that far#2016-06-0118:46arohnerMy point is just that 1) any predicate is usable in spec 2) s/keys uses no special machinery 3) you can build a predicate to validate maps of strings 4) you can build a test.check generator to generate maps of strings#2016-06-0118:46arohners/keys is the way it is because Rich is being opinionated, not that other map predicates shouldn’t exist#2016-06-0118:57seancorfieldSome of the machinery you’d need is private in clojure.spec (e.g., res) and some of the macros that you’d need to replicate as functions are a lot of code. So, yeah, "it’ll take a bit of work" is certainly true.#2016-06-0119:13arohnerok, you’re right that you’d lose the per-field error messages#2016-06-0119:16arohnerbecause currently, a predicate would look like (defn struct? [m] (and (map? m) (contains? m ā€œtypeā€) …)#2016-06-0119:18arohneractually#2016-06-0119:18arohnerone sec#2016-06-0119:20arohner
(s/conform (s/and map? (fn [m] (contains? m "type"))) {"type" "structā€})
#2016-06-0119:20arohners/keys just builds up a set of #(contains? % <key>) predicates. So you just (s/and map? <chain-of-key-checking-preds>)#2016-06-0119:21arohnerhrm, so that gets you key checking, but doesn’t conform values yet#2016-06-0119:24arohneryou could use more ands to check values, but ideally we could reuse the pass-specs-to-check-values thing#2016-06-0119:31seancorfieldā˜ļø:skin-tone-2: the sign of a Clojure developer with too much time on his hands… šŸ˜† It’s easy to get drawn into the "Hmm, that’s an interesting problem!" rabbit-hole!#2016-06-0121:00kennyWhich is preferred?
(s/conform (s/cat :v (s/spec (s/* number?))) [[1 2 3]])
=> {:v [1 2 3]}
(s/conform (s/cat :v (s/coll-of number? [])) [[1 2 3]])
=> {:v [1 2 3]}
I assume the latter as it feels cleaner.
#2016-06-0121:03mario@kenny: As far as I understand, the former will allow any seq, while the latter will only allow vectors. So depends on what you want.#2016-06-0121:04kenny@mario: Nope:
(s/valid? (s/cat :v (s/coll-of number? [])) '((1 2 3))) => true
#2016-06-0121:05kennyThe vec at the end is just the value s/conform conforms to.#2016-06-0121:06marioI see. Then I don’t know šŸ™‚#2016-06-0121:17jebberjeb@kenny: why are you using the s/cat there? Do you really want to spec that it's a collection with only one other nested collection (of numbers)?#2016-06-0121:24kenny@jebberjeb: Yes.#2016-06-0121:33stathissiderisis there any way to say that I want 2 optional elements at the end of a cat but they have to be both present or not at all?
(s/conform (s/cat :e1 string? :e2 (s/? keyword?) :e3 (s/? keyword?)) ["foo"])
#2016-06-0121:34stathissiderisa bit like that, but is e2 is there, e3 should be there as well#2016-06-0121:34jebberjeb@kenny @mario That second argument to s/coll-of is only used when working with the generator for that spec, I believe. This succeeds but probably isn't a good idea: (s/conform (s/? (s/coll-of number? '())) [[1 2 3]])#2016-06-0121:38jebberjeb@stathissideris: think you want (s/cat :s string? :opt (s/? (s/cat :k1 keyword? :k2 keyword?)))#2016-06-0121:38stathissideris@jebberjeb: that looks like it could work, let me try!#2016-06-0121:39stathissiderisworks, many thanks šŸ™‚#2016-06-0121:41jebberjeb@stathissideris: yw#2016-06-0122:31arohnerstathissideris: use s/& around the cat to pass in a pred#2016-06-0122:32stathissideris@arohner: thanks! that's an option too, but I think I'm happy with @jebberjeb 's solution#2016-06-0122:33arohnerah yes, I missed the ā€˜at the end’ part, so you can use an optional cat#2016-06-0122:33arohnerin general, & is used to add extra constraints to a seq#2016-06-0123:03kennyWhat is the best way to spec multiple arity functions where the args in one overload aren't necessarily related to the args in other overloads. Example:
(defn triangular-matrix
  "Returns a triangular matrix created from `coll`. `coll` is a 1D vector where each element will be used to create the
  triangular matrix. `upper?` is set to true to create an upper triangular matrix, false for a lower triangular matrix.
  `diagonal` is a 1D vector of elements on the diagonal. `off-diagonal` is a 1D vector of upper or lower matrix elements."
  ([coll upper?]
    ;;impl
    )
  ([diagonal off-diagonal upper?]
    ;;impl
    ))
#2016-06-0123:07arohnerI think that’s: (s/alt (s/cat :coll (s/coll-of …) :upper? boolean?) (s/cat :diagonal (s/coll-of …) :off-diagonal ….))#2016-06-0123:08arohner@kenny: ^^#2016-06-0123:09kennyYeah that makes sense to me#2016-06-0123:11kennyWhy is the common pattern to use cat instead of tuple for spec'ing args?#2016-06-0123:33stathissiderisspec looks amazing so far, the only thing missing is coercion#2016-06-0123:33stathissideristhis is the only point where I think schema is a bit more useful#2016-06-0123:33stathissiderisbut still, spec is much more expressive#2016-06-0123:50kenny@arohner: The only thing slightly annoying about that method is needed to specify keys for alt. The keys in this case are pretty arbitrary.#2016-06-0123:52stathissiderisI have a map that looks like this, but if the key is say :x I have a special spec for that value only: (s/map-of (s/or :k keyword? :k string?) ::s/any)#2016-06-0123:52stathissiderishow would I express that?#2016-06-0123:53stathissideriscustom predicate?#2016-06-0123:59stathissiderisvalidation works with a custom predicate:
(s/def ::simple-attr-map
  (s/map-of (s/or :k keyword? :k string?) ::s/any))

(s/def ::attr-map
  (fn [{:keys [transform] :as m}]
    (and (s/valid? ::simple-attr-map (dissoc m :transform))
         (or (not transform)
             (s/valid? ::transform transform)))))
#2016-06-0200:00stathissiderisbut as I expected, you lose conform destructuring and explain can't explain the problem anymore#2016-06-0200:00stathissideris...so different solutions would be welcome#2016-06-0200:01george.w.singerClojurescript is failing to compile my file if my fdefs are located above their corresponding defns. Is this the way things are intended?#2016-06-0200:01george.w.singerI.E., you can't spec your function before you define it?#2016-06-0200:02kenny@george.w.singer: Yes. The var is not defined yet. You could declare the var if you must write the spec above the function. I have just been writing my specs below my function definitions.#2016-06-0200:03george.w.singerOk, thanks.#2016-06-0200:05george.w.singerMacro sugar needs to be added to spec IMO#2016-06-0200:05george.w.singerSomething like#2016-06-0200:07george.w.singer
(fn-spec
 fn-name ::arg1, ::arg2 -> ::ret-spec 
                             :such-that    ::ret-value-greater-than-max-of-its-inputs)
This would at once declare our fn-name and annotate it with corresponding specs using intuitive Haskell-like syntax.
#2016-06-0200:13stathissideris@george.w.singer: you could write that macro šŸ™‚#2016-06-0200:14george.w.singeryea šŸ˜„#2016-06-0200:23arohner@george.w.singer: I think that’s a CLJS-impl bug, I’d report it#2016-06-0200:24arohnerin CLJ, you can (s/fdef new-fn) ahead of the var#2016-06-0200:26kenny@george.w.singer: Does it work if you do
(s/fdef 'your-sym
        :args ...
        :ret ...)
#2016-06-0200:35george.w.singerIt does sometimes but then the whole thing becomes buggy. Sometimes it doesn't compile. All tests pass#2016-06-0200:35george.w.singerReally strange behavior is exhibited#2016-06-0200:36george.w.singerThe strange behavior goes away if put them before#2016-06-0200:54kennyI seem to be getting a strange error when creating a spec for a function with a type hinted argument. Error:
java.lang.ClassCastException: clojure.spec$spec_checking_fn$fn__11414 cannot be cast to clojure.lang.IFn$DO
Example:
(defn bar
  [f]
  (f 0.0))

(s/fdef bar
        :args (s/cat :f fn?)
        :ret number?)

(defn foo
  [^double value]
  (bar (fn [_] value)))

(s/fdef foo
        :args (s/cat :v number?)
        :ret number?)

(defn foo2
  [value]
  (bar (fn [_] value)))

(s/fdef foo2
        :args (s/cat :v number?)
        :ret number?)

(foo 1.0) => error
(foo2 1.0) => 1.0

(s/instrument-all)
#2016-06-0200:57kennyProblem exists in 1.9.0-alpha4.#2016-06-0202:05seancorfieldLooks like primitive hinting encodes the argument and return type to be encoded in the underlying type (if you add ^double as a return type hint, the cast is to IFn$DD, for example).#2016-06-0202:51seancorfieldVery surprised to see that calling an instrumented function can actually load and call clojure.test.check to do generative testing as part of the conformance test, without explicitly calling clojure.spec.test/run-all-tests etc.#2016-06-0202:51seancorfieldhttp://dev.clojure.org/jira/browse/CLJ-1936#2016-06-0207:59stathissiderisis this a bug?
spec> (gen/sample (s/gen #{1 0}) 10)
(0 0 1 0 0 0 1 0 1 0)
spec> (gen/sample (s/gen #{false true}) 10)
(true true true true true true true true true true)
#2016-06-0208:02hiredmans/gen takes a spec, which is a predicate#2016-06-0208:02hiredman#{false true} will only return true, as a predicate, for true#2016-06-0208:14stathissideris@hiredman: thanks for your explanation, but I still don't get how I would generate true/false randomly#2016-06-0208:17slipsetI’ve defined these specs and vars#2016-06-0208:17slipset
(s/def ::point (s/cat :x number? :y number?))
(s/def ::body (s/+ ::point))
(s/conform ::point [2 3])
(s/conform ::body [[2 3]])
(s/explain ::body [[2 3]])
#2016-06-0208:18slipset::point conforms nicely, but ::body does not conform, and the explanation is:#2016-06-0208:18slipsetIn: [0] val: [2 3] fails spec: :snake-game.utils/point at: [:x] predicate: number?#2016-06-0208:19slipsetI would have thought that (s/+ ::point) would mean one or more points?#2016-06-0208:31stathissideris@slipset: (s/def ::point (s/spec (s/cat :x number? :y number?)))#2016-06-0208:32stathissiderisThe combination of s/+ and s/cat means that body is defined as a series of :xs and :ys without nesting#2016-06-0208:32slipsetthanks šŸ™‚#2016-06-0208:37stathissiderisfor the record, I ended up with this for booleans:
(defn- boolean? [x] (instance? Boolean x))
(s/def ::boolean (s/with-gen boolean? #(gen/boolean)))
#2016-06-0209:25stathissiderisis there any way to make s/cat specs generate a vector?#2016-06-0209:57slipsetWhat @stathissideris said :)#2016-06-0220:38surreal.analysisThis is a pretty minor note, but this appears to be one of the only channels with _ separation instead of -. Any chance an admin could change that?#2016-06-0220:39arohnerand as we all know, clojure names should prefer - to _
#2016-06-0220:40seancorfieldI see no stinkin’ underscores here...#2016-06-0311:16jcfI'm having some trouble spec'ing a fn that walks maps allowing you to hyphenate keys etc. I wonder if someone can point out my mistake as I'm having a hard time decrypting the error message…
(defn walk-map
  "Recursively apply a function to all map entries. When map is nil returns an
  empty map."
  [f m]
  (if m
    (walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)
    {}))

(s/fdef walk-map
  :args (s/cat :f (s/fspec :args (s/tuple ::s/any ::s/any) :ret ::kv)
               :m ::maybe-any-map)
  :ret ::any-map)

(defn hyphenate-keys
  "Recursively transforms all map keys from underscored strings to hyphenated
  keywords."
  [m]
  (walk-map (fn [[k v]] [(hyphenated-keyword k) v]) m))

(s/fdef hyphenate-keys
  :args (s/cat :m ::maybe-any-map)
  :ret ::any-map)
#2016-06-0311:16jcfThe error:
ERROR in (t-hyphenate-keys) (core.clj:4631)
expected: (= (sut/hyphenate-keys {:a 1, "a" 2}) {:a 2})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ({}) fails at: [:args :f] predicate: (apply fn),  nth not supported on this type: PersistentArrayMap
:clojure.spec/args  (#object[example.common$hyphenate_keys$fn__20568 0x754a38a0 "
#2016-06-0311:17jcfI've tried a s/cat to capture ::s/any but I'm sure I'm missing something.#2016-06-0311:18jcfWith the s/cat that I'd expect I need:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn),  clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args  (#object[example.common_test$fn__21912$fn__21926 0x7787a602 "
#2016-06-0311:20jcf
(s/fdef walk-map
  :args (s/cat :f (s/fspec :args (s/cat :kv ::s/any) :ret ::kv)
               :m ::maybe-any-map)
  :ret ::any-map)
Hmm.
#2016-06-0311:22jcfShouldn't an fspec with :args (s/cat :kv ::s/any) match any and all args?#2016-06-0312:27jcfThis is what's confusing me. This spec looks right, but doesn't work in the fspecs :args.
(s/explain-data (s/cat :kv (s/tuple ::s/any ::s/any)) [[:a {:b 2}]])
#2016-06-0312:31manutter51what is your hyphenated-keyword fn doing? I’m not sure I’m following what’s happening there, but it looks like it’s blowing up in that function, and that’s causing the spec failure#2016-06-0312:33jcf
(defn- hyphenated-keyword
  [x]
  (if (or (string? x) (keyword? x))
    (-> x keyword->string infl/hyphenate keyword)
    x))
@manutter51: I've used this code in a number of projects for a few years now. The fns are pretty reliable.
#2016-06-0312:36jcfThe relevant specs I'm using:
(s/def ::any-map
  (s/map-of (s/nilable ::s/any) (s/nilable ::s/any)))

(s/def ::maybe-any-map
  (s/nilable ::any-map))
#2016-06-0312:36jcfNot sure if ::s/any is already nilable… šŸ™‚#2016-06-0312:41jcfIn the walk-map test I get a cast exception:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn),  clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args  (#object[example.common_test$fn__21912$fn__21926 0x5ea46e79 "
In the tests that rely on walk-map:
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys {"a-b" 1}) {"a_b" 1})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args  (#object[example.common$underscore_keys$fn__20550 0x1f36bed9 "
#2016-06-0312:42jcfThese error messages aren't intuitive. I've got a feeling this will be another thing newcomers to Clojure really struggle with.#2016-06-0312:45jcfThis looks like it might help with the vague error message: https://github.com/clojure/clojure/commit/68fe71f0153bb6062754442c0a61c075b58fd9bc#2016-06-0312:45jcfIt recurses into a ::pcat to expand out the error explanation, and I think I'm looking at a ::pcat above.#2016-06-0312:48jcfOkay, so only the commented out test fails:
(deftest t-walk-map
  (are [f m x] (= (sut/walk-map f m) x)
    identity nil    {}
    identity {}     {}
    identity {:a 1} {:a 1}

    ;; (fn [[k ^long v]] [(name k) (if (number? v) (inc v) v)])
    ;; {:a 1 :b {:c 2 :d {:e 3}}}
    ;; {"a" 2 "b" {"c" 3 "d" {"e" 4}}}
    ))
#2016-06-0312:53jcfOkay. Looks like spec was exercising my walk-map with keys like [] and (). I was then calling name on vectors etc.#2016-06-0312:54jcfFixed that by making my test fn valid. šŸ™‚
(deftest t-walk-map
  (are [f m x] (= (sut/walk-map f m) x)
    identity nil    {}
    identity {}     {}
    identity {:a 1} {:a 1}

    (fn [[k v]]
      [(if (named? k) (name k) k)
       (if (number? v) (inc ^long v) v)])
    {:a 1 :b {:c 2 :d {:e 3}}}
    {"a" 2 "b" {"c" 3 "d" {"e" 4}}}))
#2016-06-0312:54jcfOne down. One to go!
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys nil) {})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args  (#object[example.common$underscore_keys$fn__20550 0x6dd8e64b "
#2016-06-0312:56jcfI wonder if it's the same problem. I need to make sure I have named keys.#2016-06-0314:01jcfIt was. Because I had enabled instrumentation my functions were being called with vectors, sets, etc. and those weren't supported.#2016-06-0314:50benzapso i'm converting from schema, and i'm having some issues with respect to preventing clutter within my namespace. maybe i'm not structuring my data correctly#2016-06-0314:51benzapex. `(def TextBlock {:type (schema/eq "Text") :foreground-color Color :background-color Color :style {(schema/optional-key :bold) schema/Bool (schema/optional-key :underline) schema/Bool (schema/optional-key :italic) schema/Bool} :text Letter}) (s/def ::foreground-color ::color) (s/def ::background-color ::color) (s/def ::style (s/keys :opt-un [::bold ::underline ::italic])) (s/def ::textblock (s/keys :req-un [::type ::foreground-color ::background-color ::style]))`#2016-06-0314:52benzaphow would I apply s/def to ::bold, ::underline and ::italic, without s/def? I don't want those in the namespace#2016-06-0314:52benzapif I did have them in the namespace, i'd prefer to have them called something like ::font-style-underline, but the underlying spec would need to accept :underline as the style key#2016-06-0314:54benzapIs there any way to do this? I'm confused on whether I should be inlining the spec stuff. I'm sticking to what I did with plumatic.schema by placing all of my schemas in one file called schemas.cljs#2016-06-0315:20jcf@benzap: I don't think you can. You need to def the keywords in order to enable reuse.#2016-06-0315:21jcfAnd I'd think you only want really common schema in a global namespace as spec is embracing namespaced keywords.#2016-06-0315:22jcfSo, :font/bold and :font/underline instead of :schemas/font-style-underline etc.#2016-06-0315:24jcfI've used specs like :common/any-map because I want nilable maps in a lot of places, but there might be a better way I haven't yet found.#2016-06-0315:25jcf(And I've done away completely with my shared schema namespaces for now.)#2016-06-0315:29benzap@jcf: I see, so I guess i'll have to start embracing namespaced keywords as well šŸ™‚#2016-06-0315:30benzapI already have a font.cljs, would it be common to place spec in-line with the code?#2016-06-0315:33jcfThat's what I'm doing, and what I've seen in all the examples.#2016-06-0315:34jcfIt makes sense keeping everything starting with font in font.cljs; that's where I'd look at least!#2016-06-0319:08nwjsmithHrm, is there a way to specify a transform to a spec's conformed data?#2016-06-0319:09nwjsmithI'm speccing datomic query, which accepts either a list or a map, and I'd like them to conform the same way#2016-06-0319:32nwjsmithFound it! Looks like clojure.spec/conformer is what I was after.#2016-06-0321:03hiredmanI haven't seen this discussed elsewhere, but as a warning, protocol functions called in a non-higher order way (where clojure knows the protocol function is being called an creates an optimized call site for it) don't properly check args and returns against specs if instrumented#2016-06-0415:48nwjsmithI'd like to ship a library without requiring test.check, and have the generators dynamically loaded as they are with clojure.spec, but I can't figure out how to do that. Do I wrap all calls to clojure.spec.gen functions in clojure.spec.gen/delay?#2016-06-0415:50nwjsmith(the library is essentially a bunch of specs, some of which require custom generators)#2016-06-0418:40hiredmanthat is kind of interesting to think about, if you are defining custom specs with generators in your source, then test.check needs to be a regular dependency not a test scoped dependency#2016-06-0418:41hiredmana delay delays runtime code from running, not compile time stuff#2016-06-0418:41hiredmanlike var resolution#2016-06-0418:42hiredmanyou likely need some combination of something like dynaload from clojure.spec.gen and the delay there and maybe clojure.core/delay#2016-06-0418:43bbloomor just put your generators in another namespace?#2016-06-0418:45hiredmanthat would still have issues with any kind of tooling that loads all namespaces (in the absence of test.check)#2016-06-0418:45bbloomwhat kind of tooling does that? seems kinda like a bad idea šŸ˜›#2016-06-0418:47hiredmanI was thinking of stuff like lein check, but that only loads stuff from the current project#2016-06-0418:49bbloomon another track…. 1st order predicate expressions and regular languages are closed under intersection — would be cool if somebody wrote an ā€˜intersect’ function for spec, could be useful for static analysis#2016-06-0418:52arohner@bbloom: elaborate?#2016-06-0418:55bbloom@arohner: well you could just write an abstract interpreter that walks the clojure AST and instead of an environment of symbols -> values, have an environment of symbols -> specs#2016-06-0418:55arohneroh, yeah#2016-06-0418:55bbloomeach time you use something, you call intersect on it#2016-06-0418:55bbloomif the result is void, you have a "type error"#2016-06-0418:55arohneryeah, just wasn’t sure what you meant by intersect#2016-06-0418:55arohnerI have about half of that written#2016-06-0418:56arohnercurrently reimplementing the regex part of spec to handle it#2016-06-0418:56bbloomif each spec defines a set, intersect = set intersection#2016-06-0418:57bbloomintersecting two predicates is just and-ing them, but you could do better for some predciates, cat, repeat, alt, etc#2016-06-0418:58arohneryeah, it’s a lot more work than that šŸ™‚#2016-06-0418:59bbloomcertainly not trivial, but should be at least straightforward, no?#2016-06-0418:59arohneryeah#2016-06-0418:59arohnerparsing specs and the regex stuff is straightforward, but a ton of code#2016-06-0419:00arohnersame w/ flowing predicates through a fn body#2016-06-0419:01bbloomyeah, i have a clojure interpreter that i’ve copy/pasted and modified 4 or 5 times for code walking purposes - someday i’ll make it a library#2016-06-0419:01bbloomin the mean time: https://github.com/brandonbloom/metaclj/tree/master/src/metaclj/impl#2016-06-0419:02bbloomthe transform namespace is an example of a code walk on that ast & you could replace that with a type check walk#2016-06-0419:03bbloomhere’s an interpreter: https://github.com/brandonbloom/eclj/blob/master/src/eclj/interpret/meta.eclj#2016-06-0419:03arohnerI’m already walking the tools.analyzer output, so it’d be a good amount of work to swap#2016-06-0419:04bbloomnot suggesting that you switch to my thing šŸ˜› just showing that it can be much less code than tools analyzer, which does all the analysis all at once#2016-06-0419:26bbloomanyway @arohner: excited to see what you come up with šŸ™‚#2016-06-0419:27arohneryeah, me too. Just need to finish the regex matching crap#2016-06-0421:04gfredericksthe first big question about clojure.spec that I'm going to try to dive into is how I can get functionality similar to https://github.com/gfredericks/schema-bijections#2016-06-0421:05gfredericksso I suspect (ha!) I'll be reading a bunch of the clojure.spec source soon#2016-06-0421:20gfredericksis it weird that s/tuple takes preds instead of specs?#2016-06-0422:11benzapI thought preds were specs#2016-06-0422:12benzapI have a tuple '(s/def ::color (s/tuple ::byterange ::byterange ::byterange ::byterange))' that seems to work @gfredericks#2016-06-0422:19benzapas of now, I have successfully converted my project from plumatic.schema to cljs.spec!#2016-06-0422:19benzapstill have a lot to learn about it, but it sure is nice#2016-06-0423:19gfredericksbenzap: yeah it does seem to work, but that means the docs are misleading I think: preds are specs but specs aren't preds#2016-06-0500:08Alex Miller (Clojure team)the terminology drifted a bit during dev#2016-06-0502:55adambrosIs there anything planned in terms of having predicates/specs with custom error messages/data? I’m speaking from a validation perspective, because the alternative is to wrap explain-data#2016-06-0502:56adambrosit also seems that for related reasons, I would be writing a conform-or-explain! function that throws with explain-data when the conform fails#2016-06-0503:02adambrosWhile I’m asking, it would be a useful feature to be able to somehow get the path at which a predicate is being run. This one is coming up as I’m trying to secure a datomic transaction wrapper by looking up if a user has access to a certain :db/id that’s potentially nested within a map.#2016-06-0503:25seancorfield@adambros: which alpha are you using? I thought they'd added a path result in explain in one of the later alpha builds.#2016-06-0503:27adambrosi’m referring to accessing the path from within the predicate#2016-06-0503:28adambrosmight be bad idea (TM) but its something that would ameliorate my problem#2016-06-0503:30seancorfieldBut each predicate only knows what it is called on -- it can't know the whole path, since a predicate is a general function.#2016-06-0503:32seancorfieldIf your predicate is string? for example...#2016-06-0503:32adambrosit could if there was a dynamic var *path* or something. I think i could solve this another way so its fine#2016-06-0503:33seancorfieldDynamic vars are evil šŸ˜†#2016-06-0503:33adambros*evil*!#2016-06-0503:33seancorfieldšŸ™‚#2016-06-0503:33adambrosim more interested in my other questions, so i probably shouldn’t have asked about it#2016-06-0503:37seancorfieldOk, what are the other questions? I think I missed those...#2016-06-0503:38adambroscustom error message for predicates/specs something like (s/def :title (s/either string? {:message ā€œinvalid title"}))#2016-06-0503:39adambroswhere either’s predicate either succeeds or puts the 2nd argument in explain-data’s result#2016-06-0503:40adambrosand the other was about a potential builtin for validation, conform-or-explain! that throws if it cant conform#2016-06-0503:41seancorfieldI get the impression from the discussions here that explain-data is indeed the preferred basis for any custom error messages. I think there is a JIRA ticket for something like conform-or-explain tho'... let me go look...#2016-06-0503:46seancorfieldHmm, no, maybe it was just a discussion on the clojure-dev mailing list.#2016-06-0503:47adambrosseems like it’d be a simple wrapper, with an extra arity for a function to call with the data when it fails#2016-06-0503:47adambros(defn conform-or-explain [spec x & [on-fail]] ...)#2016-06-0503:48adambrosby default it’d throw the error in an ex-info#2016-06-0503:51seancorfieldAh, I remember now, it was a valid or explain wrapper that was discussed -- for use in :pre assertions.#2016-06-0503:55seancorfieldBut, yeah, I think the wrapper is so easy to write -- and some folks would want to throw explain-data and some would throw explain as a string so I don't think there's a one size fits all here.#2016-06-0503:56adambrosfair enough, sounds like I just need to have some sort of spec namespace that declares these wrappers#2016-06-0503:58seancorfieldOne of the great things about Clojure: it's easy to write an adapter layer to make the supplied libraries match each specific application's needs šŸ™‚#2016-06-0503:58seancorfieldWe have a bunch of wrappers at World Singles. We have several of our own string functions in a namespace that supplements clojure.string.#2016-06-0504:07seancorfieldWe also have some collection functions, a bunch of date stuff, wrappers around system level stuff. Wrappers around JDBC stuff (and, heck, I maintain clojure.java.jdbc!).#2016-06-0513:30benzapanyone know of a good way to define a spec for an atom holding a piece of structured data?#2016-06-0513:31benzapseems like I would need to deref in order to perform the spec, so a lot of wrapping even for simple checks#2016-06-0514:01nwjsmith@benzap: I think someone asked this a few days ago and the answer was "There isn't because atoms aren't 'data'". IIRC there was also a suggestion to take a look at coll-of, which might have some bits you could crib to implement an atom-of. I might look into this today.#2016-06-0515:09Alex Miller (Clojure team)personally, I think you should work with state by a) defining your problem in terms of data b) writing nearly every function as a pure function from data to data c) isolating the use of stateful entities to as small number of functions as possible d) focusing on writing great specs for b in terms of a#2016-06-0515:10Alex Miller (Clojure team)not every function has to have a spec#2016-06-0518:50gfrederickshas anybody started a clojure.spec utility library yet because I keep thinking about doing that#2016-06-0518:51gfredericksshould probably call it spook#2016-06-0519:00Alex Miller (Clojure team)you may want to check your urban dictionary before you venture down that path#2016-06-0519:00gfredericksurban dictionary has all the best words#2016-06-0519:01gfredericksspock#2016-06-0519:01Alex Miller (Clojure team)totally on a tangent, the urban dictionary was hiring for a full stack clojure position recently#2016-06-0519:02Alex Miller (Clojure team)spock is much better :)#2016-06-0519:02Alex Miller (Clojure team)although I think there is a scala (or groovy?) testing library called that already#2016-06-0519:02Alex Miller (Clojure team)https://github.com/spockframework/spock#2016-06-0519:03Alex Miller (Clojure team)spackle?#2016-06-0519:06gfredericksshpec#2016-06-0519:07gfrederickssplec, shplec#2016-06-0519:08bronsashpec sounds like something sean connery would say#2016-06-0519:08gfredericksokay so we already have a mascot#2016-06-0519:09gfredericks(require [com.gfredericks.shpec :as s'])#2016-06-0519:10bronsajust realized 'foo' is valid clojure#2016-06-0519:10gfredericksI only wish the syntax highlighters were more often aware of that#2016-06-0519:10gfredericks...because I use that all the time#2016-06-0519:34mfikesReplete (standalone ClojureScript iOS app) has an update in the App Store with support for cljs.spec.#2016-06-0520:16gfredericksśpƩć#2016-06-0520:55stathissiderisis there any way to make s/cat specs generate a vector?#2016-06-0520:59gfredericksstathissideris: both s/coll-of and s/tuple can generate vectors -- are either of those usable for you?#2016-06-0521:00stathissideris@gfredericks: tuple would work for my validation, but ideally I would like to keep the destructuring that s/cat provides#2016-06-0521:01gfrederickswhat you want is a way to gen/fmap the default generator#2016-06-0521:01Alex Miller (Clojure team)use with-gen to override the generator with fmap#2016-06-0521:01gfredericksalexmiller: but how do you get the default generator to pass to fmap?#2016-06-0521:02Alex Miller (Clojure team)well you could call s/gen on the original spec but I guess you may end up repeating yourself#2016-06-0521:02gfredericksthat was the hack I was about to suggest#2016-06-0521:03gfrederickswe can put a helper for this into shpec#2016-06-0521:04stathissideristhank you both! I'm not very well-versed with test.check, but I'm saving this part of the log so that I can go do some reading. It will hopefully make more sense after that šŸ™‚#2016-06-0521:05gfredericksstathissideris: we're suggesting (s/def ::foo <definition>) (s/def ::foo (s/with-gen <definition> (let [g (s/gen ::foo)] #(gen/fmap vec g))))#2016-06-0521:05gfredericksand <definition> being there twice is the repetition alex mentioned#2016-06-0521:07stathissiderisgreat thanks, I'll give it a go. In my case <definition> is not too long, so it could be ok#2016-06-0521:08gfrederickscould even try factoring it out to a let and see if that works#2016-06-0521:08stathissideris...or even a macro is that doesn't work#2016-06-0521:11gfredericksalexmiller: the default generator for s/and is necessarily pretty hacky; should there be some documentation somewhere suggesting an override?#2016-06-0521:12gfrederickssomehow overriding test.check's normal error message for such-that would be a fancy approach#2016-06-0521:17Alex Miller (Clojure team)I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no#2016-06-0521:18Alex Miller (Clojure team)but I'm sure Rich would have more to say about that#2016-06-0521:20gfredericksthat sounds sooper fancy#2016-06-0521:21gfredericksalexmiller: do you know if recursive specs is the only reason for the no-arg-function-returning-a-generator pattern?#2016-06-0521:22Alex Miller (Clojure team)Stu worked on that but I assume the dynamic loading is another aspect#2016-06-0521:22gfredericksoh that sounds plausible#2016-06-0612:08mandersonCurious if there has been any thought given to adding a doc string parameter to clojure.spec/def like there is in clojure.core/def? I understand the spec should be self-explanatory, but it's nice to be able to capture additional information/description about the spec. I've found myself adding comments that would be nice to have as doc strings that could then also be parsed programatically.#2016-06-0612:09Alex Miller (Clojure team)yes, there has been thought about it#2016-06-0612:11Alex Miller (Clojure team)not sure if anyone noticed, but you can now call doc with a registered spec keyword to get it's spec definition. that's a place that docstring could show up.#2016-06-0612:11Alex Miller (Clojure team)but it's something Rich is still thinking about afaik#2016-06-0612:14mandersoncool, thanks Alex. I think the built in doc string could be really useful.#2016-06-0612:14mandersonAlso, didn't notice the doc addition. That's great.#2016-06-0613:53mike_ananevHello all! I learn clojurescript. Today I discovered that clojurescript doesn't throw Exceptions in code above like Clojure. Can clojure.spec help in such situations? I use reagent and want prevent state updating from using inappropriate function.#2016-06-0614:02nwjsmith@mike1452: Hey Mike, welcome to ClojureScript! clojure.spec can definitely help you here:#2016-06-0614:03mike_ananev@nwjsmith: Thank you!!#2016-06-0615:26gfredericksnwjsmith: the spec needs to be (s/cat :arg integer?)' instead of just integer?`, right?#2016-06-0615:26gfrederickss/integer/number/#2016-06-0615:27nwjsmithThat's what I thought at first, but maybe this is a situation where a pred is implicitly converted to a spec?#2016-06-0615:29gfredericksI guess I'll just go try it :)#2016-06-0615:29nwjsmithwhen passed to valid? or conform, predicates will be implicitly converted to a spec, which I think is what happens here.#2016-06-0615:31gfredericksit throws an error when you call it with a number as well#2016-06-0615:31gfredericksit's not about spec vs predicate it's about spec-for-a-single-arg vs spec-for-an-arglist#2016-06-0615:33nwjsmithooooh#2016-06-0615:36nwjsmith@mike1452 looks like the example I gave above is wrong. The call to s/fdef should be:
(s/fdef inc :args (s/cat :x number?))
thanks @gfredericks
#2016-06-0615:41gfredericksnp#2016-06-0616:47mike_ananev@nwjsmith: got it! Thanks!#2016-06-0618:03angusiguessIs there a nice way to spec the following?#2016-06-0618:04angusiguess"I want a map of arbitrary keywords whose values must conform to some predicate"#2016-06-0618:05nwjsmithmap-of#2016-06-0618:06angusiguess@nwjsmith: Beauty, thank you šŸ™‚#2016-06-0618:06nwjsmithnp#2016-06-0622:42wilkerluciowith clojure.spec I'm finding myself wanting to use some kind long namespaces for my keywords, they are just getting to be a borden to write, for example (s/keys :req [:my-app.some-ns.clock/at, :my-app.some-ns.clock/namespace]), the namespace my-app.some-ns.clock doesn't actually exists, it's just helpful to avoid name clashing, I'm writing those in the my-app.some-ns#2016-06-0622:42wilkerlucioI was thinking that would be useful to be able to alias those namespaces so they can be used in a simpler way, for example:#2016-06-0622:44wilkerlucioit's possible to alias like this with the currently clojure features?#2016-06-0622:45noonianI believe if you require a namespace like (ns my-ns (:require [my-app.some-ns.clock :as clock])) you can use ::clock/foo and it will resolve to :my-app.some-ns.clock.#2016-06-0622:46wilkerlucio@noonian: yes, but that would require me to create a file with that namespace, that's what I would like to avoid#2016-06-0622:47noonianAh got it, sorry I didn’t read your message carefully enough.#2016-06-0622:47jr@wilkerlucio: see this proposed feature http://dev.clojure.org/jira/browse/CLJ-1910#2016-06-0622:48wilkerluciothanks @jr, I had saw that one, and I agree that will help, but only on map cases... when you are setting it as keywords for lists (on the s/keys for example) you can't use that, so we would still have to repeat the typing, I believe aliasing is a more general solution here that could be very useful with long namespaces#2016-06-0622:48hiredmanyou don't actually need to create a file now#2016-06-0622:49hiredmancreate-ns#2016-06-0622:51hiredman
user=> (create-ns 'my-app.some-ns.clock)
#object[clojure.lang.Namespace 0x3b7d3a38 "my-app.some-ns.clock"]
user=> (alias 'clock 'my-app.some-ns.clock)
nil
user=> ::clock/foo
:my-app.some-ns.clock/foo
user=> 
#2016-06-0623:19wilkerlucio@hiredman: thanks, I'm going to try that soon, do you think that can work on cljs as well?#2016-06-0623:21hiredmancljs is a different kettle of fish, I am not sure#2016-06-0623:31wilkerlucioyeah, I have to fix some compilation issues here and then I'll try it out#2016-06-0702:33Alex Miller (Clojure team)@wilkerlucio: in addition to the new namespaced support in CLJ-1910 and CLJ-1919 we are considering changes related to aliasing of non-existent namespaces too#2016-06-0702:34wilkerlucio@alexmiller: cool, that would be nice, specially for cljs where I can't do the trick mentioned by @hiredman#2016-06-0702:37Alex Miller (Clojure team)I did some recon on it a few weeks ago and we talked about it briefly again today#2016-06-0702:39wilkerlucioI'm happy to hear that this is being considered, I think with clojure.spec the usage of fully namespaced keywords is going to have a great increase in usage#2016-06-0708:08jcfFound some clojure.spec benchmarks over on Reddit: https://muhuk.github.io/validation-benchmark/ N.B. I haven't independently verified any of the work.#2016-06-0708:29slipsetplaying with specs and tests in Clojurescript I get this error:#2016-06-0708:29slipset actual: #error {:message "Call to [object Object] did not conform to spec#2016-06-0708:29slipsetThe error is probably correct, but the message leaves something to be desired.#2016-06-0708:58slipsetHaving said that, specs are awesome!#2016-06-0711:41mishadoffHi, could you help me understand clojure.spec Here is the little snippet:
(s/def ::name string?)
(s/def ::specific-name #{"John" "Jack"})
(s/def ::age (s/and integer? #(<= 0 % 100)))
(s/def ::person (s/keys :req-un [::name ::age]))
(s/explain ::person {:name "John" :age 10})
Is there a way to validate that map contains specific key like :name, but value should be matched against another spec, ::specific-name?
#2016-06-0711:46slipset(s/def ::person (s/and (s/keys :req-un [::name ::age]) #(#{ā€œJohnā€ ā€œJackā€} (:name %))))#2016-06-0711:47slipsetnot answering your question, but should work.#2016-06-0711:49mishadoffyes, but would be good to use already defined spec ::specific-name instead of copying its implementation#2016-06-0711:55slipsetDon't know your code, but you could (def specific-names #{...}) and use that var in the specs#2016-06-0711:57mishadoffI can, but that not a spec anymore#2016-06-0712:12Alex Miller (Clojure team)you can do (s/def ::name ::specific-name) to just alias an existing one#2016-06-0712:12Alex Miller (Clojure team)spec will "chase" registered names like that#2016-06-0712:14Alex Miller (Clojure team)or you could define a predicate that is reused for multiple specs#2016-06-0714:12bronsalooks like spec might need some performance tuning https://muhuk.github.io/validation-benchmark/#2016-06-0714:27Alex Miller (Clojure team)I don't think that test is very good, but that may also be true. will be looking at it today#2016-06-0714:31seancorfieldThe use case for clojure.spec is different to Schema tho', right?#2016-06-0714:33seancorfieldYou might have conform in several places for destructuring and some calls to valid? but you're not going to have everything instrumented in production code. #2016-06-0714:33seancorfieldWhereas folks do have Schema enabled in production don't they? Hence the focus on performance there. #2016-06-0714:34gfredericksschema aims to be used similarly though#2016-06-0714:34seancorfieldAh, so not in production code then?#2016-06-0714:34gfrederickswhen you decorate functions with schemas they don't run by default#2016-06-0714:34gfredericksand similar to spec you can turn them all on#2016-06-0714:34gfredericksfor testing#2016-06-0714:35gfredericksand you can call them explicitly to validate in/out in production#2016-06-0714:35gfredericksso maybe they focus on performance primarily for that last case?#2016-06-0714:44Alex Miller (Clojure team)we still want spec to have good performance, even when not used in production :)#2016-06-0715:09bronsa@seancorfield: we disable schema in prod, performance hit is way too much#2016-06-0715:19wilkerlucio@mishadoff: like alexmiller said, you can create a new one, if you still need that key on the current namespace, make up a new namespace, eg: (s/def :other-ns.anything/name ::specific-name) and then (s/keys :req-un [:other-ns.anything/name])#2016-06-0715:21mishadoff@wilkerlucio: thanks, for now I’ve decided to use same names for keys and specs#2016-06-0715:22wilkerlucio@mishadoff: cool, that is the preferred way I believe šŸ™‚#2016-06-0715:23mishadoff@wilkerlucio: yes, but sometimes you have maps with unqualified keys from third-party libs#2016-06-0715:24wilkerluciothat's true, maybe with the clojure.spec people will start using more fully qualified keywords from now one, and possibly with specs already coming from the library itself, will be awesome šŸ™‚#2016-06-0715:53gfredericksis :opt in s/keys just for documentation purposes?#2016-06-0715:53gfredericksI can't figure out how else it affects anything#2016-06-0716:09angusiguess@gfredericks: does it work with generators?#2016-06-0716:09gfredericksoh that's probably true#2016-06-0716:10gfredericksyes indeed#2016-06-0716:11gfrederickscool.#2016-06-0716:42Alex Miller (Clojure team)yes, those reasons :)#2016-06-0716:44Alex Miller (Clojure team)bunch of new predicates (most with gen support) just landed in master https://github.com/clojure/clojure/commit/58227c5de080110cb2ce5bc9f987d995a911b13e#2016-06-0716:45Alex Miller (Clojure team)also, the long-awaited seqable? :)#2016-06-0717:01andrewhrAlso, a nice bonus to see an abstraction for Inst šŸ™‚#2016-06-0717:05seancorfieldAnd Alpha 5 is coming… when? 😸#2016-06-0717:29gfredericksalexmiller: oh boy this inst generator#2016-06-0717:30Alex Miller (Clojure team)@seancorfield: winding its way through the tubes#2016-06-0717:31gfredericksTIL that java.util.Date does weird things with negative years#2016-06-0717:31Alex Miller (Clojure team)ha#2016-06-0717:31Alex Miller (Clojure team)probably better to use something like inst-in with that :)#2016-06-0717:33gfrederickscoming up with reasonable generators in a lot of cases is difficult :/#2016-06-0717:33gfredericksreasonable defaults I mean#2016-06-0717:34gfredericksstrings are my best example of that#2016-06-0717:34Alex Miller (Clojure team)yeah#2016-06-0717:34Alex Miller (Clojure team)here's a bunch of stuff that in no way resembles reality #2016-06-0717:34gfredericksa uniform distribution over unicode characters would give you 99% unprintable things#2016-06-0717:40Alex Miller (Clojure team)1.9.0-alpha5 https://groups.google.com/d/msg/clojure/D_s9Drua6D4/CTWk12cXDQAJ#2016-06-0717:41Alex Miller (Clojure team)notably for spec is the new unform which lets you conform ... backwards#2016-06-0717:42Alex Miller (Clojure team)also in addition to the vast quantity of new predicates in core, there are now specs for long, double, and instant ranges in spec#2016-06-0717:42Alex Miller (Clojure team)and everything gens yay#2016-06-0717:45gfredericksyay#2016-06-0717:46seancorfieldThat’s an awesome new release! Thank you (to everyone involved)!#2016-06-0717:52angusiguessyessssss#2016-06-0718:10brabsterhey folks, I have a question on map-of in spec that I'd appreciate some help with#2016-06-0718:10brabsterhopefully this is a good forum, even if it turns out to just be me being dumb!#2016-06-0718:11brabsterif I use s/or in the value spec in a map-of, I lose the qualifier I get when I use it on its own#2016-06-0718:12brabstereg. as per docs (s/def ::thing (s/or :name string? :id integer?)) conforms eg. (s/conform ::thing "bob") to [:name "bob"]#2016-06-0718:12brabsterbut... (s/def ::map-of-kw-to-thing (s/map-of keyword? ::thing)) doesn't behave the same was under conform#2016-06-0718:13brabster(s/conform ::map-of-kw-to-thing {:foo "bob"}) gives {:foo "bob"}, not {:foo [:name "bob"]} as I was expecting#2016-06-0718:14brabsterAm I missing something? I expected ::thing to behave the same way regardless of where it was used#2016-06-0718:16brabsteroh sorry I see a message in the history that I missed from @alexmiller: Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this - sorry!#2016-06-0718:20brabsteris there somewhere I can go to see whether this is planned or to ask for it?#2016-06-0718:22gfredericksqueue the warnings from all your favorite libraries that clojure.core/boolean? is being replaced by their local copy#2016-06-0718:25pheuteryep.#2016-06-0718:26Alex Miller (Clojure team)hopefully all benign if the work we did in 1.7 was successful :)#2016-06-0718:29gfrederickswhat sort of work?#2016-06-0718:30hiredmanisn't http://dev.clojure.org/jira/browse/CLJ-1591 still open? I didn't accidentally re-open it did I?#2016-06-0718:32Alex Miller (Clojure team)it's open but it's very narrow in scope iirc - only when defining a new fn of the same name in terms of the old function of the same name, right?#2016-06-0718:32Alex Miller (Clojure team)the general case of just overlapping the name was resolved afaik#2016-06-0718:33gfredericksah#2016-06-0718:33Alex Miller (Clojure team)prior issues came up when we added "update"#2016-06-0718:33bronsayeah, I don't actually think that's a bug#2016-06-0718:33Alex Miller (Clojure team)we have since added other things that were in general use without issue (although I can't remember an example now)#2016-06-0718:34bronsathe fact that def declares the var at analysis time is the only way to write recursive functions with defn#2016-06-0718:34hiredmanmmm, I guess I am note sure what version of clojure the guy was running when I w as helping him debug this#2016-06-0718:34Alex Miller (Clojure team)@hiredman: I'm not sure if your final example in the comments is actually the same thing or not, not sure?#2016-06-0718:34bronsaFWIW I couldn't reproduce that example#2016-06-0718:35gfredericksshould it be possible to write a function that verifies that none of the keyword-references used in any specs have typos?#2016-06-0718:35gfredericksusing keywords instead of vars makes the whole thing feel a lot more typo-sensitive#2016-06-0718:36hiredmanyeah, I didn't look at the history to see the current state of the bug, just spent a while trying to figure out why this guys defmulti was resulting in an unbound var, and after way too long I remember that bug, and he said changing the name of his function fixed it, but that is kind of loose, who knows what is happening on the other end of irc#2016-06-0718:36gfredericksby "typos" I mean a keyword that is supposed to refer to a spec but no spec has been registered#2016-06-0718:36Alex Miller (Clojure team)that's not a typo#2016-06-0718:36Alex Miller (Clojure team)necessarily#2016-06-0718:37Alex Miller (Clojure team)as long as it's registered by the time you use it, it's fine#2016-06-0718:37gfredericksright, so I mean "never registered"#2016-06-0718:37Alex Miller (Clojure team)well never is a long time#2016-06-0718:37gfredericks:)#2016-06-0718:37gfredericksI could have a test in a codebase that looks at this for example#2016-06-0718:38gfredericksso it would have the responsibility to ensure that all pertinent code is loaded already
#2016-06-0718:38pheuterSo I’m trying to play around with the new clojure.spec generators, and I keep getting this error: Var clojure.test.check.generators/large-integer is not on the classpath. I definitely have test.check as a dep and i can access various vars under the clojure.test.check.generators namespace, but large-integer isn’t one of them.#2016-06-0718:39gfrederickspheuter: did you add test.check magically later?#2016-06-0718:39gfredericksthat's when I get that error#2016-06-0718:39pheuterno, we had it as a dep for a while#2016-06-0718:39gfrederickspheuter: is it an older version? large-integer is new#2016-06-0718:39Alex Miller (Clojure team)are you using test.check 0.9.0 ?#2016-06-0718:39pheuteryeah, 0.9.0#2016-06-0718:39Alex Miller (Clojure team)do you have an older version also on the classpath?#2016-06-0718:39pheuterlet me clear deps and try to re fetch#2016-06-0718:39Alex Miller (Clojure team)lein deps :tree#2016-06-0718:40gfredericksyeah do tree ↑ and look for conflicts#2016-06-0718:40gfredericks(at the top of the output)#2016-06-0718:42pheutergood call, will take a look and make some exclusions, thanks!#2016-06-0718:50Alex Miller (Clojure team)@brabster: I asked Rich, he said the point of that was that conform samples (does not check every value) for perf reasons and that is unlikely to change. we might make the docs better around it though#2016-06-0718:53brabster@alexmiller: thanks for the info, would suggest mentioning it with an example in the docs as that's not the behaviour I'd expect from map-of - working around it at the moment but would have been a nice multi-method dispatch off the conformed inputs, looks ugly in comparison now. Will share when pushed up to github, maybe I'm just misusing the functionality!#2016-06-0718:55Alex Miller (Clojure team)I making some updates to the guide for alpha5 right now - I'll try to add a note about that#2016-06-0718:56brabsterbtw spec is great imho - really neat and well thought through after a few hours of using it#2016-06-0719:06gfredericksalexmiller: the thing that makes me uneasy is when I have to type a fully qualified keyword referencing a spec#2016-06-0719:06gfredericksbut where it's actually defined elsewhere using ::#2016-06-0719:07gfredericksor regardless actually#2016-06-0719:07gfrederickswith var namespaces the compiler will catch renamings that were only partially done#2016-06-0719:07gfrederickswith specs it just looks like something isn't defined yet#2016-06-0719:58pheuterany ideas on how to use generators, possibly custom, to generate Datomic facts? More specifically, I’d like to generate legitimate temp ids, but it’s not clear to me how to express that as a predicate.#2016-06-0719:59gfrederickstemp ids are a custom datomic type aren't they?#2016-06-0719:59pheutera thought is to use a set predicate that’s populated with a bunch of tempids#2016-06-0719:59gfredericksso the predicate would be #(instance? ThatType %)#2016-06-0720:00gfrederickswhat's the API to create a tmp id? it's a 0-arg function and it returns a unique object each time?#2016-06-0720:00pheutereven simpler, they can be verified as longs! the problem is being able to actually transact them against a test db#2016-06-0720:00pheuter(tempid part)#2016-06-0720:00gfrederickslong?#2016-06-0720:00gfrederickslongs?#2016-06-0720:00gfredericksthat can't be right...#2016-06-0720:01gfredericks(tempid part) returns a long?#2016-06-0720:01pheuterwell not exactly, it returns a :db/id reader value, but you can pull out an :idx long from it#2016-06-0720:01pheuterthe problem is getting the generate to generate values in sequential order#2016-06-0720:01gfrederickssequential?#2016-06-0720:02gfrederickswhy does that matter?#2016-06-0720:02pheuterperhaps that part doesn’t matter, that’s just my interpretation of the error I get: db.error/not-a-partition Entity id -1000441 is not in a valid partition#2016-06-0720:02pheuterwhere -1000441 is a sample temp id#2016-06-0720:02gfredericksare you passing a valid partition to tempid when you call it?#2016-06-0720:03gfredericksoh um#2016-06-0720:03pheuterso how would i call it?#2016-06-0720:03gfredericksyou just generate a random long and construct the thing that way rather than calling tempid?#2016-06-0720:04pheuterright, since -1 < n < -100000 is reserved for tempids in the user partition#2016-06-0720:04gfredericksbut -1000441 is out of range, does that mean you're accidentally generating out of range numbers?#2016-06-0720:04pheutersorry, -1000000 is the actual bound#2016-06-0720:04gfredericksstill out of range#2016-06-0720:04pheuteryou’re right, let me double check#2016-06-0720:05pheuteri’ll feel very silly if that’s the case šŸ˜•#2016-06-0720:05gfredericksfeeling silly is what programming is all about#2016-06-0720:06mishahttp://blog.cognitect.com/cognicast/103#2016-06-0720:08pheuter :db.error/not-a-partition Entity id -3427 is not in a valid partition ĀÆ\(惄)/ĀÆ#2016-06-0720:10pheuterah, it may be how i’m transacting that value, since it expects it in a #db/id reader value#2016-06-0720:10pheuterat least when using the map form#2016-06-0720:20zaneFor anyone following along, the approach we're gonna try is using clojure.test.check.generators/fmap in conjunction with datomic.api/tempid and clojure.test.check.generators/choose.#2016-06-0720:24gfrederickszane: I assume you are also pheuter; now that I think about it I suspect you'll have to be careful about uniqueness?#2016-06-0720:25pheuteryes#2016-06-0720:25gfredericksyou can use gen/vector-distinct to get a collection of distinct things#2016-06-0720:25pheuterthat was my follow-up :relaxed:#2016-06-0720:25pheuteroh awesome#2016-06-0720:26zaneGood observation!#2016-06-0720:27zaneYes, @pheuter and I are colleagues.#2016-06-0720:35pheuterHm, looks like gen/vector-distinct is not one of the lazy combinators in clojure.spec.gen#2016-06-0720:36gfredericksyou can use it directly from clojure.test.check.generators/vector-distinct#2016-06-0720:36gfrederickswrap it in a function if you have to#2016-06-0720:36pheuteryeah, makes sense#2016-06-0720:36pheuterthanks#2016-06-0804:13shaunxcodein the cognicast espisode rich mentions a new syntax for default namespaces for keywords in a map - do we know what that looks like yet?#2016-06-0804:17noonian@shaunxcode: this jira item has some info http://dev.clojure.org/jira/browse/CLJ-1910?page=com.atlassian.streams.streams-jira-plugin:activity-stream-issue-tab#2016-06-0812:13moxaj@alexmiller It's a shame that conform does not (and probably won't) flow into map-of and coll-of šŸ˜ž. I have a use case where performance is mostly irrelevant and inner values should be conformed. It seems the functionality is there, but instead i'll have to resort to some error-prone, hairy tree transformation. Would you perhaps consider a dynamic var or an optional argument to map-of and coll-of to enable such transformations (at the expense of performance)?#2016-06-0812:41Alex Miller (Clojure team)@moxaj there actually is 'coll-check-limit' now #2016-06-0812:42Alex Miller (Clojure team)Defaults to 100 but you could set it higher#2016-06-0812:42Alex Miller (Clojure team)I'm not sure whether that changes what flows though#2016-06-0812:49moxaj@alexmiller It does not, currently. Just to be clear, only the size restriction is settled on, the flowing stuff might be subject to change?#2016-06-0813:06Alex Miller (Clojure team)Let's just say I'm not the one making the decision :)#2016-06-0813:59richhickeya conforming coll spec would not have the same name - either you are exhaustive/conform or neither, not going to be a dynamic magic thing#2016-06-0814:26dominicmIs there a way to specify in spec, that a key is the focus of my validation, (or where the path of the failure would be), but take other values from the map? Useful for having two fields that match for example.#2016-06-0814:32Alex Miller (Clojure team)you can write a custom predicate that validates anything you like#2016-06-0814:32Alex Miller (Clojure team)like #(= (:foo %) (:bar %))#2016-06-0814:32Alex Miller (Clojure team)not sure if I'm answering your question though#2016-06-0814:55wilkerlucio@dominicm: I guess you want something like (s/and (s/keys :req [::password ::password-confirmation]) #(= (::password %) (::password-confirmation %)))#2016-06-0814:57dominicm@alexmiller @wilkerlucio Do these methods set the paths? So that path would/could be set to [::password-confirmation]#2016-06-0814:58wilkerlucio@dominicm: sorry, I don't understand what you mean by set the paths, can you please clarify?#2016-06-0814:58dominicm@wilkerlucio: In the failure, you get a path when you do a key.#2016-06-0814:59wilkerluciowell, you always have to be composing, so it may depend where the failure is#2016-06-0814:59wilkerlucioif it's a key with a problem, it will point directly to it#2016-06-0815:00wilkerlucioif it's on the last checker (the match one) it will point to the map and say that the fail comes from the map and point the predicate out#2016-06-0815:00wilkerluciodoes that answer your question?#2016-06-0815:02wilkerlucio@dominicm: I think is a good idea to try the (explain) on the REPL so you see if the explanations of the errors met your needs#2016-06-0815:03dominicm@wilkerlucio: Yeah. It answers my question. I'm trying to figure out how I can that the predicate after failure, actually comes from ::password-confirmation when doing explain.#2016-06-0815:08dominicm
(s/def ::a boolean?)
(s/def ::b boolean?)

(s/def ::thing (s/keys :req [::a ::b]))

(s/explain
  ::thing
  {::a true
   ::b "foo!"})
;; => In: [:foo.core/b] val: "foo!" fails spec...
#2016-06-0815:08dominicmI want to be able to set that In for say password validation. So that it's set to ::password-confirmation#2016-06-0815:18dominicmTo clarify, I want this:
(s/def ::a boolean?)
(s/def ::b boolean?)

(s/def ::thing (s/and (s/keys :req [::a ::b])
                      #(= (::a %) (::b %))))

(s/explain
  ::thing
  {::a true
   ::b false})
To result in: ;; => => In: [:foo.core/b] val: "foo!" fails spec...
#2016-06-0815:59wilkerlucio@dominicm: not sure if there is an easy way to do that, you might get some insight by checking the sources for (s/keys)#2016-06-0816:00dominicm@wilkerlucio: Looks like it works on reification. Fun fun fun.#2016-06-0816:15brabsteris there any way to tell what spec was used to conform a value?#2016-06-0816:15Alex Miller (Clojure team)no?#2016-06-0816:17brabsterI was looking for a way to define a function to transform conformed output#2016-06-0816:18brabsterso if I had a way of extracting the keyword that defined the spec I could use a multimethod to do that - probably misthinking it#2016-06-0816:20brabstersure I can find another way, thought it was worth asking anyway. thanks @alexmiller?#2016-06-0816:22nwjsmith@brabster: if you would like to customize a spec's conform function, you can do it by implementing the Spec protocol: https://github.com/nwjsmith/datomic-spec/blob/master/src/datomic_spec.clj#L286-L293#2016-06-0816:24nwjsmithAlthough I'm not sure that's the best way to go about it, I just wanted my Datomic queries to conform the same whether in map or list form#2016-06-0816:25Alex Miller (Clojure team)I think you're treading on impl details there that can change without warning, fyi#2016-06-0816:29nwjsmithšŸ˜“#2016-06-0816:37nwjsmithI'm going to try the multi-spec route to see if I can get similar results. Maybe I don't want conformed map and list queries to be the same.#2016-06-0816:42brabster@nwjsmith: thanks for offering up the suggestion anyway šŸ™‚#2016-06-0818:17borkdudeWe're having a meetup about spec and are wondering about the output in this snippet:
(s/explain-data (s/cat :cat1 keyword?
              :cat2 (s/& (s/cat :cat1 integer?
                                :cat2 integer?)
                         #(> (:cat1 %) (:cat2 %)))
              :cat3 keyword?)
       [:foo 42 43 :baz])
#2016-06-0818:28fentonhaving issues just getting specs going... lein repl complains about spec not being defined#2016-06-0818:34nwjsmith@fenton have you tried lein clean?#2016-06-0818:34nwjsmithWhat is the output of lein deps :tree?#2016-06-0818:36fentonthank you...i had it over-ridden in ~/.lein/profiles.clj ! doh!#2016-06-0819:02richhickey@borkdude: what are you wondering?#2016-06-0819:03richhickeythe fact that it reports the error at :baz ?#2016-06-0819:05richhickeythat’s because :cat2 has eaten the ints, then failed its secondary pred. Currently & can’t know that its preds won’t be made happy by some future input, so it’s still working on the & and then can’t eat :baz#2016-06-0819:09richhickeycurrently regex ops can communicate that they can succeed w/o more input, but not that they are ā€˜full’, which is what would be required for & to bail out#2016-06-0819:22richhickeythat particular case could probably be handled though#2016-06-0819:36dryewowhat does (s/cat :a :b) mean? It compiles, but matches only empty lists#2016-06-0819:36dryewoto me it does not seem a valid spec, it should complain#2016-06-0819:37gfredericksyeah the :b seems like the potentially invalid part#2016-06-0819:37gfrederickswell though#2016-06-0819:37dryewoI’m asking because I had smth like (s/cat :url ::url) and I forgot to define ::url above#2016-06-0819:37gfrederickskeywords are predicates#2016-06-0819:37gfredericksso it's probably being treated that way#2016-06-0819:38gfredericksbut arguably it would be worth not treating them that way since you can't used namespaced keywords as predicates anyhow#2016-06-0819:38gfredericksthough if it matches empty lists then I'm probably mistaken about the treating-it-like-a-predicate part#2016-06-0819:38dryewo(s/valid? :a {:a 1}) throws an exception#2016-06-0819:46ikitommicould there be spec-enabled versions of fn and defn?#2016-06-0819:46borkdude@richhickey: thanks!#2016-06-0819:47ikitommiwriting a function (defn age-plus1 [{:keys [::age]}] (inc age)) already defines something about it’s inputs, the specs could be extracted from the source?#2016-06-0820:00richhickey@borkdude: next alpha will do this ^#2016-06-0820:00richhickeythanks for the report#2016-06-0820:11angusiguess@ikitommi: If I understand correctly, the reason for fdef and explicit instrumentation is to prevent default instrumentation of functions.#2016-06-0820:39stathissideris@richhickey: In the cognicast you mentioned a bit about the possibility of generating specs because they're data. It's not obvious to me how I would be able to do that, could you point me the right direction?#2016-06-0820:42richhickeythey are data as code is data, you’d generate them as you would any other code#2016-06-0820:42stathissiderisoh that's what you meant! thanks šŸ™‚#2016-06-0820:43richhickeycall s/form on a spec for an example#2016-06-0820:44stathissideristhat's very useful... I'm playing with a (simple) heuristic that would produce specs from example data#2016-06-0820:44stathissiderishard to get right, but maybe I can come up with something that's good enough#2016-06-0820:46richhickeyspecs for spec will help, e.g. the spec for s/cat:#2016-06-0820:46stathissiderisare those somewhere in the code already, or still unreleased?#2016-06-0820:47richhickeynot yet released#2016-06-0820:47stathissiderisbut that's the 2-stage generate you mentioned in the cognicast#2016-06-0820:47richhickeyI have to go through them with @alexmiller still#2016-06-0820:48richhickeyright, that’s for testing mostly. Working from example data is interesting, but not possible in the general case#2016-06-0820:48richhickeytesting and parsing I should say#2016-06-0820:48richhickeyalthough now with unform, the toolchains get interesting#2016-06-0820:49stathissiderisI'm inspired by F#'s type providers... if they can do it to a useful extent, I think the idea has some mileage#2016-06-0820:50richhickeyif not regex-requiring sequences, not so bad. keysets mostly and leaf scalar types#2016-06-0820:51richhickeystill you won’t be able to tell required from optional, etc#2016-06-0820:51stathissiderisyeah figuring out regexes would take some crazy sequence alignment algorithms (as in biology)#2016-06-0820:52stathissideriswell, if a key appears in only half the maps, we can assume it's optional... I'm thinking this could be a cool way to inspect data#2016-06-0820:55richhickeysounds like fun#2016-06-0820:56stathissideriswe'll see how far I can take it! Imagine applying this to functions while exercising an existing codebase to figure out signatures.#2016-06-0821:01nwjsmithAs I understand from the docs multi-spec is meant for tagged maps, but would be appropriate for specs that apply to either maps or sequences?#2016-06-0821:02nwjsmithAn example:
(defmulti query-form (fn [query] (if (map? query) :map :list)))

(defmethod query-form :map [_]
  (s/keys :req-un [::find] :opt-un [::with ::in ::where]))

(defmethod query-form :list [_]
  (s/cat :find (s/cat :find-kw #{:find} :spec ::find-spec)
         :with (s/? (s/cat :with-kw #{:with} :variables (s/+ ::variable)))
         :in (s/? (s/cat :in-kw #{:in} :inputs (s/+ ::input)))
         :where (s/? (s/cat :where-kw #{:where} :clauses (s/+ ::clause)))))

(s/def ::query
  (s/multi-spec query-form (fn [g _] g)))
#2016-06-0821:02nwjsmithWhat do I lose by dropping the re-tagging in multi-spec?#2016-06-0821:04richhickeyyou can do any discrimination you can with a multimethod - tag keys in maps is just an example. Also note that retag could be polymorphic#2016-06-0821:05sparkofreasonSaw an earlier comment that instrumenting protocol implementations doesn't always result in the spec being validated. Will this be supported at some point? Or does it even make sense, i.e. would it be "better" to spec a map with function specs instead of using protocols?#2016-06-0821:05richhickeymulti-spec is for data-dependent specs, generally#2016-06-0821:06richhickeyspec + protocols tbd. Certainly protocol behind API fn is good practice and supported (spec the API), but many people expose their protocols directly#2016-06-0821:26dryewois there a way to disallow non-mentioned keys? (s/valid? (s/keys :opt-un [::foo ::bar]) {:foo 1 :baz 2})#2016-06-0821:44nwjsmithAn and with a map-of spec and your keys spec will do it I think#2016-06-0821:45dryewomap-of is for homogenous maps#2016-06-0821:46nwjsmithYes, but you can use map-of's kpred to ensure the keys are the ones you accept#2016-06-0821:47dryewoah, you mean put a set there?#2016-06-0821:47nwjsmithyes#2016-06-0821:48dryewoindeed, it works#2016-06-0821:48dryewothanks#2016-06-0821:49richhickeyyou can use s/and with s/keys as well#2016-06-0821:50richhickey(s/and (s/keys …) some-further-constraints)#2016-06-0821:51richhickeymap-of not nearly as powerful, won’t do per-key validation etc#2016-06-0821:52richhickeymap-of is for map-as-collection, keys for map-as-information-object#2016-06-0821:53dryewoI see#2016-06-0821:53dryewonow I’m trying to support :opt in this case#2016-06-0821:53richhickeyyou should always question why you are limiting keys#2016-06-0821:54richhickeyjust creating brittleness#2016-06-0821:55dryewowell, I’m just exploring the possibilities#2016-06-0821:55richhickeybut people keep asking for this, maybe we’ll have a :closed option to keys#2016-06-0821:55dryewothis works:
(s/valid? (s/and (s/keys :req-un [::foo] :opt-un [::bar])
                   (s/map-of #{:foo :bar} nil))
            {:foo 1})
#2016-06-0821:56richhickeymap-of samples, will not test every key, so a set not a good predicate#2016-06-0821:56dryewobut keys will#2016-06-0821:57seancorfieldA use case I can think of is where you might want to limit keys on a map you expose as a data structure outside your code boundary: for example something you convert to JSON to return from a web service or when you insert! into a SQL database (using clojure.java.jdbc for example, where additional columns would lead to an exception). But I’d question whether those are places where you should rely on something like clojure.spec to catch the error (instead of a more explicit approach).#2016-06-0821:58seancorfieldWe have select-keys for that sort of scenario, right?#2016-06-0821:59richhickeyyeah#2016-06-0822:00richhickey@dryewo: keys is open, so will let other keys through, and map-of won’t necessarily catch them, it just samples for type consistency, not particular values#2016-06-0822:01richhickeyyou could use #(every? #{:foo :bar} (keys %)) as the second pred#2016-06-0822:05dryewoI tried this: (s/valid? (s/map-of #{:foo :bar} nil) {:foo 1 :baz 2}) and figured that map-of limits possible keys to those that satisfy the set. But you say it’s something that can change in the future?#2016-06-0822:06dryewothe solution with every? looks quite ultimate#2016-06-0822:06dryewoI found *coll-check-limit* in the sources, that explains#2016-06-0822:07richhickeyneither coll-of nor map-of do exhaustive checking, nor conforming. We will make the docs clearer#2016-06-0822:07richhickeythey are ok for very large collections#2016-06-0822:07dryewothanks, that makes sense#2016-06-0822:07ghadii don't want closed sets of keys without a compelling example#2016-06-0822:08ghadihaven't seen one yet#2016-06-0822:08richhickeyenough people seem to want exhaustive conforming version so we’ll likely have both#2016-06-0822:08richhickey@ghadi: well, that’s why open is how it is, I agree šŸ™‚#2016-06-0822:08gfredericksthe only thing that comes to mind for me is the principle of being conservative about the data you return from an API#2016-06-0822:09richhickeyyou just engender brittle code on the other end#2016-06-0822:09gfredericksoh if the closedness is publicized then yeah#2016-06-0822:09gfredericksI was just imagining using it for checking that you aren't accidentally leaking extra stuff#2016-06-0822:10richhickeywell as @seancorfield said, you can call select-keys#2016-06-0822:10gfredericksbut I suppose specs are intended to be publicized#2016-06-0822:10richhickeyright#2016-06-0822:10gfrederickscool, I'm on board#2016-06-0822:11richhickeythere’s a bit in the podcast interview about change, while maybe not made clear yet, this kind of thing is an important part of the design#2016-06-0822:12gfredericksbecause it means adding new keys is backwards compatible?#2016-06-0822:13richhickeythere are separate semantics for accept/return, but they align with co-contravariance, and the keyset and regex system will let us say when things accept/return compatible things#2016-06-0822:13gfrederickshow will predicate equality work in that case? check that the forms are the same?#2016-06-0822:14richhickeyi.e. :req less on what you accept is compatible, :req (essentially guaranteeing) more on return compatible, etc#2016-06-0822:15richhickeyregexes can be tested to see that they new accepts everything old does (accept) and old accepts everything new does (return)#2016-06-0822:15richhickeybase predicates presumed immutable, yes forms#2016-06-0822:16richhickeywhat I hope people find is that at the fine granularity of spec, things rarely change, and if the do they must compatibly#2016-06-0822:18richhickeyand it could get fancier (generative testing to do the non-spec preds)#2016-06-0822:18gfredericksrichhickey: I asked on the ML about tactics for moving libraries forward; in the podcast you mentioned something about adding numbers at the end of function names, do you imagine that being a sort of last resort?#2016-06-0822:18gfredericksstu also seemed to mention creating entirely new namespaces, so I'm curious how common you imagine each of those tactics being#2016-06-0822:20richhickeyhow often do you change the sigs of public fns? I think for algorithmic things rarely and for informational things only in spec-compatible ways. It’s the breaking change that;s the last resort, the numbering of fn names is just being honest with your users#2016-06-0822:22richhickeyThe problem with libraries is that a lib has 50 fns and when you add 2 new ones it gets a different ā€˜version’ and no one knows what’s different. What they actually depend upon are the individual fns that they call, not the lib. Stable specs will show the true granularity of change (usually additive)#2016-06-0822:24richhickeyI’d much rather deal with foo-2 (which btw can co-exist with old foo) than ā€˜version 2’ of foo.#2016-06-0822:24richhickeyI can move to foo-2 when I please#2016-06-0822:24gfredericksyeah, I definitely appreciate that#2016-06-0822:25gfredericksthere are a handful of minor warts in the test.check API that feel hard to improve without breaking#2016-06-0822:25gfredericksfor some value of "breaking"#2016-06-0822:26gfredericksat least if "the API is surprising and hard to sort through" is part of the problem, as then adding a foo-2 might maintain those downsides#2016-06-0822:26gfredericksso I suppose I'm having to accept not being able to "fix" things in that way#2016-06-0822:27richhickeyfor big changes you can also do test.check2#2016-06-0822:27gfredericksas the lib name, or namespace name?#2016-06-0822:28richhickeylib/ns same difference#2016-06-0822:28richhickeythen people can move when they please, specs will be in different nses, so they haven’t changed#2016-06-0822:29ghaditest.check2.11#2016-06-0822:30gfredericksyep#2016-06-0822:31gfredericksrichhickey: thanks#2016-06-0822:31richhickey@gfredericks: I would like to be able to convey explanation data through such-that somehow#2016-06-0822:31gfredericksrichhickey: alex mentioned something about having a fancier such-that to make the generator for s/and more robust, do you know if that's a different issue from what you just mentioned?#2016-06-0822:32gfredericks(just so I can keep track)#2016-06-0822:32richhickeymaybe a (compatible!) extra arg which is a fn to call on failing val to get exception text)#2016-06-0822:33richhickeynot sure if that’s what @alexmiller was talking about#2016-06-0822:33gfrederickswhat he said exactly was: "I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no"#2016-06-0822:34richhickeyyes, so right now the pred fails and test.check can’t say anything useful because the pred is boolean#2016-06-0822:34ghadire: open/closed keysets -- I haven't had the need to ensure a closed set, but when working with an API not under my control, it is useful to detect added information#2016-06-0822:34richhickeybut spec knows what the conditions were#2016-06-0822:35gfredericksrichhickey: okay so he was talking about the same thing? I interpreted his statement to imply some super-fancy way of giving such-that hints about other things to try#2016-06-0822:35richhickey@ghadi: yes, that’s different, and has been requested - (keys-not-in spec map)#2016-06-0822:36richhickey@gfredericks: nope, just better failure reporting, because the failure to gen could be in a nested spec we’d like to at least tell the user which one#2016-06-0822:38gfredericksrichhickey: is this a fair summary? http://dev.clojure.org/jira/browse/TCHECK-107#2016-06-0822:44ghadicould something along the lines of keys-not-in work at any path? use case is detecting a fully compatible change made on an external API#2016-06-0822:46gfredericksrichhickey: after thinking about it a bit, I'm imagining a function that's expected to return a throwable, that way you can customize the error text and the ex-data#2016-06-0822:47gfredericksI guess that means the stack trace starts in a weird place though, which I was trying to avoid :/#2016-06-0822:54kendall.buchananBig thanks to everyone who contributed to clojure.spec. Spent two days building an internal library on it, and it’s, so, so, so, so, so, so… enjoyable.#2016-06-0823:46bbloom@ghadi: another use case i just ran into in a (non-clojure) project: warning on configuration errors#2016-06-0823:47bbloomhad a config.toml file and somebody had a rogue [Section] and a Key = true line was getting put in to the wrong section, so the config setting wasn’t taking hold#2016-06-0823:47bbloomi used a feature of our toml parser to print all the unrecognized keys to the logs#2016-06-0902:53arohner@bbloom: @ghadi yes, figwheel uses that idea to great effect in config file validation#2016-06-0902:53arohnerā€œunrecognized key :foo under :bar, you probably want it in :baz#2016-06-0907:02Oliver GeorgeHello. My CLJS apps use a lot of string keys. Primarily for performance reasons, transforming JSON has a cost. I went looking for the equivalent of the :strs feature of map destructuring, perhaps something like :req-strs and :opt-strs could be added to s/keys.#2016-06-0907:02Oliver GeorgeFor now the workaround seems to be using a conformer...#2016-06-0907:02Oliver George
(def strs->keys 
  "Turn map with string keys into map of keyword keys.  Without this we can't use s/keys."
  (s/conformer #(reduce-kv (fn [m k v] (assoc m (keyword k) v)) {} %)))
(s/conform
  (s/and strs->keys
         (s/keys :req-un [::id ::options]))
  {"id" :id
   "options" [{:id 1} {}]})
#2016-06-0907:03Oliver GeorgeAm I missing anything?#2016-06-0907:35hiredman
user=> (s/valid? (s/spec #(contains? % "foo")) {"foo" 1})
true
user=> 
#2016-06-0907:35hiredmanit depends on what you are looking to get out of it, I guess#2016-06-0912:56dominicm@hiredman: You lose thikngs like path lookups doing that. I think that's the key feature that goes missing. Spec has a bit of a balancing act to perform making sure that via/paths are added. s/keys is a bit complicated in implementation, but I would be thinking that a similar-but-different implementation should surface for string keywords.#2016-06-0913:34gfredericksI've thought about string keys some too w.r.t. https://github.com/gfredericks/schema-bijections#2016-06-0913:35gfredericksI figured it could be handled with a bijection system that doesn't produce a spec on the other side, but that doesn't help if you're avoiding transformations for performance#2016-06-0914:42rickmoynihanHey... I've been very pleased with clojure.spec, which looks absolutely fantastic, and a nice step up from plumatic.schema - and I am fully intending to play with it properly (I've only really had time to read the announcement, listen to Rich on the cognicast, and looked at a few examples)... Anyway, one thing I'd really like to do is understand how this fits with JSON schema... Now, clojure.spec is clearly far superior to JSON schema, but it's not used by the javascript community etc... So I was wondering if anyone has thought about how one might be able to build a JSON schema from a clojure.spec. As spec is way more expressive, I'm guessing that to do this you'd have to stick to a subset of spec (and I guess you could probably use clojure.spec to specify that subset)... The basic idea would be to write a super-spec in spec; and output JSON schemas to interoperate with other tooling/libraries in other languages etc... e.g. swagger clients etc. If this is possible from within clojure you'd then be able reuse that subset of spec (which generated JSON schemas), and layer ontop the additional expressivity and specs you'd want clojure side.#2016-06-0915:38gfredericksrickmoynihan: the library I linked to just above has a lot to do with that problem, but it works with plumatic/schema. I've been pondering how best to translate it to clojure.spec#2016-06-0915:38gfredericksthe idea is it uses a set of transformations to turn an internal schema (the nice one) into a different schema (the coarser json-style schema in this case), with functions generated to transform data structures between the two#2016-06-0915:44rickmoynihangfredericks: interesting#2016-06-0915:45gfredericksit's still a bit half-baked, but I used it a bunch on a previous project#2016-06-0915:46gfredericksin particular it doesn't do a good job of distinguishing between bijections and non-bijections (coercions, projections, whatever you might call them); I'm thinking ou could do that interestingly using test.check generators#2016-06-0915:49gfrederickse.g., if my JSON API can accept {"unitCount":42} as well as {"unitCount":42.0}, you might specify that in the schema, but say that the first one is preferred, so that the transformation function always uses that, but could also have a generator that would take {"unitCount" 42} and generate sometimes {"unitCount" 42} and other times {"unitCount" 42.0M}#2016-06-0915:49gfredericksso it would give you a way to test flexible APIs in a robust way#2016-06-0915:51gfredericksunrelated: if I'm reading the clojure.spec source correctly, it ought to be safe to add specs to test.check (and to clojure.core) without the danger of an infinite loop during instrumentation -- does that sound right?#2016-06-0915:51rickmoynihandoes spec support coercion? I though rich said it wasn't part of its job#2016-06-0915:51gfredericksspecs can refer to themselves via the keyword#2016-06-0915:52gfredericks(s/def ::tree (s/or :leaf #{:leaf} :children (s/tuple ::tree ::tree)))#2016-06-0915:53gfredericksexample value: [[:leaf [:leaf :leaf]] [[:leaf [:leaf :leaf]] [:leaf :leaf]]]#2016-06-0915:53rickmoynihanyeah but I didn't think spec could coerce a value from "foo" into :foo - or can it?#2016-06-0915:53gfredericksoh geez sorry#2016-06-0915:53gfredericksI misread "coercion" as "recursion"#2016-06-0915:53rickmoynihanlol šŸ™‚#2016-06-0915:53gfredericks(and thus thought your question was unrelated to what I was previously talking about)#2016-06-0915:54gfredericksokay right afaik it doesn't support coercion#2016-06-0915:54gfredericksI'm not sure if the "external spec" idea would work or not#2016-06-0915:54gfrederickse.g., with json you'd want to have string keys, which isn't natural#2016-06-0915:54rickmoynihantbh I'm not too worried about the coercion case... I'd just like to write clojure.specs (even in a limited subset of spec) and output JSON schemas from them#2016-06-0915:55gfredericksI think that by itself would probably be easier, if you only support a subset of schemas#2016-06-0915:55rickmoynihanwell I'm sure that would be possible too#2016-06-0915:56rickmoynihanit doesn't need to be perfect - I'd just like to define a service in clojure, and emit a JSON schema that's good enough to catch errors between service boundaries - where client services might be implemented in javascript/ruby etc... And ideally also target swagger, to give client developers access to some swagger tooling / swagger-docs on the other side. (Though that's just an extra nice to have)#2016-06-0916:06gfredericksI'm trying to write (for fun) a variant of defn where each arg can have a spec and you can have multiple bodies with the same arity as long as the specs are different#2016-06-0916:06gfredericksi.e., you can overload on spec, not just arity#2016-06-0916:10rickmoynihaninteresting idea#2016-06-0916:10rickmoynihanI really like multispec#2016-06-0916:11rickmoynihanI'm guessing you just generate those when the arities are the same?#2016-06-0916:11gfredericksI was going to start with just one body that takes varargs and parses them#2016-06-0916:15gfredericksincidentally I have to parse arglists for this and spec is really good at that :D#2016-06-0916:15ikitommi@rickmoynihan: there is a open issue in ring-swagger to add support for spec, haven't had to to investigate yet. (Currently supports the Plumatic Schema). Could split the lib into separate submodules for the different sources models.#2016-06-0916:17rickmoynihanikitommi: does ring-swagger include use a separate library for JSON schema; or bake its own one? I'm no swagger expert - but I'd assumed that Swagger included all of JSON schema...#2016-06-0916:18rickmoynihanI just think it'd be most useful to target JSON schema directly rather than drag in swagger/ring etc too#2016-06-0916:19rickmoynihanhttps://github.com/metosin/ring-swagger/issues/95#2016-06-0916:23rickmoynihanWe're in the fortunate situation of not having a lot of legacy plumatic/schemas - mainly because we only started using schemas a month or two before spec's announcement... so I'm not too concerned about plumatic support for legacy reasons... But I'm curious whether anyone think's there's a future for plumatic/schema beyond just supporting legacy... i.e. does it have uses that clojure.spec can't support so well?#2016-06-0916:25gfredericksthere's nothing I used it for that I would keep using it or
#2016-06-0916:25gfrederickss/or/for/#2016-06-0916:26gfrederickseven if there's something minor it could do better I feel like there's more leverage to be had using The One True Tool#2016-06-0916:26gfredericksI'll make a clojure.spec utility library if I have to though, for specialized stuff that clojure.spec decides not to support directly#2016-06-0916:27gfrederickse.g., maybe for the super succinct map syntax plumatic/schema has for defining map schemas#2016-06-0916:27rickmoynihanthe only things I think plumatic/schema has over clojure.spec right now are: 1. it's perhaps a little closer to json schema - and possibly less work to bridge into json schema 2. no automated coercions#2016-06-0916:28wilkerlucioI'm wondering here about sort of generic specs, I'm thinking on the channel case, I can spec the return of a function to return a channel, maybe would be nice to be able to annotate also the expected value that will come from the channel, do you people have any thoughts on how to deal with stuff like this?#2016-06-0916:45richhickey@gfredericks: having the fn return data which you throw is better - (s/keys :opt-un [:ex-message :ex-data])?#2016-06-0917:34ikitommi@rickmoynihan: the plumatic->json schema is done within the lib (the json-schema ns, based on protocol & supporting multimethod). Swagger (OpenAPI nowadays) only supports only a "pragmaticā€ subset of JSON Schema as it’s original target was the OO-languages like Java - has client code generators for those. Things like oneOf or anyOf are not supported - the requests to add those are over 2y old now. I think having a separate pure spec<->json schema would be awesome! Having it work with swagger requires some extra work, happy to do/help with that.#2016-06-0918:30gfredericksrichhickey: yeah that's essentially what I was thinking; can you think of any args that should be passed? if we make it a one-arg function that's passed an empty map that would be better for backwards-compatibly adding more things later :) I'm wondering because if there's no need for any args then it's not clear why it even needs to be a function and not just passing the data directly#2016-06-0918:52gfredericksthis fancy defn is fun; I suppose it's a lot like core.match#2016-06-0918:56gfredericksuh oh I just made it stack overflow#2016-06-0919:03gfredericksokay here it is -- defn+spec, where each arg can be decorated with a spec and you can overload a function by spec: https://gist.github.com/gfredericks/e4a7eafe5dcf1f4feb21ebbc04b6f302#file-defn-spec-clj-L73#2016-06-0919:05gfredericksthere's a note in there about a stack overflow that I haven't tried to debug, that happens when I add a spec to the defn+spec macro itself#2016-06-0919:08fxposterhi everyone I have 2 questions about clojure.spec, which are not that obvious from the beginning: 1. is it okay to use predicates that connect to external resources for validation? ie: I have a tree of "paths" and want to validate that it matches the filesystem or posts on some website? I don't see any techncal problems with that, but maybe there are other solutions for that? 2. for example, I have 2 data structures: new {"application": {"branches": {"release": "1.0", "snapshot": "1.1"}}} and old {"application": {"branches": {"release": "0.9", "snapshot": "1.0ā€}}} and I want to actually check that the new value is a "valid update" of the old one, based on some constraints (for example: "new value contains at least all branches from the previous one" and "all versions in new value are greater than the same ones in the old one"). is there a way to at least partially express that in clojure.spec and get validation + error reporting? Thanks#2016-06-0919:12gfredericksI think the predicates are supposed to be pure functions#2016-06-0919:12gfredericksI couldn't say where the first place you would run into trouble would be though#2016-06-0920:28akielCan someone explain what I found in my snippet?#2016-06-0920:36benzapdoes replacing sp/alt with sp/or make it work?#2016-06-0920:37akielYes!#2016-06-0920:37benzaphaha, i've had the same issue#2016-06-0920:38benzapsomeone worked out the differences. It has something to do with how sp/alt is used in regex, and sp/or is used otherwise.#2016-06-0920:38akielSo sp/alt on something which is not a regexp doesn’t work? Ok I see...#2016-06-0920:38benzapSimply put yes. Someone else on here had a really good explanation, but I didn't completely follow#2016-06-0920:39benzapHe was wondering why sp/alt and sp/or were so similar, and produced the same results in some contexts#2016-06-0920:39benzaphad to do with the regex distinction#2016-06-0920:40akielso maybe we need spec for spec - I mean it’s all macros - everything can happen#2016-06-0920:41benzaphaha, i'm curious to see how much spec is used šŸ™‚#2016-06-0920:41benzapI wonder if they'll use clojure.spec on all of the clojure.core to try and get away from the java stracktraces#2016-06-0920:42akielcat and alt says: ā€œreturns a regexā€, and says: "returns a specā€ - so maybe a spec is not a regex šŸ™‚#2016-06-0920:42benzapah ok#2016-06-0920:42akieltype systems would help rant#2016-06-0920:43benzapI was talking to someone about clojure.spec on freenode#programming, and he said clojure.spec is like contracts in racket#2016-06-0920:44benzapthe idea being that they're supposed to be more powerful, since you can apply predicates#2016-06-0920:45akielyes spec is more powerful as a type system, because you can inspect actual values at runtime#2016-06-0920:45benzapya, the whole instrumentation is pretty neat#2016-06-0920:45benzapi've used it, but it's rather slow. I need to start using it per namespace#2016-06-0920:46akieljust use it only in dev and test#2016-06-0920:46benzapI have been, but it's even slow then. Maybe 20 times slower#2016-06-0920:46benzapmakes testing rather slow#2016-06-0920:46benzapIt isn't an issue if I run it once in a while, so i'll throw down instrumentation every once in a while#2016-06-0920:47benzapBut I wonder if there's any room for improvement in performance, it would make it very appealing to just leave instrumentation enabled even in production#2016-06-0920:48benzapThere's a reddit post comparing different schema/data validation libraries. No one has commented on it yet, and I wish someone more knowledgable would#2016-06-0920:48benzaphttps://www.reddit.com/r/Clojure/comments/4mxqcy/spec_performance_comparison/?ref=share&amp;ref_source=link#2016-06-0920:48akielThan I think, you apply spec to inner loop things. I would apply it only to my public API.#2016-06-0920:48benzapya, that makes sense#2016-06-0920:49benzaplike, only validate data that comes in from an external source for production?#2016-06-0920:50akielIn production I would check data coming over wire directly without instrumentation, just in normal code.#2016-06-0920:51akielI do this with Schema already.#2016-06-0920:51benzapI've been using the clojurescript version of clojure.spec, cljs.spec#2016-06-0920:51benzapoh nice#2016-06-0920:51benzapyeah, I converted my project i've been working on from schema to spec#2016-06-0920:51benzapdidn't take that long, but it did require quite a bit of refactoring#2016-06-0920:52benzapI'm still not sure where I should put the specs. I kind of placed them at the beginning of the files, and would s/fdef after each function#2016-06-0920:53benzapI suppose that's good enough, I don't think i've tested s/fdef when the function hasn't been declared yet, so I don't think I could place them in another file?#2016-06-0920:53akielI’m not completely sure either. But I have seen putting s/fdef before each funtion often.#2016-06-0920:53benzapoh, before?#2016-06-0920:53benzaphmm#2016-06-0920:54akielyes it is possible#2016-06-0920:54benzapthat's good to know#2016-06-0920:54akielits like contract first and than the implementation#2016-06-0920:54benzapthat makes sense#2016-06-0920:54benzapi'll have to consider moving my stuff into a separate folder#2016-06-0920:55benzapit's hard to get used to, really. I considered it an eye-sore at first#2016-06-0920:55benzapdefinitely useful though, I caught a lot of bugs early#2016-06-0920:56akielopposite to schema, the annotations are not inline to the function - so the possibilities are broader - like in core.typed I think - never used it#2016-06-0920:57benzapYeah, that would explain my confusion. At first, I had a hard time figuring out how to convert from schema#2016-06-0920:58benzapmade a lot more sense once I got started#2016-06-0920:58benzapwhen I used schema, I had originally defined the structures in a separate file, sortof like how you would define a jsonschema#2016-06-0920:58arohner@benzap: it’s probably slower because it’s doing generative testing at runtime. Hopefully that becomes a config setting#2016-06-0921:00benzap@arohner: that's interesting, I thought it was trying to conform each function with respect to it's defined spec. If it's also throwing in generative testing, I can see that causing some performance issues#2016-06-0921:01arohnerwell, fspec does. checking fdef now#2016-06-0921:02benzapa somewhat related topic, there's a library in clojure called specter, which compiles itself to increase performance#2016-06-0921:03benzapI wonder if the same concept could be applied to clojure.spec, where you could pre-compile the validator for your functions#2016-06-0921:19fentonwhat would a spec look like that does the following: if a map has one key, it should have another key#2016-06-0921:19bbrinckAnd in plumatic schema, you can create validators ahead of time for performance.#2016-06-0921:24fentonoops wrong description of problem. if a key in a map has a certain value, then another key should be present. writing specs can be a challeng...like a whole nother language! šŸ™‚#2016-06-0921:31angusiguess@fenton That is likely a multispec#2016-06-0921:31angusiguessWhere you write a multimethod looking for the first key, and that dispatches another spec to check for the second.#2016-06-0921:32fenton@angusiguess: ok, will look into that. thx.#2016-06-0921:32angusiguessšŸ˜…#2016-06-0921:58arohnercould also be a simple s/or#2016-06-0921:58arohner(s/or map? (s/and (s/keys [::foo ::bar]))#2016-06-0921:58arohnerdepends on how complex it needs to be#2016-06-0922:01danielcompton@rickmoynihan: we use coercions which are extremely handy#2016-06-0922:03danielcomptonIf spec (or surrounding tooling) doesn’t support automated coercions, then we’ll need to keep schemas for our boundary interfaces between webapp and RethinkDB#2016-06-0923:09rickmoynihandanielcompton: Yeah - I've used coercions before too - and I don't dispute their utility at boundaries... I guess you could easily build a specialised coercion library on top of spec though... don't know enough yet how you might do that... Regardless I much prefer what I've seen of spec to schema; and don't think coercions are enough of a feature on their own to either not use spec, or use schema as well as spec... I'd definitely much prefer something that worked with spec#2016-06-1002:23sparkofreasonIs there a straightforward to define a function spec and reuse with multiple fdef's?#2016-06-1007:43mpenet@rickmoynihan @danielcompton : same here, also free type hinting, custom explain messages, performance and the list goes on. So far we have no reason to migrate really, we're waiting to see where clj.spec is going, it's probably a bit too early#2016-06-1013:51ikitommiBesides coercions, my top feature request to spec: helper-fn to create a spec from a vanilla clojure (function) var. It would understand the Clojure destructuring syntax. Something like this:
(require '[clojure.spec :as s])

(s/def ::age integer?)
(s/def ::fullname string?)
(s/def ::role keyword?)

(defn doit [{:keys [::fullname ::age ::role] :or {:boss ::role}}] [fullname age role])

(-> #'doit s/extract-spec s/describe)
; => (keys :req [:user/age :user/fullname] :opt [:user/role])
* the responsibility to extract (and use) the specs would be on the user - I would use these on the web-tier to auto-extract docs & do coercion in the web-api tier with our libs * would not add new meta-data to vars (arguments are already in :arglists) * no need to describe the shape of the data twice (both for the function arguments & for it's spec) * (optionally the :ret and :fn could be read from the Var metadata too) Thoughts?
#2016-06-1014:01angusiguessI think this is rad, my only real question is why might we rely on spec to do it?#2016-06-1014:01angusiguessThere are pretty good hooks to pull validation information out of a defined spec and an alternate defn like this should be a macro.#2016-06-1014:02angusiguessI don't work on spec so this is grain of salt stuff, but it seems like the opinion of spec is that people could conceivably bring their own sugar but under the hood there's a common language for validation.#2016-06-1014:09Alex Miller (Clojure team)@ikitommi: you could build that from what exists now. since it wouldn't generically apply, I don't think we would do that as part of core or anything. One thing that would help is a spec for destructuring, which I have and which will be released at some point in some form (details TBD still)#2016-06-1014:16ikitommi@alexmiller: spec for destucturing sounds cool. Did a dummy version of the extractor, will play more with it. Are there any caveats in playing with :arglists? #2016-06-1014:18Alex Miller (Clojure team)there are a few cases where people have abused it a bit in what was put in it (data.generators is one that comes to mind) but generally should be fine#2016-06-1014:19Alex Miller (Clojure team)I think I would also consider allowing overrides via lookup in the registry - s/fdef registers stuff there under the fn symbol and those can be obtained via s/fn-specs#2016-06-1014:19Alex Miller (Clojure team)so you have an existing registry for overrides of things you couldn't build automatically#2016-06-1014:21Alex Miller (Clojure team)also note that CLJ-1919 will add a new syntax for namespaced key destructuring.#2016-06-1014:21Alex Miller (Clojure team)your example there is not syntactically correct btw - the keys of :or should always be unqualified symbols (matching the bindings that are created). there are some bugs in this area in current Clojure that will be fixed in CLJ-1919.#2016-06-1014:22Alex Miller (Clojure team)so that is, what you have there probably works now, but by accident not intent, and will change#2016-06-1014:24ikitommiuh, copy-paste error in the code. But thanks! will check out your pointers.#2016-06-1015:10wilkerluciowhat's the correct way to express to an fspec that a function takes no arguments?#2016-06-1016:13gfredericks(s/cat) I'd guess#2016-06-1016:13gfredericksmaybe #{()} would work too#2016-06-1016:16gfredericksalexmiller: there's no reason not to add specs to test.check is there?#2016-06-1016:17gfredericksassuming it accounts for older clojures#2016-06-1016:18gfredericksI guess this question is a superset of seancorfield's question on the ML, but also about test.check in particular since it's used in clojure.spec#2016-06-1016:24seancorfieldI haven't moved forward with that since the first cut. Want to see more discussion on the ML first. #2016-06-1016:24seancorfield(sorry if I don't follow up for a few hours -- doors closing en route for a cat show!)#2016-06-1016:28Alex Miller (Clojure team)None other than that it then requires Clojure 1.9#2016-06-1016:28Alex Miller (Clojure team)Which requires test.check#2016-06-1016:46gfredericksokay, cool; I'll probably do a separate .specs namespace like sean did#2016-06-1017:11jcfAnyone tried writing specs for stateful objects like database connections, or a Datomic database?#2016-06-1017:14jcfI can't think of a nice way to specify a function takes a db, and some other args. To generate a db I need a database connection that isn't available when I define my specs.#2016-06-1017:16jcfImagine a trivial example like this:
(s/fdef load-entity
  :args (s/cat :db ::d/db :tx ::entity-tx)
  :ret ::entity)

(defn load-entity
  [db tx]
  (d/entity db [:entity/id (:entity/id tx)]))
#2016-06-1017:22jcfI can't generate Datomic entities either. I need a DB, which needs a connection, which needs a URI.#2016-06-1017:24jcfIs there a way to say a spec can't be generated automatically so I can test other specs in this namespace maybe?#2016-06-1017:27arohnerisn’t your DB predicate just #(instance? datomic.whatever.Db %)?#2016-06-1017:27pheuter@jcf: i just went through that exercise of creating a Datomic Db#2016-06-1017:27pheuter
(s/def ::db
  (s/with-gen #(instance? datomic.db.Db %)
    (fn []
      (gen/fmap (fn [facts] (-> (helpers/empty-db)
                               (helpers/transact facts)))
                entities-generator))))
#2016-06-1017:27jcfI need a generator to go with that spec @arohner.#2016-06-1017:28pheuter
(s/def ::db
  (s/with-gen #(instance? datomic.db.Db %)
    (fn []
      (gen/fmap (fn [facts] (-> (helpers/empty-db)
                               (helpers/transact facts)))
                entities-generator))))
#2016-06-1017:28jcfI've got basic predicate fns like these:
(defn db?
  [x]
  (instance? datomic.Database x))

(defn entity?
  [x]
  (instance? datomic.Entity x))
#2016-06-1017:28jcf@pheuter: what's facts, and what does (helpers/empty-db) look like?#2016-06-1017:29pheuterwhere entities is a vector-distinct-by of :db/id values#2016-06-1017:30pheuterempty-db just creates an empty datomic db that has been primed with a schema#2016-06-1017:30jcfSo you've got some hardcoded Datomic URI or something?#2016-06-1017:30jcfYou must have global state floating around, right?#2016-06-1017:31pheuterNo, we just generate random URIs, we use in-memory databases for development / testing#2016-06-1017:32jcfThat is global state. My Datomic connection is managed with components, and they're stopped/started around tests.#2016-06-1017:33jcfI could have two Datomic databases with separate connections. That wouldn't be possible with (helpers/empty-db).#2016-06-1017:33pheuteryes, we use mount to start and stop our connections as well#2016-06-1017:34pheuterwe dont use mount for testing though, and that’s when we generate db values#2016-06-1017:34pheuteractually, we do use mount for certain state, and rely on dynamic values using test/use-fixtures#2016-06-1017:35pheuteryour predicates look fine, but you won’t be able to use them for generating out-of-the-box, will need to use s/with-gen#2016-06-1017:56dominicm https://clojurians.slack.com/archives/clojure-spec/p1465578891000846 I've been trying to figure this out also. My solutions have involved macros and sideband data. Not elegant at all. I also ignored generators. #2016-06-1018:05gfredericksseancorfield: were you thinking of having a conditional require in the .jdbc namespace? otherwise you'd have the problem of up-to-date users having to opt-in to the specs#2016-06-1018:05gfrederickswhen there are use cases for the specs besides explicit testing, e.g. clojure.repl/doc#2016-06-1019:34jcfIs there a way to merge two s/keys specs?#2016-06-1019:34jcfI must be missing something. Back to the manual!#2016-06-1019:35donaldballI’ve been idly thinking about the problem of db args as well, though in the sql context. The problem seems the same for datomic and jdbc though: a spec saying the db arg is e.g. a jdbc connection isn’t sufficient. You really want a spec that says this value is a jdbc connection to a database with at least a certain schema and maybe even a certain set of entities.#2016-06-1019:35jcfOh wait, just me being stupid apparently.#2016-06-1019:36jcf@donaldball: I'm using this at the mo:
(defn- with-datomic
  [f]
  (let [running (-> (config/read-config :test)
                    (assoc :uri (str "datomic:mem://" (UUID/randomUUID)))
                    map->Datomic
                    component/start)]
    (try
      (f running)
      (finally
        (component/stop running)))))

(defn entity?
  [x]
  (instance? datomic.Entity x))

(s/def ::d/entity
  (s/with-gen
    entity?
    (fn [] (with-datomic (fn [{:keys [conn]}]
                           (gen/fmap
                            #(d/entity (d/db conn) %)
                            gen/int))))))
#2016-06-1019:37jcfDon't love it if I'm honest. Creating a new connection for every test is pretty inefficient, but it works.#2016-06-1019:38jcfIf you've not used Component or Datomic that's probably meaningless.#2016-06-1021:02gfredericksoh woah#2016-06-1021:04gfredericksI hadn't thought about this generator setup encouraging people to use stateful resources in their generators#2016-06-1021:12dominicm@gfredericks: I'd say that the encouragement and bias towards namespace level hoisting (with s/def) takes away our ability to lexically scope and generally pass around explicit arguments. Diving too deeply into that statement takes you to mount vs component.#2016-06-1021:15gfredericksdominicm: the way I've used generators in the past is purely data-driven, so there's not even component-like stuff until the test starts running; but if you have a spec that's explicitly for a stateful thing, then you can't write a generator for it that way#2016-06-1021:16gfredericksmy gut would be to try to keep doing the data-driven thing, and so not use generators for specs that describe stateful things#2016-06-1021:16gfredericksnot sure how easy that is with datomic#2016-06-1021:16gfredericksa datomic entity is a map of attributes, is that right?#2016-06-1021:17gfredericksif that's the case you could make most of your specs just expect maps#2016-06-1021:17gfredericksand could do low-level testing with maps instead of entities#2016-06-1021:24dominicm@gfredericks: It's definitely difficult to manage with generators. The discussion (in my opinion) transcends just testing. Doing explicit s/explain-data on a runtime database is also a use-case.#2016-06-1021:24gfredericksthat should work with map specs though I would think?#2016-06-1021:29dominicm(s/explain-data {:email " The unique email checker needs a database to make it's check. If I make it part of the map data (assoc m ::db (d/db conn)), then when I check, the returned data gives me an incorrect path to the error position.#2016-06-1021:30dominicmI have just got a macro working, which would take the error generated, and readjust the path for you. But it's still somewhat unnatural.#2016-06-1021:32gfredericksso your spec is making database queries?#2016-06-1021:33ghadithat is a bad idea#2016-06-1021:36gfredericksI think specs should be pure functions#2016-06-1021:43ghadiabsolutely#2016-06-1021:44gfrederickssomething that's not a pure function can be a plain ole test :)#2016-06-1022:14eggsyntaxI need to write a script to generate some Datomic seed data, and I'm experimenting with using spec to do so. Two questions: 1) One part of the seed data I need to write is the key/val that'll generate a temporary db/id. If I were hand writing the seed data, it would look like {:db/id #db/id[:db.part/user -1015948] ... }. Can anyone help me understand how I would go about creating a spec for that? 2) ideally, it'd be nice to just spec that part of the schema, and then somehow use that to generate the seed data, but I'm not quite sure how go about that. Any hints?#2016-06-1022:15eggsyntaxIn other words, how can I spec a tagged literal like that?#2016-06-1022:30eggsyntaxGot it.
(defn db-id? [v] (instance? datomic.db.DbId v))
(s/def ::db-id db-id?)
#2016-06-1022:35eggsyntax(or at least that's the spec for the tagged literal itself. I'm not quite sure how to get from that to a spec that'll produce seed data as above. I guess I'll need to write a generator for it, but it's gonna take some experimentation for sure.#2016-06-1022:37hiredmanusing spec just to generate data seems super weird, why wouldn't you use the generators from test.check or https://github.com/clojure/data.generators directly?#2016-06-1022:39eggsyntaxWell, if I can figure out the relationship between a spec for the schema and a spec for the seed data, I can use the spec to do validation as well as generating seed data, and maybe find other uses for it as well.#2016-06-1022:40eggsyntaxAnd it's also partly a clojure.spec learning exercise for myself šŸ™‚#2016-06-1111:05borkdudeDoes spec offer anything in relation to coercing like Schema does?#2016-06-1111:36Alex Miller (Clojure team)Not really. It does have conformers but their intent is a bit different.#2016-06-1111:38Alex Miller (Clojure team)@rymndhng: this is the intended behavior - map-of samples its values for performance reasons so doesn't conform everything. This been a common question though and something will probably be added for it.#2016-06-1208:02patrkrisHi Everyone. How do you name aggregate stuff with namespaced keywords? Say I have a map that would look like this without namespaced keywords
{:first-name "John"
 :last-name "Doe"
 :address
  {:street "Example Street"
   :street-number "413"
   :city "Example city"}}
The keys at the first level might be namespaced like :customer/first-name and :customer/last-name, but how would I name the keys inside the address map?
#2016-06-1208:31hiredman
{:customer/first-name "John"
 :customer/last-name "Doe"
 :customer/street "Example Street"
 :customer/street-number "413"
 :customer/city "Example city"
 :customer/address-fields #{:customer/street
                            :customer/street-number
                            :customer/city}}
#2016-06-1208:40patrkris@hiredman: thanks. when would you say it's appropriate to create a "child namespace", e.g. :customer.address/*?#2016-06-1214:27sanderWhat is more idiomatic,
(ns work.invoice
  (:require [clojure.spec :as s]))
(s/def ::invoice (s/keys :req [::number ::date ::amount]))
or
(ns work.core
  (:require [clojure.spec :as s]
            [work.invoice :as i))
(s/def ::invoice (s/keys :req [::i/number ::i/date ::i/amount]))
? The first is less typing work, but I don't really like :work.invoice/invoice as a spec name. Maybe this?
(ns work.invoice
  (:require [clojure.spec :as s]))
(s/def :work/invoice (s/keys :req [::number ::date ::amount]))
#2016-06-1217:06danstoneDoes anyone know of a way to compose key sets? e.g something like (s/merge ::foo ::bar) edit: and works for validation, but does not yield a working generator...#2016-06-1217:10t3chnoboyHi! I’m trying to validate a javascript class and not sure how to make it work:
(s/def ::transport (s/or :websocket js/WebSocket
                         :long-poll js/Phoenix.LongPoll))

(s/conform ::transport js/WebSocket)
For objects I use #(instance? Class %) and it works as expected.
#2016-06-1217:34t3chnoboyok, I’ve just made it work:
(s/def ::transport (s/or :websocket #(= js/WebSocket %)
                         :long-poll #(= js/Phoenix.LongPoll %)))

(s/conform ::transport js/WebSocket)
#2016-06-1300:00t3chnoboyIs there a way to validate core.async chan type?#2016-06-1300:02t3chnoboyI want to define a spec for a function which returns the following map:
{:in-chan (chan)
 :out-chan (chan)}
#2016-06-1300:05jfntn@t3chnoboy: you should be able to make a predicate for a ManyToManyChannel instance, or perhaps more generically test that (satisfies? ReadPort in-chan) and (satisfies? WritePort out-chan) also both defined somewhere in core.async...#2016-06-1300:11t3chnoboy@jfntn something like:
#(instance? cljs.core.async.impl.channels.ManyToManyChannel %)
?
#2016-06-1300:12t3chnoboyDoesn’t look pretty...#2016-06-1300:17jfntnindeed, think you could import ManyToManyChannel and reference it directly though#2016-06-1300:21jfntnThe protocol check is probably a better way to go however#2016-06-1308:48dominicm@gfredericks: Yep, my specs make db queries. Spec has s/valid? to accomodate runtime validations (not just function checking in development time). I honestly think having two libraries, one for "types" and one for "db connecting checkers" would be wasted effort.#2016-06-1311:10jimmyhi guys, I want to discuss a bit about spec. Spec is amazing, I can see the workflow that we can try out things in dynamic clojure, then add spec to it later to ready for production. But would spec have an optimization behind to check for example: we define a function with the args which is an integer? then spec will also generate a function with type hint integer beside doing the validation ? Then we can have something that is very flexible at output error and very fast code as well. Just my thought.#2016-06-1312:10shemis there a way to write a spec that says "this map should contain these keywords, and only once each"?#2016-06-1312:17stathissideris@shem: but keys can only appear once each in a map... by definition!#2016-06-1312:21shemergh, right. was eyeing generator output that spat several instances of the map. need more coffee#2016-06-1312:42jimmyhi guys, how do we define a spec that can validate both namespaced key and non namespace key, for example : user/first-name and first-name one of those would be valid.#2016-06-1314:51manderson@nxqd:
(s/def ::first-name string?)
=> :user/first-name

(s/def ::map-spec (s/keys :opt [::first-name]
                          :opt-un [::first-name]))
=> :user/map-spec

(s/valid? ::map-spec {::first-name "joe"})
=> true

(s/valid? ::map-spec {:first-name "joe"})
=> true
#2016-06-1315:27settingheadhas anyone tried using core.async with clojure.spec? i ended up having a lot of functions that returns a channel in my code (`(defn [] (go …))`). i’d like to spec these functions beyond checking its type is ManyToManyChannel (i.e. checking what comes out the channel is valid based on a spec). I’m thinking of adding transducers to the returning channels. but is there a better overall approach? should i write my functions this way to begin with?#2016-06-1315:41jimmy@manderson: I meant
(s/valid? ::map-spec {:user/first-name "joe"})
and (s/valid ::map-spec {:first-name "joe"})
#2016-06-1315:43manderson::first-name is the same as :user/first-name so either should work with above spec. Is that what you're asking?#2016-06-1315:43manderson^ is the same if you are in the same namespace. :: simply appends the current namespace qualification to the keyword#2016-06-1316:18jimmy@manderson: sorry, I was out. I will check on this, I think I miss understand something here.#2016-06-1316:27jimmyyeah I did mis understand, it works fine. with user/first-name. thanks#2016-06-1316:32jimmy@manderson: ah I have another question, if I refer the spec in another ns, it shouldn't work with :user/first-name case as long as I understand ?#2016-06-1316:38mandersonusing the definition of :user/map-spec as defined above, it will validate fine for :user/first-name or :first-name. The namespaced keywords reflect the namespace they were defined in if set with ::.#2016-06-1316:41jimmyok, I see. in s/keys is there a way that we can validate something like :first-name or :user/first-name, one of those is required.#2016-06-1317:02mandersonhm, good question. this seems to work:
(s/def ::map-spec2 
  (s/or :qual (s/keys :req [::first-name] 
                        :req-un []) 
         :simple (s/keys :req [] 
                         :req-un [::first-name])))
#2016-06-1317:03mandersonthe keywords in or are just tags...#2016-06-1319:24danstoneQuestion, when should one specify a property of a function in the :fn of an fdef rather than in a proper test.check property?#2016-06-1320:48danielcomptonI’m pretty sure the answer will be ā€œnoā€, but is there any way to use spec to define newtype’s, i.e. I have a username and a full-name which are both strings. Can I spec functions to stop passing a username where a full-name is required? For this discussion, assume that both strings have no distinguishing features you could use.#2016-06-1320:50danstoneYou might be able to use conform with a custom conformer for 'full-name'. edit: never mind, it won't work - as you say such a conforming fn would never know the difference between the two values#2016-06-1403:25mfikesIt’s too bad (s/describe (s/spec #(< % 3))) has to return something that doesn’t quite look like an anonymous function literal#2016-06-1403:34mfikesI’m not sure what to make of the fact that
(s/def ::a (s/and #(> % 2) #(< % 5)))
(s/def ::b #(and (> % 2) (< % 5)))
look the same. Perhaps it is OK because they behave the same way.
#2016-06-1405:02jimmyhi guys, how do we use variable inside of (s/def (s/keys)) like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
It's a value in a macro in a macro. And I'm no macro master ...
#2016-06-1405:44jimmyI have an error while trying to generate from clojure.spec, I think this would be the try out such-that limitation in clojure.spec implementation.
(defn str-limit [lower upper] (s/and string? #(<= lower (count %) upper)))
(s/def :project/description (us/str-limit 116 1000)) ;; the gen doesn't work if the lower value is around above 100
(s/def ::project-gen (s/keys :req [:project/description]))
(gen/generate (s/gen ::project-gen))

;; -- error
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4703)
#2016-06-1407:09stathissideris@nxqd: This is because string? generates totally random strings and then checks the second predicate to see if they conform, and gives up after 100 tries. Consider wrapping the spec with s/with-gen and providing your own generator.#2016-06-1409:07jimmy@stathissideris: thanks !#2016-06-1411:26skapoorhi guys, am playing with clojure spec and running into an un-expected behavior: (s/valid? #{nil "a" "b"} nil) ;; returns false when it should be true#2016-06-1411:28minimalnil is not truthy so the set as a predicate is returning a non-truthy value#2016-06-1411:29minimal
(s/valid? #(contains? #{nil "a"} %) nil)
true
#2016-06-1411:30skapoor@minimal yeah, I tried that and it works.. but it should return true when a set is used in specs too right?#2016-06-1411:33minimalIf you are relying on the value returned from the set then it needs to be truthy to be valid#2016-06-1411:36skapoorokay, (map #{"a"} ["a" nil]) and (map #{"a" nil} ["a" nil]) both return ("a" nil)#2016-06-1411:37minimalThe not-found value of a set is also nil so it is troublesome if you don’t expicitly check using contains?.#2016-06-1411:38skapoori'm using sets in specs to define an enum of values with the possibility of nil.... so it looked like a convenient way. guess, I'll have to either use s/or nil? ... or #(contains?)#2016-06-1411:39minimalor you can use a defualt value that isn’t nil. (#{} 1) => nil (get #{} 1 :not-found) => :not-found#2016-06-1411:40minimalYeah it’s tricky#2016-06-1411:41skapoorok thanks. i'm wondering if I should file an issue for this on jira..#2016-06-1411:42minimalI don’t think it’s an issue#2016-06-1411:42skapooronly because the contains? behavior is right but when a set is used as a function call it's not.#2016-06-1411:43minimalUsing set as a predicate is a convenience but you are relying on an implicit conversion to boolean based on the value in the set#2016-06-1411:45gfrederickseither choice is surprising#2016-06-1411:46gfredericksto somebody#2016-06-1411:46gfredericksdepending on whether you expect the set to be treated specially or to be treated like a predicate#2016-06-1411:46skapoorthe implicit conversion to boolean is being done by the internal clojure implementation..#2016-06-1411:49skapoor@minimal: but I get it, it won't be considered a defect. so I'll just use a different way. thanks!#2016-06-1415:20Alex Miller (Clojure team)@mfikes from your question way back on describe - s/form is useful for distinguishing these#2016-06-1415:23Alex Miller (Clojure team)@skapoor: you could also wrap s/nilable around the set (s/nilable #{"a"})#2016-06-1417:39arohner@alexmiller: what’s the rationale behind instrument only checking :args?#2016-06-1417:53Alex Miller (Clojure team)shift in perspective - instrument is about checking invocations are correct, not verifying the function itself is correct. (this is similar to how only the :args are checked in macro fdefs). The check and test functions are for verifying functionality of the code.#2016-06-1418:05bfabryis there still a way to set up your app so every spec is checked on every invocation?#2016-06-1418:41arohner@alexmiller: this seems to severely hamper spec’ing non-pure functions (i.e. things that are difficult/impossible to write generators for)#2016-06-1418:42arohnerand IMO, validation and generative testing are still too tightly coupled#2016-06-1418:49Alex Miller (Clojure team)@bfabry: no, but that was not really the intention of instrument, which is about verifying that callers of a function are calling it correctly. If you want to test the the behavior of your functions are in accordance with your specs, you should use the spec.test functions to test your functions.#2016-06-1418:51bfabry@alexmiller: sure, and no doubt were we to use spec we would use the spec.test functions. but like @arohner mentioned not all functions are pure, and if I'm going to write all those specs then I might as well get some extra value from them for free by having them always be checked in lower environments where I do not care about performance#2016-06-1418:51Alex Miller (Clojure team)@arohner: there are still more things coming that will help with verifying aspects of non-pure functions#2016-06-1418:53arohner@alexmiller: one nice feature of schema is that you can choose to use validation at e.g. user-input boundaries, in production. Instrument seems more designed for dev-time atm#2016-06-1418:53Alex Miller (Clojure team)@arohner spec does not remove the need to write tests for your functions. those tests can use invoke specs to validate as appropriate#2016-06-1418:54Alex Miller (Clojure team)@arohner: instrument is only designed for dev time#2016-06-1418:54Alex Miller (Clojure team)you should not instrument in production#2016-06-1418:55Alex Miller (Clojure team)you can choose to explicitly conform with spec at boundaries if you like#2016-06-1418:56Alex Miller (Clojure team)there will be a conform-ex that validates or throws if not, not quite in yet#2016-06-1419:03bfabryif I explicitly conform, then it will happen in production, when I only want it in staging. it also adds a whole bunch of boilerplate to every single function. I don't really understand the reasoning here, being able to reuse the :ret and :fn specs for extra checking when performance isn't a consideration just seems like an obvious win. and I mean, I definitely can still do that, writing my own macro that wraps all functions or whatever, but it sounds like I won't be the only one#2016-06-1419:10Alex Miller (Clojure team)there is an assertion facility still coming as well#2016-06-1419:11Alex Miller (Clojure team)the point is that checking ret/fn every time should be redundant to what you have (presumably) already confirmed in testing - that your function works.#2016-06-1419:16bfabryI'm maybe a bit skeptical that adding generative testing (while definitely awesome) is going to straight away mean I stop writing functions that produce unexpected outputs when they encounter production data. and I'm a big fan of fail fast with a good error message when that does happen#2016-06-1419:19Alex Miller (Clojure team)you (will be) able to assert that return (if instrumented at dev time) or choose to explicitly validate it at production time if you want#2016-06-1419:20Alex Miller (Clojure team)it's unclear to me if you're talking about dev or prod#2016-06-1419:20bfabryactually talking about master/staging#2016-06-1419:21Alex Miller (Clojure team)fair enough - so you can turn on instrumentation in staging#2016-06-1419:21Alex Miller (Clojure team)that will check args on functions#2016-06-1419:21bfabryon my laptop/travis I run unit and generative tests, in master/staging the application runs "production-like" but with assertions turned on for :args :ret :fn, production the app runs with no assertions <-- this is my ideal scenario#2016-06-1419:22Alex Miller (Clojure team)there will be an assertion facility that you can use to (explictly) check ret/fn for instrumented functions#2016-06-1419:22danstoneAs the channel is a little bit louder this evening, I thought I'd pose a question I asked yesterday again: What is the intended usage of :fn in fdef - In the docs it says something like 'relationships between args and the return'. Is the idea here to restrict it to type-y properties (e.g arity 1 of map returning a transducer rather than a seq)? The reason I ask is it's possible to define many more general properties of functions as part of the spec. As spec gets richer I imagine it may be possible to auto-generate the code for many properties (idempotency is easy if you have spec'd the args)#2016-06-1419:23Alex Miller (Clojure team)I think what I'm talking about is close to that, but varies in that ret/fn are not automatically checked but require an explicit assert (which is not active in production)#2016-06-1419:29bfabryright. and so I'll probably end up writing a macro that wraps every function to add that explicit assert, and I've got a feeling that a whole lot of people will do that. we use plumatic/schema atm which checks arg/return values when validation is turned on, and I'd say the same number of bugs are caught by the return value checking as the arg value checking. aaaanyway, writing the macro is nbd, and maybe it'll turn out I don't actually need it or I'm the only person who does šŸ™‚#2016-06-1419:41Alex Miller (Clojure team)one question I have is whether you're getting the same level of generative testing from schema that you can get from spec (that is, whether more of those bugs could/should have been caught earlier)#2016-06-1419:41Alex Miller (Clojure team)also keep in mind that the return value often is an input to another function, which can check its args#2016-06-1419:44bfabryno, we're definitely not. but like I said I'm just a bit skeptical that generative testing will suddenly mean these issues disappear, and it costs me like an extra $5 per month to run extra validation in the staging environment so why not? the return value will likely be the input value to another function, but maybe that function doesn't have a spec yet, because we didn't feel it was worth the time to write yet, or maybe it's too broadly defined etc. If I believed I could perfectly specify every function up front so that bugs were impossible I'd be writing haskell šŸ˜†#2016-06-1419:48Alex Miller (Clojure team)maybe you should just write your code without the bugs?#2016-06-1419:48Alex Miller (Clojure team)šŸ˜€#2016-06-1419:53bfabryhaha#2016-06-1420:49wilkerluciohey people, does anyone here found/created a generator for cljs to create strings from regexps? I was hoping to use the string-from-regexp from test.chuck but I just realised it's implementation is for CLJ only#2016-06-1420:56Alex Miller (Clojure team)I would ask @gfredericks#2016-06-1421:09wilkerluciothanks Alex, I opened an issue on test.chuck, he will see it I think šŸ™‚#2016-06-1421:09wilkerlucioone more thing, given I have my fdef all spec set, I remember seeing a simple command to test that function but I'm not finding it, what's the simplest way to run generative specs on a function?#2016-06-1421:15Alex Miller (Clojure team)clojure.spec.test/check-var#2016-06-1421:21wilkerlucio@alexmiller: thanks, would you consider adding that to the clojure spec guide?#2016-06-1421:21Alex Miller (Clojure team)yeah, that's on the todo list - I was actually working on some alpha6 related updates right now#2016-06-1421:21wilkerlucionice šŸ™‚#2016-06-1421:51ikitommiIf I have understood correctly, the registry doesn’t have any tools for handling duplicate definitions?#2016-06-1421:53ikitommiso, if there are multiple definitions fos person/id, the last one stands?#2016-06-1421:56bfabry@ikitommi: yeah just replaces https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L261#2016-06-1422:10ikitommiIs this good? should the specs be immutable by default? If there are name clashes, depending on the import order of namespaces, the specs might mean different thing.#2016-06-1422:11ikitommior some hooks for the registry to resolve those clashes.#2016-06-1422:14wilkerlucio@ikitommi: this is the reason they are encouraging namespaced keywords I think, do you have a situation where the same namespace is loaded on multiple files?#2016-06-1422:15bfabryfunctionally the same as multiple (def's isn't it?#2016-06-1422:20bsimadoes anyone have examples of using fdef and check-var? I'm having trouble getting it to work in my test suite#2016-06-1422:23ikitommi@wilkerlucio: true that - the specs must have a namespace, but it doesn’t have to be a clojure namespace. One might have multiple :order/ids in a large system. Should not, but could.#2016-06-1422:23wilkerlucio@bsima: check this snippet, may help you:#2016-06-1422:25wilkerlucio@ikitommi: that's some sort of thinking shift, moving from those to fully qualified, there are going to be some nice helpers to deal with longer namespaces in 1.9#2016-06-1422:26wilkerlucioyes, the problem exists like you said, but it's the same for def as mentioned by @bfabry, I guess it's just about people starting moving towards fully qualified namespaces to avoid name clashes, I believe when it's the norm will be very positive for everyone#2016-06-1504:44jimmyhi guys, how do we use variable inside s/keys like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
#2016-06-1509:54danstone@nxqd: I don't think we can because s/keys is a macro. Though I'm not sure if this is worth it for keys, as it doesn't capture predicates, I would expect it to be a function. As far as I can tell the only thing macro'ey it does is look at forms like (or ::foo ::bar).#2016-06-1517:50Alex Miller (Clojure team)At the moment you can't do this except via eval or something#2016-06-1517:50Alex Miller (Clojure team)But there has been talk of adding a fn entry point that would allow it#2016-06-1518:57seancorfieldWith check-fn and check-var, is there a way to get a "nice" explanation of the failures, like explain produces?#2016-06-1518:57seancorfieldI'm trying to see how the changes in Alpha 6 affect the testing workflow. #2016-06-1519:47eggsyntax@nxqd: (somewhat belatedly) you can replace the var with the function that populates it (it can cache its results, if it's an expensive call), if it's a var that it makes sense to populate at macro eval time.#2016-06-1519:48eggsyntaxUnrelated: test.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one.#2016-06-1519:59eggsyntaxThe only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen), but it's pretty ugly to put in a generator and then ignore what it generates.#2016-06-1520:52jannisFolks, is it a good idea to define fdef function/macro specs next to the actual function/macro definition or would you rather define them in separate my-project.specs kind of namespace?#2016-06-1520:53jannisI'd like to associate most of my functions with specs for automatted random function testing but at the same time I don't want to clutter my code base with specs too much.
#2016-06-1520:53jannisI guess there are no best practices established yet?#2016-06-1520:54eggsyntaxI've been debating that myself, haven't come to any particular conclusion.#2016-06-1520:55eggsyntaxCurrently defining them on their own because it's still fairly experimental for us...but as we fully integrate them, I'll definitely consider moving them to live next to what they're speccing.#2016-06-1521:11seancorfieldI think we’ll have data specs in separate namespaces. Not sure yet about function specs. Part of me would like them above function definitions — that seemed natural for when we used Schema and core.typed.#2016-06-1521:12seancorfieldWe went back and forth between core.typed and Schema several times before we abandoned them. With spec being in core, I think we’re more likely to stick with it.#2016-06-1521:12jannisYeah. It also means you don't have to require the function namespaces and the function spec namespaces when you want to test functions.#2016-06-1521:13seancorfieldAlthough, https://github.com/clojure/java.jdbc/blob/master/src/test/clojure/clojure/java/test_jdbc.clj#L28-L32#2016-06-1521:13seancorfieldhttps://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2016-06-1521:14seancorfieldThere the function specs are in a separate clojure.java.jdbc.spec namespace, below the data specs.#2016-06-1521:14seancorfieldBut that was mostly to ensure pre-1.9 code can still use the library#2016-06-1521:15jannisHere's my first attempt at writing function specs alongside the actual functions: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L9#2016-06-1521:18jannisIt would feel a little more natural to define the function spec like`:pre` and :post, e.g.`(defn my-fun [arg1 arg2] {:spec {:args ... :ret ... :fn ...}} <body>)` or something like it, although there are good reasons to keep them separate.#2016-06-1523:22bbrinck@jannis: What do you think about putting the fdefs before the function? IMHO, that's a little clearer. I like having the function specs near the functions since that provides documentation when reading the function. Not sure about data specs, though.#2016-06-1523:29bbrinckor rather, put each fdef before its associated fn#2016-06-1523:45bfabrywhat have I messed up here...?
Clojure 1.9.0-alpha7
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
        Exit: Control+D or (exit) or (quit)
    Commands: (user/help)
        Docs: (doc function-name-here)
              (find-doc "part-of-name-here")
Find by Name: (find-name "part-of-name-here")
      Source: (source function-name-here)
     Javadoc: (javadoc java-object-or-class-here)
    Examples from : [clojuredocs or cdoc]
              (user/clojuredocs name-here)
              (user/clojuredocs "ns-here" "name-here")
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (require '[clojure.spec.test :as t])
nil
boot.user=> (defn foo [x] (inc x))
#'boot.user/foo
boot.user=> (s/fdef foo :args (s/cat :x integer?) :ret integer?)
boot.user/foo
boot.user=> (t/check-var foo)

java.lang.IllegalArgumentException: No :args spec for 
#2016-06-1523:51bfabry
boot.user=> (s/fn-spec foo)
nil
boot.user=> (doc foo)
-------------------------
boot.user/foo
([x])
Spec
  args: (cat :x integer?)
  ret: integer?
nil
boot.user=> (s/fn-spec 'foo)
#2016-06-1600:35seancorfield(t/check-var #’foo) — it accepts a Var.#2016-06-1600:36seancorfieldSimilarly (s/fn-spec #’foo) — although the result of that is less useful to print since it is an object.#2016-06-1600:37seancorfieldYou could also do (t/check-fn foo (s/fn-spec #’foo))#2016-06-1600:38seancorfieldcheck-fn is so you can provide an (s/fspec :args … :ret … :fn …) for an arbitrary function without fdef.#2016-06-1601:23leifpHi, all. I hacked up specs for most fns in clojure.core and 1/3rd of the macros (WIP). I don't know much about clojure.spec (of course), though, so it wasn't the good kind of hacking,. Hold your nose if necessary: https://gist.github.com/leifp/abe50082f6baa0063f8b7840e80657af#2016-06-1607:08bfabryahhhhh thanks @seancorfield#2016-06-1609:58jannisShouldn't ((instrument #'my-fun) :foo) throw an exception with an explanation if my-fun has a spec defined for it with fdef and if the returned value does not conform to the spec of my-fun?#2016-06-1609:59jannisInstead it just returns :clojure.spec/invalid.#2016-06-1610:04jannisOh, that's because that's what my-fun returns in this case. So it doesn't even fail, it just returns its value, even if that doesn't conform to the function spec.#2016-06-1610:41patrkrisI see clojure.spec uses "named arguments" in several places (e.g. (clojure.spec/fdef :args ... :ret ....)). Is there any convention as to when to use named arguments vs. a map? Maybe named arguments are primarily used for macros?#2016-06-1611:10pithylessHi, all. Has someone come across the need to have an s/keys that is more strict (does not allow extra keys)?#2016-06-1611:14jrychter@pithyless: I have. I am guessing Rich thought it isn't a good idea, as in the long term it could hurt expandability and composability. But I still think we should be able to do it in some cases.#2016-06-1611:22pithylessI'm sure it's been brought up internally; I wonder how likely it would be to get an :only option added to s/keys.#2016-06-1611:56patrkris@pithyless: some semi-related discussion here: https://groups.google.com/forum/#!topic/clojure/UVgXXcIxhJQ#2016-06-1612:00jrychterAs I suspected — "spec embraces the idea of open maps". But there are cases where you don't want open maps. A good example is the current world of lein-cljsbuild, where I regularly pull my hair out because it isn't clear if a particular setting is in the right place. Extra keys do no harm, so people tend to put config options all over the place, and you end up with a terrible mess. Figwheel recently started doing some great work towards fixing this — and this kind of config checking should really be "these keys, and these keys only".#2016-06-1612:02pithylessI've got some half-baked solution with clojure.set/difference and keys, but I wish it were simpler.#2016-06-1612:05jrychterI am hoping someone will write a validations library similar to bouncer, based on clojure.spec.#2016-06-1612:10jannisThings are getting a little meta over here... I'm spec'ing out a data format and a parse function that first uses clojure.spec/conform to validate and normalize the input and then passes it on to a transformation function. For that transformation function the input data format is the output of conform, so I now have a spec for what conform returns as well. šŸ™‚#2016-06-1612:46ghadi@pithyless: https://clojurians.slack.com/archives/clojure-spec/p1465423760000580#2016-06-1612:53pithyless@ghadi: Thanks for linking to the archives. I understand why the default is what it is, my use case falls clearly in the "don't want to accidentally be leaking stuff" camp. But it's a minor thing... clojure.spec is nice šŸ™‚#2016-06-1613:16dominicm@jrychter: As someone working on this: spec isn't suited to the task.#2016-06-1613:17jrychter@dominicm: oh šŸ˜ž I was hoping that one could build a validation system based on what explain-data returns…#2016-06-1613:17dominicmThat's the easy bit.#2016-06-1613:19jrychterI currently use bouncer, but I have to work around its problems, and as I migrated from schema to spec, I was hoping I could get rid of it entirely.#2016-06-1613:19dominicm@jrychter: When you do user registration validation, you need to check email uniqueness. So that requires 2 things: 1. Data to validate, the email 2. Sideband data, the database (connection). Spec has a funnel for 1. Not one for 2.#2016-06-1613:20dominicm@jrychter: https://github.com/leonardoborges/bouncer/issues/43 šŸ˜› I know bouncer. It would be nice.#2016-06-1613:20ghadispec has nothing to do with integrity checks (#2)#2016-06-1613:21dominicmYour only options for validating in that way, and neither of these are particularly spec-y, are dynamic vars, or the weird macro I started working on, and gave up on because it was a big ball of mud.#2016-06-1613:21jrychterAs for (2) above, I currently use global state encapsulated using mount.#2016-06-1613:22dominicm@jrychter: You may not have as much trouble.#2016-06-1613:22dominicmHowever, I land on the weavejester side of the camp. šŸ˜›#2016-06-1613:22jrychterMeaning, some of my bouncer validations actually use database or even (gasp) network connections.#2016-06-1613:22dominicmYep, it's a necessity.#2016-06-1613:22dominicm@ghadi: What kind of check does spec do?#2016-06-1613:22jrychterEU VAT numbers are an example.#2016-06-1613:28dominicm> weavejester side of the camp I should clarify this means that I use component.#2016-06-1613:48dominicm@ghadi: I'm mostly interested in the terminology and reading on the subject.#2016-06-1615:32seancorfieldAs I recall, the discussion about closed maps ended with consensus that using select-keys was appropriate for "not leaking additional keys", but there wasn't much consensus on validation of closed maps. #2016-06-1615:33seancorfieldI got the impression @richhickey is "considering" adding support for it due to repeated requests from the community but he doesn't consider it a good idea. #2016-06-1615:37seancorfieldI think designing systems to simply ignore extra keys is more robust, but I also accept there are cases where that is difficult (the example I gave is clojure.java.jdbc/insert! where a map is converted to a SQL insert statement without reflection and therefore extra keys become extra columns and will be rejected by the database; the alternative is to introspect the database and ... well, do what? Silently ignore the extra columns? Throw a different exception?)..#2016-06-1615:39seancorfieldAnd there in lies the issue: the behavior for extra keys is not knowable at that level - it would have to be an application-level decision (and the application has the means to do that reflection and call select-keys if it wants).#2016-06-1615:41seancorfieldWhich is why I side with Rich that it's not clojure.spec's job to directly support restricting keys, since it shouldn't be the norm, and you can do it via custom predicates already if you really have one of those odd cases where you really do need to check. #2016-06-1615:42donaldballGoes back to my point that if you want that level of validation, just specifying that the db arg to insert! is a Connection or a Connectable or whatever is not sufficient; it would need to specify that the database in question has such and such a schema#2016-06-1615:44seancorfieldRight, and that's a whole different thing. java.jdbc provides metadata/reflection APIs but doesn't use them internally (for performance mostly),#2016-06-1616:46bhaumanA strict key set constraint can make a lot sense depending on the situation. This depends on the api of course, but I can't imagine a higher level api, that third parties are going to rail against, not having tighter key set constraints.#2016-06-1616:48bhaumanThat being said, you can compose a strict map key set and description with spec fairly handily.#2016-06-1616:51bhaumanSpec is beautiful.#2016-06-1617:48wilkerlucio@pithyless: if you really want to get just the strict keys from a map, I would suggest using select-keys as people mentioned before, as a plus, I wrote a simple utility that can extract the keys from a keys spec, it's still a naive implementation (it doens't consider req-un or opt-un) but can be starting point if you want to go further on the idea:#2016-06-1617:52wilkerlucioso I believe if extra keys are harmful on your case, you can use this kind of trick to remove unwanted ones, so we need to force the restriction of the keys, but just eliminate the ones you can't deal with (and only if you can't silently pass it on)#2016-06-1617:59seancorfield@wilkerlucio: The s/def for ::name and ::email don’t do anything in that example…?#2016-06-1618:01seancorfieldAre you doing something else to check the values of the ::my-aggregate map conform to those specs, by convention (based on the key names)?#2016-06-1618:01wilkerlucio@seancorfield: the idea here was just to show an example of how to do a select-keys while reusing the specs, you are supposed to call s/valid? yourself before doing the strict-keys on this case#2016-06-1618:02seancorfieldOK, so you’re relying on convention that the (qualified) key names are the same as the specs that apply to their respective values?#2016-06-1618:03wilkerlucioyes#2016-06-1618:03wilkerluciobut like I said, this is a naive implementation, if you really wanna rely on it, will need more work there#2016-06-1618:04seancorfieldFYI, instead of (->> (s/form spec) (next) (partition 2) (map vec) (into {})) you could just do (->> (s/form spec) next (apply hash-map))#2016-06-1618:05wilkerlucio@seancorfield: thanks for that, I'll update the snippet šŸ™‚#2016-06-1618:11bfabryhas anyone come up with a good way of speccing non-keyword maps with known keyval pairs?#2016-06-1618:30zaneDoes spec just not play well with tools.namespace?#2016-06-1618:30zaneOr with the REPL?#2016-06-1618:32zane
dev=> (defn f [x] (inc x))
#'dev/f
dev=> (s/fdef f :ret pos?)
dev/f
dev=> (s/instrument #'f)
#'dev/f
dev=> (f 1)
2
dev=> (f -3)
-2
#2016-06-1618:33bfabryinstrument doesn't check :ret or :fn in the latest alpha#2016-06-1618:33bfabryonly :args#2016-06-1618:33zaneOuch.#2016-06-1618:33zaneIs that going to change?#2016-06-1618:33zaneDid it check :ret and :fn in previous alphas?#2016-06-1618:33bfabryyes#2016-06-1618:34zane… Huh.#2016-06-1618:34bfabrythe reasoning is that :ret and :fn are for checking whether the function is correct, which should happen when you're using the functions in clojure.spec.test. :args are for checking the function was invoked correctly#2016-06-1618:38zaneSo, we have a function that reads environment variables. That function has a :ret spec on it that validates that required environment variables are set and have valid values.#2016-06-1618:38zaneMy understanding now is that instrument will not help me here and I should use s/conform?#2016-06-1618:40bfabryyeah, s/conform or s/valid? or whatever explicitly. there's also an s/conform-ex coming iirc#2016-06-1618:41zaneWhere should I look for info on s/conform-ex?#2016-06-1618:41bfabryin a subsequent release šŸ˜†#2016-06-1618:43bfabryI'm sure it's coming soon, they've been evolving very rapidly#2016-06-1618:44zaneUnderstood.#2016-06-1618:44zaneDo you know what s/conform-ex going to do?#2016-06-1618:45bfabryconform or throw an exception on failure I assume#2016-06-1618:45zaneAh, I see.#2016-06-1618:46settingheadlooks like there was a commit 3 hours ago: https://github.com/clojure/clojure/commit/aa9b5677789821de219006ece80836bd5c6c8b9b#2016-06-1618:54leifpAs mentioned before, I've (roughly) spec'ed a good chunk of clojure.core. I made that an actual repo in case someone wants to test and/or beautify them: https://github.com/leifp/spec-play#2016-06-1618:57zane@bfabry: When is the :ret argument to fspec ever used, then?#2016-06-1618:58bfabrywhen clojure.test.* functions run generative tests#2016-06-1618:59zaneGot it. So only for generative testing.#2016-06-1618:59zaneOof.#2016-06-1619:05zaneThat seems like a very weird design choice to me.#2016-06-1619:08bfabryI'm not real sold on it either#2016-06-1619:08bfabrybut I haven't used spec on anything big enough to be confident#2016-06-1619:18robert-stuttafordso, i'm new to test.check in general. anyone know how i might generate a set of keywords, from a known set of possible keywords?#2016-06-1619:19robert-stuttaforde.g. i have #{:a :b :c :d :e} and i want generated subsets of same#2016-06-1619:20bfabry(s/exercise #{:foo :bar}) => ([:bar :bar] [:bar :bar] [:bar :bar] [:bar :bar] [:foo :foo] [:foo :foo] [:bar :bar] [:foo :foo] [:bar :bar] [:foo :foo])#2016-06-1619:21bfabrythis is probably better#2016-06-1619:21robert-stuttafordinteresting#2016-06-1619:21bfabry(s/exercise (s/coll-of #{:foo :bar} #{})) => ([#{} #{}] [#{} #{}] [#{} #{}] [#{:bar :foo} #{:bar :foo}] [#{} #{}] [#{:bar :foo} #{:bar :foo}] [#{:bar :foo} #{:bar :foo}] [#{:bar} #{:bar}] [#{:bar :foo} #{:bar :foo}] [#{:bar :foo} #{:bar :foo}])#2016-06-1619:22robert-stuttafordi'm modelling Magic the Gathering cards as an exercise#2016-06-1619:22bfabryhaha, nice#2016-06-1619:22robert-stuttaford
(s/def ::type #{:land :creature :artifact :enchantment :sorcery :instant :planeswalker})
(s/def ::types (s/with-gen
                 (s/and set? (s/+ ::type))
                 #( ? )))
#2016-06-1619:23bfabryanyway yeah s/exercise generates data that suits a spec, a spec of "sets of these keys" is (s/coll-of #{:keys} #{})#2016-06-1619:23robert-stuttaford::types works, but it can't be generated because of how s/and generators work: generate for the first and discard anything that doesn't satisfy the rest of the ands#2016-06-1619:23bfabryI think you just want (s/coll-of ::type #{})#2016-06-1619:24robert-stuttafordindeed, thank you#2016-06-1619:25bfabry
(s/def ::type #{:land :creature :artifact :enchantment :sorcery :instant :planeswalker})
=> :kafka-google-connector.runner/type
(map first (s/exercise (s/coll-of ::type #{})))
=>
(#{}
 #{:planeswalker}
 #{:artifact}
 #{:land}
 #{:creature :planeswalker}
 #{:sorcery}
 #{:land :planeswalker :sorcery}
 #{:instant :enchantment :land :planeswalker}
 #{:instant :enchantment :creature :land :planeswalker :sorcery}
 #{:land :sorcery})
#2016-06-1619:25bfabryman.. that's pretty neat#2016-06-1619:26robert-stuttafordok. my next question (which is the thing i really want to solve, now that i've softened you up šŸ™‚ ) is how might i write a generator when i'm using s/and on two s/keys specs?#2016-06-1619:26robert-stuttaford
(s/def ::base-card (s/keys :req-un [::name ::types ::metadata]
                           :opt-un [::sub-type ::legendary? ::world?]))

(s/def ::cost string?) ;; todo

(s/def ::spell (s/and ::base-card (s/keys :req-un [::cost])))
#2016-06-1619:27bfabryI've not actually looked into generators sorry#2016-06-1619:27robert-stuttafordah šŸ™‚ worth a try!#2016-06-1619:27robert-stuttafordit's a super-interesting problem to solve#2016-06-1619:27robert-stuttafordto me, anyway#2016-06-1619:47wilkerlucio@robert-stuttaford: just a suggestion, since you are modeling something new, maybe would be better to use the namespaced keys instead of clear ones, with namespaced keys you can for example validate a map keys even if you don't know the aggregate name for it#2016-06-1620:18bhaumanSpit-balling on strict keys just for the fun of it. I would love some feedback.#2016-06-1620:27leifpbhauman: What are the advantages of this vs. just (s/& (s/keys ...) #(only-these-keys % [:k ...])) ? Or a macro that expands into that.#2016-06-1620:29bhauman@leifp: the only interesting thing here is the explain data#2016-06-1620:29bhaumanwhere it points exactly to the key that failed#2016-06-1620:30bhauman
In: [:there] val: :there fails spec: :howdy/fine at: [:there] predicate: #{:builds :server-port :server-ip :http-server-root}
#2016-06-1620:34bhaumanit will create explain data for all the keys that failed#2016-06-1620:51bfabry@robert-stuttaford: I wrote this which works. I don't know how sane it is. my guess is "not very"
(defmacro extend-keys [spec-name & {:keys [req-un opt-un]}]
  (let [spec-m (apply hash-map (rest (s/form spec-name)))]
    `(s/keys :req-un ~(into (:req-un spec-m) req-un)
             :opt-un ~(into (:opt-un spec-m) opt-un))))
=> #'kafka-google-connector.runner/extend-keys
(s/def ::spell (extend-keys ::base-card :req-un [::cost]))
=> :kafka-google-connector.runner/spell
#2016-06-1621:11leifpbhauman: Hmm... I guess there is no explain equivalent of conformer or with-gen, so I can't really think of another way to do it than reifying Spec. Your impl. looks fine, but it doesn't seem to explain the extra keys if one of the required keys fails its spec.#2016-06-1621:13bhauman@leifp: I haven't looked at that, must be because of the s/and#2016-06-1621:14bhaumanbtw I've iterated on it a bit#2016-06-1621:16bhauman@leifp: what do you mean by extra keys? you mean in the explain data?#2016-06-1621:23leifp@bhauman:
user=> (s/explain (strict-keys :req [::r]) {::r "bad" ::extra 2})
In: [:user/r] val: "bad" fails spec: :user/r at: [:user/r] predicate: number?
In: [:user/extra] val: :user/extra fails at: [:user/extra] predicate: #{:user/r}  ;; <<< expected, not present
#2016-06-1621:23leifpThat output line was expected and not present, I mean.#2016-06-1621:24bhaumanoh yeah it short cuts#2016-06-1621:27bhaumans/and short cuts#2016-06-1621:27bhaumanso that makes sense#2016-06-1621:33bhaumanI would need to compose over the keys-spec to get that behavior#2016-06-1707:04Oliver GeorgeHi Specy Specers. What's the most human friendly way of viewing the output from check-var? I see that reporter-fn can be provided, is there a commonly used one? (I'm using CLJS so perhaps that makes a difference).#2016-06-1709:27jrychterI find myself writing lots of (s/and string? seq) to specify non-empty strings. Also, I'm missing a predicate for strings of length from n to m (analogous to int-in-range?).#2016-06-1709:34jannisIs it possible that s/with-gen alters the spec it defines a generator for? I have a simple (s/cat :base ... :children ...) spec that works fine but as soon as I wrap it in (s/with-gen <spec> #(gen/tuple (s/gen ...) (s/gen ...))), data that would previously conform to the spec now becomes invalid.#2016-06-1709:35jannisNote: I', not generating the data using the generator yet. I'm using hand-written data.#2016-06-1709:42jannisHere's a minimal example: https://gist.github.com/Jannis/5dcc91473f20861d154dc8be2fff2bfd#2016-06-1715:53seancorfield@olivergeorge: there's a discussion about that on the main Clojure mailing list. I'm very interested in the answers to that question. #2016-06-1716:37leifp@jannis: It looks like it's introducing a new regex context like spec does:
user=> (s/explain (s/cat :x ::pair) '[a [b c]])
Success!
user=> (s/explain (s/cat :x ::pair-with-gen) '[[a [b c]]])
Success!
#2016-06-1717:34sparkofreasonI'm working on some code that does simulations over state machines. The transition functions tend to have a lot of detailed conditional logic, and as a result test.check doesn't seem to be a great fit for validating the functions (random inputs tend to be ignored or lead to errors, and writing generators to provide valid input is essentially the same as writing the state machine model). When instrument-all checked the return value it was very useful, since I caught a lot of errors of omission, misspelled keywords, etc. I'd like to suggest we have an option to check :ret and :fn for cases like this.#2016-06-1717:50seancorfieldYeah, whilst I agree in principle with the justification @alexmiller offered as to why :ret and :fn checking was removed in Alpha 6, I also agree that there is potentially a lot of value in having the option to be able to instrument functions in a way that does conform the result at least.#2016-06-1717:50seancorfieldThis is the commit that changed the behavior: https://github.com/clojure/clojure/commit/30dd3d8554ff96f1acda7cbe31470d92df2f565a?diff=split#2016-06-1717:55sparkofreasonThanks, I may use that to hammer out my own version of instrument for now.#2016-06-1717:56bfabryI think there's gotta be something coming for spec'ing impure functions that will cover this#2016-06-1717:56bfabryotherwise you could never spec them, really, or the spec would be pointless#2016-06-1718:05Alex Miller (Clojure team)@seancorfield: Rich has some ongoing work, I'm not sure what the endpoint will be on this. you might have noticed that explain-out was made public today in master#2016-06-1718:06Alex Miller (Clojure team)@bfabry: not every function is a great candidate for generative testing via spec (but the spec may still be useful for docs or other purposes)#2016-06-1718:07eggsyntax@alexmiller: where was the explanation @seancorfield mentioned of why :ret and :fn checking was removed? I’m curious to read it.#2016-06-1718:08Alex Miller (Clojure team)mailing list#2016-06-1718:08Alex Miller (Clojure team)https://groups.google.com/d/msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ#2016-06-1718:08eggsyntaxThanks šŸ™‚#2016-06-1718:09Alex Miller (Clojure team)@seancorfield: I think Stu is looking at some testing-related mods too btw#2016-06-1718:27seancorfieldre: explain-out — I already added a comment on that commit thanking him for that šŸ™‚#2016-06-1718:29seancorfield@alexmiller: I really do appreciate the steady stream of alpha builds so we can all try this stuff out and provide feedback.#2016-06-1718:30Alex Miller (Clojure team)I'm sure there will be more :)#2016-06-1718:34seancorfieldHaving an option on instrument to use the old version of spec-checking-fn with :ret and :fn conforming would be very nice. I think checking just :args is the right choice for most cases of instrumentation, but I think being able to "fully instrument" certain functions would be very valuable — especially for functions that cannot easily be tested the generative way.#2016-06-1718:36seancorfieldFor example, working on java.jdbc’s specs, they can’t reasonably be tested generatively because many of them are side-effecting (updating the database) and writing generators that conformed to the database schema would be … a huge amount of work, if it’s even feasible (e.g., unique key constraints etc?).#2016-06-1718:36seancorfieldSo losing the ability to conform the :ret and :fn specs there is kind of a big deal, IMO.#2016-06-1718:39seancorfieldI’ll be interested to see how this all ends up since any given code base is going to have a mix of functions that can reasonably be tested generatively and functions that can’t, so (clojure.spec.test/run-all-tests) needs a way to distinguish those, right?#2016-06-1718:51seancorfield(mind you, right now I can’t run generative testing on java.jdbc because the system doesn’t know how to generate a java.sql.Connection… which might be an interesting exercise šŸ™‚ )#2016-06-1718:52seancorfield(and I’d also need a generator for a java.sql.PreparedStatement)#2016-06-1718:53seancorfieldWhat is the recommendation for stuff like that? How would you even write a generator for some of these Java objects?#2016-06-1719:34tomcWould anyone be willing to offer some guidance on conventions for attributes shared by entities of different types? For instance, I have "question" and "survey" entities, each of which can have names. Right now I'm using (s/def :entity/name string?) and entities of either type can have an :entity/name attribute along with their type-specific attributes. The alternative of course is to define both :question/name and :survey/name. I haven't seen the :entity/attr pattern elsewhere, and I'm wondering whether there's a reason for that.#2016-06-1720:03wilkerlucio@tomc: I believe you can share the attribute as long as the semantic is the same, for example, a car name may have a different semantic from a person name, but that will depend on the requirements on your system#2016-06-1720:09tomc@wilkerlucio: thanks a lot, that's helpful.#2016-06-1720:12gphilippI’m trying to generate dates within a range with spec, and I’m stucked with this piece of code which generates #inst whose year is above 9999, making them unrecognized when i try to def them manually afterwards : (gen/sample (s/gen inst?) 60) #2016-06-1720:13gphilippex of data generated: #inst"26138-06-03T15:09:43.670-00:00 #2016-06-1720:14gphilipp
(def d1 #inst"26138-06-03T15:09:43.670-00:00")
CompilerException java.lang.RuntimeException: Unrecognized date/time syntax: 26138-06-03T15:09:43.670-00:00, compiling:(/Users/gilles/dev/try-spec/src/try_spec/core.clj:23:46) 
#2016-06-1720:18wilkerlucio@gphilipp: you can try something like this:#2016-06-1720:18wilkerluciothe first number, 100000 is a cap to limit the increment, and the 1465391285642 is the ms for a start date, adjust those to met your needs#2016-06-1720:19wilkerlucioah, and this example is for CLJS, please change the Date initialisation if you are using on Clojure#2016-06-1720:25gphilipp@wilkerlucio: thx, I will try this#2016-06-1720:28leifp@gphilipp: There is also an clojure.spec/inst-in macro.#2016-06-1720:28seancorfieldThe default 100 tests for test.check can take a really long time on some fairly simple looking specs...#2016-06-1720:29seancorfield…running 50 tests wasn’t too bad but it seems to be taking more than linearly longer to do 5, 10, 25, 50, 100...#2016-06-1720:31leifp@seancorfield: Yeah, as well as being very dependent on ordering. The caveats about "generate-and-test" style vs. constraint satisfaction are in full effect here.#2016-06-1720:32seancorfieldOn the plus side, I figured out how to write generators for stuff like java.sql.Connection etc šŸ™‚#2016-06-1720:32seancorfieldAnd I also finally figured out how s/keys and the test.check integration hang together… which answered a question I’d asked someone else here before I understood what was going on...#2016-06-1720:35gphilippThanks @leifp, (gen/sample (s/gen (s/inst-in #inst "2016-01-01" #inst "2016-12-31")) 100) did the trick#2016-06-1720:39leifp@gphilipp: The end date is exclusive, remember.#2016-06-1720:39gphilippah, correct#2016-06-1720:40gphilippI still wonder why (gen/sample (s/gen inst?) 60) generates invalid instants.#2016-06-1720:44leifpSee? Generative testing is going to help us avoid the Year 26k Problem.#2016-06-1721:08seancorfieldFYI, this is the spec that takes a crazy long time to gen test: https://github.com/clojure/java.jdbc/blob/spec-gen/src/main/clojure/clojure/java/jdbc/spec.clj#L205#2016-06-1721:10seancorfield(and it took me a while to even get so far as to make run-all-tests actually start running… with-gen taking a 0-arity function that returns a generator is very counter-intuitive and I kept getting that wrong šŸ˜ž )#2016-06-1721:57leifp@seancorfield: The (s/* (s/or ...)) combo seems to be the culprit. It would probably be very fast if you could limit it to a max size. I don't know how to do that, though.#2016-06-1721:59seancorfieldI guess I could always add a custom generator around it just to avoid that?#2016-06-1721:59seancorfield(and, thanks for the pointer on that!)#2016-06-1722:18leifp@seancorfield: This works: (s/def ::column-spec (s/with-gen (s/cat :spec (s/* (s/alt :kw keyword? :str string?))) #(g/vector (g/one-of [g/string g/keyword]) 0 6))) Still not what I'd call lightning-fast, but you can bound the generation time by decreasing the max. And note that g/ refers to clojure.test.check.generators, the equivalent using clojure.spec.gen didn't work.#2016-06-1722:20leifpKind of clunky, though, and you'd need to do that everywhere you have (s/* (s/alt ...)). Maybe if enough people run into performance problems, rep will be made public, or the regex generators will be optimized.#2016-06-1722:28seancorfieldThanks @leifp#2016-06-1723:58leongrapenthinAssume I parse with conform. Then I have functions that operate on the value returned by conform. I can't get a spec for the value returned by conform (so that I can spec said functions) - It seems like this could be automated though. Imagine (s/conform-spec ::my-spec) would return the spec of the return value of calling (s/confom ::my-spec 42)#2016-06-1800:08wilkerlucioI noticed (s/exercice number?) can generate some NaN entries on cljs, is that by design?#2016-06-1800:10leongrapenthinSo conform-spec as described above can apparently be implemented quite easily via (defn conform-spec [s] (s/and (s/conformer #(s/unform s %)) s))#2016-06-1800:17leongrapenthinBut it would probably be more valuable if a spec could give a spec of what its conform* returns#2016-06-1803:55wildermuthnIs there a changelog for Spec? I’m looking at some of the recent commits to cljs.spec, and see that the docs have been updated. But would be great to know if there was another place to reference to see the updates.#2016-06-1804:05wildermuthnThe more I learn about Spec, the more I think it is going to be regarded one of the essential Clojure features, on par with immutable data. For instance, I found out about conformer, and not ten minutes later had the beginning of a solution that I’ve never had for parsing JSON enums in a sane way. I shared this on #C03S1L9DN, but interested in getting feedback if I’m going at this the wrong way:#2016-06-1808:37vikeriA question about spec and generators, is it able to generate data from a spec directly or does it generate data and test it against the spec? As an example I suppose that a generator for string will only generate strings, but what if you have some slightly more complicated predicates like strings with the length 3? Will it understand to only generate strings with the length three? I guess this would need some way of inverting an if statement, which sounds difficult. The main question in hence: How ā€smartā€ can the generator be?#2016-06-1810:21Alex Miller (Clojure team)Well it's not magic :)#2016-06-1810:23Alex Miller (Clojure team)If you have (s/and string? #(= (count %) 3)) the and generator generates based on the first pred, then filters based on the subsequent ones#2016-06-1810:24Alex Miller (Clojure team)So it will generate random strings and keep those that are length 3#2016-06-1810:25Alex Miller (Clojure team)That's probably a restrictive enough filter that you'll need to supply a custom generator#2016-06-1810:28Alex Miller (Clojure team)@wildermuthn: I list incremental changes for each release in the announcement notes in the Clojure mailing list#2016-06-1810:29Alex Miller (Clojure team)@wildermuthn: not clear to me what you're asking about with conformers above #2016-06-1812:20ghadi@alexmiller: s/every has a tiny typo in it causing this:
user=> (s/valid? (s/every #{:foo}) [:foo :foo])
true
user=> (s/valid? (s/every #{:foo}) 42)
true  ;; incorrect
#2016-06-1812:21ghadihttps://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L1095#2016-06-1812:21ghadis/:invalid/::invalid#2016-06-1812:24Alex Miller (Clojure team)Thx#2016-06-1814:31nhaWhat is wrong with the following ?
(defn add []
  "a")

(s/fdef add
        :ret int?)

(s/instrument #'add)

(add) ;;=> "a"

#2016-06-1814:32nhaI was expecting an error on the return value.#2016-06-1814:33minimalInstrument was changed to only check :args#2016-06-1814:35nhaOoh so how do I check the return value ? (ie. what would be a simple test here ?)#2016-06-1814:37nhaAlso is there a constraint on the order of things ? Can I s/fdef before defn-ing a function ? Can I s/instrument-all when I want ?#2016-06-1814:41minimalFor now you have to use the functions from the clojure.spec.test namespace like check-var#2016-06-1814:43nhaThanks I will dig into it. Those can do "normal" testing as well right ? ie. not generative testing#2016-06-1814:43minimalYou can s/fdef before. The docs say the instrument fns are idempotent#2016-06-1814:43nhaThanks šŸ˜„#2016-06-1814:44nhaAah I read that but assumed it meant it can be called many times#2016-06-1814:44minimalYeah#2016-06-1814:46nhaAny reason the :ret has been removed ? Will it come back at some point ?#2016-06-1814:46eggsyntax@nha: see https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ#2016-06-1815:03nhaSo it looks like clojure.spec.test is only for generative testing, I still have to use whatever I was using if I want to explicitly give input arguments to the tested function (ex. for things too hard to generate). Correct ?#2016-06-1815:17gfredericksnha: sounds right#2016-06-1820:56arohnerFYI, I’m reliably reproducing the ’no such var encore/bytes?’ thing that @seancorfield reported the other day#2016-06-1820:56arohnerI don’t understand the cause, but I’m experiencing the same behavior#2016-06-1821:08seancorfieldThere's an updated Encore that fixes that. Some interaction with cljx / do / defn in that case - and conflicting with a (new) core function. #2016-06-1821:09seancorfieldBut I also eliminated every single dependency conflict and that solved the other, similar problems I had after that. #2016-06-1821:09arohneryeah šŸ˜ž#2016-06-1821:09seancorfieldSo I'm not sure what the root cause is at this point. #2016-06-1821:09arohnerI’m hoping the clojure bug gets fixed#2016-06-1821:10arohneris there a way to re-use fdef or fspec on other functions?#2016-06-1821:10arohnerI’d like to say ā€œthese two defns have the same fnspec"#2016-06-1821:14arohnerah, looks like I can just (s/def ::name (s/fspec)) (s/def ::foo ::name)#2016-06-1821:15bbloom@arohner: if two functions have the same fspec, wouldn’t they just be the same function? (or your fspec contains no interesting properties?)#2016-06-1821:16arohner@bbloom no, there are lots of things that have the same spec that are interesting#2016-06-1821:16arohneri.e. simple math#2016-06-1821:16arohner+, *, - all take two numbers, return a number#2016-06-1821:16arohnerin this case, parsing#2016-06-1821:16arohnertake some input, return a parsed value and the rest of the stream#2016-06-1821:17bbloomseems like you’d want to reuse a spec for args and result, but a different spec for the property relating args to results#2016-06-1821:17bbloomthere might be a missing abstraction here#2016-06-1821:18bbloomwe’ve got specs for data and specs for functions which contain arg spec, return spec, and relation spec, but we don’t have a spec for args + return (signature?) without relation#2016-06-1821:18arohneryes, I’d like to get more specific in the future#2016-06-1821:18arohnerand I still have an open challenge out to ā€œcorrectlyā€ spec clojure.core/map#2016-06-1821:19arohnerI assert it can’t reasonably be done right now#2016-06-1821:19bbloomcorrectly = ?#2016-06-1821:19arohnerā€˜map takes a fn of type X -> Y, and a seq of X, and returns a seq of Y'#2016-06-1821:20bbloomah yeah, spec lacks logic variables#2016-06-1821:20bbloomcan’t do context sensitive parsing#2016-06-1821:21eggsyntax@arohner: how could you verify that it’s a fn of type X -> Y in a language that doesn’t do type declaration?#2016-06-1821:22eggsyntaxIs it sufficient to hand it an X, designate whatever comes out as Y, and then go from there?#2016-06-1821:22arohnerwell, I can spec that f is X->Y now, and spec that coll is coll-of X#2016-06-1821:22eggsyntaxAh, gotcha.#2016-06-1821:23eggsyntaxMakes total sense; somehow I was assuming f was unspecced.#2016-06-1821:23arohnerbut that doesn’t narrow down map’s return spec#2016-06-1821:23arohnerbecause map’s spec is just seq or whatever#2016-06-1823:42zpinterhello everyone! wondering if anybody knows of a clean way to define the spec for a map and its keys without having to repeat the list of keys again for the map spec#2016-06-1823:43zpintermy first attempt was something like this:
(def style-keys
   [
    (s/def ::padding ::dimen-spec)
    (s/def ::padding-left ::dimen-spec)
    (s/def ::padding-right ::dimen-spec)
    (s/def ::padding-top ::dimen-spec)
    (s/def ::padding-bottom ::dimen-spec)])

(s/def ::style (s/keys :opt style-keys))
#2016-06-1823:43zpinterhowever, s/keys doesn't seem to like having :opt passed as a symbol (presumably since it's a macro)#2016-06-1823:44zpinterthis version compiles and works fine
(s/def ::padding ::dimen-spec)
(s/def ::padding-left ::dimen-spec)
(s/def ::padding-right ::dimen-spec)
(s/def ::padding-top ::dimen-spec)
(s/def ::padding-bottom ::dimen-spec)

(s/def ::style (s/keys :opt [::padding ::padding-left ::padding-right ::padding-top ::padding-bottom]))
#2016-06-1823:44zpinterhowever, every time I add a new key, I have to update two different places (which I'd ideally like to avoid in this case, since there'd be a lot of keys)#2016-06-1823:45bbloomzpinter: i’m far from an expert (i haven’t tried spec properly yet) but i think you can just use s/and, right?#2016-06-1823:45zpinterthe recommendation from #C03S1L9DN was to make my own macro, which seems like a good approach, just wondering there was anything built-in#2016-06-1823:47bbloomoh, nvmd, i think i misunderstood what you were asking for#2016-06-1823:49bbloombut yeah, i’d guess making your own macro is the way to go - but depending on what you’re doing, the redundancy might make sense in order to allow decomposition later#2016-06-1823:49bbloomfor example, you may want to limit the styles on some thing to only color related styles or only spacing styles or what not#2016-06-1823:50bbloomso you’d do (s/def ::padding…, (s/def ::padding-left ... and then group those with s/and to form ::padded#2016-06-1823:50bbloomthen you can have (s/def style (s/and ::padded ::colored ::etc-etc-etc#2016-06-1823:51zpinterhmm... does s/and work with s/keys?#2016-06-1823:51zpinterI guess you could use s/and to combine multiple (s/keys :opt values....#2016-06-1823:51bbloomi’d be surprised if it doesnt, but like i said - i haven’t installed the alpha yet to try it šŸ˜›#2016-06-1823:55zpinterwill keep investigating, thanks @bbloom#2016-06-1823:56eggsyntax@zpinter: I'd be interested in seeing what you come up with šŸ™‚#2016-06-1902:10Alex Miller (Clojure team)You can compose any kind of spec with and#2016-06-1902:11Alex Miller (Clojure team)Note also that s/keys validates all keys in the map regardless of what's in req or opt#2016-06-1902:12Alex Miller (Clojure team)So you don't technically have to list them as opt keys#2016-06-1910:25akielHas someone thought about specs for ex-data?#2016-06-1913:30gfredericks(s/fspec :args ... :ret ... :throws ...)#2016-06-1913:57cigitiaIn clojure.spec, is there a way to associate custom generators with your own functions, the way that clojure.spec.gen/gen-for-pred and gen-builtins associate generators with various clojure.core functions?#2016-06-1915:00gfrederickscigitia: you can make a spec that has a particular generator associated with it using spec/with-gen#2016-06-1915:01cigitia@gfredericks: Yes, though that requires either registering the spec returned by with-gen under a keyword, or using with-gen inline every time you use the function.#2016-06-1915:02cigitiaI’m wondering whether it’s possible to have generators when using functions directly—#2016-06-1915:02cigitiaLike how int? has a generator even when you use int? directly as a predicate, due to the way gen-for-pred works.#2016-06-1915:16gfredericksI think you can say (def int? (spec/with-gen #(instance? Integer %) gen/large-integer))#2016-06-1915:16gfredericksI might be lying#2016-06-1915:59Alex Miller (Clojure team)There is more stuff coming in this area soon#2016-06-1916:16akiel@gfredericks: is :throws discussed somewhere? I can’t find something about it.#2016-06-1916:57gfredericksakiel: no I just made it up#2016-06-1916:57gfredericksI was imagining an API that related to what you were asking about
#2016-06-1917:26akiel@gfredericks: Yes that’s exactly what I’m after. @alexmiller What do you think about exception data specs? As far as I see it, exception data keys are not even documented normally.#2016-06-1917:33Alex Miller (Clojure team)We are not going to spec exceptions#2016-06-1917:34Alex Miller (Clojure team)You could of course create specs for the ex-info map keys and I can see that possibly useful in functions that receive ex-info data after an exception has happened (to generate user errors or error pages etc)#2016-06-1917:37akielDo you think that the shape of the ex-data should be public API of a function or do you are more in line with Joshua Bloch: Effective Java Item 57: Use exceptions only for exceptional conditions were he suggests that exception should not be used for decision making.#2016-06-1918:20Alex Miller (Clojure team)Yes#2016-06-1918:21Alex Miller (Clojure team)Unless you're abusing it for performance :)#2016-06-1919:24akiel@alexmiller: sorry for asking further - yes public or yes not-public?#2016-06-1919:30hiredmannot specing execeptions makes a lot of sense#2016-06-1919:34hiredman1. spec is not a type system 2. spec is primarily about specifying data and exceptions are not data, they are weird control flow operation in a language 3. specs for functions specify what valid argument data looks like, and given that valid argument data, what a valid result looks like, and that is how the generative stuff works, and in that model there is no place for exceptions#2016-06-1919:35hiredmangiven a function F, if F throws E, to ensure correctness you want to look at all callers of F, and ensure that they handle E correctly, which is exactly the opposite of how the generative testing works (if I understand correctly)#2016-06-1919:36akiel@hiredman: in case API’s don’t use exceptions for control flow I’m with you. But than we don’t need ex-info and ex-data at all and should stop build API’s which put data into exceptions#2016-06-1919:38akiels/conform is a good example for an API not using exceptions - it returns ::s/invalid#2016-06-1919:40bhaumanIs there a way to validate that you have defined all namespaced keywords in the your spec definitions? And get a warning if some are missing?#2016-06-1919:41hiredmanwrite a predicate that checks the spec registry#2016-06-1919:46bhaumanhmmm ... so say compose a macro over spec/keys, capture the key args and and then warn if if the keys are not present in the registry? or were you thinking something else?#2016-06-1919:47bhaumanif you wanted to verify everything with a single call though it seems like you would have to parse the describe of each of the registry members#2016-06-1919:47hiredman
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {:a 1})
true
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {::a 1})
false
user=> (s/def ::a ::s/any)
:user/a
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {::a 1})
true
user=> 
#2016-06-1919:49bhaumancool yep, but was thinking of a different use case, where you are inspecting the self referential integrity of the registry itself#2016-06-1919:50hiredmannot sure I follow? that s/map-of spec will fail if given a map with namespaced keys that are not speced#2016-06-1919:51hiredmanoh#2016-06-1919:51hiredmanI think I get it now#2016-06-1919:52hiredmanyeah, spec seems to be slightly later binding than clojure is#2016-06-1919:53bhaumanyeah, it's doable, but I think it would be nice to have some help beyond describe#2016-06-1919:54hiredmanyou'd like ::foo to throw an error earlier then when you run the spec, basically, like clojure does with vars#2016-06-1919:55bhaumanI'd like to do ( check-missing (clojure.spec/registry))#2016-06-1919:56bhaumanto help me in the repl so that I can see if I forgot something#2016-06-1919:58hiredmanthat might not be currently possible, I don't think specs have a way to get all the specs they depend on#2016-06-1919:58hiredmanwhich might be an interesting enhancement#2016-06-1919:59bhaumanexactly, it would be nice to have a spec walker#2016-06-1920:03bhaumanor at least a corollary to describe that returns the actual internal data#2016-06-2002:05Alex Miller (Clojure team)there will likely eventually be a deep-describe, which will require something like that#2016-06-2013:54alqvistI feel that the usefulness of s/inst-in is hampered by cljs incompability. Any words of wisdom?#2016-06-2014:43alqvistOr I could just use the latest clojurescript - Never mind#2016-06-2015:04angeliniis there anyway to reproduce the behaviour of alpha-5’s s/instrument without using spec.test? The rational of only checking :args makes sense, but it was really useful when testing code at the REPL to verify that :ret and :fn we’re correct.#2016-06-2015:32adamfreywhen you have a test.check generator that is failing a such-that predicate 100 times, is it possible to print out the failing values, just to make the generator less opaque? Nothing stood out to me in the source code for doing that.#2016-06-2017:39akielAre there plans to support some kind of validation context? I think about a spec for JSON Web Tokens which are signed. Such a validation would need the public key as context. Or do I stretch the intended usage patterns here?#2016-06-2019:41Alex Miller (Clojure team)@adamfrey nothing explicit but you can gen/sample the s/gen of some simpler part to get an idea. It's usually the first pred in an s/and that is creating things that won't pass the later preds in the s/and#2016-06-2019:41Alex Miller (Clojure team)@akiel no plans for that#2016-06-2019:42Alex Miller (Clojure team)Prob a sign you are taking things too far :)#2016-06-2020:10akiel@alexmiller thanks#2016-06-2020:38Alex Miller (Clojure team)@cigitia: this channel is for spec, please take to #off-topic #2016-06-2020:40cigitiaMy apologies; accidentally sent to wrong Slack team; meant to sent to personal friends#2016-06-2108:07borkdudemakes me wonder about the spec for get-in: (get-in {:a "lol"} nil) ;;=> {:a "lol"}#2016-06-2108:08borkdudeI guess it could be correct if nil is regarded as the empty sequence#2016-06-2111:55akiel@borkdude: nil is the empty sequence šŸ™‚ I think it should be possible to write a spec for get-in.#2016-06-2114:33jannisIs there any chance of opts being added to clojure.spec.test/run-(all-)tests? I have a couple of functions that exceed the GC limit if I run 100 tests against them. 25 or 50 is fine, so I'd like to set :num-tests globally instead of having to use check-var for every single function I want to test.#2016-06-2114:35jannisAn alternative idea would be per-`fdef` opts perhaps?#2016-06-2114:58jannisOr a way to exclude specific functions from run-(all-)tests?#2016-06-2115:06akiel@jannis why are your functions are so memory hungry? do they have side effects or are the inputs to big? for the latter, I would rather optimize the generators.#2016-06-2115:07jannisGood point, it's probably the generators generating big inputs. I'll double-check.#2016-06-2115:09jannisAlthough... the generator is already generating relatively small data.#2016-06-2115:10jannisThat's the function: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L324 and this is the generator: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L90#2016-06-2115:10angusiguesssuch-thats are a good thing to look out for.#2016-06-2115:12jannisI have overriden almost all generators because my data specs seem to be too complex to use the default generators.#2016-06-2115:12angusiguessOne thing we ran into was the following:#2016-06-2115:12angusiguessIf there's a significant distance between your generator and your validation predicate that generator tends to perform poorly.#2016-06-2115:13jannisOf course, that makes sense. That's why I overrode most of the generators so that they don't have to be tried a lot to have conforming data generated.#2016-06-2115:15angusiguessI haven't tried it, but does increasing the heap size help?#2016-06-2115:16jannisWithout knowing much about it, GC limit exceeded to me sounds like it's generating too much data too quickly for GC to catch up. Let me re-run the tests with a heap size of 2GB.#2016-06-2115:17angusiguessGC Limit exceeded usually means that the ratio of GC pauses to 'useful' computation is quite high.#2016-06-2115:18angusiguessSo more heap space can reduce pauses.#2016-06-2115:18jannisOk#2016-06-2115:23jannisIt's also a test performance issue by the way. With 100 tests, that single function test runs for ~5-10 minutes until it errors out with the GC limit. I'm aiming for a fast test suite, so being able to reduce the number of tests globally - or for this function - would help.#2016-06-2115:56jannisIncreasing the heap size avoids it failing after 5-10 minutes. Instead it has now been running for more than half an hour. šŸ˜‰#2016-06-2115:58akiel@jannis I have you source code open. But I never used boot before and I’m using cursive - so its not so easy for me to get it running.#2016-06-2115:58jannis@akiel: Once you have boot installed, all you should need is boot test-once#2016-06-2116:20akiel@jannis: the generator of ::property-spec is already very slow. Try (s/exercise ::properties-spec 15) and than with 20. The thing is that generated samples become bigger and bigger if you generate more. test.check has some internal size thing. You have to improve the generator.#2016-06-2116:21jannis@akiel: Cool, thanks for investigating. I'll see what I can do. šŸ™‚#2016-06-2121:15cigitiaIs there any particular reason that the sequence regex ops do not recognize strings as seqables of chars? For instance, ā€abā€ does not conform to (s/cat :a #(= % \a), :b #(= % \b)), while [\a \b] does.#2016-06-2121:19cigitiaIn addition, are there plans for negative-lookahead regex ops, like those from &? For instance, in PEGs / parsing expression grammars, & has a negative counterpart called !Ā . In particular, an end-of-sequence regex op would be very useful—it could be called ::end, and it could be equivalent to (! ::any). As far as I can tell, this isn’t currently possible.#2016-06-2122:25bfabrylol, @cigitia your slack-fu is struggling#2016-06-2122:25Alex Miller (Clojure team)@cigitia: you're posting news links again#2016-06-2122:25cigitiaArgh, sorry#2016-06-2122:26Alex Miller (Clojure team)@cigitia There are good tools for string regex. Spec is never going to be great for that and there are no plans to use it for that#2016-06-2122:27Alex Miller (Clojure team)No plans for negative look ahead ops #2016-06-2122:27cigitiaOkay, thanks#2016-06-2122:27Alex Miller (Clojure team)They are generally not needed for the kinds of things we expect you to do with spec regex#2016-06-2122:32cigitiaI had actually been working on a PEG library that could create grammars for generic EDN data, including but not limited to strings, before Spec was announced#2016-06-2122:34cigitiaOnce Spec was announced, it seemed like there wasn't much room for such a library anymore, and so I thought about instead making a library that would utilize specs and convert them into parser transducers#2016-06-2122:35cigitiaMaybe there's still room for the first idea, though, then, hm#2016-06-2202:59Oliver GeorgeI was expecting s/conform to return fully "conformed" data but it doesn't seem to work like that.#2016-06-2203:00Oliver George
=> (s/conform (s/coll-of (s/conformer (fn [x] (println "I am conforming" x) (str x))) []) ["1" 2 :3])
I am conforming 1
I am conforming 2
I am conforming :3
["1" 2 :3]
#2016-06-2203:01Oliver Georgeit uses the conformer in order to check the data but returns the un-conformed data. Seems to be "non-recursive"#2016-06-2203:02Oliver GeorgeIs that intended behaviour? Perhaps I'm attempting to use it incorrectly - as a data verification & transformation tool.#2016-06-2203:40Alex Miller (Clojure team)The problem here is that coll-of samples its contents and doesn't flow the entire contents#2016-06-2203:40Alex Miller (Clojure team)There are some things in this area changing in next alpha#2016-06-2203:57Oliver GeorgeThanks Alex, I think I see similar behaviour with map-of.#2016-06-2204:02Alex Miller (Clojure team)Yes#2016-06-2204:02Alex Miller (Clojure team)every and every-kv are new things already in master but not yet released#2016-06-2204:03Alex Miller (Clojure team)One of those sets will conform all args, although I'm not sure which#2016-06-2204:08Oliver GeorgePerfect. Thanks.#2016-06-2211:38akielIs there a way to spec values which will be conveyed over a channel or returned by a promise?#2016-06-2211:53Alex Miller (Clojure team)nothing built in yet#2016-06-2211:54Alex Miller (Clojure team)ultimately I expect there will be some things provided for core.async#2016-06-2211:55Alex Miller (Clojure team)you could supply a map transducer to an async channel that called s/conform or something like that though#2016-06-2212:09akielsome kind of spec wrapper which takes a spec and validates channel values on the go would be nice#2016-06-2212:11akielI only like to be sure that nothing speaks at this conceptionally before spec gets final#2016-06-2214:02eggsyntaxtest.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one. The only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen), but it's pretty ugly to put in a generator and then ignore what it generates.#2016-06-2214:04eggsyntax(I understand that shouldn't be a typical use, but it'd be good to have an escape hatch for cases where you have an existing source of randomness that you don't want to have to duplicate)#2016-06-2214:43akiel@eggsyntax: As you said already, it’s not the typical use to implement the root generator yourself. You have to understand that test.check needs to control the generated values in a way to get shrinking work. So I have no answer to your question only the suggestion: Don’t do this! šŸ™‚#2016-06-2214:44eggsyntaxHeh, fair enough.#2016-06-2214:46eggsyntaxAlthough in this case I'm still gonna do it...again, existing (complex) source of domain-specific randomness.#2016-06-2214:46eggsyntaxMaybe I'll have to figure out how to make a Rose tree from it...#2016-06-2214:47eggsyntaxI realize there's a likely cost in terms of shrinking, though.#2016-06-2218:07Alex Miller (Clojure team)@eggsyntax: have you looked at return ?#2016-06-2219:22eggsyntax@alexmiller: I have, but it looks like that'll only generate a constant value, right? I need something that'll call a fn every time it needs a new one.#2016-06-2219:39angusiguesstest.check does have a no-shrink qualifier too I believe.#2016-06-2219:39angusiguessIf the value doesn't benefit from shrinking (like uuids for ex)#2016-06-2223:06bostonaholichas anyone come up with a reasonable clojure.spec/def for a database argument?#2016-06-2223:06bostonaholicall I’m using at the moment is ::s/any#2016-06-2223:06bostonaholic
(defn get-user [db id] ...)

(s/def ::database ::s/any)

(s/def ::user (s/keys :req [::id ::email]))

(s/fdef get-user
        :args (s/cat :db ::database :id ::id)
        :ret ::user)
#2016-06-2223:07bostonaholicbut obviously that won’t work with generators#2016-06-2223:08bostonaholicI’m not sure there’s much value here, but interested in hearing what a possible solution might be#2016-06-2223:40eggsyntax@bostonaholic: what should the DB argument look like?#2016-06-2223:41bostonaholicwell, it’s a datomic db#2016-06-2223:42eggsyntaxAh, ok, you hadn’t mentioned it was datomic.#2016-06-2223:42bostonaholicyeah, sorry#2016-06-2223:42bostonaholicthat’s why I’m thinking this isn’t a good idea#2016-06-2223:44Oliver George@alexmiller picking up on my question about fully conforming yesterday. I tried every-kv this morning (cljs release) and I see the same behaviour:#2016-06-2223:45Oliver George
=> (do
  (s/def ::tree (s/or
                  :branch (s/every-kv keyword? ::tree)
                  :value ::s/any))
  (s/conform ::tree {:A {:B 2}}))

[:branch {:A {:B 2}}]
Was hoping for [:branch {:A [:branch {:B [:value 2]}]}] or similar.
#2016-06-2223:45eggsyntaxYou can certainly check the type using a predicate built from instance?#2016-06-2223:46eggsyntaxThat might be enough for your needs.#2016-06-2223:47bostonaholicyeah, that’s the only other thing I could think of#2016-06-2223:49eggsyntaxThat's enough to show it's a legit database...you could also do something like
(s/and #(instance? datomic.db.Db %)
                       (s/keys :req-un [::key1 ::key2]))
with further specs for those keys. (don't have a datomic-using project right in front of me, I'm improvising on the type, and assuming that it's a record)
#2016-06-2223:50eggsyntax(which could be dead wrong šŸ™‚ )#2016-06-2223:51bostonaholicyou’re right#2016-06-2223:56Alex Miller (Clojure team)@olivergeorge its not done yet#2016-06-2223:56eggsyntax@bostonaholic: ooh, quite a few keys there, huh? (:id :memidx :indexing :mid-index :index :history :memlog :basisT :nextT :indexBasisT :elements :keys :ids :index-root-id :index-rev :asOfT :sinceT :raw :filt)#2016-06-2223:57bostonaholicheh, yeaahhhhhhhhh#2016-06-2223:58Alex Miller (Clojure team)@olivergeorge: I believe when it is done map-of will be what you want #2016-06-2300:01Oliver GeorgeThanks again Alex. I suspected that might be the case.#2016-06-2301:08eggsyntax@alexmiller: is there a roadmap online anywhere for what changes are definitely coming? Or even something that discusses some of them?#2016-06-2301:34Alex Miller (Clojure team)no#2016-06-2301:34eggsyntaxOh well šŸ™‚#2016-06-2301:35Alex Miller (Clojure team)there are definitely changes around instrument and the c.s.test namespace coming soon#2016-06-2301:35Alex Miller (Clojure team)and map-of and coll-of work is under way#2016-06-2301:35Alex Miller (Clojure team)there are a bunch of other things on the list but I don’t how that will all pan out#2016-06-2301:36eggsyntaxExcellent! I don’t think I’ve actually said this to y’all before about spec, but thanks šŸ™‚. I’ve been working with it for about a week, and it’s just terrific work.#2016-06-2301:36eggsyntaxJust how powerful it is didn’t even fully sink in until I’d dived into it for a while.#2016-06-2301:36eggsyntax^ @alexmiller#2016-06-2301:39seancorfieldIf I have (s/keys :req-un [::foo ::bar]), am I correct that something in clojure.spec will expect ::foo and ::bar to exist as specs, for some operations? I was trying to add stuff to clojure.java.jdbc.spec to support generative testing (by writing custom generators for db-spec, java.sql.Connection, and java.sql.PreparedStatement etc) and I started getting errors that the "unqualified" keys in some of my specs didn’t resolve...#2016-06-2301:40seancorfieldThat wasn’t obvious to me from the documentation about s/keys and the -un keys in maps — but it makes sense from the p.o.v. of actually trying to generate maps to pass into functions whose arguments are spec’d as s/keys...#2016-06-2301:53seancorfield(FWIW, folks can look at the work-in-progress experiment on a gen-testable java.jdbc spec here https://github.com/clojure/java.jdbc/blob/spec-gen/src/main/clojure/clojure/java/jdbc/spec.clj )#2016-06-2302:31Alex Miller (Clojure team)if you’re trying to gen keys in a map, then yes you would need them to have a definition#2016-06-2302:31Alex Miller (Clojure team)otherwise how would you gen them?#2016-06-2302:33Alex Miller (Clojure team)are you getting this while gen’ing or during something else?#2016-06-2303:01seancorfieldDuring gen’ing. And, yes, it makes sense. It just wasn’t obvious from the docs...#2016-06-2303:02seancorfieldBecause I missed this (very important) phrase "These variants specify namespaced keys used to find their specification"#2016-06-2303:04seancorfieldI just didn’t associate the ::first-name etc with the specs given two examples above since they were "unqualified keywords" in my mind.#2016-06-2303:06seancorfieldNot sure what to suggest to make that clearer…? Perhaps repeat the specs in the :unq/person example?
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::email ::email-type)

(s/def :unq/person
  (s/keys :req-un [::first-name ::last-name ::email]
          :opt-un [::phone]))
#2016-06-2303:06seancorfield(and now I notice that ::phone is not provided as a spec anywhere on that page)#2016-06-2303:07seancorfieldObvious in hindsight, of course.#2016-06-2308:34Oliver GeorgeThis is a thought in passing. Often we want confidence that some data is fully realized. It seems like a challenging case. We have realized? as a test and could write a recursive ::realized spec. It is challenging since displaying the data in reporting might "realize" it. So the test modifies the data.#2016-06-2308:35Oliver GeorgePerhaps not a clojure.spec domain issue... data is valid, it just doesn't exist yet!#2016-06-2308:35Oliver GeorgeWe've seen cases where lazy and dynamic aspects cause surprises. Right now we're considering a class of problems where delayed evaluation makes debugging harder.#2016-06-2308:36Oliver GeorgeJust raising in case it's something of interest as spec matures.#2016-06-2308:48thomasdeutschhow can i spec this structure?
{::items-by-id {"id-1" {::id "id-1"
                  ::title "first-item"} 
          "id-2" {::id "id-2" 
                  ::title "second-item"}}}
i would like to spec that every item ::id is the same id as its key on the parent map.
#2016-06-2310:50wagjoAre optional docstrings for s/def (CLJ-1965) something you'd like to have or were they left out intentionally?#2016-06-2312:01Alex Miller (Clojure team)@wagjo: still on the table#2016-06-2312:02Alex Miller (Clojure team)@olivergeorge: I don’t expect spec is going to add anything re lazy/realized data#2016-06-2312:02Alex Miller (Clojure team)@seancorfield in my next guide pass I will see if I can do anything to bring that out#2016-06-2317:10seancorfieldMuch appreciated https://clojurians.slack.com/archives/clojure-spec/p1466683358000870#2016-06-2320:48uwoIs there a suggested way to maintain metadata about which keys a spec is concerned:
(s/def :bill/delivery inst?)
(s/def :bill/pickup inst?)
(s/def :bill/pickup-gt-delivery (fn [{:keys [bill/delivery bill/pickup]}] (> pickup delivery)))
(s/def ::bill (s/and (s/keys :req [:bill/delivery :bill/pickup])
                     :bill/pickup-gt-delivery))

;some lookup
{:bill/pickup-gt-delivery [:bill/delivery :bill/pickup]}
#2016-06-2321:40Alex Miller (Clojure team)no, I don’t think so#2016-06-2321:40Alex Miller (Clojure team)
user=> (s/explain-data ::bill {})
{:clojure.spec/problems {[] {:pred [(contains? % :bill/delivery) (contains? % :bill/pickup)], :val {}, :via [:user/bill], :in []}}}
#2016-06-2321:41Alex Miller (Clojure team)The explain-data will tell you what it’s missing in the form of predicates but that’s not what you’re asking#2016-06-2321:41Alex Miller (Clojure team)(btw, as of the next alpha, you’ll be able to rewrite pickup-gt-delivery as (fn [{:bill/keys [delivery pickup]}] (> pickup delivery)) )#2016-06-2322:07uwo@alexmiller: cool. thanks for the response!#2016-06-2402:22bsimaHow do you spec protocol functions? Is it the same as spec-ing a regular function? (I haven't actually tried it yet...)#2016-06-2402:23bsimaI tried to spec a multimethod using fdef a few days ago and it didn't work but I didn't have time to figure out why#2016-06-2402:37seancorfield@bsima: Read this issue for some information about spec and protocols http://dev.clojure.org/jira/browse/CLJ-1941 (regarding instrumentation)#2016-06-2402:42bsimamm thanks seancorfield#2016-06-2415:11jannisHow do I best write a generator for :args? It seems like even if I use (gen/cat <sequence-generating generators>) it generates a sequence that is then passed to :args as the first argument, instead of "splicing" it into the macro/function call.#2016-06-2415:13akiel@jannis Is it not sufficient for you to write individual generators? Do you have many dependencies between your args?#2016-06-2415:16jannis@akiel: This is my example: https://gist.github.com/Jannis/51ebdd54e10074fd9c574a54dd8421c9#2016-06-2415:17jannisThe args are fairly flexible and somehow what gets generated for name and & forms is not quite what it should be. Perhaps my :args spec is wrong (thinking var args perhaps need special treatment)?#2016-06-2415:21jannisEven if I just use (s/cat :name ::command-name) and drop the & forms argument in the macro, check-var complains with "Wrong number of args (1) passed to: command/defcommand". Something there is odd.#2016-06-2415:22akielYes (s/cat :name ::command-name) generates only one argument#2016-06-2415:23akielvaragrs is no special - you have already a s/* in your args#2016-06-2415:23akielwhat is ::command-name?#2016-06-2415:23jannisBut one argument should be fine for a macro that takes exactly one argument.#2016-06-2415:24jannisIt's a symbol? spec#2016-06-2415:24akielah yes you are right - one arg should be possible#2016-06-2415:25jannisI can paste the entire file once my laptop has rebooted.#2016-06-2415:30akielI get the same wrong number of args message with check-var#2016-06-2415:30akielbut if I convert the macro into a function, it works#2016-06-2415:30akielther may be some issues using char-var with macros#2016-06-2415:31akielI never spec’ed macros before#2016-06-2415:31jannisYep, I never ran into this with specs for functions.#2016-06-2415:32jannisActually, this is a Clojure + ClojureScript macro, which will just pass on the arguments to a function that does the actual work. I could move the spec to that function instead.#2016-06-2415:32jannisThat could work around the problem.#2016-06-2415:34akielbut speccing macros is an interesting feature on its own - but check-var might simply not work with it#2016-06-2415:35jannisCould be?#2016-06-2415:36akielI simply don’t know. I’ll hit this if a spec my first macro. But I have currently none.#2016-06-2415:36jannisOk šŸ™‚#2016-06-2415:37jannisMoving the spec to the function and turning the var-args into a (s/cat :name ... :forms (s/spec (s/cat :description ...))) subspec structure does the job šŸ™‚#2016-06-2415:37jannisAll the generated inputs are now useful.#2016-06-2415:40akielnice šŸ™‚#2016-06-2416:00kendall.buchananQuestion about the direction of namespaced keywords:#2016-06-2416:01kendall.buchananWith Clojurescript (Om appears to be pushing namespaced keywords) on the front end, specs in the middle, and Datomic on the end, namespaced keywords feel natural.#2016-06-2416:01kendall.buchananBut if you’re working with a Javascript front-end, that doesn’t support them. And a SQL database on the other. What’s one to do in the middle if you want to leverage specs?#2016-06-2416:01kendall.buchananJust :req-un everything?#2016-06-2416:02kendall.buchananOr create translation layers across the board?#2016-06-2416:02akiel:req-un works fine - the only thing you loose is automatic validation of all keys even if not present in req or opt#2016-06-2416:04kendall.buchananMakes sense. I’ve hesitated on spec for fear that they can’t live in ā€œthe real worldā€ (a non-full-Clojure stack environment).#2016-06-2416:05akieleven in clojure itself - nobody can change existing apis - so un-namespaced keywords will be still there#2016-06-2416:06kendall.buchananK, thanks @akiel.#2016-06-2417:49seancorfield@kendall.buchanan: I’m working on making namespaced keyword support in java.jdbc a lot easier. In the upcoming 0.6.2 release you’ll be able to specify a :qualifier "foo" option on calls to get :foo/col-name back from all queries. I need to figure out the final story for what happens if you push :foo/col-name into java.jdbc.#2016-06-2417:50kendall.buchanan@seancorfield: That’s very cool. I suspect work will need to be done on all ends of the stack to make namespaced keys more tenable for the average project.#2016-06-2417:56seancorfieldNow I look at it, I probably won’t need to do anything for pushing namespaced columns into java.jdbc:
user=> (def db-spec {:dbname "mydb" :dbtype "mysql" :user "tester" :password "secret"})
#’user/db-spec
user=> (j/query db-spec ["select * from status"])
({:id 1, :name "approved"} {:id 2, :name "new"} {:id 3, :name "rejected"})
user=> (j/query db-spec ["select * from status"] {:qualifier "status"})
({:status/id 1, :status/name "approved"} {:status/id 2, :status/name "new"} {:status/id 3, :status/name "rejected"})
user=> (j/insert! db-spec :status {:status/name "test"})
({:generated_key 13})
user=> (j/query db-spec ["select * from status"] {:qualifier "status"})
({:status/id 1, :status/name "approved"} {:status/id 2, :status/name "new"} {:status/id 3, :status/name "rejected"} {:status/id 13, :status/name "test"})
#2016-06-2417:58seancorfieldAnd due to a very recent change, even this works as expected:
user=> (j/insert! db-spec :status {:status/name "qualifier"} {:qualifier "result"})
({:result/generated_key 14})
#2016-06-2418:12Alex Miller (Clojure team)@jannis: check-var (very soon to be just ā€œtestā€ in the next alpha) is not going to work with macros#2016-06-2418:14Alex Miller (Clojure team)@kendall.buchanan: we are considering adding something in s/keys that lets you alias unaliased keys to arbitrary specs, but no guarantees#2016-06-2418:19jannis@alexmiller: Will there be an equivalent for fdef'ed macros?#2016-06-2418:19Alex Miller (Clojure team)you can fdef macros now#2016-06-2418:20Alex Miller (Clojure team)it’s just that check-var is not going to work to test them, afaik#2016-06-2418:23akielIs there a way to compose multiple s/key specs into one big one so that a map has to satisfy them all?#2016-06-2418:24bfabry@akiel: I wrote a macro that did that because I couldn't find anything. it was kinda painful#2016-06-2418:26akiel@bfabry: I need it to work with multi-spec because I have a big environment map were I have multiple subsets of varing keysets.#2016-06-2418:28bfabryI don't know enough about multi-spec to know if it would help, but here it is (very ugly) https://clojurians.slack.com/archives/clojure-spec/p1466110303000387#2016-06-2418:30akielAh s/form is nice. Never used it.#2016-06-2418:34akiel@bfabry: Thanks but your macro will only work with static s/keys specs.#2016-06-2418:34bfabryyup, I couldn't find a way around that because s/keys is a macro#2016-06-2418:39Alex Miller (Clojure team)@akiel you can just do s/and of multiple s/keys#2016-06-2418:39bfabry@alexmiller: that won't create good generators though#2016-06-2418:39Alex Miller (Clojure team)true#2016-06-2418:41akiel@alexmiller: Ok I only have to wrap all subsequent s/keys into a one-arg function and ignore the conforming value?#2016-06-2418:48akiel@alexmiller: ok it works#2016-06-2418:48Alex Miller (Clojure team)s/and will create a spec, not sure why you need to make it into a function#2016-06-2418:48akielNo my reading of the doc was just to loose.#2016-06-2418:48Alex Miller (Clojure team)the resulting conformed value should be the map#2016-06-2418:49akielI thought there would be always a predicate which gets the previous value. Bit this counts only for preds and not for specs.#2016-06-2418:51Alex Miller (Clojure team)not sure I understood that#2016-06-2418:54akielThe example is (s/and even? #(< % 42)) and the doc says: ā€œSuccessive conformed values propagate through rest of predicates.ā€ So I thought you can have one initial spec and than only predicates which are called with the previous conformed value.#2016-06-2418:54bfabry@alexmiller: out of interest is there any move to make some of these macros functions? a dynamic s/keys seems like it could be useful#2016-06-2418:55Alex Miller (Clojure team)@akiel: yes, although I think maybe you’re reading too much into the choice of ā€œspecā€ and ā€œpredicateā€ there#2016-06-2418:56Alex Miller (Clojure team)and combines specs which can be either predicates or other things#2016-06-2418:56Alex Miller (Clojure team)the conformed value flows through them#2016-06-2418:57akiel@alexmiller Ah ok. I see. The specs get also the previous conformed value and just use it to be able to return the whole thing at the end.#2016-06-2418:58Alex Miller (Clojure team)@bfabry: yes, that’s possible. things are macros so that we capture forms for reporting purposes. but there may still be function forms as well (which would fill in under the macro forms)#2016-06-2418:59bfabrycool cool#2016-06-2419:19fentonI have a multi-spec. My in data can look like: {:method :search :search-term "blah" :auth "abc123" :gps-lat 3.4 :gps-long 5.3} or {:method :get-products :product-id 12345 :auth "abc123" :gps-lat 3.4 :gps-long 5.3}. My multi-spec already keys off the :method key. My data has some shared keys: :auth, :gps-lat, :gps-long and some keys that are not shared: :search-term and :product-id. Ideally I'd like to create a spec that merges two s/keys specs into a single s/keys spec, however s/cat creates a spec that expects two hashes in a list as opposed to a single hash with merged keys.#2016-06-2419:21Alex Miller (Clojure team)I talked through some options for something like this with @luke - maybe he could elaborate on where he ended up#2016-06-2419:24Alex Miller (Clojure team)iirc, he ended up with a spec for the common keys and separate specs for each variant#2016-06-2419:24Alex Miller (Clojure team)and then specs for each variant that anded common and separate#2016-06-2419:25Alex Miller (Clojure team)and then instead of multi-spec, it’s an s/or of the concrete variants#2016-06-2419:26Alex Miller (Clojure team)with the downside that it’s a closed system#2016-06-2419:26Alex Miller (Clojure team)with multi-spec, you’d have to repeat the keys in each variant#2016-06-2419:28fenton@alexmiller: okay, i'll look into what it looks like with s/or option and compare that to repeating in a multi-spec scenario. Thanks.#2016-06-2419:29fentonthe closed system you mention doesn't use s/keys?#2016-06-2419:41Alex Miller (Clojure team)no it would#2016-06-2419:43Alex Miller (Clojure team)common = (s/keys …) A = (s/and common (s/keys A…)) B = (s/and common (s/keys B…)) all = (s/or :a A :b B)#2016-06-2419:44Alex Miller (Clojure team)that doesn’t help you with the conforming question though#2016-06-2419:45fenton@alexmiller: k i'll give that a try.#2016-06-2423:52leongrapenthin@fenton @alexmiller I ran into this too, it seems a common scenario with Datomic "polymorphism" aka {:payment/amount 42, :payment/type :payment.type/stripe, :payment.stripe/payment-id 1234} - it would be great to have a merge-keys operation that takes two s/keys and merges their implementations - what do you think?#2016-06-2423:55Alex Miller (Clojure team)Talked about it with Rich today#2016-06-2423:55Alex Miller (Clojure team)We'll see#2016-06-2501:28seancorfieldAlpha 8 is looking like a great release, based on the commits over the last few days!#2016-06-2502:14Alex Miller (Clojure team)Gonna be a big one :)#2016-06-2502:32seancorfieldAny likely ETA @alexmiller ?#2016-06-2502:46Alex Miller (Clojure team)Soon#2016-06-2502:47Alex Miller (Clojure team)Didn't quite finish everything today as expected so prob early next week#2016-06-2505:29bbloomrather than complain about having to upgrade fipp for new reader/printer forms, i’ll instead boast: open source contribution opportunities! clojure 1.9 functionality PRs welcome šŸ˜‰#2016-06-2518:26fenton@alexmiller: small typo in spec guide. the description around function specs uses an int? predicate instead of integer?#2016-06-2518:30Alex Miller (Clojure team)That's correct as is#2016-06-2518:30fentonoh, hmm, couldn't find an int? predicate#2016-06-2518:31Alex Miller (Clojure team)int? is a new predicate in 1.9 that matches any fixed precision integer#2016-06-2518:31gfredericksso in particular not bigints?#2016-06-2518:31fentonah, good to know.#2016-06-2518:31Alex Miller (Clojure team)It was briefly called long?#2016-06-2518:31gfredericksbigints are the only difference between int? and integer??#2016-06-2518:32bronsaAFAICT yes#2016-06-2518:33gfredericksinteresting#2016-06-2518:33gfredericksI wonder why that's considered more useful than the other way#2016-06-2518:34bronsawell there are a number of constructs in clojure that don't work with idxs bigger than Long/MAX_VALUE so I'm guessing int? was provided to cover those use-cases in specs#2016-06-2518:35bronsaguarding against i.e. (range 0 (inc Long/MAX_VALUE))#2016-06-2518:35gfredericksinteresting#2016-06-2518:36bronsa(although there are also other constructs that don't work with idx bigger than Integer/MAX_VALUE like dotimes IIRC?)#2016-06-2518:36bronsanope dotimes accepts longs#2016-06-2518:37bronsaah, I'm thinking of for and doseq, they're constrained by Collection.count returning int#2016-06-2519:08Alex Miller (Clojure team)Yes widening from just long to other fixed precision was the reason#2016-06-2519:08Alex Miller (Clojure team)In particular to capture int#2016-06-2519:11Alex Miller (Clojure team)Conceptually we would prefer if users primarily focus not on the Java types but on integer fixed vs integer arbitrary precision as the main "types"#2016-06-2519:40mathias_dwprobably a dumb question, but I’m having to call s/instrument-all every time I make a change to the code or the specs. Is there a way of automating that, or is my workflow simply wrong?#2016-06-2519:42mathias_dw(or is it maybe some nrepl/cider-related thing?)#2016-06-2519:47Alex Miller (Clojure team)You shouldn't have to do that #2016-06-2519:47mathias_dwok, thanks. I’ll try without the cider/nrepl stuff#2016-06-2519:48Alex Miller (Clojure team)That was true of the very first release but was fixed early on#2016-06-2519:48mathias_dwI’m using alpha5#2016-06-2519:49Alex Miller (Clojure team)Yeah shouldn't be a problem #2016-06-2519:49Alex Miller (Clojure team)Must be something in the workflow#2016-06-2519:49mathias_dwok, thanks. Will try to pinpoint it and raise it in the relevant place#2016-06-2520:45gfredericksI'm having trouble with a recursive spec on master; not a problem on alpha7: https://www.refheap.com/120803#2016-06-2520:45gfredericksI scanned the commit messages and didn't see anything suspicious#2016-06-2520:46gfredericksif I had to guess I'd say it's something to do with coll-of suggesting that there's ambiguity#2016-06-2520:48Alex Miller (Clojure team)That code is brand new and on my plate to test more on Monday morning so I will def look at it#2016-06-2520:49gfrederickscool, thanks#2016-06-2607:03seantempestais there a way to test if a spec exists for a namespaced key?#2016-06-2611:05Alex Miller (Clojure team)There is a 'registry' function #2016-06-2611:05Alex Miller (Clojure team)In master for the next alpha there will be 'get-spec'
#2016-06-2612:41jannisNeat#2016-06-2618:35fentonI have defined an api spec for front end to communicate with backend. The messages include an auth token that I strip off soon and forward the rest of the message to internal handling functions. It would be cool to be able to create a spec on my backend that takes a spec defined for the API and removes aspects of it, like for example the auth-token key of the map. I guess the generic manipulating of a spec like a regular hash or something would be interesting.#2016-06-2618:36fentonotherwise i have to duplicate the definition of the message....trying to keep it DRY.#2016-06-2618:55seancorfield@fenton: why not define the spec once for the back end, then define the spec for the front end as "back end" and "auth token"?#2016-06-2618:57fenton@seancorfield: I've pulled my api spec into a third project that is *.cljc with reader conditionals so it can be included as a library for both front and backend. I don't think I want to pull in my backend as a library to my front end....#2016-06-2618:58fentonnot sure if thats the best way to do it tho?#2016-06-2618:58seancorfieldJust the spec for the shared portion is what I'm suggesting. #2016-06-2618:59fentonahh that makes sense#2016-06-2618:59seancorfieldSo if your API and your back end traffic in a common subset of data, that spec needs to be external to both as a shared spec. #2016-06-2619:00fentonyeah, i like that...thanks for the idea!#2016-06-2619:02fentonhmmm...might run into same problem about how to make a spec like that... can't merge a spec map.#2016-06-2619:02fentonmap spec.#2016-06-2619:03seancorfieldYou use s/and to specify the API must conform to common spec and also to the auth token spec. You don't merge the maps. #2016-06-2619:04seancorfieldAlex talked about that technique here a day or two ago. #2016-06-2619:05fentonthat one?#2016-06-2619:06seancorfieldI'm on my phone so I can't dig into code. #2016-06-2619:07fentoni didn't try that approach, but wondering if that doesn't make data that looks like: [{:auth-token "abc123"} {:method :search :search-term "blah"}]#2016-06-2619:08fentonwhich is okay...i guess...#2016-06-2619:10seancorfieldSince maps are open, saying some data conforms to multiple map specs should still leave you with one map?#2016-06-2619:11seancorfield(Replete doesn't seem to have clojure.spec or I could try that out)#2016-06-2619:12fentonhmmm, not sure, making the (s/keys :req ... might be a problem the :req part...meaning each needs all?#2016-06-2619:12mfikes@seancorfield: It does, but it is under cljs.spec and slightly older at this point.#2016-06-2619:13seancorfieldAh, thanks @mfikes !#2016-06-2619:14seancorfield@fenton: if A needs key X and it needs key Y then that is additive and it needs keys X and Y - which is a merge and therefore what you want. #2016-06-2619:15gfredericksis (s/and (s/keys ...) (s/keys ...)) going to be special-cased to aid things like generation?#2016-06-2619:17seancorfield(s/conform (s/and (s/keys :req [::a]) (s/keys :req [::b])) {::a 1, ::b 2}) => {:cljs.user/a 1, :cljs.user/b 2}#2016-06-2619:18seancorfield(In cljs.spec in Replete)#2016-06-2619:19seancorfieldDang, I love that I can test that on my phone while lazing in bed on a Sunday!!!#2016-06-2619:19gfredericksI can't tell if that is supposed to answer my question or not :)#2016-06-2619:20gfredericksI can try it myself though, let's see...#2016-06-2619:20mfikes@seancorfield: And if we figure it out, clojure.spec will act as an alias for cljs.spec in bootstrapped ClojureScript šŸ™‚#2016-06-2619:20seancorfieldIt was for @fenton - I'm typing on my phone, excuse the slow responses. #2016-06-2619:20fenton@seancorfield: yeah, that looks good...#2016-06-2619:20seancorfieldReplete is truly awesome. #2016-06-2619:21fentonwhat is replete?#2016-06-2619:21gfredericksnope, the generator doesn't work#2016-06-2619:21fenton@seancorfield: ahh googled it.#2016-06-2619:21mfikesIt is for lazy people who don’t want to crawl out of bed to their computer to start a REPL.#2016-06-2619:21mfikes(Like me. šŸ™‚ )#2016-06-2619:22fentonlol!#2016-06-2619:25fentonhappy lazing folks, thanks for the help! šŸ™‚#2016-06-2619:26seancorfield@gfredericks: the generator doesn't work for and with two keys?#2016-06-2619:27gfredericksseancorfield: right, which makes sense given what I know about s/and#2016-06-2619:27gfredericksit'd have to be special-cased to have any chance of succeeding#2016-06-2619:27gfredericksbecause the s/and generator just generates the first thing and filters based on matching the rest of them#2016-06-2619:28seancorfieldwith-gen to the rescue! šŸ˜† #2016-06-2619:29seancorfieldI have nothing. #2016-06-2619:29gfredericksthere'll be a helper for this in my inevitable utility library#2016-06-2717:09angusiguess::args should be an s/cat#2016-06-2717:09angusiguess(s/fdef takes-a-map :args (s/cat :a ::A)))#2016-06-2717:09angusiguessArgs is always assumed to be a list of 0 or more arguments#2016-06-2717:10angusiguessOr rather, a spec for a list of 0 or more arguments#2016-06-2722:43bhaumanSo I'd like to do some spec work in an environment that needs to run on an earlier version of Clojure. I'd like to conditionally include the clojure.spec library if the Clojure version isn't 1.9 and if clojure.spec isn't already loaded. Is there an established smart pattern for doing this? Say requiring a library that does a conditional load?#2016-06-2723:08seancorfieldI don’t know how "established" it is, but I did this for java.jdbc’s test namespace...#2016-06-2723:08seancorfieldhttps://github.com/clojure/java.jdbc/blob/master/src/test/clojure/clojure/java/test_jdbc.clj#L28-L32#2016-06-2723:09seancorfieldAnd the specs themselves are in a separate namespace from the code https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2016-06-2723:10seancorfieldAn alternative approach is taken in clojure.core regarding Java versions: https://github.com/clojure/clojure/blob/d274b2b96588b100c70be065f949e1fdc9e7e14d/src/clj/clojure/core.clj#L6654-L6658#2016-06-2723:10seancorfield…and… https://github.com/clojure/clojure/blob/d274b2b96588b100c70be065f949e1fdc9e7e14d/src/clj/clojure/core_instant18.clj#2016-06-2723:41bhauman@seancorfield: thanks! I appreciate it. I'll take a gander :)#2016-06-2803:32Oliver GeorgeNew favourite spec trick. Wrap your spec to generate warnings instead of erroring out.
(defn warn
  "Throw warning if spec is invalid but always pass."
  [spec]
  (fn [x] (when-not (s/valid? spec x)
            (js/console.warn (s/explain-str spec x) x))
    x))
#2016-06-2806:52Oliver GeorgeAnd now not. Turns out the stack trace from normal errors is more informative (cljs).#2016-06-2808:56rauhI have within one ns two different specs for :db/id (one for data coming from datascript and one for datomic ids). How can I specify this in spec?#2016-06-2809:46rauhIs this the way to do this: https://gist.github.com/rauhs/3bf96b33f37f051859d5071cdde70148 ?#2016-06-2810:06puzzlerHow would you specify a "vector of anything"?#2016-06-2810:06puzzler(coll-of ??? [])#2016-06-2810:07puzzlerWhat goes in the ??? position?#2016-06-2810:07puzzlerIs there some sort of top-level type in spec?#2016-06-2810:08puzzlerI guess I could just use vector?, but I'd like it ideally to generate lots of random Clojure things to go in the vector.#2016-06-2810:09puzzlerSimilarly, what about a map where the keys can be anything but the vals are more restricted?#2016-06-2810:10puzzler(map-of ??? integer?)#2016-06-2810:15rauh@puzzler: (spec/coll-of ::spec/any [])#2016-06-2810:18puzzlerhanks#2016-06-2810:18puzzlerthanks#2016-06-2810:18puzzlerHmmm, looks like that conforms non-vectors too.#2016-06-2810:19puzzlerWhere is ::spec/any documented?#2016-06-2812:01jannis@puzzler: (s/spec vector?)?#2016-06-2812:03Alex Miller (Clojure team)There is new support for this in next alpha#2016-06-2812:03jannisCool#2016-06-2812:04jannisI have a lot of vector?-based specs that currently all require custom generators.#2016-06-2812:04Alex Miller (Clojure team)coll-of has many new options#2016-06-2812:17Alex Miller (Clojure team)Can now indicate the expected incoming type separately from the conformed and generated type#2016-06-2812:51Alex Miller (Clojure team)Also, Rich added s/merge yesterday to create spec that merges multiple map-validating specs and will gen as you expect (union of keys)#2016-06-2812:53Alex Miller (Clojure team)@jannis in next alpha you will be able to do (s/coll-of int? :kind []) to indicate a vector of ints#2016-06-2812:54jannis@alexmiller: Awesome šŸ™‚#2016-06-2812:58puzzlerWhen a spec is complicated and composed of many component specs, I'm having trouble figuring out, stylistically, which of those component specs to register to namespaced keys, versus just def'ing them to regular vars. Any tips on this?#2016-06-2812:59puzzlerIt seems to me that when spec'ing a bunch of optional keywords for a function, to get good error messages I really want to be able to say "These are the only keys allowed in this map of optional keywords and values". Is there any way to do that?#2016-06-2813:11Alex Miller (Clojure team)generally, I think you should not def anything to a regular var#2016-06-2813:11Alex Miller (Clojure team)and you should register specs for all map attributes and for any major ā€œtypes"#2016-06-2813:11Alex Miller (Clojure team)anything recursive will require registration at the recursion points#2016-06-2813:12Alex Miller (Clojure team)and you might want to register things at a finer granularity if you find it gives you more meaningful explain errors#2016-06-2813:13Alex Miller (Clojure team)re restricted key sets - no, this is intentionally not provided, but you can s/and a constraint like #(every? #{:allowed :keys :in :map} (keys %))#2016-06-2813:25puzzlerThanks. Early on, I predicted that a lot of people would re-invent slightly different definitions of validating or conforming combined with assertions in the event of a failure, and I've been seeing that happen. Is something along the lines of assert-conform and assert-valid? likely to make it into a future alpha?#2016-06-2814:10Alex Miller (Clojure team)yes, there are likely to be a conform-ex and some kind of assert, but I don’t think those will be in next alpha#2016-06-2815:48bhaumanI've been wanting to add an extended :reason to certain specs, especially if they have strange interdependencies#2016-06-2816:05bhaumanIf you look sideways at this you could see the possibility of specs with composable prose explanations#2016-06-2820:41kendall.buchananSuper quick question: If I’m referencing a spec in another namespace, do I need to :require that namespace too?#2016-06-2820:42kendall.buchananI’m finding the app runs fine in the REPL, but at compilation, it can’t resolve the specs.#2016-06-2820:42bhaumanyep#2016-06-2820:43bhaumanIf I'm understanding you correctly, all specs need to be required before use#2016-06-2820:43bhaumanin order for the registry to populate#2016-06-2820:44kendall.buchananOkay, I was afraid of that šŸ˜‰#2016-06-2820:44kendall.buchananThank you!#2016-06-2820:46martinklepschSo, form-validation & clojure.spec, I assume people have tried it already, are there any writeups/snippets?#2016-06-2820:48martinklepsch@bhauman: @kendall.buchanan I find it interesting that this can create an implicit dependency between namespaces that is not declared via the ns form, maybe for "good style" you should require the ns containing the spec even if you don't need to (technically)?#2016-06-2820:50kendall.buchananI’ll admit – I still find the coupling between specs and namespaces odd. Namespaced keywords seem to make sense when you’re trying to define a domain path, but sometimes those domain paths don’t match your namespace hierarchy. But Clojure basically says it has to.#2016-06-2820:50kendall.buchananStill struggling to understand it. But, seems fine.#2016-06-2820:50bhauman@kendall.buchanan: you are aware that you don't have to use an existing namespace correct?#2016-06-2820:51bhaumanjust a namespaced keyword#2016-06-2820:51kendall.buchananYeah, totally, but I guess I’m struggling to see the established path forward.#2016-06-2820:51martinklepsch> but sometimes those domain paths don’t match your namespace hierarchy#2016-06-2820:51martinklepsch:+1: (same struggle here)#2016-06-2820:53bhaumanIn terms of dependency management, I think if people look at the way they did plumatic/schema or any add hoc type validation.#2016-06-2820:54bhaumanit's like any dependency#2016-06-2820:55angusiguessWe've started defining a lot of our specs in a separate heirarchy. It's a bit early to tell how well that works but thus far it's been pretty useful in terms of reducing circular dependency issues, making specs equally usable in tests and code, etc.#2016-06-2820:55kendall.buchanan@bhauman, would you recommend (or consider as a viable path) to always use domain-based keywords school/:type and eschewing ::type (contained in my.app.model.school? Just ditching the latter pattern?#2016-06-2820:56angusiguessSo that's one possibility, is to use ::type but to place those definitions in a namespace that is a little descriptive of the domain model.#2016-06-2820:56bhaumanI wouldn't be the one to ask, but I would say no#2016-06-2820:57bhaumanA possiblity: As long as your keys are distinct, you can keep it all in one file.#2016-06-2820:57kendall.buchanan@angusiguess: Kind of like app.spec.school/type? That’s more or less what I’ve done for objects that cut across many domains.#2016-06-2820:58angusiguess@kendall.buchanan: That's precisely what I do, including a namespace for more generic types.#2016-06-2820:58angusiguessLike
app.spec/uuid-str
or similar.
#2016-06-2820:58kendall.buchananRight.#2016-06-2820:58kendall.buchananYeah, exactly what we’ve started doing.#2016-06-2820:59angusiguessThen if we have aggregates that our functions take we'll define them in the function's namespace.#2016-06-2820:59kendall.buchananAnd I think it works well. I guess I like the ::thing syntax, for brevity, and realize it ties things up into the namespace. I guess we’re learning as we’re going, though.#2016-06-2821:00kendall.buchanan(By the way, I have no proposed alternative, so not complaining. Just been a source of friction for me.)#2016-06-2821:00angusiguessDefining everything in-line bit us pretty quick.#2016-06-2821:03kendall.buchananRight, @angusiguess… which, I have to say, clojure.spec is what it claims: built for composability. I’ve been a heavy Schema user, and have to say, I’m getting more reuse out of spec by far.#2016-06-2822:17Alex Miller (Clojure team)Clojure 1.9.0-alpha8 is now out with lots and lots of spec updates https://groups.google.com/d/msg/clojure/vF3RuDWuX8I/pvn4IUuUAwAJ#2016-06-2822:41richhickeyLots of new spec stuff in alpha 8 - conforming coll support, plus count/kind control, instrument can generate stubs, gen/spec overrides in instrument/test, merge keys specs#2016-06-2822:44bfabryawesome#2016-06-2823:05Timwow sounds good#2016-06-2902:47Alex Miller (Clojure team)I’ve updated the guide for 1.9.0-alpha8 although there will be some additional updates http://clojure.org/guides/spec#2016-06-2909:20martinklepschI have a question about spec'ing maps: lets say I have two keys that should follow the same spec, is the intended way to define proxy specs which I then use as map keys?#2016-06-2911:28Alex Miller (Clojure team)You can register the map keys to point to either the same spec definition or to another registered spec. I think doing this will be common. One layer of registered base "types" and another layer of domain attributes.#2016-06-2912:41martinklepschThanks Alex#2016-06-2912:42martinklepsch
- [changed] explain-data - output is now a vector of problems with a :path element, not a map keyed by path
Whats the reasoning behind this change? Could there be multiple problems referencing the same path?
#2016-06-2912:58richhickey@martinklepsch: yes, multiple problems at same path happens with collections (was already happening with ā€˜keys' which jammed them all into same pred, now doesn’t)#2016-06-2914:05robert-stuttaford@martinklepsch: on your question from 11:20 today, do you mean like what i've done with ::pos-int here? https://github.com/robert-stuttaford/mtg/blob/master/src/mtg.clj#2016-06-2914:07martinklepsch@robert-stuttaford: yeah kind of I guess#2016-06-2914:08martinklepschif you want to use a generic spec in keys without using it's key you need a proxy like this#2016-06-2914:12robert-stuttafordyeah#2016-06-2914:21Alex Miller (Clojure team)@robert-stuttaford: there is a pos-int? predicate now btw in core#2016-06-2914:22Alex Miller (Clojure team)that ::types spec you have can now be accomplished with coll-of in alpha8 too: (s/coll-of ::type :kind set? :min-count 1)#2016-06-2914:23Alex Miller (Clojure team)that version has the benefit of automatically gen’ing too#2016-06-2914:25Alex Miller (Clojure team)And I think your ::spell, ::creature, ::planeswalker, etc can also be handled by s/merge now#2016-06-2914:25Alex Miller (Clojure team)new toys!#2016-06-2914:30robert-stuttafordoh that is rad#2016-06-2914:31robert-stuttafordthanks alex!#2016-06-2914:34robert-stuttafordhow far, as a %, would you say the core.spec slated for 1.9 is?#2016-06-2914:35robert-stuttafordit seems like you've still got a lot of meat on the bone, given all the stuff that gets added with every alpha#2016-06-2914:37richhickey@robert-stuttaford: not a lot of big things - assertions, perf tweaks, reconciling regexes and vectors, some refinement of keys*, then details#2016-06-2914:38richhickeythen moving to infrastructure around test-running etc, the whole model implied by instrument+test which is different from current practice#2016-06-2914:38ghadiOn the Complexity and Performance of Parsing with Derivatives -- PLDI 2016 https://www.youtube.com/watch?v=WPde6DDLtXg#2016-06-2914:38ghadiiteration.next of matt might & co's work#2016-06-2914:39ghadihttps://arxiv.org/pdf/1604.04695v1#2016-06-2914:39robert-stuttafordfantastic, rich! very exciting set of tools you're building, here. fundamental, like the data-oriented approach and immutability are. going to take some time to soak in#2016-06-2914:40robert-stuttafordi realise this may well be answered with 'maybe :-)', but i'm super curious how you plan to leverage spec in Datomic šŸ™‚#2016-06-2914:42richhickey@ghadi: thanks for the links#2016-06-2914:42robert-stuttafordDatalog is an obvious one, as is transaction data. wondering how else it may improve the experience of programming with it#2016-06-2914:43richhickey@robert-stuttaford: still TBD (but lots of obvious opportunities), first things first, and that’s a stable clojure.spec#2016-06-2914:43robert-stuttafordof course šŸ™‚#2016-06-2914:43richhickeyI can say we are getting a lot out of using it in the development and testing of Datomic already#2016-06-2914:44robert-stuttafordi can't think of a better way to dogfood it#2016-06-2914:45richhickeythe new stubbing support is something to check out - you can now take a version of your wire interface, spec it, and get a no-wire stub for free from spec#2016-06-2914:46ghadirad#2016-06-2914:46robert-stuttafordah - if i understand you correctly, you mean inferring a spec from an existing data set?#2016-06-2914:47robert-stuttafordif so, thanks .. i was planning to do this as a learning exercise!#2016-06-2914:48richhickeyno, let’s say you have a data-driven wire protocol that talks to a service - you spec the protocol, instrument its namespace saying :stub in the overrides, and then run tests of code that consumes the protocol. Now those tests don’t talk to a service at all but get checking of their payloads and generated returns#2016-06-2914:49ghadiaside: I really like the syntax of (s/alt :foo x :bar y). That is something I didn't get quite right in pex, my vm-based PEG parser; I made capturing things too verbose. Definitely going to adopt that approach#2016-06-2914:49richhickeyyou can selectively override gens to generate ā€˜stateful’ things like connections#2016-06-2914:50robert-stuttafordoh, right!#2016-06-2914:50richhickey@ghadi: yeah, the path system is something I think people resist initially, but it keeps paying dividends#2016-06-2914:51robert-stuttafordman. i feel like i'm going to need a spec sabbatical.#2016-06-2914:53robert-stuttafordso many new toys, but at such a fundamental place in the language.. it's going to touch everything!#2016-06-2914:53ghadicurious, why do people resist the path system?#2016-06-2914:54richhickeysometimes you don’t care about the labels or want to see them in the destructuring#2016-06-2914:54robert-stuttafordis https://clojure.github.io/clojure/branch-master/clojure.spec-api.html current as of alpha8?#2016-06-2914:54ghadiroger#2016-06-2914:55richhickeybut they let us talk about the structure of a spec all the way down, which is huge#2016-06-2914:55robert-stuttafordnot seeing anything about stub in there, yet#2016-06-2914:55ghadimy plan for better dev-time experience in the PEG library was to provide a tracing mode on all the patterns attempted#2016-06-2914:56richhickey@robert-stuttaford: (doc test/instrument)#2016-06-2914:56ghadisince backtracking is likely in a failure, and most of the ways to handle it were through heuristics like longest match#2016-06-2914:56robert-stuttafordthanks#2016-06-2914:56Alex Miller (Clojure team)@robert-stuttaford: I think those are lagging an alpha or two, I will get them updated#2016-06-2914:57richhickeyoops, didn’t realize API docs were behind#2016-06-2915:02bhaumanalright, I'm running into an issue in alpha7
(s/def ::string  (s/or :string string?))
  
  (s/def ::myid ::string)
  
  (s/explain (s/and
              (s/keys :opt-un [::myid])
              (s/keys :req-un [::myid]))
             {:myid "asdf"})
#2016-06-2915:03bhaumanResults in
In: [:myid] val: [:string "asdf"] fails spec: :strictly-specking.test-schema/myid at: [:myid :string] predicate: string?
#2016-06-2915:03bhaumanit appears to be validating against the conformed value#2016-06-2915:06bhaumanI'll check against alpha8, in a bit, i have to port some stuff#2016-06-2915:07bhaumanand it's strange because this works
(s/def ::myid string?)
  
(s/explain (s/and
              (s/keys :opt-un [::myid])
              (s/keys :req-un [::myid]))
             {:myid "asdf"})
#2016-06-2915:11bhaumanand that makes sense because the conformed value is just a string#2016-06-2915:16richhickeyā€˜and’ does flow conformed values - "Successive conformed values propagate through rest of predicates."#2016-06-2915:18richhickeythat may change for ā€˜merge’, which is what you should switch to for this application#2016-06-2915:19richhickeythere may also be and and and-> variants since sometimes you need this (e.g. when preceding pred is a regex) and sometimes not#2016-06-2915:24bhaumangreat, looking at merge now#2016-06-2915:29bhaumanthe above works with merge right now and inspection reveals that the conformed values aren't flowing right now.#2016-06-2915:32richhickey@bhauman: merge does flow through conform, not explain (that’s broken)#2016-06-2915:32bhaumanoh gotcha#2016-06-2915:33richhickeybut no reason for it to flow through conform, that will change. The big benefit of merge is that it will gen where ā€˜and’ cannot for this#2016-06-2915:34bhaumangood deal#2016-06-2915:36bhaumanmy use case is taking an existing keys spec and constraining it by moving some keys into :req#2016-06-2915:37richhickeyI understand, and that will be common#2016-06-2915:38richhickeyalso various intersections#2016-06-2915:38robert-stuttafordjust so i'm sure i understand what you're both talking about, are you talking about what i'm trying to do here? https://gist.github.com/robert-stuttaford/e1bc7bfbcdf62020277dda1a277394ca#2016-06-2915:38richhickeyso no flow is the right thing#2016-06-2915:38robert-stuttafordwhere i want to gen (s/and (s/keys ...) (s/keys ...)) automatically?#2016-06-2915:39robert-stuttafordseems like s/merge may be what i need to use instead#2016-06-2915:39richhickeyyes, that’s precisely the point of merge#2016-06-2915:39robert-stuttaford\o/ rad#2016-06-2915:39richhickeyit gens map unions#2016-06-2915:40richhickeysubject to the (temporary) flow problem @bhauman identifies above#2016-06-2915:41robert-stuttafordgotcha#2016-06-2915:42robert-stuttafordi've just seen http://dev.clojure.org/jira/browse/CLJ-1910 and http://dev.clojure.org/jira/browse/CLJ-1919. it's going to be terribly tempting to rewrite everything!#2016-06-2915:49robert-stuttafordi guess there'll be ways to add or remove namespaces from these maps?#2016-06-2915:51richhickeythis only affects reading, not the map#2016-06-2915:52richhickeyi.e. it’s not a lingering property of the map#2016-06-2915:54robert-stuttafordah, so if i wanted to add namespaces to a non-nsed map, i'm going to do so to the keys in the map myself#2016-06-2915:54richhickeythis is only reading/printing#2016-06-2915:54robert-stuttafordgotcha, thanks#2016-06-2916:16seancorfieldhttps://clojurians.slack.com/archives/clojure-spec/p1467211082001261 I hope that whatever comes out of this doesn’t make it harder to use other testing libraries (like Expectations) harder.#2016-06-2916:17seancorfield(but, yeah, great changes in Alpha 8… I updated java.jdbc to accommodate enough changes to get it running again but I haven’t dug deep into redoing the specs to leverage the new features)#2016-06-2916:51jannisAn optional piece of data inside a tuple is not supported, I assume? (s/def ::my-tuple (s/tuple (s/? keyword?) symbol?)) claims ['foo] is invalid: [foo] fails spec: :workflo.macros.command/my-tuple predicate: (= (count %) 2)#2016-06-2916:51jannisMy current "workaround" is to use (s/and vector? (s/cat :optional (s/? ...) :mandatory ...))#2016-06-2916:52jannisWhich is ok, it does mean that you have to write a custom generator for it though.#2016-06-2916:53jannisAlso, is there something like s/tuple for fixed-size non-vector sequences with a specific order of subspecs?#2016-06-2916:57Alex Miller (Clojure team)You can use s/cat, s/? etc for things like this #2016-06-2916:58Alex Miller (Clojure team)If you have optionality regex is definitely the right choice over tuple#2016-06-2916:58jannisOk#2016-06-2916:58Alex Miller (Clojure team)We are still looking at options to also enforce vector on regex in a good way#2016-06-2917:00Alex Miller (Clojure team)s/and is sufficient as you saw but needs custom gen as you said#2016-06-2917:04jannis@alexmiller: Ah, s/cat didn't work because I forgot s/spec around it to make it a subspec and require the content of s/cat to be in a child vector or sequence.#2016-06-2918:14uwomy colleagues would like to create a single source of truth to generate both datomic schema and specs. Is this advisable? I was under the impression that perhaps spec would be written mostly by hand and not codegen#2016-06-2918:48Alex Miller (Clojure team)seems low on the crazy scale to me#2016-06-2918:50uwohaha. thanks#2016-06-2918:54Alex Miller (Clojure team)code is data and all that :)#2016-06-2920:07tylerIs clojure.spec.test/test meant to be used with clojure.test?#2016-06-2920:13tylerI guess more specifically, what is the idiomatic way to set up generative tests with clojure.spec is there an equivalent of defspec from test.check for clojure.spec?#2016-06-2920:24Alex Miller (Clojure team)fully utilizing the instrumentation and test facilities in clojure.spec.test is a different kind of process from the existing clojure.test and I don’t think we will provide a linkage like defspec.#2016-06-2920:24Alex Miller (Clojure team)we have more to say about this and I expect we will have more guidance to provide before 1.9 is released#2016-06-2920:26tylerGotcha, is it expected to be a developer concern? Or will there be additional functionality added to integrate it with clojure.test? The return seems more data-centric than clojure.test.#2016-06-2920:35Alex Miller (Clojure team)I don’t currently expect anything that integrates it with clojure.test. The returns are more similar to the results from test.check as that’s what we’re actually invoking.#2016-06-2920:37Alex Miller (Clojure team)there are levels of question here - some around what’s provided in core, another level around runners, and another level around build tooling#2016-06-2920:41tylerI suppose the core of my question is around whether or not clojure.spec.test/test will integrate into the existing test tooling or is it expected that new test tooling will need to be built? instrument seems to fit into the current ecosystem but it seems like clojure.spec.test/test will require new tooling.#2016-06-2920:44puzzlerIs there a difference between (s/def ::type vector?) and (s/def ::type (coll-of ::s/any :kind vector?)) ?#2016-06-2920:47martinklepschWhen a required key is missing from a map the error looks like this:
{:pred [(contains? % :to) (contains? % :subject)],
   :val {},
   :via [:forms.app/email-form],
   :in [],
   :path []}
Is there a way to get an easier to parse description that will allow me to (programatically) figure out which keys are missing from a map?
#2016-06-2920:50angusiguessYou can get that data structure with s/explain-data#2016-06-2921:03wilkerlucio@tyler: if you want to run generative tests and have report of it in clojure.test you can use the clojure.spec.test/test like this: (is (true? (-> (clojure.spec.test/test 'my.ns/some-fn) :result)))#2016-06-2921:06tylerThanks @wilkerlucio. I had been playing around with that a little bit but the output isn’t very informative for failures for that. I think I’m going to go the route of experimenting with developing my own runner. Just wanted to make sure I wasn’t duplicating work/effort.#2016-06-2921:11martinklepsch@angusiguess: right, what I pasted is part of what s/explain-data returns, was just wondering if there's a better way to get a list of missing keys#2016-06-2921:12wilkerlucio@tyler: try using something like this:#2016-06-2921:13wilkerlucionot super pretty, but might be a good start šŸ™‚#2016-06-2921:13tylerThanks I’ll give that a go#2016-06-2921:16puzzlerIs there an explanation somewhere of how the sampling of every works? I'd like to get a sense for the probability that an error is detected if I switch from coll-of to every.#2016-06-2921:18mcorbinHello, i play with clojure-spec and i don't understand this :#2016-06-2921:19mcorbinWhy explain returns Success ? The second condition in the "s/and" should return false with a string parameter#2016-06-2921:24leongrapenthin@mcorbin I believe the lambda pred is always invoked with a vector, i. e. [:string "Hello Clojurians"]#2016-06-2921:27bhauman@mcorbin: theres a discussion of this from earlier today ...#2016-06-2921:27bhaumansee my posts above#2016-06-2921:28mcorbin@leongrapenthin: you win, thanks @bhauman i will look, thx#2016-06-2921:41Alex Miller (Clojure team)@puzzler: if you mean ā€œhow it samplesā€ - no there intentionally is not, as it may change in undefined ways in the future#2016-06-2921:43Alex Miller (Clojure team)all you should count on is that it will sample no more than *coll-check-limit* elements#2016-06-2922:03seancorfield@tyler: FYI, we use Expectations to drive test.check stuff and ended up with a pattern like this /cc @wilkerlucio
(expect (more-of {:keys [result num-tests]}
                 true result
                 100 num-tests)
        (tc/quick-check 100
                        (prop/for-all [s gen/string]
                                      (= 0 (ip->long (str/replace s #"[0-9]" "."))))))
#2016-06-2923:21puzzlerSuggestion: Right now it looks like every and coll-of check for distinctness with (apply distinct? ...). Would recommend (or (set? ...) (apply distinct? ...)) to short-circuit when a set is passed in to a function that expects a collection of distinct items.#2016-06-3000:16jjcomerI’m trying to use the new stub feature, but I can’t seem to get the stub to take hold. The println still happens for me:
(defn ding
  [x]
  (println "Hello World"))

(s/fdef ding
        :args (s/cat :x pos-int?)
        :ret nil?)

(defn adding
  [x y]
  (ding x)
  (+ x y))

(s/fdef adding
        :args (s/cat :x pos-int? :y pos-int?)
        :ret pos-int?
        :fn (fn [{:keys [args ret]}]
              (= ret (+ (:x args) (:y args)))))

(defn test-adding
  []
  (try
    (st/instrument `adding
                   {:stub #{`ding}})
    (adding 2 3)
    (finally (st/unstrument `adding))))
#2016-06-3000:29puzzlerWhat happened to instrument-all and instrument-ns in the new alpha?#2016-06-3000:33jjcomer@puzzler: All is instrument with no params and ns is passing an ns to instrument#2016-06-3000:49puzzlerthanks#2016-06-3000:50puzzlerIs there any way to keep the instrumenting on during development? It seems that when you recompile a namespace it blows away the instrumentation.#2016-06-3001:26seancorfieldSince instrument updates the Vars in place, I’d expect that (so, "no", I think is the answer).#2016-06-3001:26seancorfieldIf you recompile the namespace, you’re replacing all the Vars, so you’d need to re-run instrument.#2016-06-3001:31gfrederickspuzzler: one option is to put (s/instrument ...) at the bottom of relevant namespaces#2016-06-3001:31gfredericksoptions that don't modify the source code would depend on how you're loading your code#2016-06-3001:31puzzlerIn theory, defn could be rebound to automatically instrument functions that have specs.#2016-06-3001:33seancorfieldThat would only help if the spec was defined ahead of the function...#2016-06-3001:37puzzlerI'm not saying this should be the general definition of defn, I just mean that when you choose to instrument all, a new version of defn (and fdef to handle the case of function before spec) would ensure the instrumentation never gets out of sync at the REPL, which I'm finding to be an annoyance. Right now, you change the spec and the instrumented function doesn't see it, or you change the function and the spec-checking goes away. Too easy to forget to re-run instrumentation.#2016-06-3001:38seancorfieldI have my specs in a different namespace...#2016-06-3001:39puzzler@seancorfield: Are you saying that having specs in a different namespace alleviates this issue? Seems to me it would just make it even more complicated.#2016-06-3001:39seancorfieldI’m saying your suggestions wouldn’t address that problem.#2016-06-3001:39seancorfieldi.e., it’s not a general solution#2016-06-3001:40seancorfield(and I don’t know what the general solution is)#2016-06-3001:41seancorfieldFor the specific case of java.jdbc, I have the test namespace — which loads the specs — also run instrument.#2016-06-3001:42seancorfieldSo the code is separate from the specs, but the tests depend on them.#2016-06-3001:42puzzlerThe first time you call instrument-all, it's calling your spec name space which registers the specs in a global registry. So a defn that was aware of the specs in the registry would help that direction. Going the other direction, if you edit the spec in the separate file, that spec refers to the var in the implementation namespace, so that function could be re-instrumented. So I think it would probably work. What am I missing?#2016-06-3001:43seancorfieldI think it’s very dependent on workflow. I’m waiting to see what Clojure/core come up with (based on the discussion earlier about what’s coming for specs in the test tooling area).#2016-06-3001:44seancorfieldWhat you raise is certainly an issue that can make working with spec in the REPL pretty annoying.#2016-06-3001:44puzzlerI'm only imagining this would be helpful in a mode where all instrumentation is turned on for everything. I don't have a good handle on the workflow for spec yet, so I agree with your point that we'll see how this evolves.#2016-06-3001:46seancorfieldRight now I’m only instrumenting on a per-namespace basis as I work with / test a particular namespace. I ran into some early weirdness with instrument-all (and no longer remember the details) and backed off that… may have been back when it caused generative testing to occur on higher-order function specs?#2016-06-3001:46seancorfieldSo it may be that separating the generative part of function specs from instrumentation has addressed the issues I ran into...#2016-06-3001:47puzzlerThe only issue I have with putting specs in a separate namespace is that it diminishes their value as a form of documentation. (Actually, this is also an issue with putting tests in a separate namespace, which I've gotten used to, so maybe I'll get used to putting specs elsewhere as well).#2016-06-3001:48puzzlerIn your split-namespace approach, do you see specs in the doc strings of your functions?#2016-06-3001:49seancorfieldIf you’re working in the test namespace (where the spec has been loaded), yes.#2016-06-3001:50seancorfieldI haven’t settled on it as a normal way to work yet. We’re still exploring several options#2016-06-3001:50puzzlerBut users of your library generally wouldn't see that if they call doc on a function at the REPL, right?#2016-06-3001:50seancorfieldIf they load the specs, yes.#2016-06-3001:50seancorfieldThis allows the code to be used by pre-1.9 clients.#2016-06-3001:51seancorfieldFor java.jdbc, it supports all the way back to 1.4#2016-06-3001:51seancorfieldand even in 1.9, the specs are optional but available.#2016-06-3001:51puzzlerAh, yes, I can see how that would be the most critical thing to keep in mind for java.jdbc
#2016-06-3001:51seancorfieldwhen I run the java.jdbc tests on 1.9, it loads the specs and instruments the ns (not all, just java.jdbc)#2016-06-3001:52seancorfieldSo, for library code that might have pre-1.9 users, separate specs is pretty much the only possibility.#2016-06-3001:52puzzlerCould the main sourcecode namespace do that test on version number, and automatically load the specs for docstring purposes?#2016-06-3001:53seancorfieldFor application code that’s is bound to 1.9, I can imagine having specs intermingled — maybe.#2016-06-3001:53seancorfieldYeah, I guess java.jdbc’s source namespace could auto-load the specs on 1.9. I’m just not sure that would be what all its users would expect.#2016-06-3001:54seancorfieldAfter all, if you (instrument) your code would you expect all the external libraries to slow down due to specs?#2016-06-3001:55seancorfieldSome specs can have a really high performance overhead so I don’t think the library should dictate that.#2016-06-3001:55puzzlerI'm thinking that my number one use for specs will be to get good error messages when working with other libraries, so yes, I think that's what I personally would want. The biggest source of errors in my code is having a false expectation about how someone else's code should work.#2016-06-3002:06seancorfieldGood to know.#2016-06-3002:07seancorfieldWe’ve jumped on the Alpha builds at work (in production) in preparation for using clojure.spec fairly heavily but we’re still in the early exploration phase. We’re already leveraging the new predicates in production code tho’.#2016-06-3002:09seancorfieldWe haven’t fully decided what we’ll spec out yet vs what we won’t bother with. I’ve been exploring using java.jdbc with namespaced keys as part of this work.#2016-06-3002:10seancorfieldWe have over 30Kloc production code and over 11Kloc tests so whatever we do will need to be a gradual process...#2016-06-3002:11seancorfield…expanding what we can do with generative testing alone might bring us a good "bang for our buck".#2016-06-3002:18Tim@seancorfield: at world singles?#2016-06-3002:19seancorfieldYup. Just pushed a build to QA based on Clojure 1.9.0 Alpha 8 so that’ll go to production in our next build.#2016-06-3002:20seancorfield(we already have Alpha 7 in production)#2016-06-3003:58cap10morganI'm getting Don't know how to create ISeq from: spec_demo.core$full_name when running (stest/instrument full-name) in an ns where I have a fn full-name and have (s/fdef full-name ...). It works if I leave the argument off (so it instruments everything), but I thought it was supposed to take symbol args now too in alpha8?#2016-06-3004:22seancorfield(stest/instrument 'full-name) <-- needs to be a symbol#2016-06-3004:26seancorfieldTo explain the error message, (stest/instrument full-name) will evaluate full-name, since it's a function it evaluates to its class name which is a class full_name inside the package spec_demo.core because the Clojure is spec-demo.core/full-name.#2016-06-3004:26seancorfieldHope that helps @cap10morgan ?#2016-06-3004:50puzzlerI'm having trouble ascertaining the purpose of the retag in multi-spec. Can someone clarify when you'd actually see the retag?#2016-06-3011:35Alex Miller (Clojure team)In gen#2016-06-3011:35cap10morgan@seancorfield: ah, yes, thanks. I was trying #'full-name and getting a var but totally forgot about 'symbol.#2016-06-3011:37cap10morganI'm giving a presentation tonight on clojure.spec at Den of Clojure (Denver meetup). So putting together as many live demoes as I can.#2016-06-3011:44cap10morganHmm, but it still doesn't work. (stest/instrument 'full-name) doesn't seem to turn on instrumentation (I get an exception when an invalid arg blows up in the fn body rather than spec telling me it's invalid) and (stest/test 'full-name) just returns (). These all worked with just (stest/instrument) and (stest/test).#2016-06-3011:46Alex Miller (Clojure team)If you use back tick it will resolve the symbol to fully qualified#2016-06-3011:47Alex Miller (Clojure team)The symbol has to be fully qualified#2016-06-3011:47Alex Miller (Clojure team)Symbol, not var#2016-06-3011:49cap10morganOK, that fixed it. Thanks, @alexmiller. I think that's the first time I've used back tick outside of a macro. Makes sense that it would need to be fully-qualified since it's going into a central registry.#2016-06-3011:59cap10morganDid clojure.spec.test.check go away or change in alpha8? Trying to change how many tests get run in clojure.spec.test/test and test's docstring says to pass a map with a :clojure.spec.test.check/opts keyword.#2016-06-3012:11Alex Miller (Clojure team)That's just a keyword namespace, not an actual namespace#2016-06-3012:22cap10morganhuh, OK. not sure how to alias it, then. but I guess I don't strictly need to.#2016-06-3012:32jjcomerDoes anyone have an example of stub or replace? Neither seem to be working for me, the original fns still execute.#2016-06-3012:52puzzler@alexmiller I thought multi-specs couldn't auto-generate data since the possibilities can't be enumerated. And if you have to write your own generator, I don't see how the retag comes into play.#2016-06-3013:25bhaumanI'm sure that this has probably been discussed before, a bunch, but I am late to the game and curious about and flowing conformed values to successive predicates. Initially, it seems very surprising that a successive predicate is operating on a different__ value. This seems semantically more like and->. Is this in the Guide? It's also curious that the underlying implementation of merge is named and-preds. I know there is a reason for this and admit that I haven't looked closely enough at it.#2016-06-3014:57fenton@puzzler: you can auto-generate data for multi-specs.#2016-06-3015:47Alex Miller (Clojure team)@puzzler did you read the doc-string for multi-spec? I think it answers all these questions.#2016-06-3015:47Alex Miller (Clojure team)@bhauman: and vs and-> is under consideration. for now, and flows conformed values.#2016-06-3015:48bhaumancool, i didn't know and-> was under consideration#2016-06-3015:49Alex Miller (Clojure team)Rich mentioned it in the back chat#2016-06-3015:51bhaumanoh and a big big + šŸ’Æ on every#2016-06-3015:53flipmokidHi, I'm looking to write a spec for a map in clojurescript which is used to check user input. One of the keys requires that the input be an int and be in a set of allowed values which comes from a server (so it's an async call). What is the best way of achieving this?#2016-06-3015:59bhaumanif its deep in the spec, make the call to the server first, and then make a binding, and have your predicate read the binding#2016-06-3016:02bhaumanbut otherwise I would separate this concern from the spec#2016-06-3016:02bhaumanand do a second pass#2016-06-3016:03Alex Miller (Clojure team)you don’t have to make it a fn spec to use spec - you can explicitly call s/valid? to make that check#2016-06-3016:08flipmokid@bhauman: Thanks, that make sense (having that separate from the spec). BTW thanks for figwheel! @alexmiller I wanted to originally do something like (s/and integer? (call-service)) where call-service returns a channel which eventually returns a set of ints. I wasn't sure of the best way of doing this as that function isn't a predicate I'm fairly new to clojure btw so please bear with me!#2016-06-3016:09Alex Miller (Clojure team)have you considered just writing some Clojure code? :)#2016-06-3016:10Alex Miller (Clojure team)I’m not sure spec is buying you anything for what you’re describing#2016-06-3016:12flipmokidI will give it a go. Time to crack open Clojure Applied again šŸ™‚#2016-06-3016:14Alex Miller (Clojure team)well nothing about spec in there :)#2016-06-3016:15Alex Miller (Clojure team)those idiots were too dumb to write about it#2016-06-3016:17flipmokidLol. The book is great, thank you! I meant using the book to write some clojure code. I thought I should do the server check separately from the spec but just wanted to make sure.#2016-06-3016:17flipmokidThanks guys#2016-06-3016:17flipmokidMuch appreciated#2016-06-3016:18Alex Miller (Clojure team)you shouldn’t register a spec that’s calling a server, but you could use s/validate? to verify a value conforms to a spec (which has a predicate, which is based on a set obtained from elsewhere)#2016-06-3016:19Alex Miller (Clojure team)however, that ā€œcheckā€ is so simple, that you might as well just check if the value is in the set (without using spec)#2016-06-3016:23flipmokidWith the first option, what would be the best way of waiting for the set to come back out of the channel in clojurescript inside the predicate? The best my mind could do was poll the channel until there was something on it. The current map spec I have does have a lot of keys which are much more involved than this one, but this is the only one that needs to check against a remote resource.#2016-06-3017:10fentonwhere do we report issues found with spec?#2016-06-3017:13fentonbut change :matching-p to :matching and it works okay... seems hyphens don't work for matching values.#2016-06-3017:16fentonhmm... hold on that... maybe dirty repl#2016-06-3018:49blanceI just read about clojure.spec and it seems like a great tool for testing with test.check. To utilize it for unit testing, should I specify a spec for every functions using fdef in my application so that I just call clojure.spec.test/test to do unit testing? I've never used schema before so I'm not sure which function/args/data I should specify those and which not to#2016-06-3018:58seancorfield@blance: Have you read the Rationale for clojure.spec? It talks about when to use spec and what sort of stuff it’s designed for.#2016-06-3018:59seancorfieldhttps://clojure.org/about/spec#2016-06-3018:59bhaumanSo here is another thing, the :in path on key value errors in (s/every-kv ..) end withs a 0 1 presumably to disambiguate between the key and the value, but this renders the :in path pretty tough to use to look up a value in a nested structure. I'm thinking that intention was to have the :in path terminate with the map key and the 0 or 1 should be at the end of :path only#2016-06-3018:59eggsyntax@blance: & lots more detail in the guide: http://clojure.org/guides/spec#2016-06-3019:00eggsyntaxNotably http://clojure.org/guides/spec#_testing#2016-06-3019:01blanceYea I've read both.. It must be that I read through them too fast. I'll read them again.#2016-06-3019:02blanceI mean I know how to use them, but I'm not sure on where to use it. Does it make sense to say have a fdef for each single function in my app?#2016-06-3019:03eggsyntaxA reasonable shorthand might be: any function that you would otherwise have hand-written a test for.#2016-06-3019:03eggsyntax(and of course for fns where spec is valuable in other ways, eg runtime validation of inputs)#2016-06-3019:04eggsyntaxThat's just my take on it, though, and I'm a spec beginner too. As I guess are we all šŸ˜‰#2016-06-3019:05blanceyou would want your unit test to cover as much functions as possible right? so write spec for as much functions as possible as well?#2016-06-3019:06blanceI feel like some fns would take as long time to write a spec to validate its input&output as to write the actual function itself#2016-06-3019:06eggsyntaxMatter of taste. Personally I don't shoot for complete test coverage, just coverage of fns that a) are tricky, b) are at risk for regression bugs, c) could use a test as usage documentation.#2016-06-3019:07eggsyntaxAnd yeah, it's definitely arguable that hand-written tests are a better tool to reach for in some cases than spec is. It'll be interesting to watch as different ideas about best practices for spec emerge.#2016-06-3019:08eggsyntax(although the new stub/mock functionality makes it a tool I'm likely to reach for a lot more often)#2016-06-3019:10blancethat make sense. and I would assume you only need to use clojure.spec.test/test for unit testing? Do you still hand write test to cover corner cases?#2016-06-3019:10blancewhat's the stub/mock functionality? is that also in spec?#2016-06-3019:11blancesorry I have lots of questions as I'm new to testing as well:sweat_smile:#2016-06-3019:11eggsyntax:stub replaces a fn with a stub that checks :args, then uses the :ret spec to generate a return value. https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#2016-06-3019:12eggsyntaxTo me it seems like generative testing is, in general, more likely to catch corner cases than hand-written tests, unless there's a specific corner case you want to test for that isn't likely to be generated.#2016-06-3019:13eggsyntax(oh, :stub is an optional arg to instrument btw)#2016-06-3019:13blancethat's cool! didn't see that in the doc#2016-06-3019:13eggsyntaxIt's new as of alpha8. I think the guide isn't quite caught up yet.#2016-06-3019:14blanceIll take a look at it. Thanks!#2016-06-3019:16bfabry@fenton: there's no way and could combine the generators. there's a new function s/merge specifically for that scenario though#2016-06-3019:16fenton@bfabry: okay, will check that.#2016-06-3019:29seancorfield@blance: If you specify "every" function then your system will be brittle: changing anything will likely break specs and you’ll have to constantly rewrite them — it’ll be hard to do refactoring. I think specs make sense on "module" boundaries and on "library APIs".#2016-06-3019:31seancorfieldAs for generative testing, you can test a function against a spec (in the test suite) without needing to fdef it in the main code. That seems like a reasonable approach to me.#2016-06-3019:31fentonhow to spec for a string that should be an alpha-numeric?#2016-06-3019:31seancorfieldUse a predicate with a string regex?#2016-06-3019:33fenton@seancorfield: sorry not understand, example?#2016-06-3019:34seancorfield#(re-find #"^[a-zA-Z0-9]+$" %)#2016-06-3019:34seancorfield(there’s sure to be a shortcut for that pattern#2016-06-3019:35seancorfieldSo you’d have (def ::alphanumeric (s/and string? #(re-find #"…" %)))#2016-06-3019:37fenton@seancorfield: okay that makes sense...thx.#2016-06-3019:37seancorfield
boot.user=> (s/def ::alphanumeric (s/and string? (partial re-find #"^\w+$")))
:boot.user/alphanumeric
boot.user=> (s/exercise ::alphanumeric)
(["Q" "Q"] ["7" "7"] ["4" "4"] ["6z6" "6z6"] ["RjN" "RjN"] ["r05" "r05"] ["9" "9"] ["o8m" "o8m"] ["5j7YFcm" "5j7YFcm"] ["RFNe5Xj3" "RFNe5Xj3"])
#2016-06-3019:39fenton@seancorfield: even nicer!#2016-06-3019:39eggsyntaxIt'd be nice to do something like (s/def ::alphanumeric (s/and string? #(Integer/parseInt %))), but that'll throw an exception rather than failing...#2016-06-3019:40blance@seancorfield: I'm actually working on clojurescipt. most of my fns are api calls, transition db to new states, and UI stuffs. No need to write spec for any of them I guess?#2016-06-3019:41seancorfield@eggsyntax: you would need both a predicate to match numeric strings and a conformer to convert to Long I think?#2016-06-3019:41bhaumanI looked into the :in path problems for maps that I mentioned above, it seems like ::kfn isn't enough to eliminate the following tuple key.#2016-06-3019:41eggsyntax@seancorfield: makes sense.#2016-06-3019:41eggsyntax(& at that point the regex solution is nicer)#2016-06-3019:41seancorfield@blance: Hard to say in absolute terms. Try a few approaches and see what works best for you?#2016-06-3019:42blancesounds good. Thanks!#2016-06-3019:42seancorfieldI’d probably spec the API itself (between client and server) and then maybe some of the bigger components within each side. But I suspect you have a lot of "trivial" functions which wouldn’t be worth spec’ing?#2016-06-3019:43eggsyntaxNew motto: "Don't spec the small stuff"#2016-06-3019:43eggsyntaxšŸ˜‰#2016-06-3019:43blanceapi calls are async, or rather returning a core.async channel#2016-06-3019:43blanceso not sure how to spec that#2016-06-3019:47bhaumanits expressive enough for me to fix it in use though#2016-06-3019:48bfabryI figure fdefs probably live at a similar level to unit tests. if you unit test every function in your application it'll be annoying to change, but unit testing still holds value at some "right" level. same with fdefs#2016-06-3019:48bfabryfdefs have the added advantage that they also give you runtime contract checking and easier to understand documentation than unit tests#2016-06-3019:49puzzler@alexmiller I read the doc-string about multi-spec but (sg/generate (s/gen my-multi-spec)) throws ExceptionInfo Unable to construct gen at: [] for: so I must be misunderstanding the point about generation.#2016-06-3020:17fenton@puzzler: did u see the example i made for u?#2016-06-3020:29puzzler@fenton no, didn't see that. Where?#2016-06-3020:29puzzler@fenton, ah, I see it now.#2016-06-3020:32puzzler@fenton, hmm, not sure why it isn't working for me. Maybe one of the individual multimethods can't generate but the error message is misleading, making it seem like the overall multispec is failing.#2016-06-3020:32fenton@puzzler: did my example work for you?#2016-06-3020:38puzzler@fenton: no, I copied and pasted your code and tried it at the REPL but I'm getting NullPointerException clojure.test.check.generators/call-gen (generators.cljc:41) This is on the latest alpha. Not sure what's going on.#2016-06-3020:41puzzler@fenton, oh I called gen and generate in the wrong order on your code. So yours is now working for me. But I had it right on my example which wasn't working. So still trying to figure that out...#2016-06-3020:43puzzler@fenton, ok figured it out. I wasn't calling gen on the keyword associated with the multispec, I was calling it on the actual multispec.#2016-06-3020:43puzzler@fenton confusing error messages#2016-06-3020:54puzzlerIn some cases the docstring info provided automatically by spec isn't terribly useful, especially in the case of multi-specs. Is there any way to manually add info to the spec docstring?#2016-06-3021:19fenton@puzzler glad u got it work....dunno about ur last comment tho. šŸ™‚#2016-06-3023:20blanceWould you put all spec defination at one place? say my.appns.spec? If so, any trick to refer to a namespaced keyword?#2016-06-3023:21blanceI assume nobody want to type {:my.appname.is.very.long/key1 "foo" :my.appname.is.very.long/key2 "bar"} `#2016-06-3023:25blanceAlso i'm wondering if clojure.spec can spec arbitrary nested vector? for example, geoJSON coordinates can be [1 1] or
[
      [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ],
      [ [100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2] ]
      ]
or [ [100.0, 0.0], [101.0, 1.0] ]
#2016-06-3023:27Alex Miller (Clojure team)You have optionality and recursion, so without knowing more than those examples, yes#2016-06-3023:28Alex Miller (Clojure team)@blance if you are making a spec namespace, I would propose the plural (specs) rather than spec. We will be doing clojure.core.specs for example.#2016-06-3023:29Alex Miller (Clojure team)For namespaces, you can alias with :as in require then autoresolve with :: keywords#2016-06-3023:31blancelike my.appname.specs :as specs then ::specs/key1 right?#2016-06-3023:31Alex Miller (Clojure team)Yeah#2016-06-3023:32blancecool thanks!#2016-06-3023:41blanceOn a side note, I feel like clojure team does not want us to put all specs at one place, otherwise we would have the option to just describe un-namespaced key which is more convenient.#2016-07-0103:30seancorfieldI’m working with instants and ended up with this code:
(defn years-ago
  "Given a number of years, return an Instant that long ago."
  [y]
  (-> (LocalDate/now) (.minusYears y) .atStartOfDay (.atOffset ZoneOffset/UTC) .toInstant))

(defn age-18-120?
  "Given an Instant, return true if it represents a date of birth such
  that someone would be between 18 and 120 years old."
  [i]
  (s/inst-in-range? (years-ago 120) (years-ago 18) i))

(s/def ::date-of-birth (s/with-gen age-18-120?
                         (fn [] (s/gen (s/inst-in (years-ago 120) (years-ago 18))))))
Could this be done more cleanly? (without the duplication of the range of years)
#2016-07-0104:24bfabry@seancorfield:
(s/def ::age-18-120? (s/inst-in (years-ago 120) (years-ago 18)))
=> :kafka-google-connector.runner/age-18-120?
(defn age-18-120? [x] (s/valid? ::age-18-120? x))
=> #'kafka-google-connector.runner/age-18-120?
(s/def ::date-of-birth ::age-18-120?)
=> :kafka-google-connector.runner/date-of-birth
(s/exercise ::date-of-birth 1)
=> ([#inst"1970-01-01T00:00:00.000-00:00" #inst"1970-01-01T00:00:00.000-00:00"])
#2016-07-0104:26bfabrythough the age-18-120? fn seems totally redundant at that point, but if you still wanted it#2016-07-0104:28seancorfieldNo, because the years-ago calls must happen when the predicate is applied, not just once at compile time.#2016-07-0104:28bfabryriiiiight, makes sense#2016-07-0104:34seancorfieldAnd I'm assuming the with-gen generator-returning fn is called each whenever the spec is exercised / test-gen'd but that's not really as important since tests are short-lived, whereas the spec has to be long-lived and check the correct range of dob each time it's conformed / used for validation.#2016-07-0109:58Oliver George@seancorfield: could "now" be part of the data structure you spec/validate? that makes it a simple input rather than implied context.#2016-07-0111:13Oliver GeorgeWill it make sense to update defn to allow a spec to be provided as metadata? seems like it could overlap with pre/post...
(defn option-match
  "Default search for local datasource: case-insensitive substring match"
  [simple? option query]
  {:args (s/cat :simple? boolean? :option ::option :query string?)}
#2016-07-0111:58Alex Miller (Clojure team)No, we won't be doing that#2016-07-0113:09seantempestaSo, specs can also be used to convert invalid data into valid data? I’m reading this…
conformer
macro
Usage: (conformer f)
       (conformer f unf)
takes a predicate function with the semantics of conform i.e. it should return either a
(possibly converted) value or :clojure.spec/invalid, and returns a
spec that uses it as a predicate/conformer. Optionally takes a
second fn that does unform of result of first
So how do you use the feature to get the response (possibly converted)?
#2016-07-0113:11bhauman@seantempesta: you have to provide a function that returns :invalid or a result that is the converted value#2016-07-0113:12bhauman(conformer (fn [a] (if (= a "one") 1 :clojure.spec/invalid)))#2016-07-0113:20seantempestaholy crap, this is great!#2016-07-0113:20seantempestathanks @bhauman!#2016-07-0114:10ghadiwould love to see this answered @alexmiller https://clojurians.slack.com/files/glv/F1MQPJBQT/Specifying_an____annotated_value_map_.md#2016-07-0115:33wilkerlucio@ghadi: one simple option that I see is to move the cached data to the map meta-data, this way the map validation can stay simple#2016-07-0115:36glvBut I’d like to validate the cached data as well. For purposes of the code, it’s not valid if those keys aren’t there.#2016-07-0115:36glv(That was clumsily worded, but you know what I mean … the code expects those keys.)#2016-07-0115:44wilkerlucio@glv: I had a different idea, what you think on this:#2016-07-0115:44wilkerluciohere, I used a conformer to remove the namespaced keys before doing the map validation, but checking the keys before doing it#2016-07-0115:44wilkerluciothis way all namespaced keys will be validated, then removed just to check the rest of the map#2016-07-0115:53glvHmm … that’s better than my current horrific solution. šŸ™‚#2016-07-0116:30seancorfield@olivergeorge: Interesting idea… The data structure comes from the database and we already decorate it with some computed fields that are transient (short-lived) so adding a current timestamp to the data wouldn’t be a hardship.#2016-07-0116:56Alex Miller (Clojure team)@ghadi: sorry, moving today!#2016-07-0116:56jjcomerI’m trying to use spec-test/test in my clojure.test tests (that was a lot of test :)). When I try to realize the result of test I get the following exception
java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
#2016-07-0116:56ghadi@alexmiller: good luck!#2016-07-0116:56jjcomerIf I run the test outside of a deftest it works no problem#2016-07-0116:57jjcomerDoes anyone have a strategy they are using to use spec-test/test within a deftest?#2016-07-0117:18glv@alexmiller no worries … whenever things calm down is fine.#2016-07-0119:14akielWhat is the idiomatic way to spec an empty :args seq? I currently use (s/cat).#2016-07-0212:03leongrapenthinSeems like the "annotated value map" is also asked for here: https://stackoverflow.com/questions/38151446/how-can-i-spec-a-hybrid-map#2016-07-0314:26Alex Miller (Clojure team)This is explained in the generators with s/and section in http://clojure.org/guides/spec #2016-07-0314:27Alex Miller (Clojure team)You should probably also look at s/int-in for int ranges with gen support#2016-07-0321:50bhaumanI'm curious if it's an eventual goal to make explain's :in path work with core functions like get-in? Right now the spec'ing of maps as a sequential collection via (every (tuple k? v?)) is causing some ambiguity in the :in path.#2016-07-0408:16dhruv@alexmiller: thank you very much! That explanation clears it up for me, plus s/int-in works a treat for my purposes :+1::skin-tone-3:#2016-07-0413:51eggsyntaxIt'd be useful to have an undef fn or something like that, to clear a spec from the registry. Or some way to clear the registry entirely. Mostly just for use during development. Or, for that matter, just make the registry-ref atom publicly accessible.#2016-07-0414:03rauh@eggsyntax: Hack: (reset! @#'spec/registry-ref {})#2016-07-0414:04eggsyntax@rauh: Oooh, slick. Thanks!#2016-07-0414:07eggsyntaxI didn't realize that'd get around the private declaration. Good to know in general.#2016-07-0414:32Alex Miller (Clojure team)@eggsyntax we’ve talked about this and I expect there will be something public like this#2016-07-0414:32Alex Miller (Clojure team)eventually :)#2016-07-0414:32eggsyntaxSounds good. & Andre's hack will do me fine for now šŸ™‚#2016-07-0501:50wildermuthnReading ā€œOut of the Tarpitā€ (http://shaffner.us/cs/papers/tarpit.pdf), and maybe I’ve just got Spec on my mind, but this looks pretty familiar:#2016-07-0501:50wildermuthnhttps://www.dropbox.com/s/fjh9fi3r0e4ldt2/Screenshot%202016-07-04%2021.50.02.png?dl=0#2016-07-0502:04wildermuthnThe article’s description of "Functional Relational Programmingā€ includes a section on ā€œEssential Logicā€ that seems to correspond very well with Spec!#2016-07-0509:58Alex Miller (Clojure team)I think it's no accident that Clojure+Datomic overlap pretty well with Out of the Tarpit#2016-07-0513:06Alex Miller (Clojure team)s/and flows conformed values through the predicates and the s/+ conforms to a vector#2016-07-0513:07Alex Miller (Clojure team)Rich has mentioned a split into s/and and s/and-> to have both choices#2016-07-0514:22vikeri@alexmiller: Thanks! Yes it could probably be useful sometimes to not have it flow through the predicates but evaluate them separately.#2016-07-0515:01Alex Miller (Clojure team)separately, the need to validate this specific case of a regex and vector? is common and one I’ve run into many times in spec’ing macros and there might be some addition specifically for this#2016-07-0520:27aengelbergHas there been any talk about allowing local registries to use with clojure spec functions rather than only using the global registry? Similar to (make-hierarchy) and derive.#2016-07-0520:35Alex Miller (Clojure team)I haven't heard such talk :)#2016-07-0520:51pdlugI’m trying to use multi-spec but it doesn’t seem to work with defmethod’s :default dispatch, is this a bug or intentional?#2016-07-0520:55kidpolloI don’t think multi-spec behaves exactly like that. I believe there is no default to be had.#2016-07-0521:00pdlugThat’s unfortunate. Is there a better way to handle the validation where a field in the map determines the type and you want to explicitly constrain some types but provide a default base?#2016-07-0521:01pdlugEx: modeling analytics events, the :type provides the type of event, for :search events :terms are required, for :page-view events :url is required but for all other events there are no other constraints on the map#2016-07-0521:02kidpolloIt seems it only dispatches on key defined in s/multi-spec not on the this#2016-07-0521:05kidpollohmmm but I just tried (s/def :foo/bad-entity (s/multi-spec entity :does-not-exist)) (gen/sample (s/gen :foo/bad-entity) 1) and the program goes into an infinite loop#2016-07-0521:06kidpollowait! not infinite loop I just get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.#2016-07-0521:07kidpollomakes sense it either matches a spec or it does not. I am back to my original argument šŸ˜›#2016-07-0521:08pdlugI think it’s unexpected behavior if multi-spec works w/ multimethods but then doesn’t support the default behavior of them#2016-07-0521:08kidpollo@pdlug: yah I had the same issue I want to dispatch on type but it has to be based on a key on your map šŸ˜ž#2016-07-0521:08pdlugI still don’t see anything else that supports my example use case#2016-07-0521:11kidpolloyeah, my problem is that I want to dispatch on something else than a namespaced key on the map. I want to dispatch on meta from the map. Not always you want to expose the ā€œtypeā€ on the actual entity#2016-07-0521:13pdlugmakes sense#2016-07-0521:14pdlugah, I found this open issue: http://dev.clojure.org/jira/browse/CLJ-1935#2016-07-0521:16kidpolloright!#2016-07-0521:47Alex Miller (Clojure team)@pdlug you might also see if s/merge helps you in combining base+specific s/keys#2016-07-0522:03kidpolloYeah but you would be able to have a generic event spec for all the other types.#2016-07-0612:23ikitommiIs there a easy way to select a conformer based on external input? E.g. for the web - conform strings to numbers only for query parameters, not for json params.#2016-07-0614:51pdlug@alexmiller: I’m not familiar with the clojure bug process, that issue CLJ-1935 says the status is ā€œTriagedā€ what does that mean exactly? Is there a way to tell if it will be addressed or slated for a release?#2016-07-0614:57artemyarulinhi, what is the right way to enable spec checks globally for my project? I would like that by default everything got checked. I guess I can put something like that in project file:
:repl-options {:init (do (clojure.spec/instrument))}
and then everything should be checked when I’m in a REPL? Oh what’s the right way?
#2016-07-0614:58petterik@pdlug: You can find the entire JIRA workflow here "How a Ticket Becomes a Commit": http://dev.clojure.org/display/community/JIRA+workflow I found it very useful#2016-07-0614:59pdlug@petterik: ah that’s very helpful. Thanks!#2016-07-0615:53Alex Miller (Clojure team)@pdlug in this case, it means ā€œI think Rich should look at itā€ but he has not yet#2016-07-0616:23pdlug@alexmiller: ok, thanks#2016-07-0701:50seancorfieldI thought alias had been updated so you didn’t have to have a real namespace for the second argument?#2016-07-0701:52seancorfieldIt seems not. I was hoping to use alias to shorten a lot of qualified keywords (without having to actually load a namespace).#2016-07-0702:13Alex Miller (Clojure team)no, we’re just considering that#2016-07-0702:49seancorfieldNow we’re beginning to use namespace-qualified keywords a lot more, I can say that would be a very convenient addition! šŸ™‚#2016-07-0703:18Alex Miller (Clojure team)as a short-term workaround you can do something like this: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L17-L19#2016-07-0703:37seancorfieldThat’s an interesting hack! šŸ™‚#2016-07-0706:07mandragoranI was wondering how I would spec an async function (a function returning a core.async channel). Would I need to write an async version of fdef or is there another way?#2016-07-0708:19wagjoI'm a bit struggling to choose between :foo.vocabulary/term-uri and :foo.vocabulary.term/uri or just :term/uri. I know it's probably a bit soon, but any best practices for choosing the name(space)s?#2016-07-0709:14mandragoran@wagjo: I've been taking advantage of the new ::alias/some-keyword feature. So I'd probably call it ::term/uri and map the term alias in all namespaces using it to foo.vocabulary.term.#2016-07-0710:13vikeriHow can I reference the first argument of a function in fdef :arg when the argument does not have a name since I’m destructuring it immediately with {:keys [a b]}?#2016-07-0710:48vikerinvm, realized the keywords in :arg (s/cat are not tied to the variable names#2016-07-0711:07Alex Miller (Clojure team)@wagjo if you're in a lib or other public project, you should add enough context to be distinguishable from anyone else (something like your maven group/artifact id). If in a private app, I would probably do less just for typing sake. I think it will be useful to put specs for the domain in a small number of namespaces (maybe 1). It might be useful to separate generic specs from domain attributes (which alias them) too.#2016-07-0711:10Alex Miller (Clojure team)@mandragoran: we will probably have some async related specs eventually. Until we do I think it's better to do type checks on ReadPort and WritePort than specifically on ManyToManyChannel as those are more generic#2016-07-0712:08gfredericksad-hoc higher-order specs?#2016-07-0712:45lvhHow do I write a spec for a map of unqualified keywords where the keyword itself is something dumb that’s going to collide a lot? In particular, swagger (the api format) has a toplevel key, also called ā€œswaggerā€.#2016-07-0713:08Alex Miller (Clojure team)can you give an example? not getting the question.#2016-07-0714:06lvhSure! Let’s say I’m parsing swagger#2016-07-0714:06lvh
(def sample-swagger
  {:swagger "2.0"
   :info {:title "my sample api for puredanger"
          :version "elventy flive"}})
#2016-07-0714:07lvhLet’s say I’m in a ns defining swagger in general (doesn’t matter much, could also be :swagger/whatever). I’m doing something like:
(spec/def ::swagger
  (spec/keys :req-un [:swagger/swagger
                      ::info]))

(spec/def :swagger/swagger
  ;; The top-level key is called swagger. This feels dumb.
  #{"2.0"})

(spec/def ::info
  (spec/keys :req-un [::api-title
                      ::api-version]))

(spec/def ::api-title string?)
(spec/def ::api-version string?)
#2016-07-0714:08lvhIt seems like I’m naming :swagger/swagger what it is because :req-un expects to find the names of map keys that way, but having a ::swagger and a :swagger/swagger looks confusing#2016-07-0714:09lvhSimilarly, ::api-title actually needs to be : or something, but the tons of ns’es make it annoying to type#2016-07-0714:09lvhI guess I’m asking if there’s a way to distinguish between the key under which something appears in a map, and its spec name#2016-07-0714:10lvhIIUC there are some new reader macros in the current alpha, I don’t know if they help with this#2016-07-0714:10lvhthey might limit the typing to just the spec defs so at least the destructuring looks good, I suppose?#2016-07-0714:12lvhif I had a choice, in this code, I would refer to that specced value (esp. after conform) as ::api-title — the only reason I wouldn’t do that is because the JSON I’m getting from swagger (a standard, so I can’t change it) uses keys that don’t collaborate well with that#2016-07-0714:13lvhIf I could provide, say, an alias, or say that under key :title I’m expecting to see spec ::api-title or something, that’d help too#2016-07-0714:16lvhbailing out and using a fn to just assert that some keys are some particular value doesn’t feel very idiomatic either#2016-07-0714:33lvhI guess you can define specs in terms of other specs? That doesn’t seem to help a lot but I’ll play with it#2016-07-0715:40donaldballHas anyone run across or is writing a lib of commonly useful strong specs, e.g. for urls, hostnames, unix ports, valid sql identifiers, etc.?#2016-07-0715:52Alex Miller (Clojure team)I haven't seen anyone doing so yet but it seems like a reasonable and perhaps inevitable thing to do #2016-07-0716:40Alex Miller (Clojure team)@lvh aliasing is a good and useful thing, so creating api-title and api-version are just fine as base specs (you may want to use some other namespace though like :swagger/api-title or :swagger.api/title), then s/def code-localized versions to match the unqualified attributes that alias those base specs, like (s/def ::title :swagger/api-title)#2016-07-0716:41lvhOK, so I define the ā€œrealā€ spec under :swagger.ap/title, and then a local alias. Makes sense. Thanks!#2016-07-0716:42Alex Miller (Clojure team)if you create the keyword namespace as an actual namespace, you can alias it, then use the alias in the autoresolved keyword like ::sw/api-title (after something like (alias ā€˜sw ā€˜swagger.api))#2016-07-0716:44Alex Miller (Clojure team)the key there is that alias currently has to refer to a real (loaded) namespace. That might change still but a workaround is what you see here https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L17-L19#2016-07-0716:44Alex Miller (Clojure team)switch to (and thus create) the namespace, then switch back, and alias#2016-07-0717:32leongrapenthinToday we wanted to create a spec that validates a percentage in five percent steps. It occured to us that int-in doesn't support a step argument like range does. Has this been discussed or is a ticket worth a shot?#2016-07-0717:39sparkofreasonBefore I get too far in reinventing the wheel, is anyone else out there working on converting spec to Datomic/Datascript schema?#2016-07-0717:48Alex Miller (Clojure team)@leongrapenthin: no, it's not going to do that#2016-07-0717:49leongrapenthin@dave.dixon: I have thought about it and realized that generating a spec from Schema is likely more desirable#2016-07-0717:50donaldballI’m not sure what I’m doing wrong with this simple spec:#2016-07-0717:50donaldball
(s/def ::foo (s/spec #(= :foo %) :gen (gen/return :foo)))
:user/foo
user> (s/gen (s/get-spec ::foo))
ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn  clojure.spec/spec-impl/reify--13361 (spec.clj:800)
#2016-07-0717:50leongrapenthin@alexmiller: Ok, thanks#2016-07-0717:50Alex Miller (Clojure team)@leongrapenthin: just do (set (range 0 100 5))#2016-07-0717:51donaldballOops, I misread the contract for s/spec, carry on#2016-07-0717:51leongrapenthin@donaldball: Wrap (gen/return ...) in a (fn [] )#2016-07-0717:51Alex Miller (Clojure team)yep, or put # at the beginning#2016-07-0717:52Alex Miller (Clojure team)
(s/def ::foo (s/spec #(= :foo %) :gen #(gen/return :foo)))
(gen/sample (s/gen ::foo))
#2016-07-0717:55leongrapenthin@alexmiller: Yeah, but that's gonna be quite the error message šŸ™‚ - We came up with (s/and (s/int-in 0 101) #(zero? (mod % 5))) if I remember correctly. Generator worked. Projecting this scenario on per permille or larger spaces was what made us wonder why a step is not supported.#2016-07-0717:56Alex Miller (Clojure team)because it’s a min/max check, and you’re specifying enumerated values. that’s just not what it is intended to do.#2016-07-0717:56Alex Miller (Clojure team)I think your spec is better anyways#2016-07-0717:57Alex Miller (Clojure team)you could combine int-in with it too#2016-07-0717:57Alex Miller (Clojure team)nvm, you’re doing that#2016-07-0717:58leongrapenthinYeah, the reasoning makes sense#2016-07-0717:59Alex Miller (Clojure team)the hard part of int-in is generating uniformly across a well defined set and you’re leveraging that by building on top of it#2016-07-0718:00leongrapenthinLooking at that from a different perspective it is only enumerated because of the min-max check#2016-07-0718:00leongrapenthinYou could argue that everything in a min-max interval is enumerated then#2016-07-0718:01Alex Miller (Clojure team)well enumerations have obvious usage issues when there are a lot of values :)#2016-07-0718:04leongrapenthinMy case is a very specific enumeration and can't be discarded as just any enumeration. Every step range of allowed values is a subset of an integer range with the same boundaries.#2016-07-0718:05leongrapenthinSo from that perspective I think it still makes sense as a feature of a min/max check.#2016-07-0718:05leongrapenthinThe min/max check is also an enumeration with a step of 1, then.#2016-07-0718:14leongrapenthinBetter said, for our usecase it would be preferable if it was an in-range check than a min/max check with the word range meant in the same way as in clojure.core/range#2016-07-0718:17Alex Miller (Clojure team)well, range does a lot of stuff (reverse ranges, infinite ranges, half-open ranges, support for all number types, etc)#2016-07-0718:18Alex Miller (Clojure team)in-range implies to me a check that would need to enumerate the range to determine conformance and gen would be a lot harder.#2016-07-0718:18Alex Miller (Clojure team)int-in is intentionally much narrower#2016-07-0718:25leongrapenthinAgreed. I think what I want to say is that int-in as a name doesn't necessarily limit the spec to a min/max check#2016-07-0718:27leongrapenthinA fully featured in-range is likely an overkill for the rare amount of usecases#2016-07-0718:44leongrapenthinThanks for discussing. Having thought about it I won't be able to resist to submit a small patch over the weekend for reconsideration, though šŸ™‚#2016-07-0718:46Alex Miller (Clojure team)I think Rich is unlikely to ok that#2016-07-0718:48kidpolloI am running stest/check on an fdef that never finishes. It just keeps going and I don't know If its stuck or what is the deal. I have independently tested the input and output specs for possible problems and am getting nowhere. Any tips on how to debug this?#2016-07-0718:58kidpolloexercise-fn seems to work fine#2016-07-0719:02leongrapenthin@kidpollo: Isolating the problem by starting with dumbed down specs and adding until the problem occurs?#2016-07-0719:02kidpolloI also tried (s/exercise-fn my-fn 100) and it takes like 1 min but it works.#2016-07-0719:03leongrapenthin@kidpollo: Probably recursion in s/map-of or every-kv?#2016-07-0719:04leongrapenthinMight be that it just takes so long to print the result because it prints a huge generated value#2016-07-0719:05kidpolloCould be.#2016-07-0719:05kidpolloI am running on the repl#2016-07-0719:05kidpolloon intellij#2016-07-0719:06leongrapenthinYou could eliminate this quickly with setting print-length#2016-07-0719:07Alex Miller (Clojure team)dump stack and see what it’s doing#2016-07-0719:07Alex Miller (Clojure team)I think intellij has a button for it#2016-07-0719:08kidpolloohh let me see#2016-07-0719:10Alex Miller (Clojure team)I guess maybe that’s only in the debugger mode#2016-07-0719:14Alex Miller (Clojure team)
(map println (.dumpAllThreads (java.lang.management.ManagementFactory/getThreadMXBean) false false))
#2016-07-0719:14Alex Miller (Clojure team)will serve in a pinch#2016-07-0719:16kidpolloyah I was able to dump them in debug mode#2016-07-0719:16kidpollojust see a bunch of locked threads in generators#2016-07-0719:18kidpollodamn 700% cpu usage
#2016-07-0719:21kidpolloI guess i need to find what spec is going crazy. Start with simpler specs maybe? The input and output to this fn are multispecs. The maps are not so complicated maps.#2016-07-0719:47kidpollo@alexmiller @leongrapenthin the problem seems to be happening in alpha-9. I went back to alpha 8 and the test finishes. It takes a while but it finishes#2016-07-0719:57Alex Miller (Clojure team)I think check changed from 100 to 1000 iterations in alpha9#2016-07-0720:20kidpollooh my that would explain it#2016-07-0720:20kidpollowhen it produced errors it did end earlier#2016-07-0720:21kidpolloI guess Ill leave it over night to see how long it actually takes#2016-07-0720:40Alex Miller (Clojure team)That count is an option you can pass to check though iirc#2016-07-0721:12kidpollowhat is iirc?#2016-07-0721:13jr(if I recall correctly)#2016-07-0721:17kidpollonice! snme in spanish šŸ˜› (si no me equivoco)#2016-07-0721:18kidpollolol ^ totally does not exist in spanish šŸ˜›#2016-07-0721:26kidpolloIt seems it would look like:#2016-07-0721:28kidpolloIs it encouraged that all clojure projects start using fully qualified keywords?#2016-07-0721:47Alex Miller (Clojure team)Nothing categorical. Qualified keywords have utility - if they make sense for you, use them#2016-07-0721:48Alex Miller (Clojure team)I think spec and the other syntax changes have increased their utility and decreased their verbosity, maybe changing that equation#2016-07-0721:52danielcomptonAnd their visibility šŸ™‚#2016-07-0721:52Alex Miller (Clojure team)Well I don't think that changes their utility :)#2016-07-0721:53seancorfieldI can see us (World Singles) using qualified keywords very heavily as it will make some of our code much more explicit. Hence the recent changes to java.jdbc to make it easier to get qualified keywords back in result sets (the new :qualifier option to everything that produces result sets).#2016-07-0722:43ajssIn spec/fdef, is there any way to make one element of :args depend on another? I've got a function that takes two maps, but wants the keys of each map to be the same.#2016-07-0722:55kidpolloyou can define the relationship between inputs and outputs in :fn#2016-07-0722:55glvRight, but not constrain inputs based on relationships.#2016-07-0722:56glv(For example, I have functions that take a grid and a coordinate within that grid, and obviously the coordinate must be within the bounds of the grid to be valid. But I don’t think there’s a way of expressing that.)#2016-07-0722:58kidpolloSounds similar to the ranger-rand example in http://clojure.org/guides/spec#_spec_ing_functions#2016-07-0722:59kidpolloinstead of using :ret just use named args in :args#2016-07-0723:01ajssso (s/and (s/cat :a map? šŸ˜› map?) #(= (keys (:a %)) (keys (:b %))))?#2016-07-0723:02ajssah - ": b" = šŸ˜›#2016-07-0723:03ajsswill run out of generator iterations won't it?#2016-07-0723:23kidpollowell you would need to replace map? with a more "narrowed down" spec#2016-07-0723:28ajssah, yes that would work. Thanks šŸ™‚#2016-07-0723:31Alex Miller (Clojure team)@glv you can s/and an arbitrary predicate in the :args spec#2016-07-0723:33Alex Miller (Clojure team)and flows conformed values so it should already be nicely destructured if you add it last#2016-07-0723:35glvAh, gotcha. I’ll try to incorporate that into my specs when I get a chance.#2016-07-0723:38glv(Also, a reminder about https://clojurians.slack.com/files/glv/F1MQPJBQT/Specifying_an____annotated_value_map_.md (and @wilkerlucio’s solution https://clojurians.slack.com/archives/clojure-spec/p1467387857001732)#2016-07-0800:49donaldballI started a library with some ostensibly generally useful specs, e.g.: https://github.com/SparkFund/useful-specs/blob/master/src/specs/internet.clj#2016-07-0800:49donaldballWould very much appreciate feedback on design, etc. It’s my second day using spec in anger, as it were.#2016-07-0801:25Oliver Georgeit'd be nice if there was a way to modify how instrument reports. the default spec-checking-fn can be super verbose when turning values to str for the ex-info message.#2016-07-0801:26Oliver GeorgeHere's my quick workaround. Not perfect but gives me something nicer to investigate: https://gist.github.com/olivergeorge/c4a26886b7bbd05f2e9018614a2213ff#2016-07-0801:26Oliver GeorgeI rely on cljs-devtools to make the console.debug output to be expadable#2016-07-0802:07Alex Miller (Clojure team)@glv I have much to report re the ā€œhybrid mapā€ stuff, but not quite ready yet to do so#2016-07-0802:08Alex Miller (Clojure team)have been working on it much of yesterday and today#2016-07-0802:11Alex Miller (Clojure team)@olivergeorge: you have explain-data ex-data coming back in the exception - why not write a function that customizes output from that, rather than hacking spec-checking-fn?#2016-07-0802:13glvThat tells me that it's a legit problem that y'all are taking seriously and working on, so I'm cool with that. :-)#2016-07-0802:16bbloom@glv: There some considerable advantages to just using another map. I’m curious what @alexmiller is working on, but still: you should just add a :grid key or similar#2016-07-0802:17bbloomwhat if you want to be able to represent a board without a starting coordinate?#2016-07-0802:17bbloomyou can easily take a grid and merge in some extra options, but how easy is it to dissoc them?#2016-07-0802:18bbloomwhat if you want to change the representation? now you’ve coupled your param spec with your grid spec#2016-07-0802:18bbloomwhat advantage is there to abusing the fact that keywords and grid coordinates are disjoint? slightly shorter syntax? seems not worth the complexity#2016-07-0802:18glvThis isn't the representation of the grid; it's the result of a particular kind of analysis of the grid, which always, inherently has a starting coord.#2016-07-0802:19bbloomcomposite structures come in two flavors: homogenous and heterogenous#2016-07-0802:19bbloomof course if you have disjoint key sets, you can combine them#2016-07-0802:19bbloombut it’s just clearer to keep them separate#2016-07-0802:20bbloomalso when combined, you can’t enumerate keys without filtering extra keys#2016-07-0802:20glvI'm certainly willing to entertain arguments that I should push the coord->distance mapping down a level.#2016-07-0802:20bbloomi guess i’m making such arguments šŸ˜›#2016-07-0802:21bbloombut more importantly: what advantage is there to flattening it?#2016-07-0802:21glvSo far, I don't find them convincing. :-)#2016-07-0802:21glvI see three advantages.#2016-07-0802:21Alex Miller (Clojure team)@glv totally legit - I have a couple different solutions with existing tools and something new for just this problem that may help if it comes together#2016-07-0802:22bbloomi’m not saying hybrid maps are never useful#2016-07-0802:23bbloomi’m saying that it’s pretty rare and barring additional info, i don’t think you’ve found a justified use case šŸ™‚#2016-07-0802:25glv1: the coord->distance mapping is the primary result of the algorithm. The ::start-coord, ::max-coord, and ::max-distance entries are annotations that could be reconstructed (with some cost) if they weren't present. But the only way to make a non-hybrid map would be to leave the secondary annotations at the top level and push the primary info down into a nested map. That doesn't seem right to me.#2016-07-0802:26bbloomyou can return those keys + a ::grid key and then create a helper function which simply extracts the grid key — then the caller can decide what is ā€œprimaryā€ or not#2016-07-0802:26glv2: the code that's involved—both the code that builds the map and that which uses it—would get more complex and clumsy (and slower) that way#2016-07-0802:27bbloomreally? how?#2016-07-0802:29glv3: That's the way it is now, and it has never caused a hint of a problem; the only reason to change would be to accommodate clojure.spec. #2016-07-0802:29Alex Miller (Clojure team)I don’t think it’s that rare/weird#2016-07-0802:29Alex Miller (Clojure team)others (including me) have run into same problem#2016-07-0802:30glvHow? Isn't that obvious? The vast majority of the updates and accesses will be to the coord->distance mapping, so pushing that a level deeper means extra lookups in the vast majority of cases. #2016-07-0802:31bbloomalexmiller: any examples you can share? i don’t think i’ve ever encountered such a hybrid map that didn’t (surprisingly quickly) need a (->> hybrid keys (filter #(… that could just be replaced by (-> hybrid :foo keys)#2016-07-0802:31bbloom@glv: just build the grid and then assoc the extra keys in at the end#2016-07-0802:31Alex Miller (Clojure team)sure, I ran into it with map destructuring (so, a syntax example) - the base map is symbol->any but also special options like :keys, :syms, :or, :as etc#2016-07-0802:32bbloomalexmiller: heh, yeeaaaah i was wondering if you were going to mention macros. destructuring is like the ultimate core macro šŸ˜‰#2016-07-0802:32Alex Miller (Clojure team)it’s fun#2016-07-0802:32bbloom@alexmiller it’s funny b/c syntax is the one place where ā€œjust a little bit shorter syntaxā€ is totally worth it for a common operation#2016-07-0802:32glv@bbloom: even if that were sensible (it's not) it would only help on the creation side of the process, not the use side.#2016-07-0802:34bbloom(:grid (let [start 123, m …build up m here….] {:start start :m m}))#2016-07-0802:34glvIt's not sensible because keeping track of those values through the process (so that you could assoc them in at the end) would be clumsy compared to just tracking them in the map as I go.#2016-07-0802:34bbloomseems pretty sensible to me šŸ™‚#2016-07-0802:35bbloomer i meant: :grid m#2016-07-0802:35glv::start-coord is predefined; the other two emerge out of the process itself.#2016-07-0802:35bbloomgotcha#2016-07-0802:35bbloomi’d use mutation šŸ˜‰#2016-07-0802:37bbloom@alexmiller: one thing i’ve discovered tinkering on language design: it turns out that language designers have to do the things that they tell other people not to do, heh#2016-07-0802:37Alex Miller (Clojure team)that is a thing#2016-07-0802:38glvcf. Stu's recent tweet about why clojure.core can't be idiomatic. #2016-07-0802:40glvhttps://twitter.com/stuarthalloway/status/741860271925432324#2016-07-0802:41bbloomanyway, @glv definitely wait to see what alex et al come up with. the emitting things during the middle is IMO almost a reason to abuse the key space šŸ™‚#2016-07-0802:41glv"abuse" :-)#2016-07-0802:42bbloomi’m not above loaded language šŸ˜›#2016-07-0802:43bbloomthanks for the discussion! gotta run#2016-07-0802:43glvSure!#2016-07-0806:41Oliver George@alexmiller thanks for the suggestion. I would love to do that but don't know how. I presume you are suggesting a big try/catch around my code... my code is typically om/re-frame UI stuff which complicates that.#2016-07-0806:41Oliver GeorgePerhaps I'm missing something obvious#2016-07-0814:01donaldballJust to confirm, we don’t yet have a way to unite the specs for e.g. homogenous (`map-of`) and heterogenous (`keys`) maps?#2016-07-0814:06glvOnly by doing something like this: https://clojurians.slack.com/archives/clojure-spec/p1467387857001732#2016-07-0814:07glvWhich isn't bad at all, but could be made more expressive.#2016-07-0814:14Alex Miller (Clojure team)no, that’s bad :)#2016-07-0814:14Alex Miller (Clojure team)b/c it turns it from a declaration of truth into a process#2016-07-0814:15Alex Miller (Clojure team)there are a couple of ways to do it with existing tools#2016-07-0814:18Alex Miller (Clojure team)one other way is to treat the map as a coll-of s/or of s/tuples (where each tuple describes different kinds of map entries). For cases where the ā€œhybridā€-ness does not include registered attributes, this is probably a good choice. So something like a map that was either string->string or number->number.#2016-07-0814:19Alex Miller (Clojure team)you can handle registered attributes in the same way, but then you lose the goodness of s/keys and the attribute semantics#2016-07-0814:20Alex Miller (Clojure team)so in that case, you want to s/merge an s/keys (for the registered attributes) with something that more accurately states the truth of the map definition#2016-07-0814:20Alex Miller (Clojure team)sorry for not being detailed here - I hope to write a blog with the detail#2016-07-0814:21Alex Miller (Clojure team)and then I’m working on something new that will make this a little more palatable, but it’s not quite done yet#2016-07-0814:23glvWell, compared to my attempt at a solution, it’s not bad. šŸ™‚#2016-07-0817:29eggsyntaxIn 1.9, is there a way to disable namespaced-map output (eg for data that's going to be parsed by a process that's unaware of the idiom)? ie to get output like {:foo/bar 1} rather than #:foo{:bar 1}? [Edit:simplify example]#2016-07-0817:53eggsyntaxAnswer, if anyone's curious: CLJ-1967 will address this. For now ya gotta hack it if ya need it. http://dev.clojure.org/jira/browse/CLJ-1967#2016-07-0818:12donaldballAre double-in and friends macros so that clojure core doesn’t require test check or as a performance optimization?#2016-07-0819:56donaldballexercise-fn is pretty rad, but correct me if I’m wrong, it looks like if the fn in question has an options map specified with s/keys using :opt or :opt-un, the chances of getting a map with those keys is… small#2016-07-0819:57donaldballShould the generator for s/keys bias towards optional keys’ presence?#2016-07-0820:08jrI think they are macros because they build on other macros (like spec)#2016-07-0820:11stuarthalloway@donaldball: double-in is a macro to capture the form for error reporting, IIRC#2016-07-0820:18Alex Miller (Clojure team)in general, all of the spec creators are macros to capture forms#2016-07-0820:20donaldballI’m afraid the implication is not clear to me šŸ˜•#2016-07-0820:20donaldballI just wrote a decimal-in spec creator fn: https://github.com/SparkFund/useful-specs/blob/master/src/specs/number.clj#L6#2016-07-0820:21donaldballWhat am I missing by (lazily) writing it as a fn instead of a macro?#2016-07-0820:26Alex Miller (Clojure team)
(s/explain (s/double-in :min 0.0 :max 5.0) 20.0)
val: 20.0 fails predicate: (<= % 5.0)
=> nil
(s/explain (decimal-in :min 0.0 :max 5.0) 20.0)
nil nil 0.0 5.0
val: 20.0 fails predicate: pred
#2016-07-0820:26Alex Miller (Clojure team)that ^^#2016-07-0820:26Alex Miller (Clojure team)by using let from test.check, you’ve also made test.check a production time dependency for you#2016-07-0820:27Alex Miller (Clojure team)(which is a bummer given how nice let is - I keep wanting it too)#2016-07-0820:29Alex Miller (Clojure team)but you can easily rewrite that with gen/fmap#2016-07-0820:29donaldballYeah, I plan to replace it with fmap … jinx#2016-07-0820:30Alex Miller (Clojure team)double-in as a macro (and spec under it) allows the form of the predicate to be captured as a form as well as evaluated as code#2016-07-0820:31donaldballThanks, that makes good sense#2016-07-0820:31donaldballBtw any opinion on my optional keys generator observation?#2016-07-0820:34Alex Miller (Clojure team)did you maybe not generate enough to tell?#2016-07-0820:35Alex Miller (Clojure team)
(gen/sample (s/gen (s/keys :req [::a ::b] :opt [::c ::d])))
=>
(#:spec.examples.guide
 {:a 0, :b 0}
 #:spec.examples.guide
 {:a 0, :b 0}
 #:spec.examples.guide
 {:a 0, :b -2, :d -1, :c -1}
 #:spec.examples.guide
 {:a 0, :b -2, :c -2}
 #:spec.examples.guide
 {:a 3, :b -1}
 #:spec.examples.guide
 {:a 0, :b 3, :c 0}
 #:spec.examples.guide
 {:a 13, :b -29}
 #:spec.examples.guide
 {:a -17, :b -1}
 #:spec.examples.guide
 {:a -12, :b 4, :c 0}
 #:spec.examples.guide
 {:a 5, :b 5, :c -8, :d -9})
#2016-07-0820:35Alex Miller (Clojure team)I see optionals showing up pretty regularly there#2016-07-0820:37donaldballHmm, maybe I’m doing something else wrong. Thanks.#2016-07-0820:40Alex Miller (Clojure team)I’m not discounting the possibility of something wrong either :)#2016-07-0820:40Alex Miller (Clojure team)just that it’s not obvious to me#2016-07-0820:40donaldballHmm, looks like keys* and keys have very different behavior#2016-07-0820:41donaldball
(gen/sample (s/gen (s/keys* :opt-un [::foo])))
(() () (:_.c/*! [[{{} (Kk.B/?Q)}]]) (:AC!e._*0_/V ((({:*g #uuid "99a8b1d6-d3c3-40f7-a7c9-0fd7beaf14e8"})))) (:q?a.*kS.K*q?/?*k ()) (:x8x6?A.!/qU_m? {#uuid "5f81c97f-0e94-4a95-bdae-ff2adaba5378" 5} :T9y17p.!7p/!5a8 () :?x.d95?.*.OVnzN_/?3?J* [] :_-+U.z.F2JXV6.U070/+9a2? [()] :xQb ["<[Ê©" .EM_N 1]) (:FtROj-1 [(19 :q0P:x5F:*T*41_G:H5_G:N-xN)] :?6IS.a+.AE4/vq2* [] :G.G+.d7Z!.w_+.a.C23/? () :?P*M/tj_Zg_ ()) (:XX [()] :F_Rv+.T+*n.Vp+2VR.w/+1tJQTg () :+R+9.r.per24*!9._NNt4*+.+.d5*T!.!9c/I {} :b.mw*bbbXT.*T+*-?g._h!!3_tg.Jvx.I.*++g14/+-S?Znvq () :K.H220.A9?g?h.s71-_+42.!9.-.z9iq4D*/jr83+S {} :x0w5+v.W!l6!s+/*5a? {0 -7}) (:Y!!D9w/J- [] :+!HS_l.XG+._XK*Bw_.UNW?.FPV20-3+_.*MYrk.Z1.pk-_!_/X*v {g._Y?!.QN+9g!/q.zC00Wj4 \space, Bj_t9- J?E**, true s!E-fSL4.R3Oj-A4._!vJ1!!.Vs5iz*n9a.-dM?5!9c.U.?7.*y/?-p+a!Lc, 1.0 :kP+b2Y:D*AA:74:U:-6yKx:phX!9??:I*Is, #uuid "f34904dd-a05a-4247-ad6d-1972baa0f58f" true, false -4.0, -5 false, \þ 5} :y*!G [:*_p?d?:-!:9f9:l0-3-Cw_5:2uT7f:46-_8P!h:C49*!*?gb:QJw:x8D \Ÿ 0.5 1/6] :mH.FH_PUY/Ik!3+z? ([{([]) [[*QN.]]}]) :ez0_.qeTiE_?.?-.d11.DrQDGZXu7.j9+7ou/N3M*T {} :!vdNK.-!!cb/Gv {ZS.K_d.fWR2/f+6.e?S. y4, v.w "|u\b"}) (:j65f.Z3r*.e8V7+6Y.L5_d.l+R8.dTp.L.l0n*9Z.AmW*_W8/c___ ({{} [()]})))
user> (gen/sample (s/gen (s/keys :opt-un [::foo])))
({} {} {} {:foo 1} {:foo 0.75} {:foo 0} {} {:foo -2} {:foo -8} {})
#2016-07-0820:42Alex Miller (Clojure team)oh, there was a bug in keys* gen that Rich fixed in master today#2016-07-0820:42Alex Miller (Clojure team)you said keys :)#2016-07-0820:43donaldballSorry, I just assumed keys* was a pure wrapper macro#2016-07-0820:43Alex Miller (Clojure team)https://github.com/clojure/clojure/commit/1e236448104fc8a0fc51e26eae7cdb7e650b7ae9#2016-07-0820:44Alex Miller (Clojure team)it’s a composition of &, a conformer, and keys#2016-07-0823:15ajssnot sure if others are interested, but I've figured out the nice way to do dependent types for spec/fdef arguments : use with-gen on the :args parameter
(defn sum [a b] (+ a b)) 
(spec/fdef sum {:args (spec/with-gen (spec/tuple int? int?) (constantly (gen/fmap (fn [x] [(* x 2) (* x 3)]) gen/int)))})
#2016-07-0901:23donaldballI’m kinda wishing s/def allowed a docstring. The generated docstring is okay, but I think I’d like to be able to communicate the intent also.#2016-07-0902:01donaldballWeirdly, I had a good test running in an earlier repl with stest/check but now it’s raising#2016-07-0902:01donaldball
Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn
#2016-07-0914:35robert-stuttafordso we have s/merge to make a single s/keys out of two other s/keys. i'm wondering, how do i spec a map that has some base set of req+opt keys, but also has an optional set of keys that must appear together as if speced with s/keys :req ... ?#2016-07-0919:38arohnerhas anyone seen java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13361#2016-07-0919:38arohnerI’m doing something wrong, but I haven’t found it yet#2016-07-0919:40arohnerhrm, looks like I needed to clean after upgrading clojure#2016-07-0920:50kendall.buchanan@donaldball: I’ve been experiencing the same with check-var#2016-07-1012:17jjcomer@donaldball: I get that error when I execute in a deftest.#2016-07-1015:12donaldballI get it both when executing in a deftest and in cider, though it works fine from a repl#2016-07-1015:16donaldballI was just experimenting with ignoring Rich’s advice and expressing specs for strings as spec regexes. I find I actually quite like a couple of things that fall out: the form is easier to understand that the string regex, and you get a generator for free.#2016-07-1015:17donaldballhttps://gist.github.com/dball/e096fc7fb600fbc53fd94e4b367cf68f#2016-07-1015:19donaldballBut unless I’m missing something, you pretty much have to write one spec for the seq of chars and another spec for the string that unwraps it as a seq#2016-07-1015:20donaldballSetting aside the merits of the idea, is there a way I could do with s/and and a conformed value?#2016-07-1018:25craigyhey all, I'm trying to figure out if I can use spec for parsing strings (sentences) into structured data. I'm struggling with whether or not I will need to split around spaces, and how to handle things like phrases (sequences of words) if I use s/cat. is this kind of use case intended at all, or am I better off with something else?#2016-07-1019:02mishanot-sure-fry#2016-07-1019:07xcthulhuHow do I write my own generator for cljs.spec?#2016-07-1019:07xcthulhuI have data structures like {0 0.0, 1 0.1, 2 0.2, 3 0.3, 4 0.4, 5 1.0}#2016-07-1019:07xcthulhu(maps of integers from 0 to 5, increasing in value as the index increases, where one of the values must be 1.0 and all of the values must be within [0,1])#2016-07-1019:08xcthulhuI can generate these things with this function:
(defn generate-random-map
  "Return a random structured map"
  []
  (let [values (sort (repeatedly 6 rand))
        pivot-value (last values)]
    (into {} (map-indexed vector (map #(/ % pivot-value) values)))))
#2016-07-1020:19gfrederickshuh.#2016-07-1020:23gfredericksxcthulhu: https://www.refheap.com/121363#2016-07-1020:27xcthulhuI'd rather not rope in test.check. Too bad that clojure.spec doesn't let you register generators separately from specs#2016-07-1020:28xcthulhuBecause I wouldn't mind associating a generator with my specs in my unit tests where I do use test.check#2016-07-1020:34xcthulhuI think you can do this with just just spec since it gives you 'cat' and 'fmap'#2016-07-1020:35gfredericksxcthulhu: I think clojure.spec is designed with the intention that you rope in test.check for non-trivial generator needs#2016-07-1020:35gfredericksis it just that you don't want test.check required in a non-test namespace?#2016-07-1020:37xcthulhuYup#2016-07-1020:38xcthulhuBut like I said, they do give you enough so you don't need test.check, it's just clumsy#2016-07-1020:38gfrederickswhere's fmap?#2016-07-1020:39gfredericksclojure.spec.gen/fmap?#2016-07-1020:39gfredericksyou could probably rewrite my code using clojure.spec.gen if that's what you really want#2016-07-1020:40gfrederickslooks like gen/shuffle isn't there#2016-07-1020:41henrytill@xcthulhu: aren’t you already pulling in test.check as a transitive dep if yr using something the gen namespace?#2016-07-1020:41gfredericksnot necessarily#2016-07-1020:42gfrederickssince it's lazy loaded you can get away with not having test.check available if you never run the generators/tests#2016-07-1020:42gfrederickswhich I think is the whole point of the lazy loading#2016-07-1020:42henrytillok, makes sense#2016-07-1020:43xcthulhuYeah#2016-07-1020:43xcthulhuOr cljs.spec.impl.gen for the brave#2016-07-1020:44gfredericksI wonder how they do lazy loading in cljs#2016-07-1020:46xcthulhuDynaload#2016-07-1020:46xcthulhuhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/impl/gen.cljc#2016-07-1020:47gfredericksoh man#2016-07-1020:47gfredericksa custom var deftype#2016-07-1020:59xcthulhuYeah, it's pretty hacks. Anyway, if I read this right they lazily load vector from test.check. Sadly, clojureScript doesn't give you double*, only double#2016-07-1021:00xcthulhuAnyway, thank you I see how to do this crazy thing now#2016-07-1022:00xcthulhuHere's the generator I ended up going with BTW:
(gen/fmap
   (fn [values]
     (let [max-value (apply max values)
           min-value (apply min values)]
       (->> values
           sort
           (map #(-> % (- min-value) (/ (- max-value min-value))))
           (drop 1)
           (zipmap (range)))))
   (gen/vector
    (gen/such-that (complement #{infinity, (- infinity)}) (gen/double))
    7))
#2016-07-1022:01xcthulhuThis hacks around the fact that clojurescript doesn't have double*#2016-07-1022:02xcthulhuinfinity is defined to be Double/INFINITY and js/Number.INFINITY in the jvm and js respectively#2016-07-1022:02gfredericksthat should still give you nans though#2016-07-1022:02xcthulhureally?#2016-07-1022:02gfredericksgen/double generates nans#2016-07-1022:03xcthulhuNot with high probability, but I'll go add a restriction on to the such-that#2016-07-1022:03gfredericksshould be high enough to matter#2016-07-1022:03gfredericksunless I screwed something up#2016-07-1022:04gfredericksroughly 1% of the time#2016-07-1022:12xcthulhuHere we go:
(defn nan?
  "Tests if a value is a NaN or not"
  [x]
  #?(:clj (and (double? x) (.isNaN x))
     :cljs (js/isNaN x)))

;;;;;;;;;;;;;;;;

(gen/fmap
   (fn [values]
     (let [max-value (apply max values)
           min-value (apply min values)]
       (->> values
           sort
           (map #(-> % (- min-value) (/ (- max-value min-value))))
           (drop 1)
           (zipmap (range)))))
   (gen/vector
    (gen/such-that (complement (some-fn #{infinity, (- infinity)} nan?))
                   (gen/double))
    7))
#2016-07-1022:12xcthulhuHopefully this yak has been shaved by now#2016-07-1022:17gfredericks:)#2016-07-1102:22Alex Miller (Clojure team)There is s/double-in with infinity and nan support#2016-07-1106:32robert-stuttafordforgive the repost, folks, but i'm a little stumped šŸ™‚#2016-07-1106:34robert-stuttafordhow do i specify that an optional set of keys must either appear together or not at all?#2016-07-1108:34mandragoran@robert-stuttaford: you can write your own predicate (-> (s/keys :opt [...]) (s/and #(every (partial contains? %) [...]))#2016-07-1109:04robert-stuttafordtrue, thank you. i wonder if it's a use-case that has been considered by Rich et al?#2016-07-1112:51Alex Miller (Clojure team)Have you looked at the and/or support in s/keys :req?#2016-07-1112:59Alex Miller (Clojure team)was thinking something like (s/def ::combined (s/keys :req [(or (and ::a ::b) (and ::a ::b ::c ::d))]))#2016-07-1113:00Alex Miller (Clojure team)however b/c of open maps, I don’t think the latter two will fail as expected - for that you probably need and an additional pred#2016-07-1114:48robert-stuttafordthanks Alex#2016-07-1117:17semperosI wonder if there’s a place for an s/xor within the clojure.spec space where one, and only one, of the provided specs may conform, or if this can be concisely expressed with a combination of the existing features#2016-07-1119:31semperosAlex or others, would appreciate your thoughts if/when you have time ^^#2016-07-1120:55arohnerHere’s a fun one. If I start up a repl and clojure.test/run-tests, everything is fine. If I lein test, I get java.lang.Exception: :clojure.spec/unknown is not a fn, expected predicate fn#2016-07-1120:56arohnerAFAIK, I’m not doing anything weird. Everything is required in the proper order, not using load, etc#2016-07-1121:02arohnerAlso AFAIK, I’m not referring to any spec that hasn’t been required#2016-07-1121:25arohnermore weirdness: I run my tests from a repl, everything is fine. I run c.spec.tests/instrument, run the same tests, and I get No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13372. Reload all my source namespaces, everything works again#2016-07-1122:44Alex Miller (Clojure team)I have seen some stuff like this with mixture of aot and non aot on classpath#2016-07-1215:04akielCan someone please comment on http://dev.clojure.org/jira/browse/CLJ-1966? I would like to know if I possibly understand something wrong.#2016-07-1216:37peejaI'm afraid I'm still failing to understand the difference between keys and keys*. I understand that only keys* works in a cat, but I don't understand why. Is this explained somewhere?#2016-07-1216:39Alex Miller (Clojure team)@akiel rich hasn't looked at it yet#2016-07-1216:41Alex Miller (Clojure team)@peeja keys is a spec for a map. keys* is a spec for a sequential list of alternating keys and vals#2016-07-1216:42peejaOhhhhh!#2016-07-1216:42peejaSo they both work in the same contexts, but match different things#2016-07-1216:43Alex Miller (Clojure team)I'd say they work in different contexts to match the same thing :)#2016-07-1216:43peejaBy "context" I mean where they go in a spec#2016-07-1216:44akiel@alexmiller it’s ok, as long as it’s on your list - thanks#2016-07-1216:44peejaYeah, I was misreading the guide. Thanks!#2016-07-1218:51donaldballRevisiting the ā€œspec strings as char seqsā€ idea from the weekend, I just threw this together:#2016-07-1218:51donaldballhttps://gist.github.com/dball/d9f9bb7b6ea796e0de2ae49f6b21066e#2016-07-1218:51donaldballBad idea or simply misguided? šŸ˜›#2016-07-1219:16Alex Miller (Clojure team)@donaldball: from a spec perspective I think it's much better to use an actual regex for matching#2016-07-1219:17donaldballThe things I’m finding attractive about expressing specs on char seqs are that the spec regex forms are often more readable, but more to the point: you get working generators for free#2016-07-1219:18Alex Miller (Clojure team)There is a project for assembling readable regexes, can't remember the name#2016-07-1219:22arohnerthere’s https://github.com/cgrand/regex#2016-07-1219:23arohnerthat doesn’t use java.util.regex.Pattern though#2016-07-1220:24Alex Miller (Clojure team)yeah, that one#2016-07-1220:24Alex Miller (Clojure team)and also test.chuck has support for generating strings based on a regex#2016-07-1221:23sparkofreasonWhen writing regex specs, is there a way to get at the bindings "so far"? For instance, say I want to validate a datom [e a v], where a is keyword that represents a spec. I'd like to use that spec to validate v, and get "nice" output from explain that the failing spec is the one represented by a.#2016-07-1221:23danburtonoohh dependent specs, interesting#2016-07-1221:47Alex Miller (Clojure team)@dave.dixon custom predicate? (s/and <datom-spec> #(s/valid? (:a %) (:v %)))#2016-07-1221:47Alex Miller (Clojure team)doesn’t integrate into explain of course#2016-07-1221:50bhaumanMulti spec using second as the dispatch?#2016-07-1221:51bhaumanThen use a confirmed#2016-07-1221:51Alex Miller (Clojure team)nice#2016-07-1221:51bhaumanConformer#2016-07-1222:04danielcomptonhttps://clojurians.slack.com/archives/clojure/p1468341869003436#2016-07-1222:04danielcompton@alexmiller: what is the thinking behind it being under consideration? What are the downsides of allowing docstrings on specs?#2016-07-1222:13sparkofreason@alexmiller: would be nice if there were a way to pass explain results out of custom predicates. Though perhaps not required in most cases. #2016-07-1222:24Alex Miller (Clojure team)@danielcompton asking the wrong person :)#2016-07-1222:25Alex Miller (Clojure team)@dave.dixon: while it's tempting, I don't think Rich is inclined to support that for uniformity.#2016-07-1222:28danielcomptonhttp://dev.clojure.org/jira/browse/CLJ-1965#2016-07-1300:00sparkofreason@alexmiller: Anyway, I'd rather see support for this specific case, which seems common in Clojure, where we have a sequence of keywords followed by associated values. Just supporting the case of having the value validated against the spec named by the keyword would cover a lot of ground, as opposed to having a general dependent spec scheme. There's a nice symmetry with the map behavior then.#2016-07-1300:02Alex Miller (Clojure team)Actually you could do this with keys* couldn't you?#2016-07-1300:07Alex Miller (Clojure team)(s/cat :e any? :av (s/keys* :opt-un [::a]))#2016-07-1300:11seancorfieldOne thing I’m finding with clojure.spec as applied to user input is that I often seem to need a two-step process: the first is a very basic predicate and a conformer, the second is the real "spec" (and perhaps and conformer).#2016-07-1300:11seancorfieldExample, user inputs a country code and I’d like it to be case-insensitive.#2016-07-1300:12seancorfieldSo the first spec is that it’s a two-character string and the conformer is clojure.string/upper-case; then the second spec actually looks up the (now upper-case) two-character string and verifies it’s a valid ISO country code (and could conform it to country information).#2016-07-1300:13seancorfieldIs that somehow a poor use of clojure.spec? Or is there perhaps a better way to deal with input that needs a little "cleanup" before being validated/conformed?#2016-07-1300:17luke@seancorfield hm. If a non-uppercase string is to be considered ā€œvalidā€ input, then I’d say it’s on your country-code lookup function to handle case insensitive input. That seems like it would be easier than having two specs.#2016-07-1300:18lukeUnless you want to have a spec for your intermediate data format for some reason.#2016-07-1300:19seancorfieldWell, several of the inputs — and country is actually possible here — are sets of strings. For example, ::gender is #{"male" "female"} so it validates and generates nicely, but if you want it to be case-insensitive then you need to do "something" that isn’t necessarily easy with a single spec.#2016-07-1300:20seancorfield(`::country` could be (set (Locale/getISOCountries)) for example)#2016-07-1300:23seancorfieldI started leaning toward two specs because internally I wanted ::gender #{:male :female} for the domain model and "appropriate strings" for the input and the conformer there would be (comp keyword str/lower-case) … assuming you can validate the input strings with a different spec.#2016-07-1300:24seancorfieldPerhaps I’m making life more complicated for myself by trying to avoid with-gen here...#2016-07-1300:24lukeYeah… two specs isn’t bad if you do want separate models (one for the domain, one for the input) and are ok spec’ing them separately.#2016-07-1300:25lukeIt is also possible in theory to write a case-insensitive helper, which given a set of strings returns a (generating) spec that accepts any capitalization.#2016-07-1300:30seancorfieldHmm, I’ll have to give it more thought. So far have specs for input fields that conform to the domain model values, and specs for the domain model itself is feeling like the right way to go, as it allows me to generate data for both layers: API (input) and "system" (domain model).#2016-07-1300:32seancorfield(we’re pushing very hard to tease apart what is some rather complected logic in our current codebase — we currently do a sort of transform-validate-update operation on each field and we’ve figured out a nice clean way to separate validate from update — with a view to pushing the updates out to a queue and applying them asynchronously — but that pre-validation transform on a few fields is proving problematic 😸 )#2016-07-1300:33seancorfieldThanks for the input so far @luke#2016-07-1300:34lukeyep if you want separate layers, and you’ve fully reified both layers, then specs for each layer seems like the way to go!#2016-07-1300:34lukeno hard answers, just what comes off the top of my head when I hear the question šŸ™‚#2016-07-1301:05sparkofreason@alexmiller: Thanks, that works great. Knew there had to be a way to do that, but had my head stuck in regex-land...#2016-07-1301:15sparkofreasonSlightly shorter version: (s/cat :e any? :av (s/keys*))#2016-07-1302:22Alex Miller (Clojure team)Yup#2016-07-1303:17lvhIs there a way to specify ā€œstrings that match a particular regexā€ in a way that produces the correct generator?#2016-07-1303:18lvh(test.chuck I think has that)#2016-07-1303:20lvhit seems like test.check would have a real bad time generating strings until they accidentally match that regex#2016-07-1303:20lvh(that also seems like a general problem, which is why I’m assuming there’s a general soltuion)#2016-07-1303:23gfredericksthe test.chuck generator could be moved to test.check proper if somebody wants to rewrite the instaparse regex parser in pure clojure#2016-07-1303:24lvhWhat is it now? Bunch of Java?#2016-07-1303:24lvhAlso hi gfredericks burning the midnight Chicago oil too I see šŸ™‚#2016-07-1303:24lvhI haven’t really read the spec source code any maybe I should just go do that#2016-07-1303:25lvhI’m presuming there’s a way to map predicates to generators#2016-07-1303:25gfrederickslvh: I'm saying the test.chuck code uses instaparse#2016-07-1303:25lvhand I’m guessing that’s why spec/and is not just every-pred#2016-07-1303:25gfrederickslvh: hi#2016-07-1303:25lvhoh, right, and test.check doesn’t get to depend on instaparse#2016-07-1303:25lvhgot it#2016-07-1303:26gfredericksexactly#2016-07-1303:26gfredericksso you'd have to rewrite this in clojure somehow: https://github.com/gfredericks/test.chuck/blob/master/resources/com/gfredericks/test/chuck/regex.bnf#2016-07-1303:26lvhthat doesn’t sound very fun#2016-07-1303:27lvhhow would you feel about clojure.spec-specific things being added to test.chuck?#2016-07-1303:27lvhsay, a spec that understands how to map to generators in test.chuck, like, say, hypothetically, if someone wanted to match a regex šŸ˜‰#2016-07-1303:28gfredericksI keep thinking I'm going to want a similar clojure.spec utility library#2016-07-1303:29gfredericksclojure.schpec#2016-07-1303:29lvhšŸ’Æ#2016-07-1303:30lvhwould depend upon.#2016-07-1303:31lvhI guess I’m screwed anyway because this is a keyword, not a string, so unless there’s something that understands keywords, I’m going to manually assign a generator#2016-07-1303:32gfredericksI get the impression that manually assigning generators is not meant to be too rare#2016-07-1303:32lvhsame here#2016-07-1303:33lvhI’ll defer to you since you obviously have a lot more experience here, but just generating samples based on only predicates is going to break really badly really quickly#2016-07-1303:33lvh(basically as soon as the probability of the predicate passing is not very big)
#2016-07-1303:34gfredericksyeah I don't think filtering on predicates is expected to be a very useful approach#2016-07-1303:34gfredericksclojure.spec has that behavior in some places because there's not any other reasonable default, except maybe just refusing to try#2016-07-1303:35lvhyeah, fortunatelyi t’s mostly just for dev and test that you do that#2016-07-1303:36gfredericksmostly?#2016-07-1303:36lvhso it’s fine if the generator just shrugs and gives up after n consecutive failures#2016-07-1303:36lvhI have used test.check in other directions šŸ™‚#2016-07-1303:36lvhtechnically still a test I guesS?#2016-07-1303:36lvhspecifically, I generated sequences of events/actions on openstack (rackspace public cloud)#2016-07-1303:37lvhto then assert that autoscale (https://github.com/rackerlabs/otter) would still figure a way out#2016-07-1303:37lvhso it found things like ā€œit gets unhappy if there are two expected networks these servers should be attached to and one of them disappears"#2016-07-1303:38gfredericksfun#2016-07-1314:08Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/7/13/screencast-spec-leverage#2016-07-1314:08Alex Miller (Clojure team)First in a series of spec screencasts ^^#2016-07-1315:14peejaIs it possible to match data against a spec to find the paths at which a particular spec is used?#2016-07-1315:15peejaThat is, I want something like conform, but rather than look things up by tag, I want to look things up by spec keyword#2016-07-1315:15peeja(and find the tag-based paths to those parts)#2016-07-1315:16peeja"Find me every Foo in this data structure"#2016-07-1315:38Alex Miller (Clojure team)I’d say no, nothing like that. might help back up to the problem you’re trying to solve though#2016-07-1315:49peejaYeah, I think I'm actually down the wrong path here. (But it's hard to tell, since I haven't entirely decided on what I'm trying to do.) šŸ™‚#2016-07-1315:57Alex Miller (Clojure team)that’s usually a good first step :)#2016-07-1317:51joshmillerA question I posed originally in #C03S1KBA2: I’m having some trouble understanding how keywords work with respect to clojure.spec… Why is it that (s/def :spec-test-ns/name string?) works, but (s/def (keyword ā€œspec-test-nsā€ ā€œnameā€) string?) fails with CompilerException java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol? The type of both is clojure.lang.Keyword.#2016-07-1318:06semperosthis is the assert that is being tripped @joshmiller : (c/assert (c/and (named? k) (namespace k)) "k must be namespaced keyword or resolvable symbolā€)#2016-07-1318:10semperos(I know not fully helpful, sorry)#2016-07-1318:10joshmiller@semperos: Yeah — but (instance? clojure.lang.Named (keyword "spec-test-ns" "idā€)) is true and (namespace (keyword "spec-test-ns" "idā€)) is "spec-test-nsā€, so I don’t understand how that assert is failing.#2016-07-1318:12joshmillerI’m digging into what happens with macroexpand on s/def to see if maybe that’s the issue.#2016-07-1318:13semperosyeah, it’s essentially it’s trying (#'s/named? '(keyword "spec-test-ns" "nameā€))#2016-07-1318:14semperosand a List is not a named thing, note the ’ in the macro definition of s/def#2016-07-1318:15joshmillerYeah, I see (clojure.spec/def-impl (quote (keyword "spec-test-ns" "name")) (quote clojure.core/string?) string?) vs (clojure.spec/def-impl (quote :spec-test-ns/name) (quote clojure.core/string?) string?) when doing it the ā€œrightā€ way#2016-07-1318:43Alex Miller (Clojure team)I wanted to let everyone know that I have been continuing to update and extend the spec guide (at http://clojure.org/guides/spec) since the early days and it might have some things you haven’t seen yet in it. I made a list of things that have been added here: https://groups.google.com/d/msg/clojure/fYwDe3ygcm8/JZmmdjsVCQAJ#2016-07-1318:46wildermuthnThanks!#2016-07-1318:48Alex Miller (Clojure team)I don’t know that hardly anyone has noticed or used the instrument :stub/:replace functionality, which lets you do some pretty interesting things both for example tests or in tandem with check.#2016-07-1318:58joshmillerI ended up being able to dynamically generate the keywords for specs in other namespaces by using a macro where I could unquote the keyword generation, but if anyone knows of another way to go about it, I’d love to hear it.#2016-07-1320:04Oliver GeorgeIs there a standard way to add a generator to complement an instance? spec based on a constructor fn. I imagine you would want a spec/gen for each constructor arg. #2016-07-1320:06Oliver Georges/instance google.map.LatLng ::lat ::long #2016-07-1320:17Oliver GeorgeI think the answer is a combination of s/with-gen, s/gen and gen/fmap #2016-07-1321:08puzzlerWatching the screencast about how spec accomplishes multiple things at once got me thinking about how it is difficult to get all the listed benefits simultaneously. For example, if you put your specs in the same namespace as the code, you can't easily make custom generators because it depends on test.check which is generally only a dev dependency. If you put your specs in a separate namespace, you can't rely on predicates defined in your main namespace. I have not yet found a practical way to divide up code involving specs without giving something up. A lot of that has to do with Clojure's limitation/restriction regarding circular dependencies. I hope more info about how to use spec in the context of a real-world project will eventually be provided.#2016-07-1321:13donaldballYou can still make custom generators if you restrict yourself to the vars in clojure.spec.gen#2016-07-1321:21mike_ananevIs there any recommendation to put spec and fn's in one file or separate them?#2016-07-1321:23puzzler@donaldball: the guide says the vars in clojure.spec.gen just dynamically link to clojure.test-check, so those vars won't do anything unless you have clojure.test-check listed in your (dev) dependencies. Assuming I understand the guide correctly...#2016-07-1321:24puzzler@donaldball: Or is your point that the code will at least pass compilation, so it doesn't matter if those vars don't actually do anything? (I haven't tried that myself, so I don't know what happens if you don't have test.check active).#2016-07-1321:25donaldballThe latter#2016-07-1321:25puzzler@donaldball: Ah, that's helpful to know.#2016-07-1321:27puzzler@mike1452: Generally, people have been reporting that they have been separating the specs from the implementation, but there are limitations that come with that approach (as far as I can tell).#2016-07-1321:33mike_ananev@puzzler thanx! may be new screencasts from Cognitect will show us a code arrangement with spec.#2016-07-1321:38seancorfield@puzzler: We’re mostly keeping specs separate from code. So our spec ns requires our code ns, and any code that wants the specs applied requires the spec ns (and the code ns if needed). That’s how we’ve solved the circular reference issue.#2016-07-1322:42donaldballIs that mostly a concession to the fact that you’re working in a codebase that needs to support older versions of clojure or an explicit design choice?#2016-07-1322:43donaldballWhere I’m playing with specs, and I spec my fns, I tend to like having them adjacent to the fn definitions#2016-07-1323:03seancorfield@donaldball: The bulk of the specs are for layers of our domain model — data — so that’s independent of functions, so it made sense to have a separate namespace for them. And then to put the handful of function specs in there too.#2016-07-1323:03seancorfieldAnd as long as you load the spec namespace into the system before you run the functions that attempt to conform data to those specs, you’re good.#2016-07-1323:26wilkerlucioI'm choosing now to use they adjacent of the code, like donald is doing, I'm not sure if it's the best way but I'm enjoying it so far, will be nice to see the benefits/problems on the long run as people mature specs on their codebases#2016-07-1323:28wilkerluciofor those using on a separeted namespace, what name conventions are you using for the specs namespaces? @seancorfield @puzzler#2016-07-1323:31seancorfield@wilkerlucio: It depends on what we’re spec’ing. We have a set of specs for domain model data (in ns like ws.domain.something where something is the particular domain model "object" being described) and then we have a set of specs for input fields for APIs (in ns like ws.something.field).#2016-07-1323:31seancorfieldBut it’s early days and we’re moving code around. A lot.#2016-07-1323:32seancorfieldWe also have structures that tie things together (so we have hash maps that describe the relationship between domain model things, database tables and columns, validation and update routines — using qualified keywords that are resolved to symbols later).#2016-07-1323:33wilkerlucionice, sounds cool šŸ™‚#2016-07-1323:36wilkerlucioand for specs about functions, are you using something like: for namespace x.y.z have a x.y.z-specs, something like that?#2016-07-1323:54seancorfieldNo, those specs just go in the namespace for the "something" that those functions would (primarily) operate on. They’re fairly coarse-grained domain concepts (for our systems — "member" covers a lot of ground).#2016-07-1404:40mjmeintjesHi. Is there a way to use spec to generate maps with "dependent" properties. For example, suppose my map spec requires than both :a and šŸ˜› contain strings that have to be the same length?#2016-07-1404:42mjmeintjesThat should be - Is there a way to use spec to generate maps with "dependent" properties. For example, suppose my map spec requires than both :a and :b contain strings that have to be the same length?#2016-07-1404:56seancorfield@mjmeintjes: Well, you can use s/and to restrict what a spec generates but that might not be the easiest way to make the generator work...#2016-07-1405:05seancorfieldHmm, perhaps it's OK after all:
boot.user> (require '[clojure.spec :as s])
nil
boot.user> (s/def ::a string?)
:boot.user/a
boot.user> (s/def ::b string?)
:boot.user/b
boot.user> (s/def ::foo (s/and (s/keys :req [::a ::b]) #(= (count (::a %)) (count (::b %)))))
:boot.user/foo
boot.user> (s/exercise ::foo)
([#:boot.user{:a "", :b ""} #:boot.user{:a "", :b ""}] [#:boot.user{:a "z", :b "S"} #:boot.user{:a "z", :b "S"}] [#:boot.user{:a "GF", :b "y8"} #:boot.user{:a "GF", :b "y8"}] [#:boot.user{:a "c5kf9D7nt88PhP2zRiIj", :b "Ax9Ua1vd561LvuWZ0o5r"} #:boot.user{:a "c5kf9D7nt88PhP2zRiIj", :b "Ax9Ua1vd561LvuWZ0o5r"}] [#:boot.user{:a "", :b ""} #:boot.user{:a "", :b ""}] [#:boot.user{:a "yaLME3MI4uSh0", :b "FIxs99gp43jSe"} #:boot.user{:a "yaLME3MI4uSh0", :b "FIxs99gp43jSe"}] [#:boot.user{:a "9", :b "3"} #:boot.user{:a "9", :b "3"}] [#:boot.user{:a "f7nOvuwZ30U000f", :b "3enZpUsZXP9ZVX1"} #:boot.user{:a "f7nOvuwZ30U000f", :b "3enZpUsZXP9ZVX1"}] [#:boot.user{:a "8aDmY08S8Q", :b "rHFcc5rAN9"} #:boot.user{:a "8aDmY08S8Q", :b "rHFcc5rAN9"}] [#:boot.user{:a "Ds8zm6", :b "yM0N6p"} #:boot.user{:a "Ds8zm6", :b "yM0N6p"}])
#2016-07-1406:21mjmeintjes@seancorfield: Great, thanks! Looks promising, I'll investigate further.#2016-07-1414:43glvAfter converting some property-based tests to use spec’s generators, I noticed that some of the tests became much slower. Investigation reveals that the growth characteristics of spec’s generators can be quite different from those of test.check. Here’s an example:#2016-07-1414:46glvIn most cases I doubt the difference would be very noticeable, but in my case two such generated integer values were being used as the dimensions of a two-dimensional matrix structure, so the size really blew up quickly.#2016-07-1414:49glvI can see a rationale for making the growth a little steeper than the test.check version, but this seems a bit severe. šŸ™‚#2016-07-1417:04glvEspecially since the default for :num-tests is 1000. 😮#2016-07-1417:24seancorfieldI would say, in this case where the amount of data you test is N-squared based on random generated values, you probably want a custom generator that heavily restricts those dimension values?#2016-07-1417:24seancorfieldHow big, realistically, is a matrix you would be handling?#2016-07-1417:26glvOh, sure … I’ve done exactly that, now. Just wondering whether this is an appropriate default.#2016-07-1417:27glvGiven the actual behavior over 1000 iterations, you’d run into problems even if the amount of data was linear wrt the integer value.#2016-07-1417:29glvAnd given that the visible symptom is just ā€œmy test run seems to just stallā€ it’s a bit confusing.#2016-07-1417:55glv(Oh, and … my first attempt at constraining the generator was just to slap a #(< % 25) predicate on there, and then I got sporadic "Couldn't satisfy such-that predicate after 100 triesā€ failures. Which was easy enough to fix, but still leads me to think that the generator growth is just too fast.)#2016-07-1420:19Timwhat is the justification of having spec in separate namespaces than the actual functions?#2016-07-1420:44seancorfield@tmtwd: per my earlier comment https://clojurians.slack.com/archives/clojure-spec/p1468450981002667#2016-07-1421:52kendall.buchananWhat happened to instrument-ns in alpha10? Not possible anymore?#2016-07-1421:56glvfolded into instrument. (instrument ā€˜my.namespace)#2016-07-1421:57kendall.buchananAh, awesome.#2016-07-1421:58glvAh, I’ve figured out what’s going on with the growth rate of (s/gen pos-int?) mentioned above: some of the generators for built-in predicates use test.check functions that ignore the size guidance. This means (I think) that they won’t shrink properly when an error is found: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec/gen.clj#L128-L185#2016-07-1422:03glv@alexmiller: I’d love to hear your thinking on this when you get a chance. Discussion starts at https://clojurians.slack.com/archives/clojure-spec/p1468507432002694#2016-07-1422:04ghadiI think he'll be back tomorrow Glenn#2016-07-1422:04ghadiout#2016-07-1422:04glvOK, thanks!#2016-07-1512:13vikeriCan I spec an infinite sequence?#2016-07-1512:14gfredericksLike (s/* ___)?#2016-07-1512:14gfredericksI suppose the validation is the sticky part#2016-07-1512:14vikeriExactly#2016-07-1512:14gfredericksI think that's what s/every is for, if you need incomplete validation#2016-07-1512:15gfredericksI guess technically s/* isn't the same thing as infinite#2016-07-1512:15gfrederickss/āˆž#2016-07-1512:16vikeriWith s/+ I just got an infinite loop. (It tries to check every element in the infinite sequence I guess) s/every looks promising.#2016-07-1512:25vikeris/every did the trick! Thanks!#2016-07-1512:39vikeriOr, it works to validate it like this (s/valid? (s/every :my/pred) (repeat my-item)) but not inside :args in s/fdef. Any ideas?#2016-07-1512:55gfredericksdo you just mean that it's not validating at all because you haven't instrumented the function? or is it breaking in some way?#2016-07-1513:08vikeri@gfredericks It goes into an infinite loop when I put it in my fdef and then execute the function.'#2016-07-1513:09vikeriIf I test it in the repl with s/valid? it works fine#2016-07-1514:33rickmoynihanjust starting to play with spec... How would you say "A map must have either the keys :foo or :bar or both (but not neither)"#2016-07-1514:35rickmoynihanguessing I just need to supply all three combos#2016-07-1514:37minimalYeah use s/or the guide has an example http://clojure.org/guides/spec#_instrumentation#2016-07-1514:38minimal
(s/or :ok (s/keys :req [::result])
          :err (s/keys :req [::error])
#2016-07-1514:41rickmoynihanmakes perfect sense - thanks#2016-07-1515:38richhickey@rickmoynihan: there is direct support for or in keys:#2016-07-1515:39richhickeythe use of s/or in the example from the guide (mentioned above) is to distinguish 2 different kinds of maps#2016-07-1515:40rickmoynihanthanks richhickey šŸ™‚#2016-07-1515:42minimalooh nice#2016-07-1515:42rickmoynihanAm I right in thinking that the 'or symbol there is actually clojure.core/or - and spec interprets the symbol itself as meaning or?#2016-07-1515:43rickmoynihanahh yes keys is a macro#2016-07-1515:43richhickeyā€˜or’ and ā€˜and’ are syntax inside keys#2016-07-1515:43rickmoynihanwhen I first saw spec, I wondered why you hadn't done that! Super nice!#2016-07-1515:45rickmoynihansymbols are data too#2016-07-1515:53stuarthalloway@glv have any more info on test.check fns ā€œignoring size guidanceā€? Starting to read the source now, and it looks like we might need to explicitly opt in to sized in some places#2016-07-1515:55glvA lot of the generators for built-in predicates use large-integer* from test.check, which doesn’t use sized. Perhaps instead use the more specialized things like pos-int, s-pos-int, etc.?#2016-07-1515:58stuarthalloway@glv I am confused because large-integer* calls large-integer** which does use sized#2016-07-1515:58glvHmmm … I thought I traced through all that yesterday.#2016-07-1516:00stuarthalloway@glv your proposal still may be the right thing, I just want to understand why we are getting the current behavior#2016-07-1516:00glvRight, exactly. Investigating.#2016-07-1516:01stuarthalloway@glv it looks to me like s-pos-int use size on integers, and large-integer** uses size on bit count#2016-07-1516:01glvAh, right.#2016-07-1516:02stuarthallowayand if we switch to s-pos-int we would never get larger values#2016-07-1516:02glvThat might explain the weird, long-cycle behavior of (s/gen int?) in that example I pasted yesterday.#2016-07-1516:02stuarthalloway
(apply max (gen/sample gen/int 1000000))
=> 99
#2016-07-1516:03stuarthallowaygrr, maybe neither growth pattern is ideal#2016-07-1516:04glvRight. I think (I’m no expert) that’s why test.check has both. pos-int (and family) for integers that are used as sizes for things, and large-integer for integers used in mathematical functions.#2016-07-1516:04stuarthallowaywhereas spec has specific (int-in et al) and non-specific (pos-int et al)#2016-07-1516:05glvYes.#2016-07-1516:05glvWhich is what I switched to.#2016-07-1516:07glvAnd the only reason that kind of annoys me is that those are generators only, not specs. It doesn’t seem good to have to specify a custom generator for such a common case. (My hunch, which may be wrong, is that in most systems, functions that build and manipulate data structures are more common than functions that do complex math on integers.)#2016-07-1516:09stuarthalloway@glv in cases where you don’t specify, I would like to see it grow more slowly, but still grow#2016-07-1516:09glvAgreed.#2016-07-1516:09stuarthallowayhitting the hammock on this one#2016-07-1516:09stuarthallowayto be more honest, grabbing some lunch šŸ™‚#2016-07-1516:10stuarthallowaybut we do have a hammock in the office#2016-07-1516:10glvOF COURSE YOU DO#2016-07-1516:19glv@stuarthalloway: Oh wait … of course int-in is a spec, not just a generator. The real issue is that I want a large upper-bound in production, but smaller in tests (because a 100x100 matrix will be much slower with a negligible chance of exposing bugs that wouldn’t be seen in a 20x20 matrix over a bunch of tests. But there’s no reason to restrict it so tightly outside of the tests. (Maybe my case is just weird and special. But I do think the existing growth pattern is surprising, and I strongly suspect that it negatively affects shrinking.)